[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:08:26.540522 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:39773
I20260709 13:08:26.542738 21526 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:08:26.544111 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:26.565991 21533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:26.567651 21526 server_base.cc:1061] running on GCE node
W20260709 13:08:26.566663 21532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:26.566093 21535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:27.787595 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:27.787906 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:27.788208 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602507788171 us; error 0 us; skew 500 ppm
I20260709 13:08:27.789222 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:27.795993 21526 webserver.cc:533] Webserver started at http://127.21.5.190:41409/ using document root <none> and password file <none>
I20260709 13:08:27.797127 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:27.797426 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:27.798000 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:27.805207 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "93aaf247fdd94421ba5a1b63dd4dd326"
format_stamp: "Formatted at 2026-07-09 13:08:27 on dist-test-slave-h0rz"
I20260709 13:08:27.815389 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.006s	sys 0.005s
I20260709 13:08:27.822082 21541 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:27.823731 21526 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.002s	sys 0.001s
I20260709 13:08:27.824221 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "93aaf247fdd94421ba5a1b63dd4dd326"
format_stamp: "Formatted at 2026-07-09 13:08:27 on dist-test-slave-h0rz"
I20260709 13:08:27.824677 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:27.891342 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:27.893307 21526 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:08:27.893918 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:27.983110 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:39773
I20260709 13:08:27.983186 21592 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:39773 every 8 connection(s)
I20260709 13:08:27.992821 21593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:28.018584 21593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: Bootstrap starting.
I20260709 13:08:28.025956 21593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:28.028363 21593 log.cc:826] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: Log is configured to *not* fsync() on all Append() calls
I20260709 13:08:28.034283 21593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: No bootstrap required, opened a new log
I20260709 13:08:28.057679 21593 raft_consensus.cc:359] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER }
I20260709 13:08:28.058388 21593 raft_consensus.cc:385] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:28.058787 21593 raft_consensus.cc:740] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93aaf247fdd94421ba5a1b63dd4dd326, State: Initialized, Role: FOLLOWER
I20260709 13:08:28.059932 21593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER }
I20260709 13:08:28.060585 21593 raft_consensus.cc:399] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:28.060935 21593 raft_consensus.cc:493] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:28.061360 21593 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:28.066382 21593 raft_consensus.cc:515] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER }
I20260709 13:08:28.067514 21593 leader_election.cc:304] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93aaf247fdd94421ba5a1b63dd4dd326; no voters: 
I20260709 13:08:28.069701 21593 leader_election.cc:290] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:28.070139 21596 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:28.073017 21596 raft_consensus.cc:697] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 1 LEADER]: Becoming Leader. State: Replica: 93aaf247fdd94421ba5a1b63dd4dd326, State: Running, Role: LEADER
I20260709 13:08:28.074257 21596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER }
I20260709 13:08:28.075526 21593 sys_catalog.cc:565] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:28.086925 21597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93aaf247fdd94421ba5a1b63dd4dd326. Latest consensus state: current_term: 1 leader_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER } }
I20260709 13:08:28.087206 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93aaf247fdd94421ba5a1b63dd4dd326" member_type: VOTER } }
I20260709 13:08:28.088028 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:28.088060 21597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:28.097157 21605 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:28.112141 21605 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:28.115368 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:28.134466 21605 catalog_manager.cc:1383] Generated new cluster ID: f02dbd94f86f4ac488616e8cd9f39c72
I20260709 13:08:28.134910 21605 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:28.161350 21605 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:28.164039 21605 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:28.199239 21605 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: Generated new TSK 0
I20260709 13:08:28.200815 21605 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:28.247752 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:28.258595 21615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:28.259779 21616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.264961 21526 server_base.cc:1061] running on GCE node
W20260709 13:08:28.265436 21618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.266556 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:28.266903 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:28.267100 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602508267075 us; error 0 us; skew 500 ppm
I20260709 13:08:28.267771 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:28.272922 21526 webserver.cc:533] Webserver started at http://127.21.5.129:39215/ using document root <none> and password file <none>
I20260709 13:08:28.273550 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:28.273788 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:28.274106 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:28.275461 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "fbb830edd2d549148c23c189e4f8f1dd"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.282285 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.001s
I20260709 13:08:28.287899 21623 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.288986 21526 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:08:28.289371 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "fbb830edd2d549148c23c189e4f8f1dd"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.289710 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:28.333598 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:28.335399 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:28.338513 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:28.343772 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:28.344080 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.344471 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:28.344673 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.463611 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:41101
I20260709 13:08:28.463707 21685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:41101 every 8 connection(s)
I20260709 13:08:28.472230 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:28.483420 21690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:28.487198 21691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.496829 21526 server_base.cc:1061] running on GCE node
W20260709 13:08:28.497951 21694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.499185 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:28.499459 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:28.499665 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602508499646 us; error 0 us; skew 500 ppm
I20260709 13:08:28.500382 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:28.506181 21686 heartbeater.cc:344] Connected to a master server at 127.21.5.190:39773
I20260709 13:08:28.506904 21686 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:28.508292 21526 webserver.cc:533] Webserver started at http://127.21.5.130:34379/ using document root <none> and password file <none>
I20260709 13:08:28.508612 21686 heartbeater.cc:507] Master 127.21.5.190:39773 requested a full tablet report, sending...
I20260709 13:08:28.509284 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:28.509610 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:28.510006 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:28.511613 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "413ff94955e64c4fa0a9f520a7228c28"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.513561 21558 ts_manager.cc:194] Registered new tserver with Master: fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101)
I20260709 13:08:28.517181 21558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47444
I20260709 13:08:28.520310 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.002s	sys 0.007s
I20260709 13:08:28.527104 21698 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.528200 21526 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:08:28.528577 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "413ff94955e64c4fa0a9f520a7228c28"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.528949 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:28.548352 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:28.550027 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:28.552780 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:28.557416 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:28.557727 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.558014 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:28.558204 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.677613 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:39391
I20260709 13:08:28.677860 21760 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:39391 every 8 connection(s)
I20260709 13:08:28.685930 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:28.695618 21764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:28.698673 21765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.703361 21761 heartbeater.cc:344] Connected to a master server at 127.21.5.190:39773
I20260709 13:08:28.703986 21761 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:28.705323 21761 heartbeater.cc:507] Master 127.21.5.190:39773 requested a full tablet report, sending...
I20260709 13:08:28.708912 21558 ts_manager.cc:194] Registered new tserver with Master: 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
W20260709 13:08:28.709035 21767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:28.710151 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:28.711356 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:08:28.711251 21558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47452
W20260709 13:08:28.711663 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:28.711890 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602508711870 us; error 0 us; skew 500 ppm
I20260709 13:08:28.712685 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:28.716049 21526 webserver.cc:533] Webserver started at http://127.21.5.131:42879/ using document root <none> and password file <none>
I20260709 13:08:28.716604 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:28.716817 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:28.717139 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:28.718371 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.723870 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:28.728170 21772 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.729210 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:08:28.729646 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
format_stamp: "Formatted at 2026-07-09 13:08:28 on dist-test-slave-h0rz"
I20260709 13:08:28.730080 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:28.750222 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:28.752274 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:28.755414 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:28.759843 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:28.760174 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.760490 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:28.760715 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:28.867420 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:36491
I20260709 13:08:28.867480 21834 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:36491 every 8 connection(s)
I20260709 13:08:28.889719 21835 heartbeater.cc:344] Connected to a master server at 127.21.5.190:39773
I20260709 13:08:28.890247 21835 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:28.891502 21835 heartbeater.cc:507] Master 127.21.5.190:39773 requested a full tablet report, sending...
I20260709 13:08:28.894284 21558 ts_manager.cc:194] Registered new tserver with Master: b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:28.895084 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021034293s
I20260709 13:08:28.896281 21558 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47458
I20260709 13:08:28.967291 21558 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47474:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:28.973292 21558 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:29.101936 21650 tablet_service.cc:1511] Processing CreateTablet for tablet 9124493c897c4962b8be9c371f0061c7 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:29.104684 21649 tablet_service.cc:1511] Processing CreateTablet for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:29.105183 21651 tablet_service.cc:1511] Processing CreateTablet for tablet e8fe6d957cf74e89a781c57df8852d28 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:29.107919 21726 tablet_service.cc:1511] Processing CreateTablet for tablet e8fe6d957cf74e89a781c57df8852d28 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION VALUES < 238609294
W20260709 13:08:29.108556 21606 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:29.106173 21725 tablet_service.cc:1511] Processing CreateTablet for tablet 9124493c897c4962b8be9c371f0061c7 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:29.110841 21725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9124493c897c4962b8be9c371f0061c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.110710 21724 tablet_service.cc:1511] Processing CreateTablet for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:29.113139 21650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9124493c897c4962b8be9c371f0061c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.114943 21726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8fe6d957cf74e89a781c57df8852d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.116335 21648 tablet_service.cc:1511] Processing CreateTablet for tablet ccefdb565edc464f97ae56eb034bced5 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:29.120122 21649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.119704 21724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.126138 21648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefdb565edc464f97ae56eb034bced5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.131336 21723 tablet_service.cc:1511] Processing CreateTablet for tablet ccefdb565edc464f97ae56eb034bced5 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:29.134591 21723 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefdb565edc464f97ae56eb034bced5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.149710 21647 tablet_service.cc:1511] Processing CreateTablet for tablet 89b8e644a96e4fd4894324653af26a58 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:29.150686 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8fe6d957cf74e89a781c57df8852d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.161571 21647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b8e644a96e4fd4894324653af26a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.173565 21799 tablet_service.cc:1511] Processing CreateTablet for tablet 9124493c897c4962b8be9c371f0061c7 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:08:29.177351 21797 tablet_service.cc:1511] Processing CreateTablet for tablet ccefdb565edc464f97ae56eb034bced5 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:08:29.180897 21797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccefdb565edc464f97ae56eb034bced5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.173578 21798 tablet_service.cc:1511] Processing CreateTablet for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:08:29.184048 21800 tablet_service.cc:1511] Processing CreateTablet for tablet e8fe6d957cf74e89a781c57df8852d28 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:08:29.185124 21722 tablet_service.cc:1511] Processing CreateTablet for tablet 89b8e644a96e4fd4894324653af26a58 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:29.188354 21799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9124493c897c4962b8be9c371f0061c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.190538 21798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9de999ccf94a4fb6b7ebb98b37fe1ff2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.193475 21722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b8e644a96e4fd4894324653af26a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.195633 21800 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8fe6d957cf74e89a781c57df8852d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.241225 21796 tablet_service.cc:1511] Processing CreateTablet for tablet 89b8e644a96e4fd4894324653af26a58 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:08:29.244521 21796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89b8e644a96e4fd4894324653af26a58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.261646 21853 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.270469 21646 tablet_service.cc:1511] Processing CreateTablet for tablet 3d46b878f7c0425b9e93b6e0b2585666 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:29.276957 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d46b878f7c0425b9e93b6e0b2585666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.284451 21795 tablet_service.cc:1511] Processing CreateTablet for tablet 3d46b878f7c0425b9e93b6e0b2585666 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:29.289463 21795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d46b878f7c0425b9e93b6e0b2585666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.288494 21645 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4c184c59ba4a13afab0273b0cd3a0f (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:29.292521 21645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4c184c59ba4a13afab0273b0cd3a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.321843 21725 tablet_service.cc:1511] Processing CreateTablet for tablet 3d46b878f7c0425b9e93b6e0b2585666 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:08:29.350773 21725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d46b878f7c0425b9e93b6e0b2585666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.357739 21721 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4c184c59ba4a13afab0273b0cd3a0f (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:29.366648 21794 tablet_service.cc:1511] Processing CreateTablet for tablet 1f4c184c59ba4a13afab0273b0cd3a0f (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:08:29.369194 21794 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4c184c59ba4a13afab0273b0cd3a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.365877 21721 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f4c184c59ba4a13afab0273b0cd3a0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.400676 21853 tablet_bootstrap.cc:654] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.433107 21854 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:29.452472 21726 tablet_service.cc:1511] Processing CreateTablet for tablet af7e00f59b294f1fa1f3de0db396867b (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:29.454113 21726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7e00f59b294f1fa1f3de0db396867b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.463613 21651 tablet_service.cc:1511] Processing CreateTablet for tablet af7e00f59b294f1fa1f3de0db396867b (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:29.477911 21651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7e00f59b294f1fa1f3de0db396867b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.485984 21799 tablet_service.cc:1511] Processing CreateTablet for tablet af7e00f59b294f1fa1f3de0db396867b (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:08:29.487720 21646 tablet_service.cc:1511] Processing CreateTablet for tablet 719b58b414d64d1dba681c9bce2122c9 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:29.489204 21722 tablet_service.cc:1511] Processing CreateTablet for tablet 719b58b414d64d1dba681c9bce2122c9 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:29.490272 21855 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.493886 21722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719b58b414d64d1dba681c9bce2122c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.495857 21797 tablet_service.cc:1511] Processing CreateTablet for tablet 719b58b414d64d1dba681c9bce2122c9 (DEFAULT_TABLE table=test-workload [id=0cef549407bf4b90856d062e091918fa]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:08:29.504249 21646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719b58b414d64d1dba681c9bce2122c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.507161 21854 tablet_bootstrap.cc:654] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.508834 21799 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af7e00f59b294f1fa1f3de0db396867b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.510535 21797 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 719b58b414d64d1dba681c9bce2122c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:29.523545 21853 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.524159 21853 ts_tablet_manager.cc:1403] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.264s	user 0.017s	sys 0.009s
I20260709 13:08:29.524122 21686 heartbeater.cc:499] Master 127.21.5.190:39773 was elected leader, sending a full tablet report...
I20260709 13:08:29.530982 21853 raft_consensus.cc:359] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.532032 21853 raft_consensus.cc:385] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.532372 21853 raft_consensus.cc:740] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.533228 21853 consensus_queue.cc:260] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.545405 21855 tablet_bootstrap.cc:654] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.558020 21761 heartbeater.cc:499] Master 127.21.5.190:39773 was elected leader, sending a full tablet report...
I20260709 13:08:29.559360 21853 ts_tablet_manager.cc:1434] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.035s	user 0.017s	sys 0.008s
I20260709 13:08:29.560667 21853 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.567602 21858 raft_consensus.cc:493] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.571925 21858 raft_consensus.cc:515] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.590473 21858 leader_election.cc:290] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:29.594959 21855 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:29.595585 21855 ts_tablet_manager.cc:1403] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.106s	user 0.015s	sys 0.006s
I20260709 13:08:29.601459 21853 tablet_bootstrap.cc:654] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.602696 21855 raft_consensus.cc:359] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.607391 21855 raft_consensus.cc:385] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.607864 21855 raft_consensus.cc:740] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.606106 21854 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:29.608992 21855 consensus_queue.cc:260] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.609756 21854 ts_tablet_manager.cc:1403] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.228s	user 0.021s	sys 0.009s
I20260709 13:08:29.613907 21854 raft_consensus.cc:359] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.614792 21854 raft_consensus.cc:385] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.615108 21854 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.615999 21854 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.621433 21853 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.622011 21853 ts_tablet_manager.cc:1403] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.062s	user 0.012s	sys 0.002s
I20260709 13:08:29.627017 21854 ts_tablet_manager.cc:1434] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 13:08:29.627645 21853 raft_consensus.cc:359] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.628504 21853 raft_consensus.cc:385] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.628829 21853 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.629971 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:29.632730 21854 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
W20260709 13:08:29.632961 21702 leader_election.cc:343] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:29.636669 21853 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.642351 21853 ts_tablet_manager.cc:1434] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.005s
I20260709 13:08:29.643712 21853 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.644069 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:29.645352 21810 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:29.652293 21700 leader_election.cc:304] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:29.652961 21858 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.655638 21854 tablet_bootstrap.cc:654] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.656929 21858 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.660251 21853 tablet_bootstrap.cc:654] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.661226 21858 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:29.661849 21858 raft_consensus.cc:2804] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:29.662196 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:29.663057 21661 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:29.662236 21858 raft_consensus.cc:493] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.665989 21858 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:29.664428 21702 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:29.675603 21866 raft_consensus.cc:2804] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:29.676096 21866 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.676465 21866 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:29.678745 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:29.679979 21855 ts_tablet_manager.cc:1434] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.084s	user 0.027s	sys 0.043s
I20260709 13:08:29.678768 21835 heartbeater.cc:499] Master 127.21.5.190:39773 was elected leader, sending a full tablet report...
I20260709 13:08:29.681247 21855 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.681097 21858 raft_consensus.cc:515] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.683523 21858 leader_election.cc:290] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:29.685542 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
W20260709 13:08:29.686237 21762 tablet.cc:2406] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260709 13:08:29.688444 21702 leader_election.cc:343] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:29.693917 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:29.694710 21810 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:29.695299 21855 tablet_bootstrap.cc:654] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.695545 21854 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:29.697642 21854 ts_tablet_manager.cc:1403] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.065s	user 0.014s	sys 0.008s
W20260709 13:08:29.696056 21700 leader_election.cc:343] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:29.701443 21854 raft_consensus.cc:359] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.702548 21854 raft_consensus.cc:385] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.702978 21854 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.704107 21854 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.709913 21866 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.713524 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:29.714222 21661 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:29.718737 21854 ts_tablet_manager.cc:1434] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:08:29.719980 21854 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:29.706594 21810 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:29.721582 21700 leader_election.cc:304] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:29.722575 21858 raft_consensus.cc:2804] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:29.736291 21853 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.736950 21853 ts_tablet_manager.cc:1403] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.013s
I20260709 13:08:29.740602 21853 raft_consensus.cc:359] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.741528 21853 raft_consensus.cc:385] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.741895 21853 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.742934 21853 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.744068 21858 raft_consensus.cc:697] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:29.746294 21858 consensus_queue.cc:237] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.747423 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
W20260709 13:08:29.749040 21700 leader_election.cc:343] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:29.749454 21866 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:29.752581 21853 ts_tablet_manager.cc:1434] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 13:08:29.753863 21853 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.753693 21661 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:29.755923 21702 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:29.757740 21866 raft_consensus.cc:2804] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:29.758450 21866 raft_consensus.cc:697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:29.759516 21866 consensus_queue.cc:237] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.760465 21855 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:29.761031 21855 ts_tablet_manager.cc:1403] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.035s
I20260709 13:08:29.761047 21854 tablet_bootstrap.cc:654] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.764528 21855 raft_consensus.cc:359] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.765374 21855 raft_consensus.cc:385] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.765689 21855 raft_consensus.cc:740] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.766532 21855 consensus_queue.cc:260] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.771656 21853 tablet_bootstrap.cc:654] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.773676 21855 ts_tablet_manager.cc:1434] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:08:29.774768 21855 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.784864 21557 catalog_manager.cc:5697] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:29.787438 21855 tablet_bootstrap.cc:654] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.801225 21853 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.801837 21853 ts_tablet_manager.cc:1403] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.009s
I20260709 13:08:29.805195 21853 raft_consensus.cc:359] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.806141 21853 raft_consensus.cc:385] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.806447 21853 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.807299 21853 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.809638 21853 ts_tablet_manager.cc:1434] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:08:29.810683 21853 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.817564 21855 tablet_bootstrap.cc:492] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:29.818171 21855 ts_tablet_manager.cc:1403] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.001s
I20260709 13:08:29.820007 21854 tablet_bootstrap.cc:492] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:29.820703 21854 ts_tablet_manager.cc:1403] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.101s	user 0.038s	sys 0.018s
I20260709 13:08:29.821689 21855 raft_consensus.cc:359] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.822507 21855 raft_consensus.cc:385] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.822865 21855 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.823876 21855 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.824661 21854 raft_consensus.cc:359] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.825733 21854 raft_consensus.cc:385] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.826143 21854 raft_consensus.cc:740] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.827328 21854 consensus_queue.cc:260] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.829478 21855 ts_tablet_manager.cc:1434] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.008s
I20260709 13:08:29.830907 21855 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.835065 21853 tablet_bootstrap.cc:654] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.835875 21854 ts_tablet_manager.cc:1434] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.003s
I20260709 13:08:29.837013 21854 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:29.845593 21855 tablet_bootstrap.cc:654] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.855890 21853 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.856487 21853 ts_tablet_manager.cc:1403] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.046s	user 0.002s	sys 0.014s
I20260709 13:08:29.856491 21557 catalog_manager.cc:5697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:29.860110 21853 raft_consensus.cc:359] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.860966 21853 raft_consensus.cc:385] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.861255 21853 raft_consensus.cc:740] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.862080 21853 consensus_queue.cc:260] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.863500 21854 tablet_bootstrap.cc:654] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.864630 21853 ts_tablet_manager.cc:1434] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:29.865679 21853 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.869107 21855 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:29.869693 21855 ts_tablet_manager.cc:1403] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.039s	user 0.008s	sys 0.004s
I20260709 13:08:29.873167 21855 raft_consensus.cc:359] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.874050 21855 raft_consensus.cc:385] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.874384 21855 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.875420 21855 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.878867 21855 ts_tablet_manager.cc:1434] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:08:29.880131 21855 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.887770 21854 tablet_bootstrap.cc:492] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:29.888386 21853 tablet_bootstrap.cc:654] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.888382 21854 ts_tablet_manager.cc:1403] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.008s
I20260709 13:08:29.893551 21854 raft_consensus.cc:359] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.894503 21854 raft_consensus.cc:385] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.894920 21854 raft_consensus.cc:740] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.895866 21854 consensus_queue.cc:260] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.896445 21855 tablet_bootstrap.cc:654] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.903100 21854 ts_tablet_manager.cc:1434] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:08:29.905277 21853 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:29.905918 21853 ts_tablet_manager.cc:1403] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.003s
I20260709 13:08:29.906313 21854 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:29.909307 21853 raft_consensus.cc:359] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.911119 21853 raft_consensus.cc:385] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.911490 21853 raft_consensus.cc:740] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.912286 21853 consensus_queue.cc:260] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.914577 21853 ts_tablet_manager.cc:1434] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:08:29.915711 21853 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:29.917584 21854 tablet_bootstrap.cc:654] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.918174 21855 tablet_bootstrap.cc:492] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:29.918802 21855 ts_tablet_manager.cc:1403] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.039s	user 0.014s	sys 0.002s
I20260709 13:08:29.923482 21853 tablet_bootstrap.cc:654] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:29.922549 21855 raft_consensus.cc:359] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.931592 21855 raft_consensus.cc:385] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.931964 21855 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.932869 21855 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.935392 21855 ts_tablet_manager.cc:1434] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 13:08:29.926867 21854 tablet_bootstrap.cc:492] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:29.936331 21854 ts_tablet_manager.cc:1403] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.007s
I20260709 13:08:29.940102 21854 raft_consensus.cc:359] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.941236 21854 raft_consensus.cc:385] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:29.941658 21854 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:29.943280 21863 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.943866 21863 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.945695 21864 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.944407 21855 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:29.946228 21864 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.950173 21863 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:29.947697 21854 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:29.961395 21864 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:29.995687 21855 tablet_bootstrap.cc:654] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.020840 21853 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:30.024138 21853 ts_tablet_manager.cc:1403] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.109s	user 0.014s	sys 0.027s
I20260709 13:08:30.026959 21854 ts_tablet_manager.cc:1434] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.090s	user 0.012s	sys 0.012s
I20260709 13:08:30.028204 21863 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.029565 21863 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.041483 21887 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.040850 21854 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:30.044082 21887 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.060786 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.058681 21864 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.062337 21661 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 0.
I20260709 13:08:30.065171 21853 raft_consensus.cc:359] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.066015 21853 raft_consensus.cc:385] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.065207 21863 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:30.063261 21864 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.070206 21776 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.070291 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.073701 21660 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 0.
I20260709 13:08:30.073971 21864 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.074594 21863 raft_consensus.cc:2804] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.074759 21864 raft_consensus.cc:493] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.075173 21776 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.075079 21863 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.069387 21853 raft_consensus.cc:740] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.077589 21863 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.079711 21887 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.075523 21864 raft_consensus.cc:515] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.087621 21855 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:30.091150 21855 ts_tablet_manager.cc:1403] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.147s	user 0.020s	sys 0.007s
I20260709 13:08:30.082783 21853 consensus_queue.cc:260] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
W20260709 13:08:30.110888 21606 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:30.120893 21864 leader_election.cc:290] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.119613 21853 ts_tablet_manager.cc:1434] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.094s	user 0.011s	sys 0.005s
I20260709 13:08:30.121415 21866 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.131246 21866 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.157980 21853 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:30.159538 21894 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.159729 21866 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.167498 21855 raft_consensus.cc:359] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.162293 21858 raft_consensus.cc:493] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.176483 21854 tablet_bootstrap.cc:654] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.186255 21869 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.187682 21895 raft_consensus.cc:2804] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.187759 21863 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
W20260709 13:08:30.164465 21776 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41101, user_credentials={real_user=slave}} blocked reactor thread for 89356us
I20260709 13:08:30.168041 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.185061 21855 raft_consensus.cc:385] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.165980 21894 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.185608 21858 raft_consensus.cc:515] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.191849 21866 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:08:30.190201 21736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.190155 21869 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.190886 21895 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.199769 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:30.200197 21895 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.200443 21660 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.194855 21855 raft_consensus.cc:740] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.202834 21858 leader_election.cc:290] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.195475 21863 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:30.192979 21810 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:30.203025 21855 consensus_queue.cc:260] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.191047 21735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.207899 21735 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 0.
I20260709 13:08:30.210161 21809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.210542 21700 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:30.211673 21809 raft_consensus.cc:2468] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:30.204159 21736 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 0.
I20260709 13:08:30.213515 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:30.214529 21858 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.214880 21734 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.215081 21858 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.217541 21858 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.211611 21661 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.224956 21700 leader_election.cc:304] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:30.230391 21659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.244943 21866 raft_consensus.cc:2804] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.246721 21659 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:30.245411 21866 raft_consensus.cc:493] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.249297 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.253314 21855 ts_tablet_manager.cc:1434] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.159s	user 0.010s	sys 0.017s
I20260709 13:08:30.250538 21866 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.255445 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.253692 21736 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.253717 21735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.255316 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.261901 21855 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:30.260485 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.260888 21810 raft_consensus.cc:2393] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 413ff94955e64c4fa0a9f520a7228c28 in current term 1: Already voted for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in this term.
I20260709 13:08:30.259733 21736 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.269544 21700 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28; no voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:30.271441 21624 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.260946 21869 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.281338 21660 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 1.
I20260709 13:08:30.272477 21735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.282979 21893 raft_consensus.cc:2804] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.283761 21893 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.284222 21893 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.283506 21776 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.281607 21895 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
W20260709 13:08:30.272140 21702 leader_election.cc:343] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:30.293836 21810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.294860 21810 raft_consensus.cc:2393] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbb830edd2d549148c23c189e4f8f1dd in current term 1: Already voted for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in this term.
I20260709 13:08:30.288545 21863 raft_consensus.cc:2804] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.310372 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.311591 21807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.304499 21894 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:30.311973 21808 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.317498 21735 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.309266 21863 raft_consensus.cc:697] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:30.310483 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.310950 21858 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.330475 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.319214 21734 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 1.
I20260709 13:08:30.339233 21660 raft_consensus.cc:2393] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b7ddf4ab5d0e4f6397d4240c58659e03 in current term 1: Already voted for candidate 413ff94955e64c4fa0a9f520a7228c28 in this term.
I20260709 13:08:30.340425 21853 tablet_bootstrap.cc:654] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.340651 21733 raft_consensus.cc:2393] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fbb830edd2d549148c23c189e4f8f1dd in current term 1: Already voted for candidate 413ff94955e64c4fa0a9f520a7228c28 in this term.
I20260709 13:08:30.341867 21854 tablet_bootstrap.cc:492] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:30.342954 21869 raft_consensus.cc:2749] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.345084 21863 consensus_queue.cc:237] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.349721 21858 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.352133 21895 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:30.344691 21625 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.357026 21864 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.359515 21625 leader_election.cc:304] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbb830edd2d549148c23c189e4f8f1dd; no voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:30.359714 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:30.353452 21854 ts_tablet_manager.cc:1403] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.316s	user 0.018s	sys 0.027s
I20260709 13:08:30.360460 21887 raft_consensus.cc:2749] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.357609 21864 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.361179 21864 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.373240 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.374810 21806 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.373482 21869 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:08:30.378099 21854 raft_consensus.cc:359] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.379421 21854 raft_consensus.cc:385] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.381552 21809 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.368489 21866 raft_consensus.cc:515] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.385705 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:30.386389 21733 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.379822 21854 raft_consensus.cc:740] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.383885 21898 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:30.379833 21661 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:30.397979 21866 leader_election.cc:290] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.380200 21776 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:30.400384 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.401844 21808 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.397591 21854 consensus_queue.cc:260] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.405884 21855 tablet_bootstrap.cc:654] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.406128 21854 ts_tablet_manager.cc:1434] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.045s	user 0.003s	sys 0.004s
I20260709 13:08:30.407348 21900 raft_consensus.cc:493] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.415661 21806 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:30.415939 21659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:30.416622 21659 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.423982 21700 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:30.408787 21900 raft_consensus.cc:515] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.416544 21854 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:30.428197 21866 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.446997 21625 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbb830edd2d549148c23c189e4f8f1dd; no voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:30.448669 21887 raft_consensus.cc:2749] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.443919 21866 raft_consensus.cc:697] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:30.453068 21895 raft_consensus.cc:2749] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.457470 21853 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:30.457259 21808 raft_consensus.cc:2468] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:30.458851 21853 ts_tablet_manager.cc:1403] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.303s	user 0.017s	sys 0.017s
I20260709 13:08:30.457955 21893 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.450178 21866 consensus_queue.cc:237] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.476406 21661 raft_consensus.cc:2393] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b7ddf4ab5d0e4f6397d4240c58659e03 in current term 1: Already voted for candidate fbb830edd2d549148c23c189e4f8f1dd in this term.
I20260709 13:08:30.478179 21700 leader_election.cc:304] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:30.479140 21858 raft_consensus.cc:2804] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.479641 21858 raft_consensus.cc:697] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:30.467434 21659 raft_consensus.cc:2468] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:30.483529 21900 leader_election.cc:290] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.482016 21858 consensus_queue.cc:237] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.457899 21558 catalog_manager.cc:5697] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:30.488508 21900 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:30.498152 21864 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.488354 21659 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.500471 21660 raft_consensus.cc:2393] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 413ff94955e64c4fa0a9f520a7228c28 in current term 1: Already voted for candidate fbb830edd2d549148c23c189e4f8f1dd in this term.
I20260709 13:08:30.508349 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
W20260709 13:08:30.530049 21702 leader_election.cc:343] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:30.538375 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:30.545360 21807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.535228 21809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.547744 21807 raft_consensus.cc:2393] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fbb830edd2d549148c23c189e4f8f1dd in current term 1: Already voted for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in this term.
I20260709 13:08:30.541594 21855 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:30.554589 21855 ts_tablet_manager.cc:1403] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.298s	user 0.019s	sys 0.017s
I20260709 13:08:30.546674 21893 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.564723 21854 tablet_bootstrap.cc:654] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.566581 21853 raft_consensus.cc:359] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.571197 21853 raft_consensus.cc:385] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.571684 21853 raft_consensus.cc:740] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.572726 21809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.573689 21809 raft_consensus.cc:2393] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate fbb830edd2d549148c23c189e4f8f1dd in current term 1: Already voted for candidate 413ff94955e64c4fa0a9f520a7228c28 in this term.
I20260709 13:08:30.581056 21853 consensus_queue.cc:260] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.596757 21853 ts_tablet_manager.cc:1434] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.130s	user 0.012s	sys 0.001s
I20260709 13:08:30.598124 21853 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: Bootstrap starting.
I20260709 13:08:30.599715 21864 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.598894 21733 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 1.
I20260709 13:08:30.606642 21773 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:30.608525 21895 raft_consensus.cc:2804] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:30.608686 21700 leader_election.cc:343] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:30.609130 21895 raft_consensus.cc:697] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:30.609253 21700 leader_election.cc:304] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28; no voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:30.610100 21869 raft_consensus.cc:2749] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.604338 21625 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbb830edd2d549148c23c189e4f8f1dd; no voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:30.610066 21895 consensus_queue.cc:237] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.611009 21855 raft_consensus.cc:359] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.611941 21864 raft_consensus.cc:2749] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.611604 21625 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbb830edd2d549148c23c189e4f8f1dd; no voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:30.611733 21855 raft_consensus.cc:385] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.613085 21855 raft_consensus.cc:740] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.613269 21864 raft_consensus.cc:2749] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:30.616113 21855 consensus_queue.cc:260] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.617008 21854 tablet_bootstrap.cc:492] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:30.617599 21854 ts_tablet_manager.cc:1403] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.201s	user 0.016s	sys 0.019s
I20260709 13:08:30.621752 21855 ts_tablet_manager.cc:1434] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.067s	user 0.009s	sys 0.001s
I20260709 13:08:30.622189 21853 tablet_bootstrap.cc:654] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.624353 21855 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:30.634435 21854 raft_consensus.cc:359] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.635633 21854 raft_consensus.cc:385] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.636045 21854 raft_consensus.cc:740] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.635370 21555 catalog_manager.cc:5697] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.642048 21853 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: No bootstrap required, opened a new log
I20260709 13:08:30.642756 21853 ts_tablet_manager.cc:1403] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.001s
I20260709 13:08:30.642198 21854 consensus_queue.cc:260] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.648780 21853 raft_consensus.cc:359] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.649454 21855 tablet_bootstrap.cc:654] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.649026 21854 ts_tablet_manager.cc:1434] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.001s
I20260709 13:08:30.649731 21853 raft_consensus.cc:385] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.650103 21853 raft_consensus.cc:740] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.651158 21853 consensus_queue.cc:260] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.657919 21853 ts_tablet_manager.cc:1434] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:08:30.658852 21854 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:30.673761 21854 tablet_bootstrap.cc:654] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.673019 21557 catalog_manager.cc:5697] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:30.677557 21855 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:30.678156 21855 ts_tablet_manager.cc:1403] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.006s
I20260709 13:08:30.681870 21855 raft_consensus.cc:359] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.682889 21855 raft_consensus.cc:385] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.683246 21855 raft_consensus.cc:740] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.684185 21855 consensus_queue.cc:260] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.687767 21855 ts_tablet_manager.cc:1434] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:08:30.688401 21854 tablet_bootstrap.cc:492] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:30.688694 21855 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: Bootstrap starting.
I20260709 13:08:30.688892 21854 ts_tablet_manager.cc:1403] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.008s
I20260709 13:08:30.691570 21854 raft_consensus.cc:359] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.692351 21854 raft_consensus.cc:385] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.692866 21854 raft_consensus.cc:740] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.693734 21854 consensus_queue.cc:260] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.696231 21854 ts_tablet_manager.cc:1434] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:08:30.696380 21855 tablet_bootstrap.cc:654] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.695564 21558 catalog_manager.cc:5697] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.697458 21854 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: Bootstrap starting.
I20260709 13:08:30.708222 21855 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: No bootstrap required, opened a new log
I20260709 13:08:30.708778 21855 ts_tablet_manager.cc:1403] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:08:30.709165 21854 tablet_bootstrap.cc:654] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:30.717139 21855 raft_consensus.cc:359] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.717988 21855 raft_consensus.cc:385] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.718320 21855 raft_consensus.cc:740] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.719074 21855 consensus_queue.cc:260] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.722302 21855 ts_tablet_manager.cc:1434] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:08:30.725864 21854 tablet_bootstrap.cc:492] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: No bootstrap required, opened a new log
I20260709 13:08:30.726426 21854 ts_tablet_manager.cc:1403] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.003s
I20260709 13:08:30.729835 21854 raft_consensus.cc:359] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.730765 21854 raft_consensus.cc:385] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:30.731076 21854 raft_consensus.cc:740] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Initialized, Role: FOLLOWER
I20260709 13:08:30.731930 21854 consensus_queue.cc:260] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.733855 21854 ts_tablet_manager.cc:1434] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 13:08:30.774281 21864 raft_consensus.cc:493] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.774904 21864 raft_consensus.cc:515] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.776976 21864 leader_election.cc:290] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.777983 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.778326 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.778894 21733 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.779034 21808 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.780045 21624 leader_election.cc:304] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.780850 21864 raft_consensus.cc:2804] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.781236 21864 raft_consensus.cc:493] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.781541 21864 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.788663 21864 raft_consensus.cc:515] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.791464 21864 leader_election.cc:290] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.792584 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:30.792795 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.793383 21808 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.793363 21733 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.801391 21733 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 1.
I20260709 13:08:30.801391 21808 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 1.
I20260709 13:08:30.803192 21624 leader_election.cc:304] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.804205 21864 raft_consensus.cc:2804] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.804790 21864 raft_consensus.cc:697] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:30.805612 21864 consensus_queue.cc:237] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.808202 21893 raft_consensus.cc:493] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.808902 21893 raft_consensus.cc:515] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.811558 21893 leader_election.cc:290] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.813537 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.814325 21808 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.815025 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28" is_pre_election: true
I20260709 13:08:30.816185 21625 leader_election.cc:304] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.816900 21733 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 0.
I20260709 13:08:30.817152 21893 raft_consensus.cc:2804] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.817667 21893 raft_consensus.cc:493] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.818074 21893 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.816926 21556 catalog_manager.cc:5697] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 0 to 1, leader changed from <none> to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 1 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.826357 21893 raft_consensus.cc:515] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.830996 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:30.832525 21733 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.833532 21893 leader_election.cc:290] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.834648 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.835579 21808 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.835930 21895 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.842713 21733 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 1.
I20260709 13:08:30.844264 21624 leader_election.cc:304] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.844367 21808 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 1.
I20260709 13:08:30.845292 21893 raft_consensus.cc:2804] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.845834 21893 raft_consensus.cc:697] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:30.847635 21893 consensus_queue.cc:237] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.862870 21895 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.884198 21866 raft_consensus.cc:493] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.884867 21866 raft_consensus.cc:515] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.888106 21858 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.891665 21866 leader_election.cc:290] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.890966 21557 catalog_manager.cc:5697] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 0 to 1, leader changed from <none> to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 1 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:30.893766 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" is_pre_election: true
I20260709 13:08:30.894980 21808 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:30.896627 21700 leader_election.cc:304] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:30.898361 21858 raft_consensus.cc:2804] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:30.898821 21858 raft_consensus.cc:493] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.899176 21858 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.893930 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd" is_pre_election: true
I20260709 13:08:30.904117 21660 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 0.
I20260709 13:08:30.907163 21869 consensus_queue.cc:1048] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.910979 21869 consensus_queue.cc:1048] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.923252 21858 raft_consensus.cc:515] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.939918 21858 leader_election.cc:290] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:30.940292 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:30.941007 21808 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.942147 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:30.942870 21658 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:30.943627 21904 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:30.949844 21808 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:30.957003 21658 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 1.
I20260709 13:08:30.958518 21702 leader_election.cc:304] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:30.959689 21904 raft_consensus.cc:2804] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:30.960224 21904 raft_consensus.cc:697] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:30.961234 21904 consensus_queue.cc:237] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:30.994385 21556 catalog_manager.cc:5697] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.069408 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60790
I20260709 13:08:31.070302 21734 raft_consensus.cc:606] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:31.076550 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60790
I20260709 13:08:31.077278 21734 raft_consensus.cc:606] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:31.083184 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60790
I20260709 13:08:31.083873 21734 raft_consensus.cc:606] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:31.101212 21808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:56620
I20260709 13:08:31.101890 21808 raft_consensus.cc:606] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:31.107911 21808 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:56620
I20260709 13:08:31.108486 21808 raft_consensus.cc:606] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:31.136603 21660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:50994
I20260709 13:08:31.137249 21660 raft_consensus.cc:606] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:31.143564 21660 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:50994
I20260709 13:08:31.144152 21660 raft_consensus.cc:606] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Received request to transfer leadership to TS b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:31.169984 21863 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:31.184284 21927 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.196317 21927 raft_consensus.cc:993] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:31.196966 21895 raft_consensus.cc:1081] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:31.199241 21734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60776
I20260709 13:08:31.199899 21734 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.200300 21734 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.208256 21734 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.210484 21893 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.211880 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.212533 21808 raft_consensus.cc:3055] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.212910 21808 raft_consensus.cc:740] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:31.213781 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:31.214004 21734 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 2 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.214419 21660 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.213796 21808 consensus_queue.cc:260] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.218298 21808 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.220690 21864 raft_consensus.cc:993] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:31.222105 21660 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 2.
I20260709 13:08:31.222307 21893 raft_consensus.cc:1081] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:31.225131 21702 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.226761 21858 raft_consensus.cc:2804] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.227273 21858 raft_consensus.cc:697] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:31.228143 21858 consensus_queue.cc:237] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.225884 21734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60788
I20260709 13:08:31.231067 21734 raft_consensus.cc:493] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.231617 21734 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.237627 21808 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 2.
I20260709 13:08:31.244473 21864 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.258118 21734 raft_consensus.cc:515] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.262439 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:31.263667 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.264091 21660 raft_consensus.cc:3055] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.264578 21660 raft_consensus.cc:740] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.264676 21808 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.265545 21660 consensus_queue.cc:260] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.266858 21660 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.267774 21555 catalog_manager.cc:5697] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 1 to 2, leader changed from b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.273536 21808 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 2.
I20260709 13:08:31.275054 21700 leader_election.cc:304] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:31.275401 21660 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 2.
I20260709 13:08:31.276139 21858 raft_consensus.cc:2804] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.276659 21858 raft_consensus.cc:697] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:31.277071 21734 leader_election.cc:290] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 2 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.277580 21858 consensus_queue.cc:237] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.287919 21556 catalog_manager.cc:5697] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 1 to 2, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.302049 21895 raft_consensus.cc:993] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.302559 21863 raft_consensus.cc:1081] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.304349 21660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50980
I20260709 13:08:31.304926 21660 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.305294 21660 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.311156 21660 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.313256 21660 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.314438 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:31.314672 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.315217 21734 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.315351 21808 raft_consensus.cc:3055] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.315721 21808 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:31.316536 21808 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.317597 21808 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.323563 21808 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.323568 21734 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.325093 21625 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.326114 21942 raft_consensus.cc:2804] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.326608 21942 raft_consensus.cc:697] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.327526 21942 consensus_queue.cc:237] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.336015 21556 catalog_manager.cc:5697] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 1 to 2, leader changed from b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 2 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.359009 21942 consensus_queue.cc:1048] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:31.372960 21942 consensus_queue.cc:1048] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.395090 21942 raft_consensus.cc:993] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:31.395619 21942 raft_consensus.cc:1081] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Signalling peer b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:31.397593 21808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:56618
I20260709 13:08:31.398406 21808 raft_consensus.cc:493] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.398905 21808 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.406795 21808 raft_consensus.cc:515] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.409881 21808 leader_election.cc:290] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 2 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:31.410463 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:31.411226 21660 raft_consensus.cc:3055] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.411597 21660 raft_consensus.cc:740] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.410563 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "719b58b414d64d1dba681c9bce2122c9" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:31.412398 21734 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.412521 21660 consensus_queue.cc:260] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.413733 21660 raft_consensus.cc:3060] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.420969 21858 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.426749 21660 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 2.
I20260709 13:08:31.427186 21904 raft_consensus.cc:993] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.427693 21904 raft_consensus.cc:1081] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.429325 21776 leader_election.cc:304] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.429872 21734 raft_consensus.cc:2468] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 2.
I20260709 13:08:31.430709 21863 raft_consensus.cc:2804] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.431222 21863 raft_consensus.cc:697] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:31.432104 21863 consensus_queue.cc:237] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.435201 21866 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:31.448467 21660 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:31.457298 21555 catalog_manager.cc:5697] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 1 to 2, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:31.452550 21660 raft_consensus.cc:493] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.475395 21660 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.480753 21866 raft_consensus.cc:993] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.481330 21858 raft_consensus.cc:1081] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.483297 21661 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:31.483996 21661 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.484432 21661 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.492164 21661 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.494581 21661 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.495739 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:31.496344 21734 raft_consensus.cc:3055] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.496577 21734 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:31.497283 21734 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.497594 21660 raft_consensus.cc:515] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.498168 21734 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.499924 21660 leader_election.cc:290] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.501049 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:31.501763 21733 raft_consensus.cc:3055] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.501966 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.502692 21808 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.505147 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.505931 21806 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.502146 21733 raft_consensus.cc:740] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:31.510947 21808 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.511499 21733 consensus_queue.cc:260] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.512240 21625 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.512933 21733 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.513054 21941 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.513605 21941 raft_consensus.cc:697] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.514354 21941 consensus_queue.cc:237] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.522686 21904 raft_consensus.cc:993] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.523265 21904 raft_consensus.cc:1081] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:31.529042 21733 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.529280 21658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:31.530210 21556 catalog_manager.cc:5697] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 1 to 2, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 2 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.538245 21658 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:31.538754 21658 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.538961 21624 leader_election.cc:304] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.540016 21941 raft_consensus.cc:2804] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.540786 21941 raft_consensus.cc:697] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.541783 21941 consensus_queue.cc:237] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.536055 21734 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.537354 21806 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.549031 21658 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.551929 21658 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:31.552944 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:31.553782 21733 raft_consensus.cc:3055] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:31.553918 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:31.554373 21733 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:31.554662 21806 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.555784 21733 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.557374 21733 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.561252 21555 catalog_manager.cc:5697] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 1 to 2, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 2 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.564364 21806 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.565997 21625 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:31.566792 21733 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:31.567163 21941 raft_consensus.cc:2804] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:31.567818 21941 raft_consensus.cc:697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:31.568794 21941 consensus_queue.cc:237] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:31.581979 21555 catalog_manager.cc:5697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 1 to 2, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 2 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:31.670521 21658 raft_consensus.cc:1275] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.672212 21858 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:31.683195 21806 raft_consensus.cc:1275] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.685292 21904 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:31.751020 21808 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.752715 21941 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:31.764412 21733 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.766871 21942 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:31.821527 21658 raft_consensus.cc:1275] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.823050 21904 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:31.833930 21808 raft_consensus.cc:1275] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.837502 21904 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:31.938371 21808 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.939828 21942 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:31.948025 21734 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.950047 21864 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:32.005296 21658 raft_consensus.cc:1275] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.007000 21895 consensus_queue.cc:1048] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:32.008961 21734 raft_consensus.cc:1275] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.010440 21733 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.010648 21895 consensus_queue.cc:1048] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:32.012084 21864 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:32.028560 21808 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.030495 21941 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:32.109510 21808 raft_consensus.cc:1275] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.111191 21941 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:32.112877 21734 raft_consensus.cc:1275] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:32.114452 21941 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:34.181391 21526 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:08:34.182403 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:34.205178 21733 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:34.205832 21733 raft_consensus.cc:606] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Received request to transfer leadership to TS b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:34.218134 21658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:51004
I20260709 13:08:34.218847 21658 raft_consensus.cc:606] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Received request to transfer leadership to TS b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:34.220181 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:34.220741 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:34.409636 21941 raft_consensus.cc:993] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:34.410178 21957 raft_consensus.cc:1081] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Signalling peer b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:34.411913 21806 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:56618
I20260709 13:08:34.412475 21806 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.412814 21806 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.418146 21806 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.420156 21806 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:34.421379 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:34.421379 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:34.422258 21733 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.422413 21658 raft_consensus.cc:3055] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:34.422878 21658 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:34.423709 21658 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.424904 21658 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.429100 21733 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 3.
I20260709 13:08:34.430564 21773 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:34.431046 21658 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 3.
I20260709 13:08:34.431605 21895 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.433202 21895 raft_consensus.cc:697] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:34.434144 21895 consensus_queue.cc:237] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.443519 21960 raft_consensus.cc:993] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:34.444032 21858 raft_consensus.cc:1081] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Signalling peer b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:34.443356 21555 catalog_manager.cc:5697] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 2 to 3, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.445837 21806 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:56614
I20260709 13:08:34.446395 21806 raft_consensus.cc:493] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:34.446779 21806 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.453562 21806 raft_consensus.cc:515] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.457316 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:34.456837 21806 leader_election.cc:290] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 2 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:34.458043 21658 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.457479 21733 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:34.458719 21733 raft_consensus.cc:3055] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:34.459035 21733 raft_consensus.cc:740] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:34.459780 21733 consensus_queue.cc:260] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.460764 21733 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:34.464335 21658 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 2.
I20260709 13:08:34.466116 21776 leader_election.cc:304] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:34.467020 21895 raft_consensus.cc:2804] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:34.467460 21895 raft_consensus.cc:697] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:34.467499 21733 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 2.
I20260709 13:08:34.468340 21895 consensus_queue.cc:237] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:34.476806 21555 catalog_manager.cc:5697] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 1 to 2, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:34.867359 21733 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:34.868762 21895 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:34.881631 21658 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:34.884043 21895 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:34.924981 21733 raft_consensus.cc:1275] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:34.926558 21958 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:34.937419 21658 raft_consensus.cc:1275] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:34.939934 21895 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:35.221812 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:35.233301 21806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:56620
I20260709 13:08:35.233966 21806 raft_consensus.cc:606] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:35.240157 21658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:50994
I20260709 13:08:35.240832 21658 raft_consensus.cc:606] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:35.246107 21658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:50994
I20260709 13:08:35.246775 21658 raft_consensus.cc:606] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:35.252550 21658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:50994
I20260709 13:08:35.253190 21658 raft_consensus.cc:606] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:35.256561 21974 raft_consensus.cc:993] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.257113 21957 raft_consensus.cc:1081] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.258965 21733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60788
I20260709 13:08:35.259593 21733 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.259992 21733 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.265769 21806 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
mode: GRACEFUL
new_leader_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:56620
I20260709 13:08:35.266369 21806 raft_consensus.cc:606] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Received request to transfer leadership to TS 413ff94955e64c4fa0a9f520a7228c28
I20260709 13:08:35.267143 21733 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.269711 21733 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:35.270332 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:35.270860 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:35.271003 21806 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.271512 21658 raft_consensus.cc:3055] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:35.271806 21658 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:35.272677 21658 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.274010 21658 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.277897 21806 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.279361 21700 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:35.280368 21858 raft_consensus.cc:2804] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.280544 21658 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.281694 21858 raft_consensus.cc:697] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:35.282578 21858 consensus_queue.cc:237] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.290439 21556 catalog_manager.cc:5697] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 2 to 3, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.316138 21969 raft_consensus.cc:993] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.316674 21895 raft_consensus.cc:1081] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.318442 21733 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60776
I20260709 13:08:35.319149 21733 raft_consensus.cc:493] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.319543 21733 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.320662 21895 raft_consensus.cc:993] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.321142 21969 raft_consensus.cc:1081] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.323148 21734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60776
I20260709 13:08:35.323707 21734 raft_consensus.cc:493] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.324054 21734 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.326306 21733 raft_consensus.cc:515] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.329164 21733 leader_election.cc:290] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 4 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:35.329696 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:35.329859 21806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8fe6d957cf74e89a781c57df8852d28" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:35.330514 21806 raft_consensus.cc:3055] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:35.330475 21658 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.331070 21806 raft_consensus.cc:740] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:35.331094 21734 raft_consensus.cc:515] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.332046 21806 consensus_queue.cc:260] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.333469 21806 raft_consensus.cc:3060] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.334877 21734 leader_election.cc:290] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:35.334802 21660 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:35.335055 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f4c184c59ba4a13afab0273b0cd3a0f" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:35.335682 21660 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.335716 21808 raft_consensus.cc:3055] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:35.336251 21808 raft_consensus.cc:740] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:35.337193 21808 consensus_queue.cc:260] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.338111 21808 raft_consensus.cc:3060] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.341965 21660 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.342015 21806 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 4.
I20260709 13:08:35.344573 21702 leader_election.cc:304] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:35.345541 21858 raft_consensus.cc:2804] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.346030 21858 raft_consensus.cc:697] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:35.346385 21700 leader_election.cc:304] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:35.347003 21858 consensus_queue.cc:237] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.347481 21975 raft_consensus.cc:2804] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:35.348599 21658 raft_consensus.cc:2468] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 4.
I20260709 13:08:35.352402 21808 raft_consensus.cc:2468] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.353299 21974 raft_consensus.cc:993] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.353873 21974 raft_consensus.cc:1081] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.354744 21975 raft_consensus.cc:697] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:35.355914 21734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60788
I20260709 13:08:35.355715 21975 consensus_queue.cc:237] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.356549 21734 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.357004 21734 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.356753 21556 catalog_manager.cc:5697] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 2 to 3, leader changed from b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.364343 21734 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.367651 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:35.366928 21734 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:35.368276 21658 raft_consensus.cc:3055] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:35.368323 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:35.368727 21658 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:35.368968 21808 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.369648 21658 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.371079 21658 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.372372 21556 catalog_manager.cc:5697] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 3 to 4, leader changed from b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.376493 21808 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.377897 21700 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:35.378784 21658 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.378986 21975 raft_consensus.cc:2804] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.379504 21975 raft_consensus.cc:697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:35.380422 21975 consensus_queue.cc:237] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.389082 21556 catalog_manager.cc:5697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 2 to 3, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.683791 21957 raft_consensus.cc:993] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: : Instructing follower 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.684324 21974 raft_consensus.cc:1081] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Signalling peer 413ff94955e64c4fa0a9f520a7228c28 to start an election
I20260709 13:08:35.686040 21734 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
 from {username='slave'} at 127.0.0.1:60788
I20260709 13:08:35.686703 21734 raft_consensus.cc:493] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.687029 21734 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.692875 21734 raft_consensus.cc:515] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.694995 21734 leader_election.cc:290] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:35.695984 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:35.696174 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9124493c897c4962b8be9c371f0061c7" candidate_uuid: "413ff94955e64c4fa0a9f520a7228c28" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:35.696601 21658 raft_consensus.cc:3055] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:35.696708 21808 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.696928 21658 raft_consensus.cc:740] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:35.697695 21658 consensus_queue.cc:260] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.698750 21658 raft_consensus.cc:3060] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.701924 21808 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.703125 21700 leader_election.cc:304] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413ff94955e64c4fa0a9f520a7228c28, b7ddf4ab5d0e4f6397d4240c58659e03; no voters: 
I20260709 13:08:35.703992 21975 raft_consensus.cc:2804] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.704418 21975 raft_consensus.cc:697] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Leader. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:35.704595 21658 raft_consensus.cc:2468] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413ff94955e64c4fa0a9f520a7228c28 in term 3.
I20260709 13:08:35.705291 21975 consensus_queue.cc:237] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:35.711432 21808 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.712769 21975 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.712929 21556 catalog_manager.cc:5697] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 reported cstate change: term changed from 2 to 3, leader changed from fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129) to 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "413ff94955e64c4fa0a9f520a7228c28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:35.723791 21658 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.725354 21975 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:35.823271 21808 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:35.824560 21975 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:35.834107 21658 raft_consensus.cc:1275] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:35.836314 21808 raft_consensus.cc:1275] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.837049 21975 consensus_queue.cc:1048] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:35.838131 21988 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.854049 21658 raft_consensus.cc:1275] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.859346 21989 consensus_queue.cc:1048] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.861320 21658 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.864032 21975 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:35.874995 21808 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:35.876536 21975 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:36.302632 21660 raft_consensus.cc:1275] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.303906 21975 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:36.315078 21808 raft_consensus.cc:1275] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Refusing update from remote peer 413ff94955e64c4fa0a9f520a7228c28: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:36.316954 21858 consensus_queue.cc:1048] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:38.276687 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:38.285842 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:38.286739 21734 raft_consensus.cc:606] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:38.290376 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:38.291134 21734 raft_consensus.cc:606] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:38.294035 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:38.294636 21734 raft_consensus.cc:606] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Received request to transfer leadership to TS b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:38.297399 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:38.297994 21734 raft_consensus.cc:606] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Received request to transfer leadership to TS b7ddf4ab5d0e4f6397d4240c58659e03
I20260709 13:08:38.300736 21734 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
mode: GRACEFUL
new_leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:60796
I20260709 13:08:38.301299 21734 raft_consensus.cc:606] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Received request to transfer leadership to TS fbb830edd2d549148c23c189e4f8f1dd
I20260709 13:08:38.302518 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:38.303043 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:38.324775 21989 raft_consensus.cc:993] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.325723 21995 raft_consensus.cc:1081] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.327452 21658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:38.327965 21658 raft_consensus.cc:493] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.328311 21658 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:38.335644 21658 raft_consensus.cc:515] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.337553 21658 leader_election.cc:290] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:38.338864 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:38.338994 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d46b878f7c0425b9e93b6e0b2585666" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:38.339967 21734 raft_consensus.cc:3055] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:38.339967 21808 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:38.340525 21734 raft_consensus.cc:740] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:38.341528 21734 consensus_queue.cc:260] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.343202 21734 raft_consensus.cc:3060] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:38.349081 21808 raft_consensus.cc:2468] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:38.350919 21625 leader_election.cc:304] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:38.351233 21734 raft_consensus.cc:2468] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 2.
I20260709 13:08:38.352977 21997 raft_consensus.cc:2804] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:38.354329 21997 raft_consensus.cc:697] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:38.355530 21997 consensus_queue.cc:237] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.357182 21996 raft_consensus.cc:993] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.357869 21995 raft_consensus.cc:1081] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.360003 21658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:38.360764 21658 raft_consensus.cc:493] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.361203 21658 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.365152 21555 catalog_manager.cc:5697] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 1 to 2, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 2 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:38.366782 21658 raft_consensus.cc:515] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.368600 21658 leader_election.cc:290] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 4 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:38.369722 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:38.369877 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccefdb565edc464f97ae56eb034bced5" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:38.370404 21808 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.370512 21734 raft_consensus.cc:3055] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:38.370908 21734 raft_consensus.cc:740] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:38.371865 21734 consensus_queue.cc:260] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.372795 21734 raft_consensus.cc:3060] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.379369 21808 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 4.
I20260709 13:08:38.380743 21625 leader_election.cc:304] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:38.381060 21734 raft_consensus.cc:2468] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 4.
I20260709 13:08:38.382097 21997 raft_consensus.cc:2804] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:38.382920 21997 raft_consensus.cc:697] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 4 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:38.384217 21997 consensus_queue.cc:237] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.392242 21555 catalog_manager.cc:5697] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 3 to 4, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 4 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:38.491616 21996 raft_consensus.cc:993] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:38.492115 21995 raft_consensus.cc:1081] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Signalling peer b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:38.493909 21808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:56614
I20260709 13:08:38.494365 21808 raft_consensus.cc:493] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.494606 21808 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.499766 21808 raft_consensus.cc:515] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.501436 21808 leader_election.cc:290] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 3 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:38.502655 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:38.502660 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af7e00f59b294f1fa1f3de0db396867b" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:38.503392 21658 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.503417 21734 raft_consensus.cc:3055] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:38.503801 21734 raft_consensus.cc:740] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:38.504508 21734 consensus_queue.cc:260] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.505435 21734 raft_consensus.cc:3060] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.508455 21658 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 3.
I20260709 13:08:38.509856 21776 leader_election.cc:304] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:38.510423 21734 raft_consensus.cc:2468] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 3.
I20260709 13:08:38.511026 21969 raft_consensus.cc:2804] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:38.512560 21969 raft_consensus.cc:697] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:38.513376 21969 consensus_queue.cc:237] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.521536 21555 catalog_manager.cc:5697] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 2 to 3, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:38.600360 21996 raft_consensus.cc:993] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:38.600996 21996 raft_consensus.cc:1081] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Signalling peer b7ddf4ab5d0e4f6397d4240c58659e03 to start an election
I20260709 13:08:38.603111 21808 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2"
dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
 from {username='slave'} at 127.0.0.1:56614
I20260709 13:08:38.603703 21808 raft_consensus.cc:493] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.604035 21808 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.609133 21808 raft_consensus.cc:515] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.611088 21808 leader_election.cc:290] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 4 election: Requested vote from peers fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101), 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391)
I20260709 13:08:38.612022 21658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
I20260709 13:08:38.612154 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9de999ccf94a4fb6b7ebb98b37fe1ff2" candidate_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:38.612757 21658 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.612830 21734 raft_consensus.cc:3055] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:38.613197 21734 raft_consensus.cc:740] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:38.613880 21734 consensus_queue.cc:260] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.614923 21734 raft_consensus.cc:3060] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:38.618352 21658 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 4.
I20260709 13:08:38.619648 21734 raft_consensus.cc:2468] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ddf4ab5d0e4f6397d4240c58659e03 in term 4.
I20260709 13:08:38.619648 21776 leader_election.cc:304] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:38.620728 21969 raft_consensus.cc:2804] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:38.621151 21969 raft_consensus.cc:697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 LEADER]: Becoming Leader. State: Replica: b7ddf4ab5d0e4f6397d4240c58659e03, State: Running, Role: LEADER
I20260709 13:08:38.621999 21969 consensus_queue.cc:237] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.628571 21995 raft_consensus.cc:993] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: : Instructing follower fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.629120 21996 raft_consensus.cc:1081] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Signalling peer fbb830edd2d549148c23c189e4f8f1dd to start an election
I20260709 13:08:38.630204 21555 catalog_manager.cc:5697] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 reported cstate change: term changed from 3 to 4, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:38.630827 21658 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58"
dest_uuid: "fbb830edd2d549148c23c189e4f8f1dd"
 from {username='slave'} at 127.0.0.1:50970
I20260709 13:08:38.631430 21658 raft_consensus.cc:493] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.631820 21658 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.638849 21658 raft_consensus.cc:515] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.640822 21658 leader_election.cc:290] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 3 election: Requested vote from peers 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391), b7ddf4ab5d0e4f6397d4240c58659e03 (127.21.5.131:36491)
I20260709 13:08:38.641929 21808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03"
I20260709 13:08:38.641930 21734 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89b8e644a96e4fd4894324653af26a58" candidate_uuid: "fbb830edd2d549148c23c189e4f8f1dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "413ff94955e64c4fa0a9f520a7228c28"
I20260709 13:08:38.642573 21808 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.643180 21734 raft_consensus.cc:3055] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:38.643527 21734 raft_consensus.cc:740] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 413ff94955e64c4fa0a9f520a7228c28, State: Running, Role: LEADER
I20260709 13:08:38.644343 21734 consensus_queue.cc:260] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.645354 21734 raft_consensus.cc:3060] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:38.648170 21808 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 3.
I20260709 13:08:38.649664 21625 leader_election.cc:304] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7ddf4ab5d0e4f6397d4240c58659e03, fbb830edd2d549148c23c189e4f8f1dd; no voters: 
I20260709 13:08:38.650547 21997 raft_consensus.cc:2804] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:38.650772 21734 raft_consensus.cc:2468] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbb830edd2d549148c23c189e4f8f1dd in term 3.
I20260709 13:08:38.651127 21997 raft_consensus.cc:697] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 3 LEADER]: Becoming Leader. State: Replica: fbb830edd2d549148c23c189e4f8f1dd, State: Running, Role: LEADER
I20260709 13:08:38.652117 21997 consensus_queue.cc:237] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } }
I20260709 13:08:38.662537 21555 catalog_manager.cc:5697] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd reported cstate change: term changed from 2 to 3, leader changed from 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130) to fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129). New cstate: current_term: 3 leader_uuid: "fbb830edd2d549148c23c189e4f8f1dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:38.795624 21734 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:38.797044 21997 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:38.809723 21808 raft_consensus.cc:1275] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:38.811431 21997 consensus_queue.cc:1048] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:38.892812 21808 raft_consensus.cc:1275] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:38.894143 21997 consensus_queue.cc:1048] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:38.904801 21734 raft_consensus.cc:1275] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:38.907016 21997 consensus_queue.cc:1048] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:38.942704 21658 raft_consensus.cc:1275] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:38.944108 21969 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:38.954736 21734 raft_consensus.cc:1275] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:38.956751 22019 consensus_queue.cc:1048] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:39.030495 21734 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:39.032145 22019 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:39.044685 21658 raft_consensus.cc:1275] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Refusing update from remote peer b7ddf4ab5d0e4f6397d4240c58659e03: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:39.046423 22005 consensus_queue.cc:1048] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbb830edd2d549148c23c189e4f8f1dd" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:39.103842 21808 raft_consensus.cc:1275] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:39.105521 22016 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ddf4ab5d0e4f6397d4240c58659e03" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 36491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:39.116945 21734 raft_consensus.cc:1275] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Refusing update from remote peer fbb830edd2d549148c23c189e4f8f1dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:39.118785 22016 consensus_queue.cc:1048] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "413ff94955e64c4fa0a9f520a7228c28" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:39.304139 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:39.310357 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
W20260709 13:08:39.318578 21776 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:08:39.325963 21776 consensus_peers.cc:597] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 -> Peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Couldn't send request to peer fbb830edd2d549148c23c189e4f8f1dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:39.342393 21702 consensus_peers.cc:597] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 -> Peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Couldn't send request to peer fbb830edd2d549148c23c189e4f8f1dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.356979 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:39.357894 21526 tablet_replica.cc:333] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.358718 21526 raft_consensus.cc:2243] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.359293 21526 raft_consensus.cc:2272] T 719b58b414d64d1dba681c9bce2122c9 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.361923 21526 tablet_replica.cc:333] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.362493 21526 raft_consensus.cc:2243] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.363090 21526 raft_consensus.cc:2272] T e8fe6d957cf74e89a781c57df8852d28 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.365002 21526 tablet_replica.cc:333] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.365741 21526 raft_consensus.cc:2243] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.366294 21526 raft_consensus.cc:2272] T af7e00f59b294f1fa1f3de0db396867b P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.369457 21526 tablet_replica.cc:333] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.370071 21526 raft_consensus.cc:2243] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.370556 21526 raft_consensus.cc:2272] T 1f4c184c59ba4a13afab0273b0cd3a0f P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.372452 21526 tablet_replica.cc:333] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.373005 21526 raft_consensus.cc:2243] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.373469 21526 raft_consensus.cc:2272] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.376031 21526 tablet_replica.cc:333] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.376621 21526 raft_consensus.cc:2243] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.377538 21526 raft_consensus.cc:2272] T 89b8e644a96e4fd4894324653af26a58 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.380023 21526 tablet_replica.cc:333] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.380609 21526 raft_consensus.cc:2243] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.381460 21526 raft_consensus.cc:2272] T 3d46b878f7c0425b9e93b6e0b2585666 P fbb830edd2d549148c23c189e4f8f1dd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.384212 21526 tablet_replica.cc:333] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.384863 21526 raft_consensus.cc:2243] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.385440 21526 raft_consensus.cc:2272] T 9124493c897c4962b8be9c371f0061c7 P fbb830edd2d549148c23c189e4f8f1dd [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.385927 21776 consensus_peers.cc:597] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 -> Peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Couldn't send request to peer fbb830edd2d549148c23c189e4f8f1dd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.387428 21526 tablet_replica.cc:333] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd: stopping tablet replica
I20260709 13:08:39.387974 21526 raft_consensus.cc:2243] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.388835 21526 raft_consensus.cc:2272] T ccefdb565edc464f97ae56eb034bced5 P fbb830edd2d549148c23c189e4f8f1dd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.497612 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
W20260709 13:08:39.499467 21776 consensus_peers.cc:597] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 -> Peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Couldn't send request to peer fbb830edd2d549148c23c189e4f8f1dd. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.521414 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
W20260709 13:08:39.552701 21702 consensus_peers.cc:597] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 -> Peer fbb830edd2d549148c23c189e4f8f1dd (127.21.5.129:41101): Couldn't send request to peer fbb830edd2d549148c23c189e4f8f1dd. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41101: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.555419 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:39.556213 21526 tablet_replica.cc:333] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.557233 21526 raft_consensus.cc:2243] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.557948 21526 raft_consensus.cc:2272] T 719b58b414d64d1dba681c9bce2122c9 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.560561 21526 tablet_replica.cc:333] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.561308 21526 raft_consensus.cc:2243] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.561849 21526 raft_consensus.cc:2272] T ccefdb565edc464f97ae56eb034bced5 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.564270 21526 tablet_replica.cc:333] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.565035 21526 raft_consensus.cc:2243] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.566038 21526 raft_consensus.cc:2272] T 1f4c184c59ba4a13afab0273b0cd3a0f P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.568909 21526 tablet_replica.cc:333] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.569545 21526 raft_consensus.cc:2243] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.570081 21526 raft_consensus.cc:2272] T af7e00f59b294f1fa1f3de0db396867b P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:39.573871 21773 consensus_peers.cc:597] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 -> Peer 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391): Couldn't send request to peer 413ff94955e64c4fa0a9f520a7228c28. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.574005 21526 tablet_replica.cc:333] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.574947 21526 raft_consensus.cc:2243] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.575541 21526 raft_consensus.cc:2272] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.578547 21526 tablet_replica.cc:333] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.579195 21526 raft_consensus.cc:2243] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.579643 21526 raft_consensus.cc:2272] T 89b8e644a96e4fd4894324653af26a58 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.581993 21526 tablet_replica.cc:333] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.582484 21526 raft_consensus.cc:2243] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.583400 21526 raft_consensus.cc:2272] T e8fe6d957cf74e89a781c57df8852d28 P 413ff94955e64c4fa0a9f520a7228c28 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.585556 21526 tablet_replica.cc:333] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.586282 21526 raft_consensus.cc:2243] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.586825 21526 raft_consensus.cc:2272] T 3d46b878f7c0425b9e93b6e0b2585666 P 413ff94955e64c4fa0a9f520a7228c28 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.589282 21526 tablet_replica.cc:333] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28: stopping tablet replica
I20260709 13:08:39.589826 21526 raft_consensus.cc:2243] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.590746 21526 raft_consensus.cc:2272] T 9124493c897c4962b8be9c371f0061c7 P 413ff94955e64c4fa0a9f520a7228c28 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.620438 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
I20260709 13:08:39.641203 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
W20260709 13:08:39.668761 21773 consensus_peers.cc:597] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 -> Peer 413ff94955e64c4fa0a9f520a7228c28 (127.21.5.130:39391): Couldn't send request to peer 413ff94955e64c4fa0a9f520a7228c28. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39391: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:39.670145 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:39.672803 21526 tablet_replica.cc:333] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.673615 21526 raft_consensus.cc:2243] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.674525 21526 raft_consensus.cc:2272] T 719b58b414d64d1dba681c9bce2122c9 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.676658 21526 tablet_replica.cc:333] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.677312 21526 raft_consensus.cc:2243] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.678262 21526 raft_consensus.cc:2272] T af7e00f59b294f1fa1f3de0db396867b P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.680727 21526 tablet_replica.cc:333] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.681360 21526 raft_consensus.cc:2243] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.681866 21526 raft_consensus.cc:2272] T 1f4c184c59ba4a13afab0273b0cd3a0f P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.683914 21526 tablet_replica.cc:333] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.684451 21526 raft_consensus.cc:2243] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.684932 21526 raft_consensus.cc:2272] T 89b8e644a96e4fd4894324653af26a58 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.687294 21526 tablet_replica.cc:333] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.687866 21526 raft_consensus.cc:2243] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.688710 21526 raft_consensus.cc:2272] T 9de999ccf94a4fb6b7ebb98b37fe1ff2 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.691066 21526 tablet_replica.cc:333] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.691753 21526 raft_consensus.cc:2243] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.692225 21526 raft_consensus.cc:2272] T ccefdb565edc464f97ae56eb034bced5 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.694746 21526 tablet_replica.cc:333] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.695233 21526 raft_consensus.cc:2243] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.695664 21526 raft_consensus.cc:2272] T e8fe6d957cf74e89a781c57df8852d28 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.697547 21526 tablet_replica.cc:333] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.698096 21526 raft_consensus.cc:2243] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.698520 21526 raft_consensus.cc:2272] T 3d46b878f7c0425b9e93b6e0b2585666 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.700727 21526 tablet_replica.cc:333] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03: stopping tablet replica
I20260709 13:08:39.701191 21526 raft_consensus.cc:2243] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:39.701718 21526 raft_consensus.cc:2272] T 9124493c897c4962b8be9c371f0061c7 P b7ddf4ab5d0e4f6397d4240c58659e03 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.729904 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:08:39.749859 21526 master.cc:562] Master@127.21.5.190:39773 shutting down...
I20260709 13:08:39.971706 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:39.972468 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:39.972882 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 93aaf247fdd94421ba5a1b63dd4dd326: stopping tablet replica
I20260709 13:08:39.994933 21526 master.cc:584] Master@127.21.5.190:39773 shutdown complete.
413ff94955e64c4fa0a9f520a7228c28  4
b7ddf4ab5d0e4f6397d4240c58659e03  1
fbb830edd2d549148c23c189e4f8f1dd  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (13510 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:08:40.047022 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:37419
I20260709 13:08:40.048429 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.056032 22030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.056051 22032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.056186 22029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.057519 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:40.058583 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.059147 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.059399 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602520059369 us; error 0 us; skew 500 ppm
I20260709 13:08:40.060109 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.063465 21526 webserver.cc:533] Webserver started at http://127.21.5.190:43633/ using document root <none> and password file <none>
I20260709 13:08:40.064167 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.064419 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.064757 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.066289 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "2029440bbfee488f8f11cffdc621525b"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.071784 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:40.075836 22037 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.076669 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:40.077000 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "2029440bbfee488f8f11cffdc621525b"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.077343 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.094421 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.096035 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.143472 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:37419
I20260709 13:08:40.143581 22088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:37419 every 8 connection(s)
I20260709 13:08:40.148167 22089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.159807 22089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b: Bootstrap starting.
I20260709 13:08:40.165601 22089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.171137 22089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b: No bootstrap required, opened a new log
I20260709 13:08:40.174031 22089 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER }
I20260709 13:08:40.174504 22089 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.174742 22089 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2029440bbfee488f8f11cffdc621525b, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.175341 22089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER }
I20260709 13:08:40.175827 22089 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:40.176043 22089 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:40.176283 22089 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:40.181761 22089 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER }
I20260709 13:08:40.182435 22089 leader_election.cc:304] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2029440bbfee488f8f11cffdc621525b; no voters: 
I20260709 13:08:40.183827 22089 leader_election.cc:290] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:40.184230 22092 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:40.185824 22092 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 1 LEADER]: Becoming Leader. State: Replica: 2029440bbfee488f8f11cffdc621525b, State: Running, Role: LEADER
I20260709 13:08:40.186761 22092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER }
I20260709 13:08:40.187333 22089 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:40.191895 22093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2029440bbfee488f8f11cffdc621525b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER } }
I20260709 13:08:40.192541 22094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2029440bbfee488f8f11cffdc621525b. Latest consensus state: current_term: 1 leader_uuid: "2029440bbfee488f8f11cffdc621525b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029440bbfee488f8f11cffdc621525b" member_type: VOTER } }
I20260709 13:08:40.192966 22093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:40.193455 22094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:40.199232 22100 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:40.206367 22100 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:40.207669 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:40.217391 22100 catalog_manager.cc:1383] Generated new cluster ID: 82221ca3758f4979b809bff4655f36ad
I20260709 13:08:40.217787 22100 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:40.242128 22100 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:40.243736 22100 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:40.261804 22100 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b: Generated new TSK 0
I20260709 13:08:40.262570 22100 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:40.274504 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.281678 22110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.283069 22111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.284643 22113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.284921 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:40.285943 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.286155 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.286301 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602520286287 us; error 0 us; skew 500 ppm
I20260709 13:08:40.286900 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.289772 21526 webserver.cc:533] Webserver started at http://127.21.5.129:38601/ using document root <none> and password file <none>
I20260709 13:08:40.290305 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.290483 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.290899 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.292126 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "60ef22fed7c54bc497e6a0bc5947f768"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.297840 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:08:40.301733 22118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.302596 21526 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:08:40.303000 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "60ef22fed7c54bc497e6a0bc5947f768"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.303351 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.315053 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.316504 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.318197 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.320747 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.320981 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.321269 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.321473 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.371183 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:39147
I20260709 13:08:40.371276 22180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:39147 every 8 connection(s)
I20260709 13:08:40.377307 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.385905 22185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.388423 22186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.391566 21526 server_base.cc:1061] running on GCE node
W20260709 13:08:40.391821 22188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.393252 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.393489 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.393644 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602520393600 us; error 0 us; skew 500 ppm
I20260709 13:08:40.394402 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.397778 21526 webserver.cc:533] Webserver started at http://127.21.5.130:36945/ using document root <none> and password file <none>
I20260709 13:08:40.398530 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.398542 22181 heartbeater.cc:344] Connected to a master server at 127.21.5.190:37419
I20260709 13:08:40.398929 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.399120 22181 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.399322 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.400036 22181 heartbeater.cc:507] Master 127.21.5.190:37419 requested a full tablet report, sending...
I20260709 13:08:40.400874 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "7af91453bcd64a8c8d739d595c316978"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.402966 22054 ts_manager.cc:194] Registered new tserver with Master: 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147)
I20260709 13:08:40.405273 22054 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54144
I20260709 13:08:40.407073 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:08:40.411408 22193 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.412488 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:08:40.412915 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "7af91453bcd64a8c8d739d595c316978"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.413266 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.448480 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.449966 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.451668 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.454095 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.454348 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.454643 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.454849 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:08:40.507027 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:35421
I20260709 13:08:40.507095 22255 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:35421 every 8 connection(s)
I20260709 13:08:40.513327 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:40.521520 22259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.524755 22260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:40.525234 22262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:40.525511 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:40.526939 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:40.527275 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:40.527484 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602520527469 us; error 0 us; skew 500 ppm
I20260709 13:08:40.528074 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:40.528469 22256 heartbeater.cc:344] Connected to a master server at 127.21.5.190:37419
I20260709 13:08:40.528952 22256 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.530030 22256 heartbeater.cc:507] Master 127.21.5.190:37419 requested a full tablet report, sending...
I20260709 13:08:40.531433 21526 webserver.cc:533] Webserver started at http://127.21.5.131:39001/ using document root <none> and password file <none>
I20260709 13:08:40.532158 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:40.532435 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:40.532760 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:40.532894 22054 ts_manager.cc:194] Registered new tserver with Master: 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:40.534143 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "75a79c6e85aa4a86b6d295f495af93c6"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.534849 22054 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54148
I20260709 13:08:40.541067 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:08:40.545370 22267 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.546603 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:08:40.547190 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "75a79c6e85aa4a86b6d295f495af93c6"
format_stamp: "Formatted at 2026-07-09 13:08:40 on dist-test-slave-h0rz"
I20260709 13:08:40.547546 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:40.563855 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:40.565378 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:40.567231 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:40.569855 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:40.570142 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.570430 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:40.570657 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:40.620679 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:35223
I20260709 13:08:40.620788 22329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:35223 every 8 connection(s)
I20260709 13:08:40.636696 22330 heartbeater.cc:344] Connected to a master server at 127.21.5.190:37419
I20260709 13:08:40.637190 22330 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:40.638103 22330 heartbeater.cc:507] Master 127.21.5.190:37419 requested a full tablet report, sending...
I20260709 13:08:40.640707 22054 ts_manager.cc:194] Registered new tserver with Master: 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:40.641822 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017606261s
I20260709 13:08:40.642361 22054 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54154
I20260709 13:08:40.672972 22054 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54158:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:40.676497 22054 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:40.769858 22221 tablet_service.cc:1511] Processing CreateTablet for tablet 2b2bda3c8cd7401c99749d553cf6c450 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.771886 22221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b2bda3c8cd7401c99749d553cf6c450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.772867 22218 tablet_service.cc:1511] Processing CreateTablet for tablet e8ab137f040944af8f345448de34a044 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.772867 22217 tablet_service.cc:1511] Processing CreateTablet for tablet 5e125e5edd154f5791623a8e2eca5f43 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.775456 22220 tablet_service.cc:1511] Processing CreateTablet for tablet 11bbc686e33d4dee9b035c5ad2654458 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.775936 22217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e125e5edd154f5791623a8e2eca5f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.776947 22220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bbc686e33d4dee9b035c5ad2654458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.777618 22218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ab137f040944af8f345448de34a044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.779290 22219 tablet_service.cc:1511] Processing CreateTablet for tablet 47da534c68a8403381813fb9c8ddbe28 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.780803 22219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47da534c68a8403381813fb9c8ddbe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.795056 22216 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce54de8688e45e8bcbf01308b2ce481 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.796903 22216 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce54de8688e45e8bcbf01308b2ce481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.797538 22295 tablet_service.cc:1511] Processing CreateTablet for tablet 2b2bda3c8cd7401c99749d553cf6c450 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.799185 22295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b2bda3c8cd7401c99749d553cf6c450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.802567 22294 tablet_service.cc:1511] Processing CreateTablet for tablet 11bbc686e33d4dee9b035c5ad2654458 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.803382 22293 tablet_service.cc:1511] Processing CreateTablet for tablet 47da534c68a8403381813fb9c8ddbe28 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.804287 22294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bbc686e33d4dee9b035c5ad2654458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.805418 22293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47da534c68a8403381813fb9c8ddbe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.807579 22215 tablet_service.cc:1511] Processing CreateTablet for tablet a56f68092fd14422ac5e080668a180cf (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.808380 22292 tablet_service.cc:1511] Processing CreateTablet for tablet e8ab137f040944af8f345448de34a044 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.812395 22290 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce54de8688e45e8bcbf01308b2ce481 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.815791 22291 tablet_service.cc:1511] Processing CreateTablet for tablet 5e125e5edd154f5791623a8e2eca5f43 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.809176 22215 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f68092fd14422ac5e080668a180cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.810457 22292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ab137f040944af8f345448de34a044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.824014 22289 tablet_service.cc:1511] Processing CreateTablet for tablet a56f68092fd14422ac5e080668a180cf (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.825583 22289 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f68092fd14422ac5e080668a180cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.831264 22290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce54de8688e45e8bcbf01308b2ce481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.835206 22291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e125e5edd154f5791623a8e2eca5f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.842548 22218 tablet_service.cc:1511] Processing CreateTablet for tablet 17f378fdf4c44e25bab64d95990d0fed (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.844198 22218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f378fdf4c44e25bab64d95990d0fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.850562 22345 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:40.856420 22288 tablet_service.cc:1511] Processing CreateTablet for tablet 17f378fdf4c44e25bab64d95990d0fed (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.858188 22288 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f378fdf4c44e25bab64d95990d0fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.872795 22146 tablet_service.cc:1511] Processing CreateTablet for tablet 2b2bda3c8cd7401c99749d553cf6c450 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:08:40.876034 22145 tablet_service.cc:1511] Processing CreateTablet for tablet 11bbc686e33d4dee9b035c5ad2654458 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:08:40.875910 22143 tablet_service.cc:1511] Processing CreateTablet for tablet e8ab137f040944af8f345448de34a044 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:08:40.878850 22146 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b2bda3c8cd7401c99749d553cf6c450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.878445 22142 tablet_service.cc:1511] Processing CreateTablet for tablet 5e125e5edd154f5791623a8e2eca5f43 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:08:40.880404 22143 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8ab137f040944af8f345448de34a044. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.881582 22141 tablet_service.cc:1511] Processing CreateTablet for tablet 0ce54de8688e45e8bcbf01308b2ce481 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:08:40.883080 22145 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11bbc686e33d4dee9b035c5ad2654458. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.884639 22140 tablet_service.cc:1511] Processing CreateTablet for tablet a56f68092fd14422ac5e080668a180cf (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:08:40.890488 22140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56f68092fd14422ac5e080668a180cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.886330 22144 tablet_service.cc:1511] Processing CreateTablet for tablet 47da534c68a8403381813fb9c8ddbe28 (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:08:40.895308 22345 tablet_bootstrap.cc:654] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:40.896440 22144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47da534c68a8403381813fb9c8ddbe28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.900925 22141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ce54de8688e45e8bcbf01308b2ce481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.901710 22142 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e125e5edd154f5791623a8e2eca5f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.917052 22139 tablet_service.cc:1511] Processing CreateTablet for tablet 17f378fdf4c44e25bab64d95990d0fed (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:08:40.920112 22138 tablet_service.cc:1511] Processing CreateTablet for tablet e97e258839854a01bf8dd90ae6b6bd3c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.926991 22139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f378fdf4c44e25bab64d95990d0fed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.928392 22138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97e258839854a01bf8dd90ae6b6bd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.929888 22220 tablet_service.cc:1511] Processing CreateTablet for tablet e97e258839854a01bf8dd90ae6b6bd3c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.931437 22220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97e258839854a01bf8dd90ae6b6bd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.927292 22346 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:40.937381 22290 tablet_service.cc:1511] Processing CreateTablet for tablet e97e258839854a01bf8dd90ae6b6bd3c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:08:40.943813 22290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e97e258839854a01bf8dd90ae6b6bd3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.960304 22137 tablet_service.cc:1511] Processing CreateTablet for tablet c97b9b4ea6ae400493181b22c3f6213c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.962006 22137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97b9b4ea6ae400493181b22c3f6213c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.969318 22219 tablet_service.cc:1511] Processing CreateTablet for tablet c97b9b4ea6ae400493181b22c3f6213c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.970963 22219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97b9b4ea6ae400493181b22c3f6213c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.983623 22295 tablet_service.cc:1511] Processing CreateTablet for tablet c97b9b4ea6ae400493181b22c3f6213c (DEFAULT_TABLE table=test-workload [id=88e99bdd934f40de8f25d629db429471]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:08:40.985260 22295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c97b9b4ea6ae400493181b22c3f6213c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:40.989349 22345 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:40.990221 22345 ts_tablet_manager.cc:1403] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.140s	user 0.016s	sys 0.011s
I20260709 13:08:40.994125 22345 raft_consensus.cc:359] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:40.996289 22345 raft_consensus.cc:385] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:40.996714 22345 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:40.997783 22345 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.009500 22348 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.025904 22346 tablet_bootstrap.cc:654] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.041867 22348 tablet_bootstrap.cc:654] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.051786 22345 ts_tablet_manager.cc:1434] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.061s	user 0.012s	sys 0.005s
I20260709 13:08:41.052170 22256 heartbeater.cc:499] Master 127.21.5.190:37419 was elected leader, sending a full tablet report...
I20260709 13:08:41.052969 22345 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.067768 22346 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.068049 22345 tablet_bootstrap.cc:654] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.068450 22346 ts_tablet_manager.cc:1403] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.142s	user 0.007s	sys 0.011s
I20260709 13:08:41.072680 22346 raft_consensus.cc:359] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.073518 22346 raft_consensus.cc:385] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.073858 22346 raft_consensus.cc:740] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.074185 22348 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.074857 22348 ts_tablet_manager.cc:1403] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.014s
I20260709 13:08:41.075027 22346 consensus_queue.cc:260] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.079092 22348 raft_consensus.cc:359] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.080334 22348 raft_consensus.cc:385] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.080855 22348 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.081773 22345 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.082409 22345 ts_tablet_manager.cc:1403] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.005s
I20260709 13:08:41.082022 22348 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.083359 22346 ts_tablet_manager.cc:1434] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:08:41.084247 22330 heartbeater.cc:499] Master 127.21.5.190:37419 was elected leader, sending a full tablet report...
I20260709 13:08:41.085901 22345 raft_consensus.cc:359] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.086745 22345 raft_consensus.cc:385] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.087087 22345 raft_consensus.cc:740] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.088127 22345 consensus_queue.cc:260] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.088795 22348 ts_tablet_manager.cc:1434] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:08:41.089704 22181 heartbeater.cc:499] Master 127.21.5.190:37419 was elected leader, sending a full tablet report...
I20260709 13:08:41.091485 22346 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.098955 22345 ts_tablet_manager.cc:1434] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:08:41.099833 22346 tablet_bootstrap.cc:654] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.101624 22345 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.108381 22345 tablet_bootstrap.cc:654] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.112294 22348 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.113101 22346 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.113667 22346 ts_tablet_manager.cc:1403] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.001s
I20260709 13:08:41.116068 22345 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.116717 22345 ts_tablet_manager.cc:1403] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.003s
I20260709 13:08:41.117354 22346 raft_consensus.cc:359] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.118261 22346 raft_consensus.cc:385] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.118590 22346 raft_consensus.cc:740] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.119623 22346 consensus_queue.cc:260] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.120728 22348 tablet_bootstrap.cc:654] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.120630 22345 raft_consensus.cc:359] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.121716 22345 raft_consensus.cc:385] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.122128 22345 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.122471 22346 ts_tablet_manager.cc:1434] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:08:41.123703 22346 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.123399 22345 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.130694 22345 ts_tablet_manager.cc:1434] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:08:41.132189 22345 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.137219 22346 tablet_bootstrap.cc:654] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.139216 22348 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.139878 22348 ts_tablet_manager.cc:1403] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.028s	user 0.020s	sys 0.006s
I20260709 13:08:41.140496 22345 tablet_bootstrap.cc:654] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.144402 22348 raft_consensus.cc:359] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.146061 22353 raft_consensus.cc:493] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.145664 22348 raft_consensus.cc:385] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.148114 22348 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.149394 22348 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.152504 22348 ts_tablet_manager.cc:1434] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.004s
I20260709 13:08:41.146925 22353 raft_consensus.cc:515] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.153716 22348 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.168654 22348 tablet_bootstrap.cc:654] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.168860 22346 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.169440 22346 ts_tablet_manager.cc:1403] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.046s	user 0.016s	sys 0.007s
I20260709 13:08:41.169627 22353 leader_election.cc:290] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.169739 22354 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.170334 22354 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.172919 22346 raft_consensus.cc:359] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.173755 22346 raft_consensus.cc:385] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.174072 22346 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.180475 22354 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.183473 22346 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.184664 22345 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.185274 22345 ts_tablet_manager.cc:1403] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.011s
I20260709 13:08:41.186381 22346 ts_tablet_manager.cc:1434] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:08:41.187669 22346 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.195706 22345 raft_consensus.cc:359] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.196960 22345 raft_consensus.cc:385] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.197382 22345 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.200866 22348 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.201633 22348 ts_tablet_manager.cc:1403] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.011s
W20260709 13:08:41.202085 22102 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:41.198526 22345 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.205658 22346 tablet_bootstrap.cc:654] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.205933 22348 raft_consensus.cc:359] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.206817 22348 raft_consensus.cc:385] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.207170 22348 raft_consensus.cc:740] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.207988 22348 consensus_queue.cc:260] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.213750 22348 ts_tablet_manager.cc:1434] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:08:41.214166 22345 ts_tablet_manager.cc:1434] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.004s
I20260709 13:08:41.215440 22348 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.223484 22345 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.230002 22354 raft_consensus.cc:493] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.230651 22354 raft_consensus.cc:515] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.233059 22354 leader_election.cc:290] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.234112 22348 tablet_bootstrap.cc:654] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.243398 22349 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.244045 22349 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.252998 22345 tablet_bootstrap.cc:654] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.262763 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.264386 22156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
W20260709 13:08:41.269361 22270 leader_election.cc:343] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.278003 22230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bbc686e33d4dee9b035c5ad2654458" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.278298 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.280138 22231 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 0.
W20260709 13:08:41.283761 22121 leader_election.cc:343] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.284859 22121 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
W20260709 13:08:41.286474 22269 leader_election.cc:343] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.287184 22269 leader_election.cc:304] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6; no voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.288321 22353 raft_consensus.cc:2749] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.289000 22354 raft_consensus.cc:2804] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.289542 22354 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.290045 22354 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.262844 22349 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.302479 22354 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.309381 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:41.310105 22231 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.311365 22346 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.311934 22346 ts_tablet_manager.cc:1403] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.125s	user 0.035s	sys 0.029s
I20260709 13:08:41.315467 22346 raft_consensus.cc:359] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.316278 22346 raft_consensus.cc:385] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.316602 22346 raft_consensus.cc:740] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.318466 22346 consensus_queue.cc:260] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.319515 22349 raft_consensus.cc:493] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.320060 22349 raft_consensus.cc:515] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.320993 22348 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.321382 22346 ts_tablet_manager.cc:1434] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:08:41.321533 22348 ts_tablet_manager.cc:1403] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.106s	user 0.027s	sys 0.044s
I20260709 13:08:41.322551 22346 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.322934 22349 leader_election.cc:290] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.324515 22354 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.326707 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bbc686e33d4dee9b035c5ad2654458" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.327431 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.327525 22304 raft_consensus.cc:2468] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 0.
I20260709 13:08:41.329239 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.330780 22231 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 1.
W20260709 13:08:41.331907 22121 leader_election.cc:343] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.331995 22155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.332693 22345 tablet_bootstrap.cc:492] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.332782 22155 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:41.333273 22345 ts_tablet_manager.cc:1403] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.030s
I20260709 13:08:41.333200 22156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.334198 22156 raft_consensus.cc:2393] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7af91453bcd64a8c8d739d595c316978 in current term 1: Already voted for candidate 60ef22fed7c54bc497e6a0bc5947f768 in this term.
I20260709 13:08:41.336848 22348 raft_consensus.cc:359] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.337355 22346 tablet_bootstrap.cc:654] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.337733 22348 raft_consensus.cc:385] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.332803 22121 leader_election.cc:304] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.338268 22348 raft_consensus.cc:740] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
W20260709 13:08:41.339322 22121 leader_election.cc:343] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.339123 22348 consensus_queue.cc:260] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.339488 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.340958 22195 leader_election.cc:304] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.341292 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.341605 22345 raft_consensus.cc:359] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.342392 22349 raft_consensus.cc:2804] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.342994 22349 raft_consensus.cc:493] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.343669 22349 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.344998 22121 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:41.345631 22348 ts_tablet_manager.cc:1434] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.007s
I20260709 13:08:41.343026 22354 raft_consensus.cc:2804] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.346149 22354 raft_consensus.cc:493] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.346658 22354 raft_consensus.cc:3060] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.348006 22372 raft_consensus.cc:2804] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.342414 22345 raft_consensus.cc:385] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.346762 22348 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
W20260709 13:08:41.349035 22196 leader_election.cc:343] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.356827 22372 raft_consensus.cc:697] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:41.357288 22196 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7af91453bcd64a8c8d739d595c316978; no voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:41.357975 22372 consensus_queue.cc:237] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.355957 22349 raft_consensus.cc:515] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.356423 22346 tablet_bootstrap.cc:492] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.360649 22346 ts_tablet_manager.cc:1403] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.001s
I20260709 13:08:41.361248 22349 leader_election.cc:290] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.361510 22354 raft_consensus.cc:515] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.361907 22349 raft_consensus.cc:2749] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.356700 22345 raft_consensus.cc:740] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.365834 22345 consensus_queue.cc:260] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.364423 22354 leader_election.cc:290] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.368929 22345 ts_tablet_manager.cc:1434] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.035s	user 0.014s	sys 0.001s
I20260709 13:08:41.370061 22345 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.364370 22346 raft_consensus.cc:359] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.370534 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bbc686e33d4dee9b035c5ad2654458" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:41.370811 22346 raft_consensus.cc:385] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.371133 22346 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.371235 22231 raft_consensus.cc:3060] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.372145 22346 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.377065 22346 ts_tablet_manager.cc:1434] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:08:41.378171 22346 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
W20260709 13:08:41.379379 22196 leader_election.cc:343] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.381361 22348 tablet_bootstrap.cc:654] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.382812 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.382886 22156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.379202 22231 raft_consensus.cc:2468] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 1.
I20260709 13:08:41.383545 22305 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.383808 22156 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.385082 22121 leader_election.cc:304] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.386088 22354 raft_consensus.cc:2804] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.386693 22354 raft_consensus.cc:697] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:41.387676 22354 consensus_queue.cc:237] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.390082 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11bbc686e33d4dee9b035c5ad2654458" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.390877 22304 raft_consensus.cc:3060] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.393463 22345 tablet_bootstrap.cc:654] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.395509 22156 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.404654 22054 catalog_manager.cc:5697] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.409366 22354 raft_consensus.cc:493] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.409924 22354 raft_consensus.cc:515] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.413595 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.415534 22348 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.407047 22195 leader_election.cc:304] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.416554 22349 raft_consensus.cc:2804] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.417114 22349 raft_consensus.cc:697] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:41.418032 22349 consensus_queue.cc:237] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
W20260709 13:08:41.415416 22121 leader_election.cc:343] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.423390 22348 ts_tablet_manager.cc:1403] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.001s
I20260709 13:08:41.405881 22304 raft_consensus.cc:2468] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 1.
I20260709 13:08:41.425644 22354 leader_election.cc:290] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.427184 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
W20260709 13:08:41.429571 22121 leader_election.cc:343] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.430297 22121 leader_election.cc:304] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768; no voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.431572 22354 raft_consensus.cc:2749] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.432610 22348 raft_consensus.cc:359] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.437664 22348 raft_consensus.cc:385] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.402607 22305 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.441118 22348 raft_consensus.cc:740] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.440572 22345 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.442166 22345 ts_tablet_manager.cc:1403] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.072s	user 0.013s	sys 0.038s
I20260709 13:08:41.442096 22348 consensus_queue.cc:260] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.443290 22346 tablet_bootstrap.cc:654] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.444504 22348 ts_tablet_manager.cc:1434] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:08:41.445407 22348 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.445782 22345 raft_consensus.cc:359] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.446702 22345 raft_consensus.cc:385] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.447099 22345 raft_consensus.cc:740] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.449195 22345 consensus_queue.cc:260] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.452733 22348 tablet_bootstrap.cc:654] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.453526 22345 ts_tablet_manager.cc:1434] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:08:41.454777 22345 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.460017 22346 tablet_bootstrap.cc:492] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.458065 22052 catalog_manager.cc:5697] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.460731 22346 ts_tablet_manager.cc:1403] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.083s	user 0.012s	sys 0.008s
I20260709 13:08:41.463296 22345 tablet_bootstrap.cc:654] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.469800 22346 raft_consensus.cc:359] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.471185 22346 raft_consensus.cc:385] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.471639 22346 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.478348 22348 tablet_bootstrap.cc:492] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.479116 22348 ts_tablet_manager.cc:1403] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.010s
I20260709 13:08:41.475250 22346 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.482152 22348 raft_consensus.cc:359] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.483160 22348 raft_consensus.cc:385] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.483520 22348 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.484701 22348 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.484122 22054 catalog_manager.cc:5697] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.487865 22348 ts_tablet_manager.cc:1434] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.006s
I20260709 13:08:41.489236 22348 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.491726 22353 raft_consensus.cc:493] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.492818 22353 raft_consensus.cc:515] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.498872 22348 tablet_bootstrap.cc:654] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.509990 22156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.511168 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.511929 22231 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 0.
I20260709 13:08:41.512063 22156 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 0.
I20260709 13:08:41.513789 22269 leader_election.cc:304] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:41.516834 22388 raft_consensus.cc:2804] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.517345 22388 raft_consensus.cc:493] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.517736 22388 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.518338 22346 ts_tablet_manager.cc:1434] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.000s
I20260709 13:08:41.522241 22345 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.510957 22353 leader_election.cc:290] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.522921 22345 ts_tablet_manager.cc:1403] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.068s	user 0.025s	sys 0.039s
I20260709 13:08:41.534062 22346 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.526399 22345 raft_consensus.cc:359] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.536185 22345 raft_consensus.cc:385] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.536597 22345 raft_consensus.cc:740] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.537722 22345 consensus_queue.cc:260] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.541126 22345 ts_tablet_manager.cc:1434] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.003s
I20260709 13:08:41.542359 22345 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.543834 22388 raft_consensus.cc:515] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.546327 22346 tablet_bootstrap.cc:654] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.547537 22156 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.548206 22388 leader_election.cc:290] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.548309 22156 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.549367 22348 tablet_bootstrap.cc:492] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.549958 22348 ts_tablet_manager.cc:1403] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.028s
I20260709 13:08:41.550033 22345 tablet_bootstrap.cc:654] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.547730 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:41.551173 22231 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.554075 22348 raft_consensus.cc:359] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.555104 22348 raft_consensus.cc:385] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.555471 22348 raft_consensus.cc:740] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.556346 22348 consensus_queue.cc:260] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.557174 22345 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.557689 22345 ts_tablet_manager.cc:1403] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.016s	user 0.004s	sys 0.008s
I20260709 13:08:41.558951 22348 ts_tablet_manager.cc:1434] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:08:41.560177 22348 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.560811 22345 raft_consensus.cc:359] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.561506 22345 raft_consensus.cc:385] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.561780 22345 raft_consensus.cc:740] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.562691 22345 consensus_queue.cc:260] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.567229 22345 ts_tablet_manager.cc:1434] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 13:08:41.567957 22348 tablet_bootstrap.cc:654] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.581748 22349 raft_consensus.cc:493] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.582195 22348 tablet_bootstrap.cc:492] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.582814 22388 raft_consensus.cc:493] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.582928 22348 ts_tablet_manager.cc:1403] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.007s
I20260709 13:08:41.583069 22375 raft_consensus.cc:493] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.583380 22388 raft_consensus.cc:515] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.582355 22349 raft_consensus.cc:515] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.584149 22375 raft_consensus.cc:515] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.586261 22348 raft_consensus.cc:359] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.587044 22348 raft_consensus.cc:385] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.587361 22348 raft_consensus.cc:740] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.587738 22230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.588505 22230 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 0.
I20260709 13:08:41.588840 22156 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 1.
I20260709 13:08:41.589422 22155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.595361 22155 raft_consensus.cc:2468] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:41.594882 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.588295 22348 consensus_queue.cc:260] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.590325 22270 leader_election.cc:304] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.589473 22349 leader_election.cc:290] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.591261 22375 leader_election.cc:290] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.591909 22388 leader_election.cc:290] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.597508 22195 leader_election.cc:304] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.598024 22388 raft_consensus.cc:2804] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.598413 22388 raft_consensus.cc:493] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.598520 22375 raft_consensus.cc:2804] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.598680 22348 ts_tablet_manager.cc:1434] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.001s
I20260709 13:08:41.598845 22388 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.592411 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.600378 22348 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.600512 22269 leader_election.cc:304] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:41.596323 22153 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 0.
I20260709 13:08:41.598975 22375 raft_consensus.cc:493] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.602312 22375 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.592469 22353 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.607371 22353 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.609201 22348 tablet_bootstrap.cc:654] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.612964 22346 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.613636 22346 ts_tablet_manager.cc:1403] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.080s	user 0.025s	sys 0.010s
I20260709 13:08:41.583297 22345 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.616153 22348 tablet_bootstrap.cc:492] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.616793 22348 ts_tablet_manager.cc:1403] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:08:41.617467 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.620334 22348 raft_consensus.cc:359] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.621243 22348 raft_consensus.cc:385] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.621495 22345 tablet_bootstrap.cc:654] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.621623 22348 raft_consensus.cc:740] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.622745 22348 consensus_queue.cc:260] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.623296 22353 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.624020 22353 raft_consensus.cc:2804] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.624522 22353 raft_consensus.cc:697] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:41.625695 22348 ts_tablet_manager.cc:1434] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:41.594901 22231 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 1.
I20260709 13:08:41.625734 22353 consensus_queue.cc:237] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.635813 22348 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: Bootstrap starting.
I20260709 13:08:41.645383 22375 raft_consensus.cc:515] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.650655 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.651947 22153 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.654057 22348 tablet_bootstrap.cc:654] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.656160 22345 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.656899 22345 ts_tablet_manager.cc:1403] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.074s	user 0.023s	sys 0.016s
W20260709 13:08:41.660434 22270 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:35421, user_credentials={real_user=slave}} blocked reactor thread for 70229.3us
I20260709 13:08:41.662210 22348 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: No bootstrap required, opened a new log
I20260709 13:08:41.662849 22348 ts_tablet_manager.cc:1403] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.002s
I20260709 13:08:41.662583 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.662732 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.663853 22231 raft_consensus.cc:2393] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75a79c6e85aa4a86b6d295f495af93c6 in current term 1: Already voted for candidate 60ef22fed7c54bc497e6a0bc5947f768 in this term.
I20260709 13:08:41.664319 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.666153 22348 raft_consensus.cc:359] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.666985 22348 raft_consensus.cc:385] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.667259 22348 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.667310 22346 raft_consensus.cc:359] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.668104 22346 raft_consensus.cc:385] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.667956 22348 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.668452 22346 raft_consensus.cc:740] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.666330 22270 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6; no voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.669364 22346 consensus_queue.cc:260] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.670310 22348 ts_tablet_manager.cc:1434] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:41.671335 22353 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.672032 22346 ts_tablet_manager.cc:1434] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.058s	user 0.006s	sys 0.003s
W20260709 13:08:41.672330 22196 leader_election.cc:343] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.673146 22346 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.673959 22153 raft_consensus.cc:2468] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.665493 22388 raft_consensus.cc:515] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.674312 22052 catalog_manager.cc:5697] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.675935 22305 raft_consensus.cc:2393] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7af91453bcd64a8c8d739d595c316978 in current term 1: Already voted for candidate 75a79c6e85aa4a86b6d295f495af93c6 in this term.
I20260709 13:08:41.676898 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
W20260709 13:08:41.678808 22196 leader_election.cc:343] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.679590 22375 leader_election.cc:290] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.680644 22353 raft_consensus.cc:2749] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.680873 22196 leader_election.cc:304] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7af91453bcd64a8c8d739d595c316978; no voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:41.682480 22375 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.682565 22345 raft_consensus.cc:359] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.683388 22345 raft_consensus.cc:385] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.683669 22345 raft_consensus.cc:740] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.685390 22195 leader_election.cc:304] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:41.686137 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.686821 22349 raft_consensus.cc:2804] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.686904 22153 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.695370 22388 leader_election.cc:290] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.684656 22345 consensus_queue.cc:260] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.700735 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:41.701243 22349 raft_consensus.cc:697] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:41.702473 22349 consensus_queue.cc:237] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.705564 22153 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 1.
I20260709 13:08:41.707185 22269 leader_election.cc:304] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:41.708479 22388 raft_consensus.cc:2804] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.709100 22388 raft_consensus.cc:697] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:41.709359 22375 raft_consensus.cc:2749] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.710160 22388 consensus_queue.cc:237] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.712205 22346 tablet_bootstrap.cc:654] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.715888 22398 raft_consensus.cc:493] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.716482 22345 ts_tablet_manager.cc:1434] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.059s	user 0.003s	sys 0.009s
I20260709 13:08:41.716643 22398 raft_consensus.cc:515] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.717664 22345 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: Bootstrap starting.
I20260709 13:08:41.719635 22231 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 1.
I20260709 13:08:41.720670 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.721642 22154 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:41.723580 22195 leader_election.cc:304] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.724817 22375 raft_consensus.cc:2804] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.725271 22375 raft_consensus.cc:493] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.725705 22375 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.725886 22345 tablet_bootstrap.cc:654] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.732165 22346 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.731634 22052 catalog_manager.cc:5697] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:41.732730 22346 ts_tablet_manager.cc:1403] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.060s	user 0.008s	sys 0.004s
I20260709 13:08:41.735049 22354 raft_consensus.cc:493] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.735616 22354 raft_consensus.cc:515] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.736025 22346 raft_consensus.cc:359] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.736785 22346 raft_consensus.cc:385] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.737085 22346 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.737687 22375 raft_consensus.cc:515] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.737860 22346 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.739450 22398 leader_election.cc:290] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.740273 22375 leader_election.cc:290] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.741065 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.741811 22346 ts_tablet_manager.cc:1434] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 13:08:41.738206 22354 leader_election.cc:290] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.742944 22346 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.734144 22054 catalog_manager.cc:5697] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:41.743438 22121 leader_election.cc:343] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:41.744328 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.744946 22304 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.745236 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.745859 22153 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.747078 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.747792 22303 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 0.
I20260709 13:08:41.750379 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.751104 22346 tablet_bootstrap.cc:654] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.751161 22121 leader_election.cc:304] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.755291 22354 raft_consensus.cc:2804] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.755868 22354 raft_consensus.cc:493] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.756366 22354 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.765709 22304 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.768564 22196 leader_election.cc:304] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.768363 22153 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.772318 22375 raft_consensus.cc:2804] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.772923 22375 raft_consensus.cc:697] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:41.773288 22345 tablet_bootstrap.cc:492] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: No bootstrap required, opened a new log
I20260709 13:08:41.773882 22375 consensus_queue.cc:237] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.773882 22345 ts_tablet_manager.cc:1403] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: Time spent bootstrapping tablet: real 0.056s	user 0.006s	sys 0.010s
I20260709 13:08:41.775249 22354 raft_consensus.cc:515] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.778085 22345 raft_consensus.cc:359] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.778880 22345 raft_consensus.cc:385] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.779181 22345 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.779644 22354 leader_election.cc:290] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.780045 22346 tablet_bootstrap.cc:492] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.779959 22345 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.780433 22398 raft_consensus.cc:493] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.780587 22346 ts_tablet_manager.cc:1403] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.038s	user 0.026s	sys 0.008s
I20260709 13:08:41.780654 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:41.781008 22398 raft_consensus.cc:515] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.781469 22231 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.783456 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.784130 22304 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.786018 22345 ts_tablet_manager.cc:1434] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:08:41.787783 22346 raft_consensus.cc:359] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.788559 22346 raft_consensus.cc:385] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.788839 22346 raft_consensus.cc:740] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.789611 22346 consensus_queue.cc:260] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.789710 22398 leader_election.cc:290] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.792066 22346 ts_tablet_manager.cc:1434] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:08:41.793589 22346 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: Bootstrap starting.
I20260709 13:08:41.793818 22304 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 1.
I20260709 13:08:41.794951 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97e258839854a01bf8dd90ae6b6bd3c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:41.795933 22305 raft_consensus.cc:2468] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:41.798898 22121 leader_election.cc:304] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:41.799840 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97e258839854a01bf8dd90ae6b6bd3c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.800297 22346 tablet_bootstrap.cc:654] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:41.800352 22196 leader_election.cc:304] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.801029 22153 raft_consensus.cc:2468] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:41.801692 22354 raft_consensus.cc:2804] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.803911 22375 raft_consensus.cc:2804] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:41.804404 22375 raft_consensus.cc:493] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.804802 22375 raft_consensus.cc:3060] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.805944 22354 raft_consensus.cc:697] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:41.807875 22354 consensus_queue.cc:237] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.810798 22231 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 1.
I20260709 13:08:41.823832 22354 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.824625 22375 raft_consensus.cc:515] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.824746 22052 catalog_manager.cc:5697] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.827524 22346 tablet_bootstrap.cc:492] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: No bootstrap required, opened a new log
I20260709 13:08:41.828102 22346 ts_tablet_manager.cc:1403] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: Time spent bootstrapping tablet: real 0.035s	user 0.015s	sys 0.015s
I20260709 13:08:41.828655 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97e258839854a01bf8dd90ae6b6bd3c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:41.829392 22153 raft_consensus.cc:3060] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.836194 22375 leader_election.cc:290] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:41.838029 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e97e258839854a01bf8dd90ae6b6bd3c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:41.839778 22304 raft_consensus.cc:3060] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.838174 22346 raft_consensus.cc:359] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.844283 22346 raft_consensus.cc:385] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:41.844663 22346 raft_consensus.cc:740] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Initialized, Role: FOLLOWER
I20260709 13:08:41.845463 22346 consensus_queue.cc:260] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.847914 22346 ts_tablet_manager.cc:1434] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:08:41.848361 22153 raft_consensus.cc:2468] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.848642 22304 raft_consensus.cc:2468] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:41.850003 22195 leader_election.cc:304] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:41.851079 22375 raft_consensus.cc:2804] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:41.851601 22375 raft_consensus.cc:697] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:41.852594 22375 consensus_queue.cc:237] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.853614 22054 catalog_manager.cc:5697] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.855094 22372 consensus_queue.cc:1048] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.860788 22374 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.868897 22388 raft_consensus.cc:493] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.869773 22388 raft_consensus.cc:515] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:41.872546 22374 consensus_queue.cc:1048] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:41.874601 22388 leader_election.cc:290] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:41.878271 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:41.879827 22153 raft_consensus.cc:2393] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75a79c6e85aa4a86b6d295f495af93c6 in current term 1: Already voted for candidate 7af91453bcd64a8c8d739d595c316978 in this term.
I20260709 13:08:41.879535 22230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7af91453bcd64a8c8d739d595c316978" is_pre_election: true
I20260709 13:08:41.895275 22270 leader_election.cc:304] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6; no voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:41.897392 22388 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:41.909078 22053 catalog_manager.cc:5697] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:41.932838 22388 raft_consensus.cc:2749] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:41.989176 22375 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.001696 22349 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.024477 22349 raft_consensus.cc:493] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:42.025095 22349 raft_consensus.cc:515] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.027390 22349 leader_election.cc:290] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.028364 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768" is_pre_election: true
I20260709 13:08:42.028661 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6" is_pre_election: true
I20260709 13:08:42.029240 22305 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:42.029206 22153 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 0.
I20260709 13:08:42.030766 22196 leader_election.cc:304] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.031523 22349 raft_consensus.cc:2804] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:42.031991 22349 raft_consensus.cc:493] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.032325 22349 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:42.037878 22349 raft_consensus.cc:515] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.040131 22349 leader_election.cc:290] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.041244 22153 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:42.041437 22305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.042078 22305 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:42.042052 22153 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:42.050551 22153 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:42.050551 22305 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 1.
I20260709 13:08:42.052342 22195 leader_election.cc:304] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.053131 22349 raft_consensus.cc:2804] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:42.053702 22349 raft_consensus.cc:697] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.054514 22349 consensus_queue.cc:237] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.063333 22054 catalog_manager.cc:5697] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.077304 22388 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.089941 22392 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.125101 22349 consensus_queue.cc:1048] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.137522 22349 consensus_queue.cc:1048] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.193280 22230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:42.193943 22230 raft_consensus.cc:606] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:42.203621 22230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:42.204497 22230 raft_consensus.cc:606] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:42.222270 22305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8ab137f040944af8f345448de34a044"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:53062
I20260709 13:08:42.223009 22305 raft_consensus.cc:606] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:42.239746 22305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
mode: GRACEFUL
new_leader_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:53062
I20260709 13:08:42.240500 22305 raft_consensus.cc:606] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Received request to transfer leadership to TS 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:42.244943 22372 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.262225 22354 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.262826 22153 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:33018
I20260709 13:08:42.263507 22153 raft_consensus.cc:606] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Received request to transfer leadership to TS 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:42.271152 22392 consensus_queue.cc:1048] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.274097 22354 raft_consensus.cc:993] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:42.274703 22354 raft_consensus.cc:1081] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Signalling peer 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:42.277433 22231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:49140
I20260709 13:08:42.278151 22231 raft_consensus.cc:493] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.278685 22231 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.281541 22229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:42.282325 22229 raft_consensus.cc:606] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Received request to transfer leadership to TS 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:42.288556 22388 consensus_queue.cc:1048] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.287076 22231 raft_consensus.cc:515] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.295826 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:42.296574 22154 raft_consensus.cc:3055] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.296911 22154 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:42.297744 22154 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.302438 22154 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.302939 22349 consensus_queue.cc:1048] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.311141 22231 leader_election.cc:290] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 2 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.316035 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.316797 22304 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.316747 22154 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 2.
I20260709 13:08:42.322072 22195 leader_election.cc:304] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.323702 22397 raft_consensus.cc:2804] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.324259 22397 raft_consensus.cc:697] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.325239 22397 consensus_queue.cc:237] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.326504 22353 raft_consensus.cc:993] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.327045 22353 raft_consensus.cc:1081] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.326814 22375 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.329530 22153 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8ab137f040944af8f345448de34a044"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:32990
I20260709 13:08:42.330184 22153 raft_consensus.cc:493] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.330607 22153 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.330788 22349 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.333392 22304 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 2.
I20260709 13:08:42.340718 22153 raft_consensus.cc:515] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.343819 22153 leader_election.cc:290] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.350652 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:42.351038 22397 consensus_queue.cc:1048] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.351403 22231 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.353976 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8ab137f040944af8f345448de34a044" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.354682 22303 raft_consensus.cc:3055] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.355022 22303 raft_consensus.cc:740] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:42.355964 22303 consensus_queue.cc:260] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.357530 22303 raft_consensus.cc:3060] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.359695 22054 catalog_manager.cc:5697] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 1 to 2, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.364670 22231 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.366534 22121 leader_election.cc:304] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.367528 22372 raft_consensus.cc:2804] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.368036 22372 raft_consensus.cc:697] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:42.368947 22372 consensus_queue.cc:237] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.378759 22303 raft_consensus.cc:2468] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.394418 22052 catalog_manager.cc:5697] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 1 to 2, leader changed from 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.550244 22388 raft_consensus.cc:993] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: : Instructing follower 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:42.550244 22349 raft_consensus.cc:993] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.550954 22388 raft_consensus.cc:1081] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Signalling peer 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:42.551008 22349 raft_consensus.cc:1081] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.553310 22155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:33006
I20260709 13:08:42.553539 22231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:49156
I20260709 13:08:42.554032 22155 raft_consensus.cc:493] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.554250 22231 raft_consensus.cc:493] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.554523 22155 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.554829 22231 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.561721 22231 raft_consensus.cc:515] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.562579 22155 raft_consensus.cc:515] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.564869 22155 leader_election.cc:290] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.565615 22397 raft_consensus.cc:993] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.566157 22397 raft_consensus.cc:1081] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:42.566957 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.564973 22231 leader_election.cc:290] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 2 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.567255 22229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:42.567725 22304 raft_consensus.cc:3055] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.568051 22229 raft_consensus.cc:3055] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.568118 22304 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:42.568468 22229 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.569150 22304 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.569366 22229 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.570428 22304 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.571283 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.571590 22154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:33006
I20260709 13:08:42.571816 22155 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:42.571962 22303 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.572163 22154 raft_consensus.cc:493] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.572455 22155 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.572604 22154 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.573906 22229 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.579582 22304 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 2.
I20260709 13:08:42.581277 22155 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 2.
I20260709 13:08:42.581434 22196 leader_election.cc:304] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.581653 22154 raft_consensus.cc:515] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.582506 22398 raft_consensus.cc:2804] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.583114 22398 raft_consensus.cc:697] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.584249 22154 leader_election.cc:290] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:42.584074 22398 consensus_queue.cc:237] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.585896 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:42.586669 22231 raft_consensus.cc:3055] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.586848 22303 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.587054 22231 raft_consensus.cc:740] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.587786 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47da534c68a8403381813fb9c8ddbe28" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:42.588419 22304 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.588075 22231 consensus_queue.cc:260] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.589764 22231 raft_consensus.cc:3060] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.591434 22229 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.592780 22121 leader_election.cc:304] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:42.593704 22354 raft_consensus.cc:2804] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.594259 22354 raft_consensus.cc:697] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:42.595211 22053 catalog_manager.cc:5697] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 1 to 2, leader changed from 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131) to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.597185 22304 raft_consensus.cc:2468] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.596182 22354 consensus_queue.cc:237] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.598592 22121 leader_election.cc:304] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:42.599578 22372 raft_consensus.cc:2804] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.600102 22372 raft_consensus.cc:697] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:42.600528 22231 raft_consensus.cc:2468] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:42.601048 22372 consensus_queue.cc:237] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.607975 22054 catalog_manager.cc:5697] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 1 to 2, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.621640 22054 catalog_manager.cc:5697] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 1 to 2, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:42.637715 22398 consensus_queue.cc:1048] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:42.646220 22349 raft_consensus.cc:993] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:42.646261 22397 consensus_queue.cc:1048] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:42.646883 22349 raft_consensus.cc:1081] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Signalling peer 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:42.648962 22304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:53048
I20260709 13:08:42.649690 22304 raft_consensus.cc:493] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:42.650130 22304 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.658994 22304 raft_consensus.cc:515] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.660332 22303 raft_consensus.cc:1240] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Rejecting Update request from peer 7af91453bcd64a8c8d739d595c316978 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:42.663069 22304 leader_election.cc:290] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 2 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:42.664316 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:42.664745 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c97b9b4ea6ae400493181b22c3f6213c" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:42.665007 22154 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.665396 22231 raft_consensus.cc:3055] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:42.665783 22231 raft_consensus.cc:740] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:42.666733 22231 consensus_queue.cc:260] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:42.667990 22231 raft_consensus.cc:3060] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.673506 22154 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 2.
I20260709 13:08:42.675278 22269 leader_election.cc:304] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:42.676210 22231 raft_consensus.cc:2468] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 2.
I20260709 13:08:42.676537 22388 raft_consensus.cc:2804] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:42.677300 22388 raft_consensus.cc:697] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:42.678517 22388 consensus_queue.cc:237] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
W20260709 13:08:42.674036 22398 consensus_queue.cc:1175] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "75a79c6e85aa4a86b6d295f495af93c6" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7af91453bcd64a8c8d739d595c316978 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:08:42.687983 22053 catalog_manager.cc:5697] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 1 to 2, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.852599 22154 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.854166 22398 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:42.863211 22304 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.865111 22398 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:42.907078 22304 raft_consensus.cc:1275] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.907078 22231 raft_consensus.cc:1275] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:42.908960 22354 consensus_queue.cc:1048] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:42.909677 22372 consensus_queue.cc:1048] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.017414 22304 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.018916 22372 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.030330 22231 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.032259 22372 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:43.035601 22154 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.037662 22398 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:43.047894 22304 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.050014 22375 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:43.073127 22231 raft_consensus.cc:1275] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.074817 22372 consensus_queue.cc:1048] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.084961 22304 raft_consensus.cc:1275] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.087383 22372 consensus_queue.cc:1048] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:43.299064 22231 raft_consensus.cc:1275] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.300539 22388 consensus_queue.cc:1048] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:43.306089 22154 raft_consensus.cc:1275] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:43.308466 22353 consensus_queue.cc:1048] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:45.317574 21526 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:08:45.318461 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:45.339763 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:45.340530 22154 raft_consensus.cc:606] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Received request to transfer leadership to TS 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:45.343313 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:45.343915 22154 raft_consensus.cc:606] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Received request to transfer leadership to TS 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:45.345080 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:08:45.345573 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:45.714352 22444 raft_consensus.cc:993] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:45.714994 22443 raft_consensus.cc:1081] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Signalling peer 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:45.716936 22303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:53036
I20260709 13:08:45.717494 22303 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:45.717831 22303 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:45.724176 22303 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.726029 22303 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 2 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:45.726976 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:45.727149 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:45.727733 22154 raft_consensus.cc:3055] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:45.727778 22231 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:45.728124 22154 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:45.728950 22154 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.729970 22154 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:45.733127 22231 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 2.
I20260709 13:08:45.734412 22270 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:45.735348 22353 raft_consensus.cc:2804] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:45.736012 22154 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 2.
I20260709 13:08:45.737727 22353 raft_consensus.cc:697] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:45.739130 22444 raft_consensus.cc:993] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:45.738771 22353 consensus_queue.cc:237] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.739656 22444 raft_consensus.cc:1081] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Signalling peer 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:45.741544 22303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:53036
I20260709 13:08:45.742084 22303 raft_consensus.cc:493] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:45.742456 22303 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:45.749168 22303 raft_consensus.cc:515] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.753126 22303 leader_election.cc:290] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 3 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147)
I20260709 13:08:45.753959 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:45.754691 22231 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:45.754544 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:45.755291 22154 raft_consensus.cc:3055] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:45.755657 22154 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:45.756527 22154 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.757872 22154 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:45.758397 22054 catalog_manager.cc:5697] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 1 to 2, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:45.763132 22231 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 3.
I20260709 13:08:45.764221 22154 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 3.
I20260709 13:08:45.764501 22270 leader_election.cc:304] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:45.765506 22353 raft_consensus.cc:2804] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:45.766033 22353 raft_consensus.cc:697] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:45.767009 22353 consensus_queue.cc:237] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:45.781409 22054 catalog_manager.cc:5697] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 2 to 3, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:46.195406 22231 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:46.196972 22353 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:46.210515 22154 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:46.212590 22455 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:46.237951 22231 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:46.239681 22455 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:46.240394 22155 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:46.242079 22455 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:46.346741 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:46.359226 22303 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:53062
I20260709 13:08:46.359891 22303 raft_consensus.cc:606] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:46.376902 22303 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:53062
I20260709 13:08:46.377559 22303 raft_consensus.cc:606] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:46.383507 22231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:46.384087 22231 raft_consensus.cc:606] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:46.389957 22231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:46.390731 22231 raft_consensus.cc:606] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:46.398161 22231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
mode: GRACEFUL
new_leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:49160
I20260709 13:08:46.399000 22231 raft_consensus.cc:606] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Received request to transfer leadership to TS 60ef22fed7c54bc497e6a0bc5947f768
I20260709 13:08:46.447219 22375 raft_consensus.cc:993] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.447711 22461 raft_consensus.cc:1081] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.449630 22155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:33006
I20260709 13:08:46.450290 22155 raft_consensus.cc:493] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:46.450675 22155 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.457206 22155 raft_consensus.cc:515] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.459342 22155 leader_election.cc:290] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:46.460574 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:46.460577 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:46.461279 22231 raft_consensus.cc:3055] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:46.461553 22303 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.461627 22231 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:46.462567 22231 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.463595 22231 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.468792 22231 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.469036 22303 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.470196 22121 leader_election.cc:304] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:46.471112 22444 raft_consensus.cc:2804] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:46.472189 22444 raft_consensus.cc:697] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:46.473105 22444 consensus_queue.cc:237] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.480893 22054 catalog_manager.cc:5697] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 2 to 3, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.608865 22375 raft_consensus.cc:993] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.609462 22461 raft_consensus.cc:1081] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.611454 22155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:33006
I20260709 13:08:46.612290 22155 raft_consensus.cc:493] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:46.612854 22155 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.661128 22155 raft_consensus.cc:515] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.666663 22353 raft_consensus.cc:993] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.667367 22353 raft_consensus.cc:1081] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.668337 22155 leader_election.cc:290] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:46.669618 22353 raft_consensus.cc:993] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.670308 22353 raft_consensus.cc:1081] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.672075 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:46.673310 22231 raft_consensus.cc:3055] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:46.673743 22231 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:46.674094 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:46.675089 22304 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.675208 22155 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:32990
I20260709 13:08:46.675704 22154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:32990
I20260709 13:08:46.675300 22231 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.676326 22154 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:46.676913 22154 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.676267 22155 raft_consensus.cc:493] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:46.677685 22155 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:46.679620 22231 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.686964 22304 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.687264 22155 raft_consensus.cc:515] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.687280 22154 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.688694 22231 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.689535 22121 leader_election.cc:304] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 75a79c6e85aa4a86b6d295f495af93c6; no voters: 
I20260709 13:08:46.690034 22155 leader_election.cc:290] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 4 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:46.690176 22154 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:46.691080 22444 raft_consensus.cc:2804] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:46.691664 22444 raft_consensus.cc:697] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:46.693308 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:46.693019 22444 consensus_queue.cc:237] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.693166 22229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:46.694875 22231 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.695257 22229 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:46.694996 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:46.695988 22303 raft_consensus.cc:3055] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:46.696360 22303 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:46.697650 22303 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.700109 22303 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:46.703083 22375 raft_consensus.cc:993] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: : Instructing follower 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.704118 22375 raft_consensus.cc:1081] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Signalling peer 60ef22fed7c54bc497e6a0bc5947f768 to start an election
I20260709 13:08:46.705228 22231 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.706670 22121 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:46.707711 22462 raft_consensus.cc:2804] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:46.710325 22229 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 4.
I20260709 13:08:46.712623 22462 raft_consensus.cc:697] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:46.712862 22303 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 3.
I20260709 13:08:46.713817 22462 consensus_queue.cc:237] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.716180 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b2bda3c8cd7401c99749d553cf6c450" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:46.714522 22121 leader_election.cc:304] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:46.717950 22154 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
 from {username='slave'} at 127.0.0.1:33006
I20260709 13:08:46.718816 22304 raft_consensus.cc:3055] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:46.719378 22154 raft_consensus.cc:493] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:46.719633 22304 raft_consensus.cc:740] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:46.720055 22154 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:46.721261 22304 consensus_queue.cc:260] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.723278 22304 raft_consensus.cc:3060] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:46.724968 22462 raft_consensus.cc:2804] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:46.725764 22462 raft_consensus.cc:697] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:46.726985 22462 consensus_queue.cc:237] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.732300 22054 catalog_manager.cc:5697] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 2 to 3, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.734531 22154 raft_consensus.cc:515] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.737172 22154 leader_election.cc:290] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Requested vote from peers 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:46.738597 22229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:46.739574 22229 raft_consensus.cc:3055] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:46.740031 22229 raft_consensus.cc:740] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:46.739594 22303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "60ef22fed7c54bc497e6a0bc5947f768" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:46.741387 22303 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:46.741321 22229 consensus_queue.cc:260] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.742276 22304 raft_consensus.cc:2468] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 4.
I20260709 13:08:46.743402 22229 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:46.752583 22303 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:46.752583 22229 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60ef22fed7c54bc497e6a0bc5947f768 in term 2.
I20260709 13:08:46.753934 22121 leader_election.cc:304] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60ef22fed7c54bc497e6a0bc5947f768, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:46.754796 22462 raft_consensus.cc:2804] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:46.755322 22462 raft_consensus.cc:697] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Leader. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:46.756222 22462 consensus_queue.cc:237] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:46.758122 22054 catalog_manager.cc:5697] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 2 to 3, leader changed from 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.759552 22054 catalog_manager.cc:5697] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 3 to 4, leader changed from 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 4 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.775085 22054 catalog_manager.cc:5697] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 reported cstate change: term changed from 1 to 2, leader changed from 7af91453bcd64a8c8d739d595c316978 (127.21.5.130) to 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "60ef22fed7c54bc497e6a0bc5947f768" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:46.981995 22229 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:46.983547 22462 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:46.996488 22303 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:46.999729 22462 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:47.123154 22303 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:47.124678 22462 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:47.135644 22229 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:47.137580 22462 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:47.142786 22304 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:47.142808 22231 raft_consensus.cc:1275] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:47.145840 22469 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:47.147341 22444 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:47.160950 22229 raft_consensus.cc:1275] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:47.166571 22303 raft_consensus.cc:1275] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:47.170037 22462 consensus_queue.cc:1048] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:47.173972 22462 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:47.179814 22303 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:47.182642 22469 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:47.191620 22229 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Refusing update from remote peer 60ef22fed7c54bc497e6a0bc5947f768: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:47.193625 22444 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.409360 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:49.418198 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:49.418910 22154 raft_consensus.cc:606] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Received request to transfer leadership to TS 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:49.421733 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:49.422291 22154 raft_consensus.cc:606] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Received request to transfer leadership to TS 75a79c6e85aa4a86b6d295f495af93c6
I20260709 13:08:49.424901 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:49.425472 22154 raft_consensus.cc:606] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Received request to transfer leadership to TS 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:49.428431 22154 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
mode: GRACEFUL
new_leader_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:58496
I20260709 13:08:49.429014 22154 raft_consensus.cc:606] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Received request to transfer leadership to TS 7af91453bcd64a8c8d739d595c316978
I20260709 13:08:49.430323 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:49.430889 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:49.533366 22480 raft_consensus.cc:993] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:49.533891 22481 raft_consensus.cc:1081] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Signalling peer 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:49.535583 22231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:49140
I20260709 13:08:49.536114 22231 raft_consensus.cc:493] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:49.536407 22231 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.541531 22231 raft_consensus.cc:515] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.543557 22231 leader_election.cc:290] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 4 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:49.544627 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:49.544590 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e125e5edd154f5791623a8e2eca5f43" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:49.545365 22304 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.545466 22154 raft_consensus.cc:3055] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.545890 22154 raft_consensus.cc:740] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:49.546782 22154 consensus_queue.cc:260] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.547892 22154 raft_consensus.cc:3060] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.551095 22304 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 4.
I20260709 13:08:49.552618 22196 leader_election.cc:304] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:49.553586 22154 raft_consensus.cc:2468] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 4.
I20260709 13:08:49.553668 22461 raft_consensus.cc:2804] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:49.554858 22461 raft_consensus.cc:697] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 4 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:49.555745 22461 consensus_queue.cc:237] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.563828 22054 catalog_manager.cc:5697] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 3 to 4, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:49.570549 22480 raft_consensus.cc:993] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:49.571337 22481 raft_consensus.cc:1081] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Signalling peer 7af91453bcd64a8c8d739d595c316978 to start an election
I20260709 13:08:49.573302 22231 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf"
dest_uuid: "7af91453bcd64a8c8d739d595c316978"
 from {username='slave'} at 127.0.0.1:49140
I20260709 13:08:49.574023 22231 raft_consensus.cc:493] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:49.574358 22231 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.582002 22231 raft_consensus.cc:515] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.583673 22231 leader_election.cc:290] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 4 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131:35223)
I20260709 13:08:49.584689 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:49.584682 22304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a56f68092fd14422ac5e080668a180cf" candidate_uuid: "7af91453bcd64a8c8d739d595c316978" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
I20260709 13:08:49.585393 22154 raft_consensus.cc:3055] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.585440 22304 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.585667 22154 raft_consensus.cc:740] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:49.586454 22154 consensus_queue.cc:260] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.587442 22154 raft_consensus.cc:3060] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.591765 22304 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 4.
I20260709 13:08:49.592718 22196 leader_election.cc:304] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:49.593456 22461 raft_consensus.cc:2804] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:49.593639 22154 raft_consensus.cc:2468] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af91453bcd64a8c8d739d595c316978 in term 4.
I20260709 13:08:49.593858 22461 raft_consensus.cc:697] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 4 LEADER]: Becoming Leader. State: Replica: 7af91453bcd64a8c8d739d595c316978, State: Running, Role: LEADER
I20260709 13:08:49.594738 22461 consensus_queue.cc:237] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.601634 22054 catalog_manager.cc:5697] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 reported cstate change: term changed from 3 to 4, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 7af91453bcd64a8c8d739d595c316978 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "7af91453bcd64a8c8d739d595c316978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:49.638793 22480 raft_consensus.cc:993] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:49.639341 22481 raft_consensus.cc:1081] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Signalling peer 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:49.640872 22304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:53036
I20260709 13:08:49.641312 22304 raft_consensus.cc:493] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:49.641546 22304 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.646394 22304 raft_consensus.cc:515] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.648590 22304 leader_election.cc:290] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 4 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:49.649582 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:49.649698 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f378fdf4c44e25bab64d95990d0fed" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:49.650375 22231 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.650375 22154 raft_consensus.cc:3055] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.650861 22154 raft_consensus.cc:740] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:49.651716 22154 consensus_queue.cc:260] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.652980 22154 raft_consensus.cc:3060] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.658207 22231 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 4.
I20260709 13:08:49.659852 22270 leader_election.cc:304] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:49.659958 22154 raft_consensus.cc:2468] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 4.
I20260709 13:08:49.660786 22455 raft_consensus.cc:2804] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:49.662328 22455 raft_consensus.cc:697] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:49.663283 22455 consensus_queue.cc:237] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.671748 22054 catalog_manager.cc:5697] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 3 to 4, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.727306 22480 raft_consensus.cc:993] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: : Instructing follower 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:49.727852 22481 raft_consensus.cc:1081] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Signalling peer 75a79c6e85aa4a86b6d295f495af93c6 to start an election
I20260709 13:08:49.729336 22304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481"
dest_uuid: "75a79c6e85aa4a86b6d295f495af93c6"
 from {username='slave'} at 127.0.0.1:53036
I20260709 13:08:49.729851 22304 raft_consensus.cc:493] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:49.730166 22304 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:49.735210 22304 raft_consensus.cc:515] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.736944 22304 leader_election.cc:290] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 3 election: Requested vote from peers 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147), 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421)
I20260709 13:08:49.738255 22154 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "60ef22fed7c54bc497e6a0bc5947f768"
I20260709 13:08:49.738593 22231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ce54de8688e45e8bcbf01308b2ce481" candidate_uuid: "75a79c6e85aa4a86b6d295f495af93c6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7af91453bcd64a8c8d739d595c316978"
I20260709 13:08:49.738946 22154 raft_consensus.cc:3055] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:49.739136 22231 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:49.739264 22154 raft_consensus.cc:740] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 60ef22fed7c54bc497e6a0bc5947f768, State: Running, Role: LEADER
I20260709 13:08:49.740074 22154 consensus_queue.cc:260] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.741150 22154 raft_consensus.cc:3060] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:49.746945 22231 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 3.
I20260709 13:08:49.748337 22154 raft_consensus.cc:2468] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a79c6e85aa4a86b6d295f495af93c6 in term 3.
I20260709 13:08:49.749019 22270 leader_election.cc:304] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a79c6e85aa4a86b6d295f495af93c6, 7af91453bcd64a8c8d739d595c316978; no voters: 
I20260709 13:08:49.750218 22455 raft_consensus.cc:2804] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:49.750737 22455 raft_consensus.cc:697] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Becoming Leader. State: Replica: 75a79c6e85aa4a86b6d295f495af93c6, State: Running, Role: LEADER
I20260709 13:08:49.752022 22455 consensus_queue.cc:237] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } }
I20260709 13:08:49.761168 22054 catalog_manager.cc:5697] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 reported cstate change: term changed from 2 to 3, leader changed from 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129) to 75a79c6e85aa4a86b6d295f495af93c6 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "75a79c6e85aa4a86b6d295f495af93c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:50.062891 22231 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.064136 22455 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.075850 22154 raft_consensus.cc:1275] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.079051 22455 consensus_queue.cc:1048] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:50.079748 22304 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.081465 22461 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.097774 22154 raft_consensus.cc:1275] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.099717 22482 consensus_queue.cc:1048] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:50.142042 22154 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.143597 22461 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:50.153231 22304 raft_consensus.cc:1275] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Refusing update from remote peer 7af91453bcd64a8c8d739d595c316978: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:50.156986 22461 consensus_queue.cc:1048] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a79c6e85aa4a86b6d295f495af93c6" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 35223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:50.174228 22231 raft_consensus.cc:1275] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:50.175796 22455 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7af91453bcd64a8c8d739d595c316978" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 35421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:50.187806 22154 raft_consensus.cc:1275] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Refusing update from remote peer 75a79c6e85aa4a86b6d295f495af93c6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:50.189348 22494 consensus_queue.cc:1048] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60ef22fed7c54bc497e6a0bc5947f768" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:50.431964 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:50.439050 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
I20260709 13:08:50.471438 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:50.472259 21526 tablet_replica.cc:333] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.472945 21526 raft_consensus.cc:2243] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.473474 21526 raft_consensus.cc:2272] T 0ce54de8688e45e8bcbf01308b2ce481 P 60ef22fed7c54bc497e6a0bc5947f768 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.476186 21526 tablet_replica.cc:333] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.476884 21526 raft_consensus.cc:2243] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.477418 21526 raft_consensus.cc:2272] T a56f68092fd14422ac5e080668a180cf P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.480366 21526 tablet_replica.cc:333] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.481097 21526 raft_consensus.cc:2243] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.482062 21526 raft_consensus.cc:2272] T 47da534c68a8403381813fb9c8ddbe28 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.484261 21526 tablet_replica.cc:333] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.484856 21526 raft_consensus.cc:2243] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.485770 21526 raft_consensus.cc:2272] T e8ab137f040944af8f345448de34a044 P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.487905 21526 tablet_replica.cc:333] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.488490 21526 raft_consensus.cc:2243] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.489029 21526 raft_consensus.cc:2272] T c97b9b4ea6ae400493181b22c3f6213c P 60ef22fed7c54bc497e6a0bc5947f768 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.491000 21526 tablet_replica.cc:333] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.491501 21526 raft_consensus.cc:2243] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.492317 21526 raft_consensus.cc:2272] T 11bbc686e33d4dee9b035c5ad2654458 P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.494287 21526 tablet_replica.cc:333] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.494935 21526 raft_consensus.cc:2243] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.495419 21526 raft_consensus.cc:2272] T 17f378fdf4c44e25bab64d95990d0fed P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.497829 21526 tablet_replica.cc:333] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.498396 21526 raft_consensus.cc:2243] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.498915 21526 raft_consensus.cc:2272] T e97e258839854a01bf8dd90ae6b6bd3c P 60ef22fed7c54bc497e6a0bc5947f768 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.500962 21526 tablet_replica.cc:333] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.501513 21526 raft_consensus.cc:2243] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.501969 21526 raft_consensus.cc:2272] T 5e125e5edd154f5791623a8e2eca5f43 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.504498 21526 tablet_replica.cc:333] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768: stopping tablet replica
I20260709 13:08:50.505124 21526 raft_consensus.cc:2243] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.506018 21526 raft_consensus.cc:2272] T 2b2bda3c8cd7401c99749d553cf6c450 P 60ef22fed7c54bc497e6a0bc5947f768 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.518867 22269 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:08:50.526559 22269 connection.cc:573] client connection to 127.21.5.129:39147 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:08:50.527341 22269 consensus_peers.cc:597] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 -> Peer 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147): Couldn't send request to peer 60ef22fed7c54bc497e6a0bc5947f768. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.546140 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
I20260709 13:08:50.569294 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
I20260709 13:08:50.601459 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:50.602388 21526 tablet_replica.cc:333] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
W20260709 13:08:50.603184 22269 consensus_peers.cc:597] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 -> Peer 60ef22fed7c54bc497e6a0bc5947f768 (127.21.5.129:39147): Couldn't send request to peer 60ef22fed7c54bc497e6a0bc5947f768. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:39147: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.603338 21526 raft_consensus.cc:2243] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.604104 21526 raft_consensus.cc:2272] T 0ce54de8688e45e8bcbf01308b2ce481 P 7af91453bcd64a8c8d739d595c316978 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.606525 21526 tablet_replica.cc:333] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.607200 21526 raft_consensus.cc:2243] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.607647 21526 raft_consensus.cc:2272] T 47da534c68a8403381813fb9c8ddbe28 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.609727 21526 tablet_replica.cc:333] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.610283 21526 raft_consensus.cc:2243] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.611219 21526 raft_consensus.cc:2272] T a56f68092fd14422ac5e080668a180cf P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.614190 21526 tablet_replica.cc:333] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.614871 21526 raft_consensus.cc:2243] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.615571 21526 raft_consensus.cc:2272] T c97b9b4ea6ae400493181b22c3f6213c P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.617652 21526 tablet_replica.cc:333] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.618252 21526 raft_consensus.cc:2243] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.618847 21526 raft_consensus.cc:2272] T e8ab137f040944af8f345448de34a044 P 7af91453bcd64a8c8d739d595c316978 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:08:50.619343 22270 consensus_peers.cc:597] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 -> Peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Couldn't send request to peer 7af91453bcd64a8c8d739d595c316978. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.621042 21526 tablet_replica.cc:333] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.621677 21526 raft_consensus.cc:2243] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.622144 21526 raft_consensus.cc:2272] T 11bbc686e33d4dee9b035c5ad2654458 P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.624087 21526 tablet_replica.cc:333] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.624658 21526 raft_consensus.cc:2243] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.625133 21526 raft_consensus.cc:2272] T 17f378fdf4c44e25bab64d95990d0fed P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.627645 21526 tablet_replica.cc:333] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.628217 21526 raft_consensus.cc:2243] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.629108 21526 raft_consensus.cc:2272] T e97e258839854a01bf8dd90ae6b6bd3c P 7af91453bcd64a8c8d739d595c316978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.631162 21526 tablet_replica.cc:333] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.631700 21526 raft_consensus.cc:2243] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.632566 21526 raft_consensus.cc:2272] T 5e125e5edd154f5791623a8e2eca5f43 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.634882 21526 tablet_replica.cc:333] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978: stopping tablet replica
I20260709 13:08:50.635424 21526 raft_consensus.cc:2243] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.635875 21526 raft_consensus.cc:2272] T 2b2bda3c8cd7401c99749d553cf6c450 P 7af91453bcd64a8c8d739d595c316978 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.658660 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
I20260709 13:08:50.680990 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
W20260709 13:08:50.689930 22270 consensus_peers.cc:597] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 -> Peer 7af91453bcd64a8c8d739d595c316978 (127.21.5.130:35421): Couldn't send request to peer 7af91453bcd64a8c8d739d595c316978. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:35421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:50.710870 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:08:50.711848 21526 tablet_replica.cc:333] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.712599 21526 raft_consensus.cc:2243] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.713167 21526 raft_consensus.cc:2272] T 47da534c68a8403381813fb9c8ddbe28 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.715261 21526 tablet_replica.cc:333] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.715875 21526 raft_consensus.cc:2243] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.716778 21526 raft_consensus.cc:2272] T 17f378fdf4c44e25bab64d95990d0fed P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.719260 21526 tablet_replica.cc:333] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.719890 21526 raft_consensus.cc:2243] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.720373 21526 raft_consensus.cc:2272] T e97e258839854a01bf8dd90ae6b6bd3c P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.722384 21526 tablet_replica.cc:333] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.722990 21526 raft_consensus.cc:2243] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.723462 21526 raft_consensus.cc:2272] T 5e125e5edd154f5791623a8e2eca5f43 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.725682 21526 tablet_replica.cc:333] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.726284 21526 raft_consensus.cc:2243] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.726835 21526 raft_consensus.cc:2272] T a56f68092fd14422ac5e080668a180cf P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.729022 21526 tablet_replica.cc:333] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.729597 21526 raft_consensus.cc:2243] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.730036 21526 raft_consensus.cc:2272] T 2b2bda3c8cd7401c99749d553cf6c450 P 75a79c6e85aa4a86b6d295f495af93c6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.731909 21526 tablet_replica.cc:333] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.732456 21526 raft_consensus.cc:2243] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.733294 21526 raft_consensus.cc:2272] T 0ce54de8688e45e8bcbf01308b2ce481 P 75a79c6e85aa4a86b6d295f495af93c6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.735592 21526 tablet_replica.cc:333] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.736131 21526 raft_consensus.cc:2243] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.737010 21526 raft_consensus.cc:2272] T c97b9b4ea6ae400493181b22c3f6213c P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.739387 21526 tablet_replica.cc:333] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.739915 21526 raft_consensus.cc:2243] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.740391 21526 raft_consensus.cc:2272] T e8ab137f040944af8f345448de34a044 P 75a79c6e85aa4a86b6d295f495af93c6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.742157 21526 tablet_replica.cc:333] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6: stopping tablet replica
I20260709 13:08:50.742692 21526 raft_consensus.cc:2243] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:50.743119 21526 raft_consensus.cc:2272] T 11bbc686e33d4dee9b035c5ad2654458 P 75a79c6e85aa4a86b6d295f495af93c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.766426 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:08:50.788553 21526 master.cc:562] Master@127.21.5.190:37419 shutting down...
I20260709 13:08:50.811035 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:08:50.811705 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:50.812086 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2029440bbfee488f8f11cffdc621525b: stopping tablet replica
I20260709 13:08:50.832789 21526 master.cc:584] Master@127.21.5.190:37419 shutdown complete.
60ef22fed7c54bc497e6a0bc5947f768  5
75a79c6e85aa4a86b6d295f495af93c6  1
7af91453bcd64a8c8d739d595c316978  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10835 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:08:50.882241 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:44025
I20260709 13:08:50.883607 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:50.889747 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:50.889983 22509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:50.892036 22511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:50.892666 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:50.893766 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:50.894008 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:50.894183 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602530894165 us; error 0 us; skew 500 ppm
I20260709 13:08:50.894927 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:50.897815 21526 webserver.cc:533] Webserver started at http://127.21.5.190:32855/ using document root <none> and password file <none>
I20260709 13:08:50.898387 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:50.898649 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:50.898977 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:50.900271 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "71605ba8587a4b5193d8d69d9967c1af"
format_stamp: "Formatted at 2026-07-09 13:08:50 on dist-test-slave-h0rz"
I20260709 13:08:50.905841 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:08:50.910205 22516 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:50.911342 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:08:50.911680 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "71605ba8587a4b5193d8d69d9967c1af"
format_stamp: "Formatted at 2026-07-09 13:08:50 on dist-test-slave-h0rz"
I20260709 13:08:50.911975 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:50.941635 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:50.943111 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:50.991107 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:44025
I20260709 13:08:50.991227 22567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:44025 every 8 connection(s)
I20260709 13:08:50.995911 22568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.008564 22568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Bootstrap starting.
I20260709 13:08:51.014174 22568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:51.019281 22568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: No bootstrap required, opened a new log
I20260709 13:08:51.022006 22568 raft_consensus.cc:359] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER }
I20260709 13:08:51.022490 22568 raft_consensus.cc:385] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:51.022816 22568 raft_consensus.cc:740] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71605ba8587a4b5193d8d69d9967c1af, State: Initialized, Role: FOLLOWER
I20260709 13:08:51.023455 22568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER }
I20260709 13:08:51.023975 22568 raft_consensus.cc:399] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:08:51.024232 22568 raft_consensus.cc:493] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:08:51.024534 22568 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:51.029510 22568 raft_consensus.cc:515] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER }
I20260709 13:08:51.030177 22568 leader_election.cc:304] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71605ba8587a4b5193d8d69d9967c1af; no voters: 
I20260709 13:08:51.031543 22568 leader_election.cc:290] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:08:51.031945 22571 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:51.033402 22571 raft_consensus.cc:697] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 1 LEADER]: Becoming Leader. State: Replica: 71605ba8587a4b5193d8d69d9967c1af, State: Running, Role: LEADER
I20260709 13:08:51.034178 22571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER }
I20260709 13:08:51.034981 22568 sys_catalog.cc:565] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:08:51.040342 22572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71605ba8587a4b5193d8d69d9967c1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER } }
I20260709 13:08:51.040390 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71605ba8587a4b5193d8d69d9967c1af. Latest consensus state: current_term: 1 leader_uuid: "71605ba8587a4b5193d8d69d9967c1af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71605ba8587a4b5193d8d69d9967c1af" member_type: VOTER } }
I20260709 13:08:51.041774 22572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:51.041761 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [sys.catalog]: This master's current role is: LEADER
I20260709 13:08:51.044992 22578 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:08:51.051088 22578 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:08:51.055709 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:08:51.062304 22578 catalog_manager.cc:1383] Generated new cluster ID: f63cb98d7bce46bb9c84300b4d2dcd25
I20260709 13:08:51.062707 22578 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:08:51.082124 22578 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:08:51.083688 22578 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:08:51.099014 22578 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Generated new TSK 0
I20260709 13:08:51.099745 22578 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:08:51.123916 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:51.131111 22589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:51.132570 22590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:51.134142 22592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:51.134446 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:51.135577 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:51.135802 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:51.135931 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602531135918 us; error 0 us; skew 500 ppm
I20260709 13:08:51.136493 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:51.139392 21526 webserver.cc:533] Webserver started at http://127.21.5.129:45915/ using document root <none> and password file <none>
I20260709 13:08:51.140064 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:51.140321 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:51.140664 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:51.141886 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "1f99ad34cc414233929e0a92f8c3d8db"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.147255 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:08:51.151292 22597 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.152266 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:08:51.152671 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "1f99ad34cc414233929e0a92f8c3d8db"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.153028 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:51.183393 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:51.184947 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:51.186926 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:51.189692 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:51.189952 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.190251 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:51.190452 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.239282 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:37283
I20260709 13:08:51.239369 22659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:37283 every 8 connection(s)
I20260709 13:08:51.245164 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:51.253806 22664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:51.256130 22665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:51.259295 22667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:51.259356 21526 server_base.cc:1061] running on GCE node
I20260709 13:08:51.261109 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:51.261499 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:51.261703 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602531261683 us; error 0 us; skew 500 ppm
I20260709 13:08:51.262780 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:51.266111 22660 heartbeater.cc:344] Connected to a master server at 127.21.5.190:44025
I20260709 13:08:51.266542 22660 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:51.267176 21526 webserver.cc:533] Webserver started at http://127.21.5.130:45621/ using document root <none> and password file <none>
I20260709 13:08:51.267524 22660 heartbeater.cc:507] Master 127.21.5.190:44025 requested a full tablet report, sending...
I20260709 13:08:51.267773 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:51.268077 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:51.268369 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:51.269886 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "91ebb78bf12f41baae84669c1effc423"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.270146 22533 ts_manager.cc:194] Registered new tserver with Master: 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:08:51.272200 22533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38102
I20260709 13:08:51.275683 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:08:51.279775 22672 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.280827 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:51.281121 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "91ebb78bf12f41baae84669c1effc423"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.281395 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:51.295706 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:51.297055 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:51.298681 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:51.301154 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:51.301429 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.301710 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:51.301884 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.350889 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:34681
I20260709 13:08:51.351003 22734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:34681 every 8 connection(s)
I20260709 13:08:51.356405 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:08:51.364919 22738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:08:51.366745 22739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:51.369277 21526 server_base.cc:1061] running on GCE node
W20260709 13:08:51.369244 22741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:08:51.370136 22735 heartbeater.cc:344] Connected to a master server at 127.21.5.190:44025
I20260709 13:08:51.370574 22735 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:51.370836 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:08:51.371101 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:08:51.371284 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602531371266 us; error 0 us; skew 500 ppm
I20260709 13:08:51.371628 22735 heartbeater.cc:507] Master 127.21.5.190:44025 requested a full tablet report, sending...
I20260709 13:08:51.371968 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:08:51.374184 22533 ts_manager.cc:194] Registered new tserver with Master: 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:51.375432 21526 webserver.cc:533] Webserver started at http://127.21.5.131:36971/ using document root <none> and password file <none>
I20260709 13:08:51.376149 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:08:51.376434 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:08:51.376339 22533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38108
I20260709 13:08:51.376869 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:08:51.378520 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.383817 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:08:51.387693 22746 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.388762 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:08:51.389287 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
format_stamp: "Formatted at 2026-07-09 13:08:51 on dist-test-slave-h0rz"
I20260709 13:08:51.389626 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:08:51.409066 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:08:51.410815 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:08:51.412269 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:08:51.415014 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:08:51.415243 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.415529 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:08:51.415678 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:08:51.467383 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:39075
I20260709 13:08:51.467474 22808 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:39075 every 8 connection(s)
I20260709 13:08:51.487241 22809 heartbeater.cc:344] Connected to a master server at 127.21.5.190:44025
I20260709 13:08:51.487715 22809 heartbeater.cc:461] Registering TS with master...
I20260709 13:08:51.488576 22809 heartbeater.cc:507] Master 127.21.5.190:44025 requested a full tablet report, sending...
I20260709 13:08:51.490828 22533 ts_manager.cc:194] Registered new tserver with Master: a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:51.491782 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020725774s
I20260709 13:08:51.492700 22533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38114
I20260709 13:08:51.526782 22533 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38126:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:08:51.533849 22533 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:08:51.797029 22625 tablet_service.cc:1511] Processing CreateTablet for tablet fee10603c0c042fd8e28683daeb86a99 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:51.798974 22622 tablet_service.cc:1511] Processing CreateTablet for tablet 99c51f61e4f74c1593ebf98255a83b4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:51.800015 22621 tablet_service.cc:1511] Processing CreateTablet for tablet 3e474e7307e645259a4f6d201b451bef (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:51.801852 22623 tablet_service.cc:1511] Processing CreateTablet for tablet 91be8db7516b4bca939f439f045b40eb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:51.802862 22774 tablet_service.cc:1511] Processing CreateTablet for tablet fee10603c0c042fd8e28683daeb86a99 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:51.803013 22624 tablet_service.cc:1511] Processing CreateTablet for tablet bfb716d0bf184679a89d2dde7563865d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:51.803778 22700 tablet_service.cc:1511] Processing CreateTablet for tablet fee10603c0c042fd8e28683daeb86a99 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:08:51.804683 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 99c51f61e4f74c1593ebf98255a83b4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:51.805349 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee10603c0c042fd8e28683daeb86a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.805441 22772 tablet_service.cc:1511] Processing CreateTablet for tablet 91be8db7516b4bca939f439f045b40eb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:51.806260 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c51f61e4f74c1593ebf98255a83b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.808146 22773 tablet_service.cc:1511] Processing CreateTablet for tablet bfb716d0bf184679a89d2dde7563865d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:51.809851 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb716d0bf184679a89d2dde7563865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.810031 22770 tablet_service.cc:1511] Processing CreateTablet for tablet 3e474e7307e645259a4f6d201b451bef (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:51.811256 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91be8db7516b4bca939f439f045b40eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.811017 22699 tablet_service.cc:1511] Processing CreateTablet for tablet bfb716d0bf184679a89d2dde7563865d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:08:51.812482 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb716d0bf184679a89d2dde7563865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.813385 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e474e7307e645259a4f6d201b451bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.814591 22620 tablet_service.cc:1511] Processing CreateTablet for tablet 60c24d6125434f4caf1190c8bdba2f12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:51.815752 22698 tablet_service.cc:1511] Processing CreateTablet for tablet 91be8db7516b4bca939f439f045b40eb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:08:51.817304 22698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91be8db7516b4bca939f439f045b40eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.817973 22774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee10603c0c042fd8e28683daeb86a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.819166 22625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee10603c0c042fd8e28683daeb86a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.820150 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfb716d0bf184679a89d2dde7563865d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.820986 22622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c51f61e4f74c1593ebf98255a83b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.823057 22621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e474e7307e645259a4f6d201b451bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.823207 22769 tablet_service.cc:1511] Processing CreateTablet for tablet 60c24d6125434f4caf1190c8bdba2f12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:51.824728 22620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c24d6125434f4caf1190c8bdba2f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.826702 22697 tablet_service.cc:1511] Processing CreateTablet for tablet 99c51f61e4f74c1593ebf98255a83b4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:08:51.827960 22623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91be8db7516b4bca939f439f045b40eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.828305 22697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c51f61e4f74c1593ebf98255a83b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.824779 22769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c24d6125434f4caf1190c8bdba2f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.832633 22696 tablet_service.cc:1511] Processing CreateTablet for tablet 3e474e7307e645259a4f6d201b451bef (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:08:51.834255 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e474e7307e645259a4f6d201b451bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.845002 22695 tablet_service.cc:1511] Processing CreateTablet for tablet 60c24d6125434f4caf1190c8bdba2f12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:08:51.846911 22695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c24d6125434f4caf1190c8bdba2f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.854043 22824 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:51.863922 22824 tablet_bootstrap.cc:654] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:51.864257 22694 tablet_service.cc:1511] Processing CreateTablet for tablet 71723544b43c4cb99dbeaf0321d14da6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:08:51.862779 22624 tablet_service.cc:1511] Processing CreateTablet for tablet 71723544b43c4cb99dbeaf0321d14da6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:08:51.907899 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71723544b43c4cb99dbeaf0321d14da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.883514 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 71723544b43c4cb99dbeaf0321d14da6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:08:51.883913 22619 tablet_service.cc:1511] Processing CreateTablet for tablet b8291ea7e33a40b5afaebc02d6ce7e9d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:51.911039 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71723544b43c4cb99dbeaf0321d14da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.911129 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8291ea7e33a40b5afaebc02d6ce7e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.912061 22825 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:51.913884 22693 tablet_service.cc:1511] Processing CreateTablet for tablet b8291ea7e33a40b5afaebc02d6ce7e9d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:51.907904 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71723544b43c4cb99dbeaf0321d14da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.917433 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8291ea7e33a40b5afaebc02d6ce7e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.929584 22770 tablet_service.cc:1511] Processing CreateTablet for tablet b8291ea7e33a40b5afaebc02d6ce7e9d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:08:51.931195 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8291ea7e33a40b5afaebc02d6ce7e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.932358 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac0029002284e64a5fde523ffaf77fb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:51.934036 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac0029002284e64a5fde523ffaf77fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.937659 22692 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac0029002284e64a5fde523ffaf77fb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:51.939601 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac0029002284e64a5fde523ffaf77fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.953742 22824 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:51.954489 22824 ts_tablet_manager.cc:1403] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.000s
I20260709 13:08:51.962750 22768 tablet_service.cc:1511] Processing CreateTablet for tablet 9ac0029002284e64a5fde523ffaf77fb (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:08:51.964581 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac0029002284e64a5fde523ffaf77fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.967437 22699 tablet_service.cc:1511] Processing CreateTablet for tablet cc8b169704cc4157ad021d774f83419e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:51.968567 22617 tablet_service.cc:1511] Processing CreateTablet for tablet cc8b169704cc4157ad021d774f83419e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:51.970206 22617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8b169704cc4157ad021d774f83419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.976459 22767 tablet_service.cc:1511] Processing CreateTablet for tablet cc8b169704cc4157ad021d774f83419e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:08:51.978345 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8b169704cc4157ad021d774f83419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.979931 22826 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:51.987454 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 1918c6e39d7d46139a210c432e0f4d84 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:51.989152 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1918c6e39d7d46139a210c432e0f4d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:51.995682 22825 tablet_bootstrap.cc:654] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:51.969033 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8b169704cc4157ad021d774f83419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.001344 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 1918c6e39d7d46139a210c432e0f4d84 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:51.958027 22824 raft_consensus.cc:359] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.007004 22824 raft_consensus.cc:385] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.007432 22824 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.011106 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1918c6e39d7d46139a210c432e0f4d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.011055 22824 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.041589 22773 tablet_service.cc:1511] Processing CreateTablet for tablet 1918c6e39d7d46139a210c432e0f4d84 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:08:52.050216 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1918c6e39d7d46139a210c432e0f4d84. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:52.050414 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:52.050597 22692 tablet_service.cc:1511] Processing CreateTablet for tablet a840ecd041fa46a198e4fb917ef79b2a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:52.054841 22660 heartbeater.cc:499] Master 127.21.5.190:44025 was elected leader, sending a full tablet report...
I20260709 13:08:52.055521 22824 ts_tablet_manager.cc:1434] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.101s	user 0.009s	sys 0.003s
I20260709 13:08:52.056128 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a840ecd041fa46a198e4fb917ef79b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.056697 22824 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.057058 22772 tablet_service.cc:1511] Processing CreateTablet for tablet a840ecd041fa46a198e4fb917ef79b2a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:52.060678 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a840ecd041fa46a198e4fb917ef79b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.065534 22824 tablet_bootstrap.cc:654] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.064359 22825 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:52.066354 22825 ts_tablet_manager.cc:1403] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.155s	user 0.017s	sys 0.000s
I20260709 13:08:52.058929 22618 tablet_service.cc:1511] Processing CreateTablet for tablet a840ecd041fa46a198e4fb917ef79b2a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:08:52.076673 22826 tablet_bootstrap.cc:654] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.075487 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a840ecd041fa46a198e4fb917ef79b2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.080789 22825 raft_consensus.cc:359] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.089318 22825 raft_consensus.cc:385] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.093258 22619 tablet_service.cc:1511] Processing CreateTablet for tablet e6d9cbf659974b358ba6d364ca9ca431 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:52.094869 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d9cbf659974b358ba6d364ca9ca431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.094527 22693 tablet_service.cc:1511] Processing CreateTablet for tablet e6d9cbf659974b358ba6d364ca9ca431 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:52.095940 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d9cbf659974b358ba6d364ca9ca431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.090476 22825 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.099249 22825 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.108740 22824 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:52.109427 22824 ts_tablet_manager.cc:1403] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.053s	user 0.016s	sys 0.017s
I20260709 13:08:52.114435 22824 raft_consensus.cc:359] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.114789 22773 tablet_service.cc:1511] Processing CreateTablet for tablet e6d9cbf659974b358ba6d364ca9ca431 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:08:52.115294 22824 raft_consensus.cc:385] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.115645 22824 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.115998 22616 tablet_service.cc:1511] Processing CreateTablet for tablet b907163bc2174a2aa11df858ccae00ac (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:52.116544 22824 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.117496 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907163bc2174a2aa11df858ccae00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.119174 22824 ts_tablet_manager.cc:1434] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.007s
I20260709 13:08:52.120338 22824 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.126745 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d9cbf659974b358ba6d364ca9ca431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.128381 22824 tablet_bootstrap.cc:654] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.128993 22699 tablet_service.cc:1511] Processing CreateTablet for tablet b907163bc2174a2aa11df858ccae00ac (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:52.131125 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907163bc2174a2aa11df858ccae00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.131908 22767 tablet_service.cc:1511] Processing CreateTablet for tablet b907163bc2174a2aa11df858ccae00ac (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:08:52.133745 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b907163bc2174a2aa11df858ccae00ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.146026 22617 tablet_service.cc:1511] Processing CreateTablet for tablet 66c163cd962d4a568a04fc251b913b43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:52.147018 22692 tablet_service.cc:1511] Processing CreateTablet for tablet 66c163cd962d4a568a04fc251b913b43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:52.148898 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c163cd962d4a568a04fc251b913b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.159322 22772 tablet_service.cc:1511] Processing CreateTablet for tablet 66c163cd962d4a568a04fc251b913b43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:08:52.160984 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c163cd962d4a568a04fc251b913b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.163909 22617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c163cd962d4a568a04fc251b913b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.166116 22825 ts_tablet_manager.cc:1434] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.099s	user 0.002s	sys 0.009s
I20260709 13:08:52.166144 22694 tablet_service.cc:1511] Processing CreateTablet for tablet c015ab1d14c443258576403d9d39c485 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:52.167156 22809 heartbeater.cc:499] Master 127.21.5.190:44025 was elected leader, sending a full tablet report...
I20260709 13:08:52.167824 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c015ab1d14c443258576403d9d39c485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.173161 22825 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:52.175925 22619 tablet_service.cc:1511] Processing CreateTablet for tablet c015ab1d14c443258576403d9d39c485 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:52.177563 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c015ab1d14c443258576403d9d39c485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.184791 22824 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:52.185026 22826 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:52.185488 22824 ts_tablet_manager.cc:1403] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.065s	user 0.004s	sys 0.012s
I20260709 13:08:52.185609 22826 ts_tablet_manager.cc:1403] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.206s	user 0.018s	sys 0.024s
I20260709 13:08:52.189472 22826 raft_consensus.cc:359] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.190369 22826 raft_consensus.cc:385] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.193277 22826 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.197124 22824 raft_consensus.cc:359] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.194345 22826 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.197954 22824 raft_consensus.cc:385] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.200971 22825 tablet_bootstrap.cc:654] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.198321 22824 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.208989 22772 tablet_service.cc:1511] Processing CreateTablet for tablet c015ab1d14c443258576403d9d39c485 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:08:52.210947 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c015ab1d14c443258576403d9d39c485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.210590 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 92329501c20843f68ffa5b3fdfc8f83d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:52.212297 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92329501c20843f68ffa5b3fdfc8f83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.215121 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 92329501c20843f68ffa5b3fdfc8f83d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:52.223637 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92329501c20843f68ffa5b3fdfc8f83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.213423 22824 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.228798 22824 ts_tablet_manager.cc:1434] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
I20260709 13:08:52.235868 22828 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.236501 22828 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.220780 22768 tablet_service.cc:1511] Processing CreateTablet for tablet 92329501c20843f68ffa5b3fdfc8f83d (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:08:52.239765 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92329501c20843f68ffa5b3fdfc8f83d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.241106 22824 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.219055 22735 heartbeater.cc:499] Master 127.21.5.190:44025 was elected leader, sending a full tablet report...
I20260709 13:08:52.218336 22826 ts_tablet_manager.cc:1434] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.001s
I20260709 13:08:52.247797 22826 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:52.249243 22828 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.262024 22700 tablet_service.cc:1511] Processing CreateTablet for tablet a937bd6b17c04d5199f368b1d595cf8b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:52.263680 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a937bd6b17c04d5199f368b1d595cf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.265874 22824 tablet_bootstrap.cc:654] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.267483 22618 tablet_service.cc:1511] Processing CreateTablet for tablet a937bd6b17c04d5199f368b1d595cf8b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:52.271533 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a937bd6b17c04d5199f368b1d595cf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.279372 22826 tablet_bootstrap.cc:654] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.289294 22767 tablet_service.cc:1511] Processing CreateTablet for tablet a937bd6b17c04d5199f368b1d595cf8b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:08:52.292863 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:52.300360 22601 leader_election.cc:343] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.307363 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a937bd6b17c04d5199f368b1d595cf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.311556 22825 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:52.312197 22825 ts_tablet_manager.cc:1403] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.000s
I20260709 13:08:52.311985 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 11f8cc3feaf345e397b685036c912e13 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:52.313517 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8cc3feaf345e397b685036c912e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.316748 22617 tablet_service.cc:1511] Processing CreateTablet for tablet 11f8cc3feaf345e397b685036c912e13 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:52.318301 22617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8cc3feaf345e397b685036c912e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.321200 22824 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:52.321738 22824 ts_tablet_manager.cc:1403] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.081s	user 0.017s	sys 0.007s
I20260709 13:08:52.328799 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:52.330992 22600 leader_election.cc:343] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.331574 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:52.332939 22828 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:52.336071 22825 raft_consensus.cc:359] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.336988 22825 raft_consensus.cc:385] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.337736 22773 tablet_service.cc:1511] Processing CreateTablet for tablet 11f8cc3feaf345e397b685036c912e13 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:08:52.337309 22825 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.339257 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8cc3feaf345e397b685036c912e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.337848 22824 raft_consensus.cc:359] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:52.339689 22824 raft_consensus.cc:385] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.340890 22824 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.341723 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:52.340059 22825 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.343247 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.341794 22824 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:52.346454 22824 ts_tablet_manager.cc:1434] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 13:08:52.349727 22824 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.351362 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:52.344499 22825 ts_tablet_manager.cc:1434] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.000s
I20260709 13:08:52.352943 22825 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:52.355284 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.362891 22824 tablet_bootstrap.cc:654] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.363332 22772 tablet_service.cc:1511] Processing CreateTablet for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:08:52.364531 22826 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:52.364948 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.365124 22826 ts_tablet_manager.cc:1403] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.118s	user 0.020s	sys 0.009s
I20260709 13:08:52.369096 22826 raft_consensus.cc:359] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.369988 22826 raft_consensus.cc:385] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.375816 22832 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.376682 22832 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.370339 22826 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.380484 22826 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.385208 22826 ts_tablet_manager.cc:1434] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 13:08:52.385043 22828 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.397397 22699 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:52.398978 22828 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.399464 22832 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:52.399756 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.401351 22828 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.404415 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:52.405700 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:52.406467 22784 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:52.398268 22824 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:52.408028 22824 ts_tablet_manager.cc:1403] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.015s
I20260709 13:08:52.408254 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:08:52.410427 22601 leader_election.cc:343] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.411572 22828 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:52.411972 22828 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.412284 22828 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.412148 22824 raft_consensus.cc:359] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.413262 22824 raft_consensus.cc:385] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.413671 22824 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.422257 22825 tablet_bootstrap.cc:654] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.421759 22828 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.422555 22617 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:52.428112 22617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.425133 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:08:52.430543 22700 tablet_service.cc:1511] Processing CreateTablet for tablet d6544eab6fd54f95baf389a8b15729cd (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:52.430815 22616 tablet_service.cc:1511] Processing CreateTablet for tablet d6544eab6fd54f95baf389a8b15729cd (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:52.431519 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a4b81cf2b9c48b59c92dd3ace99c8db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.432107 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6544eab6fd54f95baf389a8b15729cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.432193 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6544eab6fd54f95baf389a8b15729cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.426733 22826 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:52.444087 22768 tablet_service.cc:1511] Processing CreateTablet for tablet d6544eab6fd54f95baf389a8b15729cd (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:08:52.445626 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6544eab6fd54f95baf389a8b15729cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.447062 22692 tablet_service.cc:1511] Processing CreateTablet for tablet 1d580d080ca644b2af67c9e48cbd3b78 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:52.448729 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d580d080ca644b2af67c9e48cbd3b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.450071 22619 tablet_service.cc:1511] Processing CreateTablet for tablet 1d580d080ca644b2af67c9e48cbd3b78 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:52.450520 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:52.451555 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d580d080ca644b2af67c9e48cbd3b78. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:52.452205 22601 leader_election.cc:343] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.452453 22824 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.459640 22770 tablet_service.cc:1511] Processing CreateTablet for tablet 1d580d080ca644b2af67c9e48cbd3b78 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:08:52.460616 22828 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.461764 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d580d080ca644b2af67c9e48cbd3b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.464036 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:52.464707 22784 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.479851 22620 tablet_service.cc:1511] Processing CreateTablet for tablet 34f27615de694e7faaba89794bf9b5c6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:52.482077 22620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f27615de694e7faaba89794bf9b5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.488618 22828 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.489583 22828 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.503748 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:52.505960 22635 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:52.503993 22825 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:52.507786 22825 ts_tablet_manager.cc:1403] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.155s	user 0.024s	sys 0.016s
I20260709 13:08:52.508548 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:52.509694 22832 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:52.510211 22832 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.510636 22832 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.515869 22834 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.516808 22834 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.502256 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 34f27615de694e7faaba89794bf9b5c6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:52.531131 22826 tablet_bootstrap.cc:654] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.526201 22824 ts_tablet_manager.cc:1434] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.118s	user 0.009s	sys 0.006s
I20260709 13:08:52.528133 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 34f27615de694e7faaba89794bf9b5c6 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:08:52.532568 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f27615de694e7faaba89794bf9b5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.532714 22824 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.543043 22825 raft_consensus.cc:359] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.544006 22825 raft_consensus.cc:385] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.544342 22825 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.541757 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34f27615de694e7faaba89794bf9b5c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.545392 22825 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.548070 22834 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.551628 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:52.552714 22710 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:52.554589 22601 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:52.557508 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:52.557734 22624 tablet_service.cc:1511] Processing CreateTablet for tablet ec9764b0b7dd4b67a5b50a652203c60a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:52.559182 22828 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.559557 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9764b0b7dd4b67a5b50a652203c60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.571084 22784 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:52.570873 22832 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.572804 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:52.574050 22832 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:52.573706 22773 tablet_service.cc:1511] Processing CreateTablet for tablet ec9764b0b7dd4b67a5b50a652203c60a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:52.574463 22845 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:52.576260 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9764b0b7dd4b67a5b50a652203c60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.576678 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:52.578900 22694 tablet_service.cc:1511] Processing CreateTablet for tablet ec9764b0b7dd4b67a5b50a652203c60a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:08:52.560048 22828 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:52.594298 22635 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.607774 22783 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:52.620441 22699 tablet_service.cc:1511] Processing CreateTablet for tablet 295bf86c5a444ebc9285e902fd5e9ae0 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:52.623637 22635 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:52.626225 22828 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.626842 22828 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:52.627454 22825 ts_tablet_manager.cc:1434] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.119s	user 0.023s	sys 0.014s
I20260709 13:08:52.628660 22825 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:52.632350 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:52.633813 22832 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:52.634061 22848 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.634539 22832 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:52.634693 22848 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:52.635835 22832 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.637565 22848 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:52.639331 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:52.641477 22601 leader_election.cc:343] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.642872 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9764b0b7dd4b67a5b50a652203c60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.643646 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295bf86c5a444ebc9285e902fd5e9ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.657963 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:52.658852 22709 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.659366 22824 tablet_bootstrap.cc:654] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.602802 22845 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:52.665526 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:52.666350 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 295bf86c5a444ebc9285e902fd5e9ae0 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:52.668043 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295bf86c5a444ebc9285e902fd5e9ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.675885 22772 tablet_service.cc:1511] Processing CreateTablet for tablet 295bf86c5a444ebc9285e902fd5e9ae0 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:08:52.676192 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:52.677111 22783 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:52.677453 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295bf86c5a444ebc9285e902fd5e9ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.679733 22695 tablet_service.cc:1511] Processing CreateTablet for tablet ffde8dbdd2d54723860d280e3e916e60 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:52.681674 22618 tablet_service.cc:1511] Processing CreateTablet for tablet ffde8dbdd2d54723860d280e3e916e60 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:52.685236 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffde8dbdd2d54723860d280e3e916e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.687301 22695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffde8dbdd2d54723860d280e3e916e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.689720 22770 tablet_service.cc:1511] Processing CreateTablet for tablet ffde8dbdd2d54723860d280e3e916e60 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:08:52.680250 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:52.681067 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:52.697331 22834 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:52.697821 22834 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.697487 22845 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.698201 22834 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:52.699033 22675 leader_election.cc:343] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.704337 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:52.707049 22825 tablet_bootstrap.cc:654] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:52.716732 22600 leader_election.cc:343] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.717332 22600 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:52.702076 22617 tablet_service.cc:1511] Processing CreateTablet for tablet 53c258bc80ce4d88bac804cb01a34b32 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:08:52.720324 22617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c258bc80ce4d88bac804cb01a34b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.721894 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 53c258bc80ce4d88bac804cb01a34b32 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:08:52.723450 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c258bc80ce4d88bac804cb01a34b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.730880 22709 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:52.730480 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 53c258bc80ce4d88bac804cb01a34b32 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:08:52.718086 22828 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.734321 22850 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:52.739413 22826 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:52.740481 22826 ts_tablet_manager.cc:1403] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.314s	user 0.017s	sys 0.012s
I20260709 13:08:52.739508 22834 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.709947 22531 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:52.746405 22620 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6afadc577c4fd08e4345a82ae50e5f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:52.748137 22620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6afadc577c4fd08e4345a82ae50e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.748697 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:52.749395 22783 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:52.750833 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:08:52.752563 22675 leader_election.cc:343] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.755025 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffde8dbdd2d54723860d280e3e916e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.758021 22850 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.757100 22694 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6afadc577c4fd08e4345a82ae50e5f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:52.763196 22826 raft_consensus.cc:359] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.764743 22783 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:52.765951 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:52.746482 22834 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.769059 22850 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.771020 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6afadc577c4fd08e4345a82ae50e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.775872 22825 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:52.781100 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c258bc80ce4d88bac804cb01a34b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.772861 22826 raft_consensus.cc:385] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.775830 22824 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:52.794018 22709 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:52.775518 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:52.802062 22619 tablet_service.cc:1511] Processing CreateTablet for tablet 05be2cb1839f447296223033d364a7cc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:52.774029 22624 tablet_service.cc:1511] Processing CreateTablet for tablet bf401fd06552416592129b334cdf8167 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:08:52.773725 22828 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.797540 22826 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.806941 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:08:52.812469 22834 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:52.810822 22850 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:52.813207 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:52.811056 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05be2cb1839f447296223033d364a7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.783932 22767 tablet_service.cc:1511] Processing CreateTablet for tablet bf401fd06552416592129b334cdf8167 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
W20260709 13:08:52.815126 22601 leader_election.cc:343] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.813184 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:52.818142 22600 leader_election.cc:343] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:52.818722 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:52.807837 22825 ts_tablet_manager.cc:1403] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.179s	user 0.012s	sys 0.010s
I20260709 13:08:52.828125 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf401fd06552416592129b334cdf8167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.821486 22828 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:52.833334 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf401fd06552416592129b334cdf8167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.780694 22692 tablet_service.cc:1511] Processing CreateTablet for tablet bf401fd06552416592129b334cdf8167 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:08:52.836100 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf401fd06552416592129b334cdf8167. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.820148 22824 ts_tablet_manager.cc:1403] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.288s	user 0.035s	sys 0.015s
I20260709 13:08:52.777454 22773 tablet_service.cc:1511] Processing CreateTablet for tablet 4f6afadc577c4fd08e4345a82ae50e5f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:08:52.835036 22825 raft_consensus.cc:359] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:52.856405 22825 raft_consensus.cc:385] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.856814 22825 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.858398 22625 tablet_service.cc:1511] Processing CreateTablet for tablet af344e0ef1b1489a9bc301ba3b375d4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:52.858573 22696 tablet_service.cc:1511] Processing CreateTablet for tablet af344e0ef1b1489a9bc301ba3b375d4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:52.858398 22772 tablet_service.cc:1511] Processing CreateTablet for tablet 05be2cb1839f447296223033d364a7cc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:52.860255 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af344e0ef1b1489a9bc301ba3b375d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.860985 22772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05be2cb1839f447296223033d364a7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.865326 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 05be2cb1839f447296223033d364a7cc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:08:52.875273 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6afadc577c4fd08e4345a82ae50e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.875797 22621 tablet_service.cc:1511] Processing CreateTablet for tablet 160d972ad1454f9db922aa360a0e7436 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:52.876204 22768 tablet_service.cc:1511] Processing CreateTablet for tablet af344e0ef1b1489a9bc301ba3b375d4c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:08:52.849447 22826 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.877604 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af344e0ef1b1489a9bc301ba3b375d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.882545 22625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af344e0ef1b1489a9bc301ba3b375d4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.883014 22825 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:52.886162 22825 ts_tablet_manager.cc:1434] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.002s
I20260709 13:08:52.879798 22697 tablet_service.cc:1511] Processing CreateTablet for tablet 160d972ad1454f9db922aa360a0e7436 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:52.887557 22825 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:52.889235 22533 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:52.851997 22834 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:52.892390 22621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d972ad1454f9db922aa360a0e7436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.891875 22834 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.897384 22826 ts_tablet_manager.cc:1434] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.153s	user 0.006s	sys 0.005s
I20260709 13:08:52.894815 22709 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:08:52.904361 22601 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:52.905963 22845 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:52.906525 22845 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:52.907513 22845 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.910933 22826 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:52.913391 22769 tablet_service.cc:1511] Processing CreateTablet for tablet 160d972ad1454f9db922aa360a0e7436 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:08:52.914008 22824 raft_consensus.cc:359] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.914841 22824 raft_consensus.cc:385] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:52.915235 22769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d972ad1454f9db922aa360a0e7436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.915127 22618 tablet_service.cc:1511] Processing CreateTablet for tablet a79a6f618f4944b1a11b9c1df766ae7f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:52.916707 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a79a6f618f4944b1a11b9c1df766ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.915154 22824 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:52.920198 22824 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:52.924875 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05be2cb1839f447296223033d364a7cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.925621 22697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d972ad1454f9db922aa360a0e7436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.931579 22824 ts_tablet_manager.cc:1434] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.002s
I20260709 13:08:52.933010 22824 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:52.937772 22694 tablet_service.cc:1511] Processing CreateTablet for tablet a79a6f618f4944b1a11b9c1df766ae7f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:52.955801 22770 tablet_service.cc:1511] Processing CreateTablet for tablet a79a6f618f4944b1a11b9c1df766ae7f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:08:52.958036 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a79a6f618f4944b1a11b9c1df766ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.968339 22825 tablet_bootstrap.cc:654] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:52.987100 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a79a6f618f4944b1a11b9c1df766ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:52.990464 22824 tablet_bootstrap.cc:654] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.021023 22826 tablet_bootstrap.cc:654] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.022420 22696 tablet_service.cc:1511] Processing CreateTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:53.035043 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f4adfa257c34bb88d79bda9d39c5146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.043998 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:53.045673 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f4adfa257c34bb88d79bda9d39c5146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.028416 22619 tablet_service.cc:1511] Processing CreateTablet for tablet 7235e142d8ec45adb46a8c389a2962fa (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
W20260709 13:08:53.068347 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:53.048462 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:08:53.073954 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f4adfa257c34bb88d79bda9d39c5146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.074824 22825 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:53.079092 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235e142d8ec45adb46a8c389a2962fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.082365 22824 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:53.082296 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 7235e142d8ec45adb46a8c389a2962fa (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:08:53.083048 22824 ts_tablet_manager.cc:1403] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.009s
I20260709 13:08:53.083858 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235e142d8ec45adb46a8c389a2962fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.079514 22825 ts_tablet_manager.cc:1403] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.192s	user 0.021s	sys 0.010s
I20260709 13:08:53.090711 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 7235e142d8ec45adb46a8c389a2962fa (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:08:53.094079 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235e142d8ec45adb46a8c389a2962fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.094332 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 887cfff73fab4d52b2e39f5b6089133a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:53.096235 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887cfff73fab4d52b2e39f5b6089133a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.106757 22826 tablet_bootstrap.cc:492] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:53.107436 22826 ts_tablet_manager.cc:1403] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.209s	user 0.020s	sys 0.005s
I20260709 13:08:53.110085 22825 raft_consensus.cc:359] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:53.111145 22601 consensus_peers.cc:597] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:53.111481 22825 raft_consensus.cc:385] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.111826 22825 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.112749 22825 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.114291 22635 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:53.115279 22825 ts_tablet_manager.cc:1434] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.001s
I20260709 13:08:53.116349 22825 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:53.117007 22847 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.118814 22847 consensus_queue.cc:1243] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 2.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260709 13:08:53.122154 22832 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:53.122825 22832 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:53.124210 22832 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.125726 22832 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:53.143823 22826 raft_consensus.cc:359] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.144675 22826 raft_consensus.cc:385] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.144999 22826 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.145813 22826 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.086486 22824 raft_consensus.cc:359] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.150658 22824 raft_consensus.cc:385] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.151005 22824 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.150149 22697 tablet_service.cc:1511] Processing CreateTablet for tablet 887cfff73fab4d52b2e39f5b6089133a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:53.157716 22697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887cfff73fab4d52b2e39f5b6089133a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.151870 22824 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.156472 22826 ts_tablet_manager.cc:1434] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 13:08:53.162871 22845 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.163442 22845 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.164728 22826 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:53.166651 22845 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:53.168335 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:53.169453 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:53.170545 22600 leader_election.cc:343] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:53.171515 22601 leader_election.cc:343] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.172302 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:53.169365 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 227f18995af4452ab6e5db39ef780c8a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:53.176486 22828 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:53.181859 22696 tablet_service.cc:1511] Processing CreateTablet for tablet 227f18995af4452ab6e5db39ef780c8a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:53.185043 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 227f18995af4452ab6e5db39ef780c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.191097 22824 ts_tablet_manager.cc:1434] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.107s	user 0.004s	sys 0.004s
I20260709 13:08:53.197340 22824 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:53.199417 22770 tablet_service.cc:1511] Processing CreateTablet for tablet 227f18995af4452ab6e5db39ef780c8a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:08:53.199412 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 227f18995af4452ab6e5db39ef780c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.188985 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 887cfff73fab4d52b2e39f5b6089133a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:08:53.207996 22532 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:53.209599 22832 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.211921 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:53.213814 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 227f18995af4452ab6e5db39ef780c8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.220286 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.222836 22832 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.215389 22826 tablet_bootstrap.cc:654] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.226452 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:53.226912 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:53.227577 22635 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:53.227867 22709 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:53.229008 22824 tablet_bootstrap.cc:654] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.229178 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:53.230363 22866 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:53.230809 22866 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.231204 22866 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.235096 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887cfff73fab4d52b2e39f5b6089133a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.209316 22619 tablet_service.cc:1511] Processing CreateTablet for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:53.241791 22866 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.242796 22825 tablet_bootstrap.cc:654] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.248559 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:53.249387 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.249670 22635 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.242960 22832 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.261456 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:08:53.265799 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.267868 22618 tablet_service.cc:1511] Processing CreateTablet for tablet ca809087a6724923ab06ace347425d31 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:53.268954 22700 tablet_service.cc:1511] Processing CreateTablet for tablet ca809087a6724923ab06ace347425d31 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:53.270529 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca809087a6724923ab06ace347425d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.269452 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca809087a6724923ab06ace347425d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.274734 22824 tablet_bootstrap.cc:492] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:53.275317 22824 ts_tablet_manager.cc:1403] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.005s
I20260709 13:08:53.281842 22824 raft_consensus.cc:359] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.286772 22824 raft_consensus.cc:385] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.287179 22824 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.287942 22824 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.290088 22824 ts_tablet_manager.cc:1434] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:08:53.283031 22832 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.292963 22832 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:53.294824 22866 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.296178 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:53.297101 22709 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.301501 22824 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:53.296375 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:53.304440 22634 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:53.303750 22828 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.305720 22828 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.306861 22826 tablet_bootstrap.cc:492] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:53.307443 22826 ts_tablet_manager.cc:1403] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.143s	user 0.025s	sys 0.000s
I20260709 13:08:53.307231 22533 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:53.306581 22749 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:53.312129 22828 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:53.313156 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:53.319175 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:53.313930 22710 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
W20260709 13:08:53.323062 22600 leader_election.cc:343] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.323747 22832 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.324389 22601 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:53.316234 22866 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:53.326709 22624 tablet_service.cc:1511] Processing CreateTablet for tablet ed3a2e4658ea473ba5c2b68313aeaf3b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:08:53.328295 22866 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.328723 22866 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.329124 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3a2e4658ea473ba5c2b68313aeaf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.332661 22845 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:53.333227 22845 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.333602 22845 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.333925 22773 tablet_service.cc:1511] Processing CreateTablet for tablet ed3a2e4658ea473ba5c2b68313aeaf3b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:08:53.335458 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3a2e4658ea473ba5c2b68313aeaf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.334239 22635 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:53.342324 22709 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:53.339599 22828 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.341595 22621 tablet_service.cc:1511] Processing CreateTablet for tablet 26e6130713d74c0ea9492f5151053c42 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:53.359701 22621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e6130713d74c0ea9492f5151053c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.361236 22845 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.351593 22825 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:53.353535 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:53.310401 22826 raft_consensus.cc:359] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.385535 22825 ts_tablet_manager.cc:1403] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.269s	user 0.028s	sys 0.029s
I20260709 13:08:53.385846 22826 raft_consensus.cc:385] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.386209 22826 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.387189 22826 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.388280 22832 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:53.388880 22832 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:53.354517 22770 tablet_service.cc:1511] Processing CreateTablet for tablet ca809087a6724923ab06ace347425d31 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:08:53.389909 22845 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:53.349093 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:53.366071 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 22e83136ddd24d71ad143a6f5fc28dd9 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:08:53.390836 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca809087a6724923ab06ace347425d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.389830 22832 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.393049 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e83136ddd24d71ad143a6f5fc28dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.369041 22866 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:53.370672 22693 tablet_service.cc:1511] Processing CreateTablet for tablet 22e83136ddd24d71ad143a6f5fc28dd9 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:08:53.371774 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 26e6130713d74c0ea9492f5151053c42 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:53.389839 22826 ts_tablet_manager.cc:1434] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 13:08:53.397541 22784 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:08:53.399565 22824 tablet_bootstrap.cc:654] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.398597 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:53.375913 22697 tablet_service.cc:1511] Processing CreateTablet for tablet 26e6130713d74c0ea9492f5151053c42 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:08:53.348260 22696 tablet_service.cc:1511] Processing CreateTablet for tablet ed3a2e4658ea473ba5c2b68313aeaf3b (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 13:08:53.398917 22750 leader_election.cc:343] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.408860 22784 pending_rounds.cc:85] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 0
I20260709 13:08:53.423897 22784 pending_rounds.cc:107] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:08:53.424309 22784 raft_consensus.cc:2889] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:53.430231 22769 tablet_service.cc:1511] Processing CreateTablet for tablet 22e83136ddd24d71ad143a6f5fc28dd9 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:08:53.431919 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 9a4549b98fd04604afe4d21f9bb0acfe (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:53.434026 22694 tablet_service.cc:1511] Processing CreateTablet for tablet 594d7db14b30467aac30a69dc1570d43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:08:53.438544 22768 tablet_service.cc:1511] Processing CreateTablet for tablet 9a4549b98fd04604afe4d21f9bb0acfe (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:53.442345 22826 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:53.422915 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3a2e4658ea473ba5c2b68313aeaf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.441910 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 594d7db14b30467aac30a69dc1570d43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
W20260709 13:08:53.443222 22600 leader_election.cc:343] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.443828 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594d7db14b30467aac30a69dc1570d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.444237 22871 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.446146 22693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e83136ddd24d71ad143a6f5fc28dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.447052 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4549b98fd04604afe4d21f9bb0acfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.448694 22697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e6130713d74c0ea9492f5151053c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.423926 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e6130713d74c0ea9492f5151053c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.452533 22825 raft_consensus.cc:359] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.453477 22825 raft_consensus.cc:385] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.453799 22825 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.455070 22694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594d7db14b30467aac30a69dc1570d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.454850 22825 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.422654 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:53.456077 22710 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.461550 22769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e83136ddd24d71ad143a6f5fc28dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.463281 22825 ts_tablet_manager.cc:1434] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.077s	user 0.009s	sys 0.000s
I20260709 13:08:53.463939 22871 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.464479 22871 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.475133 22616 tablet_service.cc:1511] Processing CreateTablet for tablet 9a4549b98fd04604afe4d21f9bb0acfe (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:08:53.481635 22866 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.484297 22709 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:53.485173 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:53.477284 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4549b98fd04604afe4d21f9bb0acfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.488099 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:53.488795 22635 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.491127 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:53.492869 22750 leader_election.cc:343] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.477001 22826 tablet_bootstrap.cc:654] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.496783 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 594d7db14b30467aac30a69dc1570d43 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:08:53.507967 22634 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.510069 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5cf575974c43e79448d00fab49d7ab (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:08:53.511487 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5cf575974c43e79448d00fab49d7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.512121 22871 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:53.518093 22871 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.524168 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a4549b98fd04604afe4d21f9bb0acfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.518275 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:53.532696 22825 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:53.533030 22781 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:08:53.535578 22866 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.546468 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:53.542634 22834 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:08:53.549314 22866 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.550392 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 594d7db14b30467aac30a69dc1570d43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.610556 22533 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:53.615900 22866 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.615692 22692 tablet_service.cc:1511] Processing CreateTablet for tablet 51e41e0d49c846d0ae7c7d737a15be9e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:53.617280 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e41e0d49c846d0ae7c7d737a15be9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.618054 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:53.619537 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5cf575974c43e79448d00fab49d7ab (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:08:53.620858 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5cf575974c43e79448d00fab49d7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.622510 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:53.639405 22824 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:53.639647 22633 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:53.639961 22824 ts_tablet_manager.cc:1403] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.339s	user 0.032s	sys 0.009s
I20260709 13:08:53.640789 22709 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.641992 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:08:53.643013 22750 leader_election.cc:343] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.653126 22635 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:53.654495 22866 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:53.661229 22866 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.661613 22866 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.658058 22624 tablet_service.cc:1511] Processing CreateTablet for tablet 51e41e0d49c846d0ae7c7d737a15be9e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:53.660732 22825 tablet_bootstrap.cc:654] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.618409 22710 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:53.675014 22695 tablet_service.cc:1511] Processing CreateTablet for tablet 56c28a1350eb4e148e6e49cab764b300 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:53.679982 22824 raft_consensus.cc:359] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.680975 22824 raft_consensus.cc:385] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.681294 22824 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.678151 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 4a5cf575974c43e79448d00fab49d7ab (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:08:53.681947 22824 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.683506 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a5cf575974c43e79448d00fab49d7ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.686590 22824 ts_tablet_manager.cc:1434] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.046s	user 0.010s	sys 0.004s
I20260709 13:08:53.688660 22826 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:53.689236 22826 ts_tablet_manager.cc:1403] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.247s	user 0.021s	sys 0.000s
I20260709 13:08:53.695590 22601 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:53.697086 22828 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:53.697640 22828 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:53.698467 22699 tablet_service.cc:1511] Processing CreateTablet for tablet 14efc068101d4b50a525206c447e1ae8 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:08:53.698580 22828 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.714385 22824 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:53.715152 22695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c28a1350eb4e148e6e49cab764b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.724521 22769 tablet_service.cc:1511] Processing CreateTablet for tablet 51e41e0d49c846d0ae7c7d737a15be9e (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:08:53.726135 22769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e41e0d49c846d0ae7c7d737a15be9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.727135 22749 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:53.727562 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 56c28a1350eb4e148e6e49cab764b300 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:53.728292 22845 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:53.728333 22832 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:53.729611 22832 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:53.732538 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51e41e0d49c846d0ae7c7d737a15be9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.737792 22621 tablet_service.cc:1511] Processing CreateTablet for tablet 14efc068101d4b50a525206c447e1ae8 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:08:53.746758 22621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14efc068101d4b50a525206c447e1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.738971 22832 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:53.745012 22773 tablet_service.cc:1511] Processing CreateTablet for tablet 14efc068101d4b50a525206c447e1ae8 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:08:53.749244 22826 raft_consensus.cc:359] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:53.749863 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14efc068101d4b50a525206c447e1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.750108 22826 raft_consensus.cc:385] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.745195 22616 tablet_service.cc:1511] Processing CreateTablet for tablet fbb33c4b77a2451499c9c5e49b83aa12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:53.738591 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14efc068101d4b50a525206c447e1ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.752449 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c28a1350eb4e148e6e49cab764b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.759981 22700 tablet_service.cc:1511] Processing CreateTablet for tablet fbb33c4b77a2451499c9c5e49b83aa12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:53.761801 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb33c4b77a2451499c9c5e49b83aa12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.767444 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb33c4b77a2451499c9c5e49b83aa12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.783596 22767 tablet_service.cc:1511] Processing CreateTablet for tablet 56c28a1350eb4e148e6e49cab764b300 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:08:53.750451 22826 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.785501 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56c28a1350eb4e148e6e49cab764b300. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.799166 22826 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:53.800644 22825 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:53.792511 22866 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.801239 22825 ts_tablet_manager.cc:1403] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.269s	user 0.009s	sys 0.023s
I20260709 13:08:53.803732 22834 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.808655 22825 raft_consensus.cc:359] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.812497 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:53.819684 22866 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:53.818084 22825 raft_consensus.cc:385] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:53.831146 22825 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:53.832331 22825 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:53.843729 22700 tablet_service.cc:1511] Processing CreateTablet for tablet d92b65b2cbd546e18ddac5ce049ea91c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:53.845592 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92b65b2cbd546e18ddac5ce049ea91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.835327 22634 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:53.886766 22826 ts_tablet_manager.cc:1434] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.197s	user 0.021s	sys 0.018s
I20260709 13:08:53.887975 22826 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:53.888546 22824 tablet_bootstrap.cc:654] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.904420 22825 ts_tablet_manager.cc:1434] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.003s
I20260709 13:08:53.906127 22825 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:53.910908 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:53.913297 22866 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:53.918403 22773 tablet_service.cc:1511] Processing CreateTablet for tablet fbb33c4b77a2451499c9c5e49b83aa12 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:08:53.920306 22618 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebec7fc99cb448ea382f8b3e769e7ca (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:53.922252 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebec7fc99cb448ea382f8b3e769e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.920600 22767 tablet_service.cc:1511] Processing CreateTablet for tablet d92b65b2cbd546e18ddac5ce049ea91c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:53.931066 22699 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebec7fc99cb448ea382f8b3e769e7ca (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:53.932909 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebec7fc99cb448ea382f8b3e769e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.934435 22533 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:53.942184 22700 tablet_service.cc:1511] Processing CreateTablet for tablet f09c3bbe948b4c7a9f50df8ec4277e9f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:53.943828 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09c3bbe948b4c7a9f50df8ec4277e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.952910 22696 tablet_service.cc:1511] Processing CreateTablet for tablet f33fee9d64384b499e55f8bc58339f69 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:53.955008 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f33fee9d64384b499e55f8bc58339f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.956408 22616 tablet_service.cc:1511] Processing CreateTablet for tablet d92b65b2cbd546e18ddac5ce049ea91c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:08:53.958010 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb33c4b77a2451499c9c5e49b83aa12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:53.960382 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52854.3us
I20260709 13:08:53.967886 22616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92b65b2cbd546e18ddac5ce049ea91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.975929 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92b65b2cbd546e18ddac5ce049ea91c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:08:53.981307 22750 leader_election.cc:343] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:53.985631 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 0ebec7fc99cb448ea382f8b3e769e7ca (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:08:53.987229 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebec7fc99cb448ea382f8b3e769e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.992506 22624 tablet_service.cc:1511] Processing CreateTablet for tablet f09c3bbe948b4c7a9f50df8ec4277e9f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:53.994091 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09c3bbe948b4c7a9f50df8ec4277e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:53.993921 22634 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:53.998901 22825 tablet_bootstrap.cc:654] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:53.996889 22619 tablet_service.cc:1511] Processing CreateTablet for tablet f33fee9d64384b499e55f8bc58339f69 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:53.995735 22530 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.038515 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f33fee9d64384b499e55f8bc58339f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.049275 22826 tablet_bootstrap.cc:654] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.031019 22832 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:08:54.025640 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:08:54.060870 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 49604.1us
I20260709 13:08:54.064545 22767 tablet_service.cc:1511] Processing CreateTablet for tablet f09c3bbe948b4c7a9f50df8ec4277e9f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:08:54.065915 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f09c3bbe948b4c7a9f50df8ec4277e9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.069563 22832 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.070225 22832 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:54.071213 22832 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.074126 22618 tablet_service.cc:1511] Processing CreateTablet for tablet ef7d2e730e694ade8b424e4ff007a48a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:54.075654 22618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7d2e730e694ade8b424e4ff007a48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.075831 22768 tablet_service.cc:1511] Processing CreateTablet for tablet f33fee9d64384b499e55f8bc58339f69 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:08:54.077380 22768 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f33fee9d64384b499e55f8bc58339f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.114082 22882 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.114761 22882 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:08:54.117039 22882 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.119694 22692 tablet_service.cc:1511] Processing CreateTablet for tablet ef7d2e730e694ade8b424e4ff007a48a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:54.120182 22824 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.120883 22824 ts_tablet_manager.cc:1403] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.407s	user 0.035s	sys 0.073s
I20260709 13:08:54.121099 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.121338 22692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7d2e730e694ade8b424e4ff007a48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.122095 22634 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:54.123478 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.124212 22824 raft_consensus.cc:359] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.124961 22824 raft_consensus.cc:385] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.125267 22824 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.126606 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.127559 22882 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.126009 22824 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.131067 22824 ts_tablet_manager.cc:1434] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:08:54.132193 22824 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.134817 22770 tablet_service.cc:1511] Processing CreateTablet for tablet ef7d2e730e694ade8b424e4ff007a48a (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:08:54.136270 22699 tablet_service.cc:1511] Processing CreateTablet for tablet 79ed72c313bf419bb01d45a59391a262 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:54.138129 22699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ed72c313bf419bb01d45a59391a262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.146411 22826 tablet_bootstrap.cc:492] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.149470 22770 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef7d2e730e694ade8b424e4ff007a48a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.155812 22773 tablet_service.cc:1511] Processing CreateTablet for tablet 79ed72c313bf419bb01d45a59391a262 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:54.157176 22773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ed72c313bf419bb01d45a59391a262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.171010 22825 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.171653 22825 ts_tablet_manager.cc:1403] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.266s	user 0.023s	sys 0.010s
I20260709 13:08:54.171443 22695 tablet_service.cc:1511] Processing CreateTablet for tablet 308af34f5c0144e185e0af82342c7f92 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:54.173046 22695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308af34f5c0144e185e0af82342c7f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.174841 22825 raft_consensus.cc:359] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.175644 22825 raft_consensus.cc:385] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.175899 22825 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.177822 22825 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.181768 22624 tablet_service.cc:1511] Processing CreateTablet for tablet 79ed72c313bf419bb01d45a59391a262 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:08:54.183331 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ed72c313bf419bb01d45a59391a262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.183127 22619 tablet_service.cc:1511] Processing CreateTablet for tablet 308af34f5c0144e185e0af82342c7f92 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:54.184340 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308af34f5c0144e185e0af82342c7f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.184965 22882 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.149478 22826 ts_tablet_manager.cc:1403] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.262s	user 0.011s	sys 0.017s
I20260709 13:08:54.199256 22825 ts_tablet_manager.cc:1434] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:08:54.200397 22825 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.204952 22828 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.205386 22866 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.205547 22828 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.205899 22866 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.207962 22866 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.215026 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.215281 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.215983 22709 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:08:54.216846 22601 leader_election.cc:343] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.216985 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.218070 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.218483 22828 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:08:54.219192 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:08:54.219529 22600 leader_election.cc:343] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.219628 22696 tablet_service.cc:1511] Processing CreateTablet for tablet a9a28904b6a34dbf81e5d1d4bb08a08c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:54.220115 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.221141 22696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a28904b6a34dbf81e5d1d4bb08a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.222162 22828 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.222318 22769 tablet_service.cc:1511] Processing CreateTablet for tablet 308af34f5c0144e185e0af82342c7f92 (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:08:54.223757 22769 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308af34f5c0144e185e0af82342c7f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.230830 22530 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:54.234297 22750 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:54.239146 22832 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.239738 22700 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3855678a8148a09ffe99d7624c9b4f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:54.241516 22824 tablet_bootstrap.cc:654] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.243611 22767 tablet_service.cc:1511] Processing CreateTablet for tablet a9a28904b6a34dbf81e5d1d4bb08a08c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:54.244973 22767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a28904b6a34dbf81e5d1d4bb08a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.248065 22624 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3855678a8148a09ffe99d7624c9b4f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:54.249624 22624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3855678a8148a09ffe99d7624c9b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.254393 22700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3855678a8148a09ffe99d7624c9b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.268110 22825 tablet_bootstrap.cc:654] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.272330 22771 tablet_service.cc:1511] Processing CreateTablet for tablet 9e3855678a8148a09ffe99d7624c9b4f (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:08:54.274178 22771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3855678a8148a09ffe99d7624c9b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.278152 22832 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.280153 22871 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.293025 22619 tablet_service.cc:1511] Processing CreateTablet for tablet a9a28904b6a34dbf81e5d1d4bb08a08c (DEFAULT_TABLE table=test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:08:54.296949 22619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9a28904b6a34dbf81e5d1d4bb08a08c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:54.298025 22826 raft_consensus.cc:359] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.298949 22826 raft_consensus.cc:385] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.299265 22826 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.305171 22871 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.317031 22826 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.320952 22826 ts_tablet_manager.cc:1434] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.122s	user 0.007s	sys 0.000s
I20260709 13:08:54.335631 22826 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.336935 22871 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.337486 22871 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.349723 22824 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.350363 22824 ts_tablet_manager.cc:1403] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.218s	user 0.011s	sys 0.005s
I20260709 13:08:54.350652 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.354730 22871 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.356014 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:54.356473 22601 leader_election.cc:343] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:54.357731 22600 leader_election.cc:343] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.358175 22600 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.358496 22825 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.359031 22825 ts_tablet_manager.cc:1403] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.022s
I20260709 13:08:54.356005 22832 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:54.361974 22825 raft_consensus.cc:359] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.362713 22825 raft_consensus.cc:385] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.363004 22825 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.363658 22825 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.364670 22871 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.353780 22824 raft_consensus.cc:359] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.366880 22824 raft_consensus.cc:385] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.367193 22824 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.374533 22824 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.378810 22826 tablet_bootstrap.cc:654] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.380712 22825 ts_tablet_manager.cc:1434] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:08:54.387854 22825 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.390126 22866 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.396159 22824 ts_tablet_manager.cc:1434] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.000s
I20260709 13:08:54.397396 22824 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.407801 22824 tablet_bootstrap.cc:654] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.429625 22825 tablet_bootstrap.cc:654] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.430907 22824 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.431493 22824 ts_tablet_manager.cc:1403] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.005s
I20260709 13:08:54.435397 22824 raft_consensus.cc:359] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.441807 22824 raft_consensus.cc:385] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.442167 22824 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.442934 22824 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.445071 22824 ts_tablet_manager.cc:1434] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:54.446069 22824 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.453337 22826 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.453936 22826 ts_tablet_manager.cc:1403] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.131s	user 0.008s	sys 0.004s
I20260709 13:08:54.455034 22824 tablet_bootstrap.cc:654] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.457093 22826 raft_consensus.cc:359] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.457928 22826 raft_consensus.cc:385] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.458294 22826 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.459125 22825 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.459678 22825 ts_tablet_manager.cc:1403] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.003s
I20260709 13:08:54.459467 22826 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.462795 22825 raft_consensus.cc:359] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.463546 22825 raft_consensus.cc:385] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.463878 22825 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.464732 22825 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.468351 22826 ts_tablet_manager.cc:1434] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 13:08:54.468809 22866 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.469105 22825 ts_tablet_manager.cc:1434] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:08:54.469575 22826 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.469368 22866 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.470144 22825 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.471725 22866 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.473719 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.474929 22635 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:54.476608 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.477346 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:54.478814 22750 leader_election.cc:343] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.479905 22866 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.480290 22866 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.480625 22866 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.486742 22826 tablet_bootstrap.cc:654] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.487665 22825 tablet_bootstrap.cc:654] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.491627 22824 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.492165 22824 ts_tablet_manager.cc:1403] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.007s
I20260709 13:08:54.493034 22826 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.493610 22826 ts_tablet_manager.cc:1403] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.001s
I20260709 13:08:54.495186 22824 raft_consensus.cc:359] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.495985 22824 raft_consensus.cc:385] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.496260 22824 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.496729 22826 raft_consensus.cc:359] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.497023 22824 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.497602 22826 raft_consensus.cc:385] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.497984 22826 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.498746 22826 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.500905 22826 ts_tablet_manager.cc:1434] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:54.502804 22824 ts_tablet_manager.cc:1434] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:08:54.502215 22866 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.503882 22824 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.506024 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:54.506451 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:54.506719 22635 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.507088 22707 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.510003 22826 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.510090 22832 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.510797 22832 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.513578 22866 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.514825 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.515588 22709 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:54.516705 22707 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.517923 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.518522 22826 tablet_bootstrap.cc:654] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.519537 22750 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.520577 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.521453 22847 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.522143 22847 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.522639 22847 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.523275 22866 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.521522 22633 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:54.521629 22832 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.533818 22901 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.534469 22901 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.521450 22825 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.536286 22825 ts_tablet_manager.cc:1403] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.012s
I20260709 13:08:54.538354 22635 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.540616 22825 raft_consensus.cc:359] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.541728 22825 raft_consensus.cc:385] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.542099 22825 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.542666 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:54.544404 22600 leader_election.cc:343] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.544059 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.544191 22901 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:08:54.546825 22601 leader_election.cc:343] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.547572 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.548826 22901 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.547580 22866 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:54.550568 22866 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.551654 22847 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.552178 22911 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:54.555200 22824 tablet_bootstrap.cc:654] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.554773 22825 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.557915 22825 ts_tablet_manager.cc:1434] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:08:54.560714 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:54.561671 22633 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.575434 22847 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.577757 22825 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.577397 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:54.578368 22707 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.579243 22826 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.579806 22826 ts_tablet_manager.cc:1403] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.070s	user 0.035s	sys 0.030s
I20260709 13:08:54.582871 22633 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.583626 22826 raft_consensus.cc:359] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.584415 22826 raft_consensus.cc:385] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.584429 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.584827 22826 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.585558 22847 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.585649 22826 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.586141 22847 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:54.588021 22826 ts_tablet_manager.cc:1434] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:54.588866 22847 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.591089 22882 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.591159 22826 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.591713 22882 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.593848 22882 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.594794 22825 tablet_bootstrap.cc:654] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.596099 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.596853 22784 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:54.597057 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.597743 22633 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:54.599213 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:54.601332 22882 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.600777 22530 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:54.601994 22882 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.603397 22882 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.614197 22707 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.632972 22826 tablet_bootstrap.cc:654] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.626067 22882 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.636507 22886 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.637194 22886 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.639146 22824 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.639729 22824 ts_tablet_manager.cc:1403] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.136s	user 0.028s	sys 0.020s
I20260709 13:08:54.639956 22886 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.642565 22707 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.643054 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.648506 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.648809 22882 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.655643 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.661721 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:54.662544 22784 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.666044 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:54.667161 22633 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.669472 22901 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.670670 22901 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.671693 22824 raft_consensus.cc:359] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.672432 22824 raft_consensus.cc:385] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.674084 22824 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.674945 22824 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.675261 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.675448 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.676751 22824 ts_tablet_manager.cc:1434] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.002s
W20260709 13:08:54.677201 22600 leader_election.cc:343] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.674257 22901 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.681553 22784 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
W20260709 13:08:54.681479 22601 leader_election.cc:343] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.684332 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:54.685452 22901 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.686954 22530 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:54.687875 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.687989 22633 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:54.688628 22882 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:54.689821 22824 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.700793 22826 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.701393 22826 ts_tablet_manager.cc:1403] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.110s	user 0.008s	sys 0.013s
I20260709 13:08:54.693626 22825 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.707130 22847 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:54.711104 22825 ts_tablet_manager.cc:1403] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.134s	user 0.024s	sys 0.020s
I20260709 13:08:54.710454 22824 tablet_bootstrap.cc:654] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.689841 22886 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.712455 22886 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:54.713388 22886 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.714249 22825 raft_consensus.cc:359] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.715070 22825 raft_consensus.cc:385] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.715379 22825 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.716070 22825 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.718366 22825 ts_tablet_manager.cc:1434] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:54.719434 22825 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.721768 22826 raft_consensus.cc:359] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.723250 22826 raft_consensus.cc:385] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.725379 22826 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.726567 22826 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.729347 22825 tablet_bootstrap.cc:654] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.732411 22826 ts_tablet_manager.cc:1434] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:08:54.733893 22826 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.735224 22824 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.735993 22824 ts_tablet_manager.cc:1403] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.046s	user 0.010s	sys 0.006s
I20260709 13:08:54.740594 22824 raft_consensus.cc:359] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.748549 22826 tablet_bootstrap.cc:654] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.741736 22824 raft_consensus.cc:385] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.753293 22824 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.754144 22824 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.754423 22530 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.757370 22825 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.757966 22825 ts_tablet_manager.cc:1403] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.039s	user 0.017s	sys 0.017s
I20260709 13:08:54.761200 22825 raft_consensus.cc:359] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.761999 22825 raft_consensus.cc:385] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.762300 22825 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.763274 22825 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.766036 22825 ts_tablet_manager.cc:1434] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:08:54.766949 22824 ts_tablet_manager.cc:1434] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:08:54.767700 22825 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.768009 22824 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.776093 22824 tablet_bootstrap.cc:654] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.781095 22826 tablet_bootstrap.cc:492] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.781719 22826 ts_tablet_manager.cc:1403] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.004s
I20260709 13:08:54.782087 22825 tablet_bootstrap.cc:654] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.785039 22826 raft_consensus.cc:359] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.785858 22826 raft_consensus.cc:385] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.786190 22826 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.786963 22826 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.789469 22826 ts_tablet_manager.cc:1434] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:54.790544 22826 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.797679 22826 tablet_bootstrap.cc:654] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.807638 22832 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.808315 22832 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.810201 22826 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.810786 22826 ts_tablet_manager.cc:1403] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 13:08:54.810964 22832 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.813688 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.813915 22826 raft_consensus.cc:359] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.814662 22826 raft_consensus.cc:385] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.814957 22826 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
W20260709 13:08:54.816145 22749 leader_election.cc:343] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:54.816557 22826 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.817160 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.817824 22707 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:54.819262 22825 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.819152 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:08:54.819293 22824 tablet_bootstrap.cc:492] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.819898 22825 ts_tablet_manager.cc:1403] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.052s	user 0.032s	sys 0.011s
I20260709 13:08:54.820091 22824 ts_tablet_manager.cc:1403] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.052s	user 0.022s	sys 0.015s
I20260709 13:08:54.820799 22832 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.821197 22832 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.821338 22826 ts_tablet_manager.cc:1434] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:08:54.821580 22832 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.822463 22826 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.823215 22825 raft_consensus.cc:359] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.823542 22824 raft_consensus.cc:359] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.823993 22825 raft_consensus.cc:385] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.824404 22824 raft_consensus.cc:385] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.824443 22825 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.824903 22824 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.825397 22825 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.827711 22825 ts_tablet_manager.cc:1434] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:54.829320 22825 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.825706 22824 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.830602 22826 tablet_bootstrap.cc:654] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.835144 22824 ts_tablet_manager.cc:1434] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:08:54.835783 22901 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.836589 22825 tablet_bootstrap.cc:654] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.836362 22901 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.838567 22824 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.839152 22901 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.840471 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.841156 22707 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:54.843771 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:54.844736 22901 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.845145 22901 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.845492 22901 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.844607 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.848970 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:54.853976 22832 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.861135 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:54.861881 22633 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.872325 22633 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.873808 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:54.874894 22847 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.875419 22847 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:54.876356 22847 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.884814 22832 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:54.882097 22901 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.887766 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:54.888562 22707 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.890223 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:54.890957 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:54.891637 22784 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.895562 22825 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.896337 22825 ts_tablet_manager.cc:1403] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.067s	user 0.023s	sys 0.018s
I20260709 13:08:54.891116 22709 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.897780 22531 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:54.899888 22825 raft_consensus.cc:359] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.900719 22825 raft_consensus.cc:385] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.901026 22824 tablet_bootstrap.cc:654] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.901083 22825 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.902184 22825 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.904320 22826 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.907034 22826 ts_tablet_manager.cc:1403] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.085s	user 0.028s	sys 0.022s
I20260709 13:08:54.904502 22825 ts_tablet_manager.cc:1434] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:54.909094 22825 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.904896 22707 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:54.914127 22826 raft_consensus.cc:359] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.915596 22826 raft_consensus.cc:385] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.915942 22826 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.916742 22826 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.919848 22826 ts_tablet_manager.cc:1434] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 13:08:54.920202 22901 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.919438 22709 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:54.921663 22826 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.922072 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:54.923055 22901 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:54.924420 22901 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:54.925511 22901 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.929287 22826 tablet_bootstrap.cc:654] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.917352 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:54.949455 22825 tablet_bootstrap.cc:654] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.950789 22826 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:54.951411 22826 ts_tablet_manager.cc:1403] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.030s	user 0.021s	sys 0.003s
I20260709 13:08:54.953989 22826 raft_consensus.cc:359] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.954581 22826 raft_consensus.cc:385] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.955044 22826 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.955854 22826 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.957854 22826 ts_tablet_manager.cc:1434] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:08:54.959712 22826 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:54.960546 22824 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:54.961115 22824 ts_tablet_manager.cc:1403] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.123s	user 0.017s	sys 0.012s
I20260709 13:08:54.964232 22824 raft_consensus.cc:359] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.965123 22824 raft_consensus.cc:385] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.965369 22825 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:54.965588 22824 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.965973 22825 ts_tablet_manager.cc:1403] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.057s	user 0.003s	sys 0.009s
I20260709 13:08:54.967074 22824 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.967697 22886 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.968597 22826 tablet_bootstrap.cc:654] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.968223 22886 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.969168 22825 raft_consensus.cc:359] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.970134 22824 ts_tablet_manager.cc:1434] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:54.970358 22825 raft_consensus.cc:385] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:54.970804 22825 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:54.971400 22886 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.971550 22824 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:54.971802 22825 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.974090 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:54.975849 22633 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:54.977568 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:54.975066 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.978847 22784 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:54.982685 22886 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.983357 22886 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.983780 22886 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:54.985256 22824 tablet_bootstrap.cc:654] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:54.985574 22825 ts_tablet_manager.cc:1434] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 13:08:54.989527 22901 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.990135 22901 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:54.992733 22901 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:54.995333 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:54.996480 22709 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:54.997215 22825 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:54.998273 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:54.998764 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:54.999305 22901 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:54.999742 22901 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.000085 22901 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:55.000957 22600 leader_election.cc:343] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.004621 22825 tablet_bootstrap.cc:654] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.011950 22530 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.018460 22825 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:55.019040 22825 ts_tablet_manager.cc:1403] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:08:55.022131 22825 raft_consensus.cc:359] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.022969 22825 raft_consensus.cc:385] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.023264 22825 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.024017 22825 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.031066 22825 ts_tablet_manager.cc:1434] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:08:55.032179 22825 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:55.038693 22825 tablet_bootstrap.cc:654] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.039822 22886 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.042333 22882 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.043116 22882 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.047572 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:55.048846 22633 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:55.051064 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:55.052461 22891 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:55.052994 22891 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.053442 22891 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.059690 22882 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.062312 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:55.063498 22784 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:55.086030 22901 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.090601 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:55.091565 22709 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.099032 22832 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.099957 22832 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.104988 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:08:55.107638 22749 leader_election.cc:343] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.110821 22832 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.111115 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:55.112856 22750 leader_election.cc:343] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.113557 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:55.114673 22832 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:55.114490 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:55.115783 22633 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.115993 22886 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.118317 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:55.119302 22784 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.119382 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:08:55.121891 22600 leader_election.cc:343] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.122841 22901 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.122910 22847 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:55.111785 22709 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:55.127926 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:55.132400 22845 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.133200 22845 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:55.141551 22845 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.143705 22824 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:55.144570 22824 ts_tablet_manager.cc:1403] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.173s	user 0.051s	sys 0.060s
I20260709 13:08:55.109805 22891 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.155431 22824 raft_consensus.cc:359] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.156477 22824 raft_consensus.cc:385] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.150983 22784 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:55.156841 22824 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.157577 22824 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.159751 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.160559 22886 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.161044 22886 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:55.161479 22824 ts_tablet_manager.cc:1434] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.002s
I20260709 13:08:55.161913 22886 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.167747 22891 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.168382 22832 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:55.170151 22826 tablet_bootstrap.cc:492] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:55.170578 22826 ts_tablet_manager.cc:1403] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.211s	user 0.079s	sys 0.082s
I20260709 13:08:55.173498 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:55.174293 22635 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.175693 22845 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.175096 22530 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.176262 22845 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.179106 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:55.183967 22901 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.184935 22784 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.184731 22901 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.189105 22824 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:55.193379 22901 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.194794 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:55.195129 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.195598 22783 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:55.197127 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.200726 22901 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:55.204312 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.210369 22825 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:55.210969 22825 ts_tablet_manager.cc:1403] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.179s	user 0.035s	sys 0.077s
I20260709 13:08:55.202441 22832 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
W20260709 13:08:55.213385 22601 leader_election.cc:343] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.208443 22901 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.215903 22901 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.216456 22832 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:08:55.222805 22825 raft_consensus.cc:359] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.223953 22825 raft_consensus.cc:385] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.224232 22784 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:55.224318 22825 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.225416 22825 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.227598 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.229712 22633 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
W20260709 13:08:55.241946 22601 leader_election.cc:343] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.248582 22825 ts_tablet_manager.cc:1434] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 13:08:55.250232 22825 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:55.252101 22891 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.252733 22891 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:55.258672 22826 raft_consensus.cc:359] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.260419 22826 raft_consensus.cc:385] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.260949 22826 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.262064 22826 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.264698 22826 ts_tablet_manager.cc:1434] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.094s	user 0.008s	sys 0.000s
I20260709 13:08:55.253927 22891 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.235424 22845 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.266379 22826 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
W20260709 13:08:55.306813 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:55.352007 22910 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.360771 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:55.365502 22600 leader_election.cc:343] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.368147 22824 tablet_bootstrap.cc:654] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.367723 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:55.357508 22635 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:55.360177 22910 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.375161 22942 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:08:55.413192 22940 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:08:55.420403 22901 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.424641 22910 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.430181 22531 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.437572 22825 tablet_bootstrap.cc:654] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.434876 22891 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 13:08:55.491276 22942 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:55.481997 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66656.9us
W20260709 13:08:55.481634 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56910.6us
I20260709 13:08:55.500770 22824 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:55.501266 22824 ts_tablet_manager.cc:1403] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.313s	user 0.027s	sys 0.043s
I20260709 13:08:55.506454 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:55.506114 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.507292 22635 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:55.520761 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:55.521930 22951 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:55.522514 22824 raft_consensus.cc:359] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.523324 22824 raft_consensus.cc:385] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.523391 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.523691 22824 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.524483 22824 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.526212 22824 ts_tablet_manager.cc:1434] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 13:08:55.527979 22824 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:55.539120 22825 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:55.540010 22825 ts_tablet_manager.cc:1403] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.290s	user 0.016s	sys 0.027s
I20260709 13:08:55.536676 22953 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:55.534375 22707 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.543220 22901 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:55.554937 22953 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.556905 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:55.556825 22954 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:08:55.558485 22953 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.558857 22784 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.564801 22886 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:55.567680 22826 tablet_bootstrap.cc:654] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.578567 22825 raft_consensus.cc:359] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.584681 22825 raft_consensus.cc:385] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.585104 22825 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.587164 22825 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.584462 22531 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.589682 22825 ts_tablet_manager.cc:1434] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 13:08:55.599255 22825 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:55.608976 22953 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:55.615595 22750 leader_election.cc:343] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.617472 22847 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:55.621076 22954 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.621677 22954 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.624287 22954 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.634749 22941 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:08:55.640679 22826 tablet_bootstrap.cc:492] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:55.641238 22826 ts_tablet_manager.cc:1403] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.375s	user 0.032s	sys 0.018s
W20260709 13:08:55.695199 22858 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:08:55.693176 22901 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.697233 22901 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.705205 22847 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.709254 22847 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.727671 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:55.730381 22600 leader_election.cc:343] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.727794 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.740530 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:55.747017 22824 tablet_bootstrap.cc:654] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.698303 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:55.674919 22784 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:55.737991 22901 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:08:55.758031 22601 leader_election.cc:343] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.739256 22707 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:55.762749 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:55.772662 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:55.773515 22634 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:55.759378 22633 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.764045 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.777763 22941 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:55.779114 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.765080 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.787492 22941 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.788075 22941 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:55.789070 22784 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.788995 22941 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.776122 22847 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.796635 22949 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:55.801848 22949 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.802312 22949 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.796365 22825 tablet_bootstrap.cc:654] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.760955 22953 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.830962 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:55.837786 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:55.838088 22750 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:55.845044 22886 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:55.834957 22826 raft_consensus.cc:359] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.846908 22826 raft_consensus.cc:385] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.847338 22826 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.848227 22826 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:55.849161 22750 leader_election.cc:343] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.851166 22826 ts_tablet_manager.cc:1434] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.210s	user 0.046s	sys 0.057s
I20260709 13:08:55.853058 22826 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:55.848062 22532 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:55.849717 22910 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:55.860949 22966 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:08:55.859524 22886 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:55.862759 22967 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.059s	user 0.002s	sys 0.000s
I20260709 13:08:55.865585 22826 tablet_bootstrap.cc:654] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.865753 22886 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:55.868649 22750 leader_election.cc:343] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.870257 22949 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.870028 22886 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:08:55.873925 22633 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:55.875524 22949 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:55.876348 22824 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:55.876920 22824 ts_tablet_manager.cc:1403] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.349s	user 0.014s	sys 0.017s
I20260709 13:08:55.877465 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:55.886582 22824 raft_consensus.cc:359] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.887496 22824 raft_consensus.cc:385] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.887784 22824 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.889159 22824 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.892027 22824 ts_tablet_manager.cc:1434] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:08:55.893085 22824 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:55.888556 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:55.896859 22633 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:55.889686 22949 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.906044 22949 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:55.907356 22826 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:55.907096 22949 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.907924 22826 ts_tablet_manager.cc:1403] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.014s
I20260709 13:08:55.918354 22826 raft_consensus.cc:359] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.919978 22826 raft_consensus.cc:385] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.920378 22826 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.921849 22826 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.924595 22825 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:55.925344 22825 ts_tablet_manager.cc:1403] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.335s	user 0.021s	sys 0.004s
I20260709 13:08:55.935539 22824 tablet_bootstrap.cc:654] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:55.937981 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:55.938832 22826 ts_tablet_manager.cc:1434] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.031s	user 0.012s	sys 0.000s
I20260709 13:08:55.943675 22826 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:55.963755 22633 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:55.965622 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:55.967021 22847 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:55.967743 22847 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:55.969014 22847 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:55.977694 22750 leader_election.cc:343] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:55.978590 22825 raft_consensus.cc:359] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.984712 22825 raft_consensus.cc:385] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:55.985196 22825 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:55.987797 22825 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:55.990880 22825 ts_tablet_manager.cc:1434] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.065s	user 0.013s	sys 0.007s
I20260709 13:08:55.995118 22825 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:56.019326 22826 tablet_bootstrap.cc:654] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.022959 22530 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:56.072769 22948 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.073802 22948 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.087965 22948 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.094971 22825 tablet_bootstrap.cc:654] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.098827 22824 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.099495 22824 ts_tablet_manager.cc:1403] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.207s	user 0.015s	sys 0.012s
I20260709 13:08:56.102126 22826 tablet_bootstrap.cc:492] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.103937 22826 ts_tablet_manager.cc:1403] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.163s	user 0.013s	sys 0.006s
I20260709 13:08:56.102722 22824 raft_consensus.cc:359] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.111006 22824 raft_consensus.cc:385] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.117838 22824 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.120013 22824 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:56.112406 22975 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.074s	user 0.013s	sys 0.010s
I20260709 13:08:56.124606 22826 raft_consensus.cc:359] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.126180 22826 raft_consensus.cc:385] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.128755 22826 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.129647 22826 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.132467 22826 ts_tablet_manager.cc:1434] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:08:56.133807 22826 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:56.135277 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.128386 22824 ts_tablet_manager.cc:1434] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:08:56.136020 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.137323 22824 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.137900 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.138586 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.141157 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.142707 22948 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.143214 22948 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.143597 22948 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.146471 22530 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:56.153613 22825 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:56.154214 22825 ts_tablet_manager.cc:1403] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.159s	user 0.013s	sys 0.005s
I20260709 13:08:56.161201 22948 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.163893 22948 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.167351 22826 tablet_bootstrap.cc:654] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.167662 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.168800 22784 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.170423 22824 tablet_bootstrap.cc:654] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.174549 22825 raft_consensus.cc:359] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.175251 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:56.175549 22825 raft_consensus.cc:385] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.175891 22825 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.176851 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:56.176667 22825 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.177486 22634 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.179050 22825 ts_tablet_manager.cc:1434] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:08:56.179919 22825 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:56.186062 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:56.187180 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.188216 22948 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.188817 22948 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:56.190358 22948 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.192871 22824 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.192905 22826 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.193549 22824 ts_tablet_manager.cc:1403] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.000s
I20260709 13:08:56.193657 22826 ts_tablet_manager.cc:1403] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.004s
I20260709 13:08:56.199313 22826 raft_consensus.cc:359] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.199934 22826 raft_consensus.cc:385] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.200196 22826 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.200266 22825 tablet_bootstrap.cc:654] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.203346 22824 raft_consensus.cc:359] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.204161 22824 raft_consensus.cc:385] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.200881 22826 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.204437 22824 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.205197 22824 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.207103 22826 ts_tablet_manager.cc:1434] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:08:56.208588 22826 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:56.212071 22824 ts_tablet_manager.cc:1434] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.005s
I20260709 13:08:56.211524 22531 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.213651 22824 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.215483 22826 tablet_bootstrap.cc:654] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.219671 22948 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.220259 22948 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.222810 22824 tablet_bootstrap.cc:654] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.224035 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.224754 22634 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.225050 22948 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.225775 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.227639 22784 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.231670 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:56.235960 22825 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:56.236166 22941 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:56.236625 22948 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.237268 22948 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.237679 22948 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.236598 22825 ts_tablet_manager.cc:1403] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.057s	user 0.013s	sys 0.006s
I20260709 13:08:56.242807 22826 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.243403 22826 ts_tablet_manager.cc:1403] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.000s
I20260709 13:08:56.245575 22948 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.246441 22826 raft_consensus.cc:359] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.247141 22826 raft_consensus.cc:385] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.247442 22826 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.248081 22826 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.248922 22948 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.250593 22826 ts_tablet_manager.cc:1434] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:56.251099 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.251754 22826 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:56.251716 22783 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.252425 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:56.253016 22634 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.256727 22891 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.257275 22891 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.261320 22826 tablet_bootstrap.cc:654] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.262669 22825 raft_consensus.cc:359] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.264145 22825 raft_consensus.cc:385] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.264494 22825 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.265193 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.265537 22824 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.266258 22824 ts_tablet_manager.cc:1403] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.003s
I20260709 13:08:56.267531 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.268460 22633 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:56.269852 22783 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:56.271016 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:56.272071 22948 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.262395 22891 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.280146 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.280560 22845 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:56.280378 22824 raft_consensus.cc:359] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.281363 22891 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.282296 22891 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:56.283560 22891 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.265242 22825 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.288764 22824 raft_consensus.cc:385] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.289386 22824 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.291368 22948 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:56.291082 22824 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.298393 22825 ts_tablet_manager.cc:1434] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:08:56.300940 22825 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:56.307484 22634 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:56.331522 22824 ts_tablet_manager.cc:1434] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.065s	user 0.011s	sys 0.000s
I20260709 13:08:56.333694 22824 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.349191 22825 tablet_bootstrap.cc:654] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.364816 22968 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.365430 22968 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:56.366067 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:56.360702 22531 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:56.368600 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.369329 22634 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:56.378437 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.351562 22824 tablet_bootstrap.cc:654] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.381752 22968 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:56.379947 22949 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.382654 22826 tablet_bootstrap.cc:492] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.383282 22826 ts_tablet_manager.cc:1403] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.132s	user 0.042s	sys 0.046s
I20260709 13:08:56.382555 22949 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.383725 22949 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.384840 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:56.389142 22826 raft_consensus.cc:359] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.390072 22826 raft_consensus.cc:385] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.390369 22826 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.390810 22845 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.391372 22845 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.391126 22826 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.392984 22826 ts_tablet_manager.cc:1434] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:08:56.393906 22826 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:56.394416 22825 tablet_bootstrap.cc:492] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:56.395000 22825 ts_tablet_manager.cc:1403] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.024s
I20260709 13:08:56.396903 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:56.400247 22845 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:08:56.400527 22601 leader_election.cc:343] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.402165 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.402844 22708 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:56.403277 22783 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:56.406759 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:56.408325 22845 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.410761 22845 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.411161 22845 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.412149 22826 tablet_bootstrap.cc:654] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.412292 22824 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.412834 22824 ts_tablet_manager.cc:1403] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.079s	user 0.020s	sys 0.001s
I20260709 13:08:56.414533 22825 raft_consensus.cc:359] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.408110 22949 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.415411 22825 raft_consensus.cc:385] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.415818 22825 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.417448 22949 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:56.416584 22825 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.420145 22825 ts_tablet_manager.cc:1434] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:08:56.420557 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:56.426451 22633 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.427176 22825 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:56.422020 22847 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:56.420733 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:56.434528 22708 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.434253 22824 raft_consensus.cc:359] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.435145 22824 raft_consensus.cc:385] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.435457 22824 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.435595 22825 tablet_bootstrap.cc:654] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.436204 22824 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.438225 22824 ts_tablet_manager.cc:1434] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:08:56.439389 22824 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.446107 22824 tablet_bootstrap.cc:654] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.447309 22709 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.451232 22891 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.452008 22891 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.455943 22847 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.456511 22847 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.459414 22942 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.459957 22942 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.462257 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.462975 22949 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:56.463287 22635 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:56.464733 22942 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.468973 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.470472 22953 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.471046 22953 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.471487 22953 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.459110 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.466032 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:56.466348 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.475198 22632 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.475201 22706 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:56.477421 22633 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:56.478790 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:56.478519 22845 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.479900 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:56.480742 22948 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.481155 22948 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.481678 22948 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.481850 22847 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:56.482432 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:56.483234 22891 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.483470 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:56.480736 22942 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.484249 22942 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:56.484354 22601 leader_election.cc:343] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.484860 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.487527 22968 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:56.491058 22825 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:56.491845 22825 ts_tablet_manager.cc:1403] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.065s	user 0.016s	sys 0.024s
I20260709 13:08:56.490579 22942 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.496582 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.497002 22968 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.497633 22968 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:56.498648 22968 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.501083 22845 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.501924 22825 raft_consensus.cc:359] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.502723 22825 raft_consensus.cc:385] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.503014 22825 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.503250 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.503706 22825 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.506932 22781 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.496577 22953 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.510234 22845 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.541602 22825 ts_tablet_manager.cc:1434] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.001s
I20260709 13:08:56.545192 22826 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.550141 22826 ts_tablet_manager.cc:1403] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.156s	user 0.012s	sys 0.004s
I20260709 13:08:56.550688 22825 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:56.514448 22948 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.547258 22845 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.555684 22783 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:56.567191 22824 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.568106 22824 ts_tablet_manager.cc:1403] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.129s	user 0.015s	sys 0.003s
I20260709 13:08:56.572539 22709 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:08:56.597155 22953 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:56.579936 22942 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.640843 22845 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.641078 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:56.643714 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:56.644523 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:56.644961 22826 raft_consensus.cc:359] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.646049 22826 raft_consensus.cc:385] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.646425 22826 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.648335 22824 raft_consensus.cc:359] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.649132 22824 raft_consensus.cc:385] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.648610 22826 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.649458 22824 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.647675 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:56.655638 22845 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:56.656116 22845 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.656512 22845 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.658380 22942 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.635525 22781 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:56.663228 22948 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.665544 22708 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:56.666895 22949 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:56.669414 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:56.670922 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:08:56.670922 22601 leader_election.cc:343] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.674530 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:56.674876 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.676102 22781 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:56.677563 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:56.678524 22634 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:56.679018 22941 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:56.680006 22941 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:56.681226 22941 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.692526 22824 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.695760 22826 ts_tablet_manager.cc:1434] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.145s	user 0.016s	sys 0.002s
I20260709 13:08:56.697598 22826 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
W20260709 13:08:56.702386 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53597.6us
I20260709 13:08:56.721597 22824 ts_tablet_manager.cc:1434] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.153s	user 0.023s	sys 0.040s
I20260709 13:08:56.727898 22824 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.722558 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:56.745530 22966 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:08:56.746007 22999 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
I20260709 13:08:56.746915 22845 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.766134 22886 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.771509 22886 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.773718 22826 tablet_bootstrap.cc:654] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.769893 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:56.779368 22968 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:56.800487 22886 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.802075 22845 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:56.855782 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:56.871858 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.873207 22781 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:56.873370 22825 tablet_bootstrap.cc:654] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.879951 22632 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:56.885982 22824 tablet_bootstrap.cc:654] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:56.885383 22530 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:56.908486 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:56.909871 22781 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:56.917552 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:56.927052 22635 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:56.936178 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:56.940830 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 215834us
I20260709 13:08:56.945853 22826 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:56.946437 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:56.947121 22826 ts_tablet_manager.cc:1403] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.250s	user 0.022s	sys 0.023s
I20260709 13:08:56.959069 22826 raft_consensus.cc:359] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.959689 22824 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:56.959973 22826 raft_consensus.cc:385] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.960348 22826 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.960489 22824 ts_tablet_manager.cc:1403] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.233s	user 0.025s	sys 0.017s
I20260709 13:08:56.961287 22826 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.966990 22826 ts_tablet_manager.cc:1434] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.008s
I20260709 13:08:56.966674 22824 raft_consensus.cc:359] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.967692 22824 raft_consensus.cc:385] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.968044 22824 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.968899 22826 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:56.969020 22824 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.971767 22824 ts_tablet_manager.cc:1434] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.002s
I20260709 13:08:56.971436 23004 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:56.972671 22675 leader_election.cc:343] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:56.973562 22824 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:56.974138 23004 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:56.980304 22825 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:56.981387 22825 ts_tablet_manager.cc:1403] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.431s	user 0.037s	sys 0.033s
I20260709 13:08:56.989681 22825 raft_consensus.cc:359] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.990729 22825 raft_consensus.cc:385] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:56.991075 22825 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:56.991858 22825 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:56.993664 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:56.996906 23004 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.001617 22781 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
W20260709 13:08:57.002602 22601 leader_election.cc:343] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.003681 22824 tablet_bootstrap.cc:654] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.007632 22886 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.009563 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.011054 22886 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.011559 22886 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.011718 22825 ts_tablet_manager.cc:1434] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:08:57.011961 22886 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.015406 22825 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.015108 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:57.017709 22966 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.018306 22966 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:57.019987 22966 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.037969 22826 tablet_bootstrap.cc:654] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.042163 22825 tablet_bootstrap.cc:654] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.042508 23001 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.056443 22824 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.057214 22824 ts_tablet_manager.cc:1403] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.084s	user 0.017s	sys 0.005s
I20260709 13:08:57.046903 22886 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.087471 22886 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.094971 22824 raft_consensus.cc:359] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.096931 22824 raft_consensus.cc:385] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.097332 22824 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.107659 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:57.108560 22633 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.111759 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:57.112596 22781 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.112354 22824 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.112792 23004 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.114936 22824 ts_tablet_manager.cc:1434] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.005s
I20260709 13:08:57.116169 22824 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.134771 22949 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.136102 23004 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.136529 23006 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.136722 23004 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.148842 23006 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.154299 23006 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.158049 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.158991 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:57.161013 22600 leader_election.cc:343] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:57.161994 22601 leader_election.cc:343] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.162644 22601 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.147217 22633 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:57.167337 23006 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.179085 22826 tablet_bootstrap.cc:492] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.179674 22826 ts_tablet_manager.cc:1403] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.211s	user 0.039s	sys 0.027s
I20260709 13:08:57.180140 22825 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.180645 22825 ts_tablet_manager.cc:1403] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.166s	user 0.012s	sys 0.007s
I20260709 13:08:57.185488 22826 raft_consensus.cc:359] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.186480 22826 raft_consensus.cc:385] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.186854 22826 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.191684 23004 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.191551 22825 raft_consensus.cc:359] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.192687 22825 raft_consensus.cc:385] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.193598 22826 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.196491 22825 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.198132 22825 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.199759 22824 tablet_bootstrap.cc:654] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.201524 22825 ts_tablet_manager.cc:1434] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.000s
I20260709 13:08:57.202899 22826 ts_tablet_manager.cc:1434] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:08:57.204228 22826 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.208850 22825 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.209820 22781 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:57.212510 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:08:57.218046 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.219713 22826 tablet_bootstrap.cc:654] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.220033 22825 tablet_bootstrap.cc:654] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.221159 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.221618 23027 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.222548 22781 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:57.222325 23027 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:57.227489 22675 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.233042 23027 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.234508 23026 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.264101 22825 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.267901 22825 ts_tablet_manager.cc:1403] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.001s
I20260709 13:08:57.268390 23026 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.270718 22826 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.271276 22826 ts_tablet_manager.cc:1403] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.067s	user 0.022s	sys 0.008s
I20260709 13:08:57.271615 22825 raft_consensus.cc:359] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.272470 22825 raft_consensus.cc:385] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.272797 22825 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.273706 22825 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.274240 22826 raft_consensus.cc:359] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.277578 22826 raft_consensus.cc:385] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.277923 22826 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.281075 22825 ts_tablet_manager.cc:1434] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:08:57.281689 22824 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.282258 22824 ts_tablet_manager.cc:1403] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.166s	user 0.011s	sys 0.007s
I20260709 13:08:57.282779 22825 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.286195 22824 raft_consensus.cc:359] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.287112 22824 raft_consensus.cc:385] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.287238 22826 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.287447 22824 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.311734 22826 ts_tablet_manager.cc:1434] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.016s
I20260709 13:08:57.312954 22826 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.290460 22531 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.315395 22949 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.315922 22949 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.318254 22949 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:57.320000 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.314930 22824 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.321616 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.331669 22966 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.332265 22966 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.322641 23006 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.327199 22633 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:57.335678 22825 tablet_bootstrap.cc:654] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.336979 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.338792 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:57.339311 22600 leader_election.cc:343] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.335505 22749 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:08:57.343501 22601 leader_election.cc:343] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.344025 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.344930 22949 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.345432 22949 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.345674 22942 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.345929 22949 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:57.352612 22750 leader_election.cc:343] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.353281 22941 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.355470 22966 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.359339 22949 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.360599 22824 ts_tablet_manager.cc:1434] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.078s	user 0.011s	sys 0.010s
I20260709 13:08:57.364724 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:57.365429 22633 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.366497 22949 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:57.371294 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:57.373550 22825 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.374567 22825 ts_tablet_manager.cc:1403] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.004s
I20260709 13:08:57.365828 22532 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:57.383020 22750 leader_election.cc:343] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.390667 22633 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:57.391569 22824 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.392860 23001 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.394948 22826 tablet_bootstrap.cc:654] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.398563 23001 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.400305 22749 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:57.401079 23001 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:57.402309 23001 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.402405 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.402952 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.403026 23001 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:57.403177 22710 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:57.404404 23001 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:57.405427 22749 leader_election.cc:343] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.407500 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:08:57.408689 22968 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.409181 22968 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.409616 22968 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.403613 22530 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:57.420701 22825 raft_consensus.cc:359] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.421518 22825 raft_consensus.cc:385] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.421806 22825 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.422590 22825 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.434517 22824 tablet_bootstrap.cc:654] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.446998 22825 ts_tablet_manager.cc:1434] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.072s	user 0.005s	sys 0.004s
I20260709 13:08:57.449237 22825 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
W20260709 13:08:57.453709 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:57.441830 22968 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.456571 22824 tablet_bootstrap.cc:492] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.457146 22824 ts_tablet_manager.cc:1403] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.066s	user 0.006s	sys 0.007s
I20260709 13:08:57.458000 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:57.460354 22826 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.460861 22826 ts_tablet_manager.cc:1403] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.148s	user 0.006s	sys 0.008s
I20260709 13:08:57.461320 22710 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.461622 22968 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:57.461833 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:57.465580 22709 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.472013 22845 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.473843 22709 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:57.475190 22825 tablet_bootstrap.cc:654] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:57.462788 22749 leader_election.cc:343] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.464277 22824 raft_consensus.cc:359] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.476646 22824 raft_consensus.cc:385] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.476996 22824 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.477761 22824 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.480142 22824 ts_tablet_manager.cc:1434] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.023s	user 0.003s	sys 0.006s
I20260709 13:08:57.483853 22824 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.487819 22845 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.489313 22532 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.490774 22824 tablet_bootstrap.cc:654] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.497944 22826 raft_consensus.cc:359] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.499197 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:08:57.502949 23001 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.503626 23001 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:57.504801 23001 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.515815 22826 raft_consensus.cc:385] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.516232 22826 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.517040 22826 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.519743 22826 ts_tablet_manager.cc:1434] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 13:08:57.520979 22826 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.538319 22826 tablet_bootstrap.cc:654] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.557304 23001 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.558264 23001 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.559015 22532 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:57.561381 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.561728 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.562803 23001 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:08:57.563448 22750 leader_election.cc:343] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:57.564744 22749 leader_election.cc:343] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.568359 22824 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.568969 22824 ts_tablet_manager.cc:1403] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.085s	user 0.015s	sys 0.025s
I20260709 13:08:57.565253 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:57.569932 23001 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.570163 22826 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.570770 22826 ts_tablet_manager.cc:1403] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.008s
I20260709 13:08:57.573278 22824 raft_consensus.cc:359] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.573706 22826 raft_consensus.cc:359] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.574100 22824 raft_consensus.cc:385] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.574561 22824 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.575546 22824 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.577868 22824 ts_tablet_manager.cc:1434] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:08:57.579324 22824 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.581419 22825 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.582073 22825 ts_tablet_manager.cc:1403] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.133s	user 0.027s	sys 0.021s
I20260709 13:08:57.586333 22824 tablet_bootstrap.cc:654] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.586072 22825 raft_consensus.cc:359] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.587105 22825 raft_consensus.cc:385] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.587535 22825 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.588621 22825 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.590513 22825 ts_tablet_manager.cc:1434] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:08:57.591952 22825 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.601583 22825 tablet_bootstrap.cc:654] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.617836 22824 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.618398 22824 ts_tablet_manager.cc:1403] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.005s
I20260709 13:08:57.620914 22825 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.621542 22825 ts_tablet_manager.cc:1403] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.004s
I20260709 13:08:57.621490 22824 raft_consensus.cc:359] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.622329 22824 raft_consensus.cc:385] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.622682 22824 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.623701 22824 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.624816 22826 raft_consensus.cc:385] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.625253 22826 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.626163 22824 ts_tablet_manager.cc:1434] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:08:57.626041 22826 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.626153 22825 raft_consensus.cc:359] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.627444 22825 raft_consensus.cc:385] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.627547 22824 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.627892 22825 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.628706 22825 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.631664 22825 ts_tablet_manager.cc:1434] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.005s
I20260709 13:08:57.633173 22825 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.635028 22824 tablet_bootstrap.cc:654] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.641258 22825 tablet_bootstrap.cc:654] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.648010 22824 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.648877 22824 ts_tablet_manager.cc:1403] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:08:57.652482 22824 raft_consensus.cc:359] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.653298 22824 raft_consensus.cc:385] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.653622 22824 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.654340 22824 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.655076 22826 ts_tablet_manager.cc:1434] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.084s	user 0.007s	sys 0.000s
I20260709 13:08:57.657363 22824 ts_tablet_manager.cc:1434] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:08:57.659638 22824 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.661600 22825 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.662163 22825 ts_tablet_manager.cc:1403] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.000s
I20260709 13:08:57.665261 22825 raft_consensus.cc:359] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.666003 22825 raft_consensus.cc:385] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.666286 22825 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.666842 22826 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.666973 22825 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.669373 22824 tablet_bootstrap.cc:654] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.669606 22825 ts_tablet_manager.cc:1434] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:08:57.670898 22825 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.688205 22825 tablet_bootstrap.cc:654] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.697929 22826 tablet_bootstrap.cc:654] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.698002 22824 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.698813 22824 ts_tablet_manager.cc:1403] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.017s
I20260709 13:08:57.702147 22824 raft_consensus.cc:359] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.703246 22824 raft_consensus.cc:385] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.703689 22824 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.703958 22825 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:57.704667 22825 ts_tablet_manager.cc:1403] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.004s
I20260709 13:08:57.704790 22824 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.708246 22825 raft_consensus.cc:359] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.709038 22825 raft_consensus.cc:385] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.709314 22825 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.711019 22825 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.713886 22825 ts_tablet_manager.cc:1434] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:08:57.715384 23001 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.715948 23001 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.720048 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.720904 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:57.721935 22824 ts_tablet_manager.cc:1434] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:08:57.722889 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.723326 22824 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.723965 22968 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.724387 22968 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.724754 22968 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.724411 22826 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.726078 22826 ts_tablet_manager.cc:1403] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.060s	user 0.021s	sys 0.013s
I20260709 13:08:57.729048 23001 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:57.730535 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.732301 22824 tablet_bootstrap.cc:654] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:57.733260 22750 leader_election.cc:343] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.735232 22968 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.735471 22826 raft_consensus.cc:359] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.736367 22826 raft_consensus.cc:385] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.736678 22826 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.737564 22826 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.738893 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:57.739599 22632 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.740015 22826 ts_tablet_manager.cc:1434] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:08:57.741199 22826 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.742127 23027 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.750959 22966 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.751573 22966 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.752274 22826 tablet_bootstrap.cc:654] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.755262 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:57.757170 22601 leader_election.cc:343] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.760802 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.761613 22781 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:57.763644 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:57.764770 23006 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.765221 23006 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.765588 23006 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.768795 22845 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.769402 22845 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.771708 22845 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.772686 22826 tablet_bootstrap.cc:492] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:57.773070 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.773417 22826 ts_tablet_manager.cc:1403] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.004s
I20260709 13:08:57.728782 22825 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:57.777484 22966 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.792488 22968 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:08:57.781136 22600 leader_election.cc:343] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.797842 23026 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.796859 22966 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.798484 23026 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.798697 22966 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.800841 23026 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.797749 22845 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.801723 22845 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.802980 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.803864 22783 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:57.804138 22845 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.805529 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.806025 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.807015 22783 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:57.807400 22941 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.807878 22941 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.808265 22941 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.808547 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.808943 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.810792 22942 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.811246 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.811329 22942 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.813853 22942 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.815027 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.815690 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:57.816284 22704 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.816510 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.817117 22709 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:57.818174 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:57.819051 22635 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:57.819489 22706 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.823247 23001 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.825042 22845 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.826145 22845 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:57.830715 22601 leader_election.cc:343] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.831251 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:57.800856 22966 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.835274 22709 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:08:57.833483 22845 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.837950 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.844554 22825 tablet_bootstrap.cc:654] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.854413 22966 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.861091 22966 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.861660 22966 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.861389 22824 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.867448 22824 ts_tablet_manager.cc:1403] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.144s	user 0.025s	sys 0.031s
I20260709 13:08:57.871186 22824 raft_consensus.cc:359] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.872016 22824 raft_consensus.cc:385] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.872323 22824 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.874109 22824 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.876232 22824 ts_tablet_manager.cc:1434] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:08:57.877293 22824 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:57.879365 23006 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.856360 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:57.887979 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.889482 22826 raft_consensus.cc:359] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.890229 22826 raft_consensus.cc:385] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:57.890532 22826 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:57.891297 22826 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.893509 22826 ts_tablet_manager.cc:1434] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.120s	user 0.007s	sys 0.002s
I20260709 13:08:57.882679 22942 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.897187 22942 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.899470 22942 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.874856 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.900620 23026 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:57.901444 23026 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:57.901733 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:57.906354 23006 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.909395 23056 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.910440 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:57.911942 22783 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.911023 22845 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.915017 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:57.917446 22781 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:57.919041 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.919370 22826 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:57.920006 23006 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:57.920432 23006 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.920500 22783 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:57.920866 23006 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.922114 22845 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.923647 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.924433 22824 tablet_bootstrap.cc:654] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:57.925918 22966 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.927008 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:57.931968 22783 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:57.930771 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.932868 22968 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.930506 23006 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.929993 22845 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:57.911258 22601 leader_election.cc:343] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.934319 22968 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:57.934826 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:57.935287 22968 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:57.936581 22600 leader_election.cc:343] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:57.945922 23001 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:57.951026 22783 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:57.953619 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:57.943509 22704 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:57.940927 22966 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:57.980551 22942 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:57.981194 22942 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:08:57.965188 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57944.5us
I20260709 13:08:57.982110 22942 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.972266 22845 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:57.992590 22824 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:57.993376 22824 ts_tablet_manager.cc:1403] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.040s
I20260709 13:08:57.996160 22845 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:57.999800 22533 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:58.000655 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:58.001433 22783 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.001662 23056 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.005563 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:58.006737 22941 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.007221 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.008577 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.009516 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.010113 22709 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.011559 22826 tablet_bootstrap.cc:654] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.004233 23006 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.014364 22941 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.052052 22824 raft_consensus.cc:359] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.053205 22824 raft_consensus.cc:385] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.053565 22824 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.054472 22824 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.057627 22824 ts_tablet_manager.cc:1434] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.064s	user 0.007s	sys 0.003s
I20260709 13:08:58.072561 22824 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:58.047446 23026 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:58.073705 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61041.5us
I20260709 13:08:58.080463 23026 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.112843 22783 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:58.073906 23023 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.049s
W20260709 13:08:58.141456 23032 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:08:58.142668 22825 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:58.114418 22709 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:58.143662 22825 ts_tablet_manager.cc:1403] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.429s	user 0.033s	sys 0.005s
I20260709 13:08:58.148301 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.137269 23026 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.156906 23069 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:58.164834 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.159468 23069 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:58.186723 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:58.187819 22784 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:08:58.189970 22966 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.190676 22966 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.168483 23069 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.171496 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:58.212033 22966 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.212373 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:08:58.203572 22601 leader_election.cc:343] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.226822 22825 raft_consensus.cc:359] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.229666 22825 raft_consensus.cc:385] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.230044 22825 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.208621 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.219264 22783 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.246174 22825 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.253471 22824 tablet_bootstrap.cc:654] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.263053 22825 ts_tablet_manager.cc:1434] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.119s	user 0.007s	sys 0.007s
I20260709 13:08:58.260174 22631 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:58.248082 22709 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:58.262338 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74326.5us
I20260709 13:08:58.260484 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:08:58.287511 22601 leader_election.cc:343] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.306088 22826 tablet_bootstrap.cc:492] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:58.309435 22826 ts_tablet_manager.cc:1403] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.390s	user 0.019s	sys 0.034s
I20260709 13:08:58.319748 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:58.328713 22601 leader_election.cc:343] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.329928 22709 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:58.321017 22780 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:58.330519 23026 raft_consensus.cc:2764] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:08:58.335207 22600 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.338136 23060 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:58.338788 23060 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.339215 23060 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.338094 22825 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:58.353691 22783 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:58.351809 23060 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.355293 23074 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.347252 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:58.359284 22845 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:08:58.359474 22601 leader_election.cc:343] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.357321 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.359802 22845 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:58.356236 22826 raft_consensus.cc:359] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.364595 22825 tablet_bootstrap.cc:654] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.365029 23060 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.388913 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:58.390683 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:58.372356 22826 raft_consensus.cc:385] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.376257 23067 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:58.395907 23067 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:58.396915 23067 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.375121 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:58.399989 22783 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.385150 22824 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:58.400841 22824 ts_tablet_manager.cc:1403] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.334s	user 0.016s	sys 0.007s
I20260709 13:08:58.402726 22826 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.373159 22845 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.413468 22949 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.411413 22826 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.380769 23069 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.417183 23069 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.420487 23069 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.406190 22824 raft_consensus.cc:359] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.421372 22824 raft_consensus.cc:385] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.421775 22824 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.421737 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:58.422803 22824 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:58.424033 22600 leader_election.cc:343] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.425215 22824 ts_tablet_manager.cc:1434] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:08:58.426393 22824 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:58.427193 22825 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:58.427703 22825 ts_tablet_manager.cc:1403] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.090s	user 0.010s	sys 0.012s
I20260709 13:08:58.432696 22826 ts_tablet_manager.cc:1434] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.123s	user 0.011s	sys 0.003s
I20260709 13:08:58.439556 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:58.440032 22826 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:58.433214 22824 tablet_bootstrap.cc:654] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.433626 22825 raft_consensus.cc:359] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.436157 23026 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.442576 22825 raft_consensus.cc:385] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.443032 22825 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.442768 23026 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.444721 22825 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.447359 22825 ts_tablet_manager.cc:1434] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.004s
I20260709 13:08:58.448586 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:58.448727 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:58.449566 22782 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:58.445163 23026 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.449239 23069 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:58.451954 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:58.453347 23065 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.454725 22825 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:58.465932 22783 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:58.467406 22600 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.468287 22845 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:58.469009 22845 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:58.469923 22845 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.471493 22949 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:58.458600 22826 tablet_bootstrap.cc:654] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:58.438108 22601 leader_election.cc:343] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.513707 23067 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.514382 23067 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:58.566635 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:58.586057 23065 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.608777 22824 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:58.609519 22824 ts_tablet_manager.cc:1403] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.183s	user 0.010s	sys 0.006s
I20260709 13:08:58.619390 23060 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.627738 22824 raft_consensus.cc:359] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.628638 22824 raft_consensus.cc:385] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.628940 22824 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
W20260709 13:08:58.607488 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 169436us
I20260709 13:08:58.629583 22824 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:58.607924 22816 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.21.5.190:44025, user_credentials={real_user=slave}} blocked reactor thread for 85095.3us
W20260709 13:08:58.633189 22601 leader_election.cc:343] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.609252 22533 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:08:58.637916 22825 tablet_bootstrap.cc:654] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.643575 22824 ts_tablet_manager.cc:1434] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
W20260709 13:08:58.645628 22601 leader_election.cc:343] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.647498 22824 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:58.647536 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:58.648644 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:58.650317 22600 leader_election.cc:343] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.654026 22845 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.658764 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:58.647176 23067 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.713452 23065 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.714293 23065 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.717234 23065 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.719919 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:58.722476 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:58.722959 22780 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:58.723192 22635 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:08:58.721683 22826 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:58.724709 22826 ts_tablet_manager.cc:1403] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.285s	user 0.022s	sys 0.034s
I20260709 13:08:58.725208 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.726702 23065 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:58.727135 23065 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.727505 23065 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.732983 22826 raft_consensus.cc:359] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.735472 22824 tablet_bootstrap.cc:654] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.735922 22825 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:58.736461 22825 ts_tablet_manager.cc:1403] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.282s	user 0.039s	sys 0.037s
W20260709 13:08:58.739074 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 67750.2us
I20260709 13:08:58.740113 22825 raft_consensus.cc:359] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.741004 22825 raft_consensus.cc:385] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.741344 22825 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.742234 22825 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.743674 22826 raft_consensus.cc:385] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.738282 23065 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:58.747529 22601 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:58.751719 22601 leader_election.cc:343] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:58.752214 22601 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:58.753229 23006 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.755496 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:58.756285 22632 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.761502 23065 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.766826 22826 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.764390 22532 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.768316 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:58.769044 22782 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:58.750103 22825 ts_tablet_manager.cc:1434] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:08:58.770473 22825 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:58.772395 23006 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.774919 22632 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:08:58.768802 22826 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.783284 22782 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:08:58.785241 22826 ts_tablet_manager.cc:1434] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.060s	user 0.006s	sys 0.005s
I20260709 13:08:58.786603 22826 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:58.784793 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:58.769371 22532 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.789955 23026 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:58.790602 23026 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:08:58.790937 22532 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.791615 23026 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.793501 22824 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:58.794103 22824 ts_tablet_manager.cc:1403] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.148s	user 0.019s	sys 0.025s
I20260709 13:08:58.798364 22826 tablet_bootstrap.cc:654] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.797923 22824 raft_consensus.cc:359] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.799320 22825 tablet_bootstrap.cc:654] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:58.803180 22601 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:58.804225 22824 raft_consensus.cc:385] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.804710 22824 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.812702 22824 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.819763 23023 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.820462 23023 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.823946 23023 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:58.824172 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:58.825095 22709 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:58.823887 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:58.831450 23067 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:58.836371 22749 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:58.840231 22825 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:58.840991 22825 ts_tablet_manager.cc:1403] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.071s	user 0.010s	sys 0.003s
I20260709 13:08:58.844575 23023 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:58.850055 22824 ts_tablet_manager.cc:1434] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.001s
I20260709 13:08:58.853902 22824 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:58.857374 22825 raft_consensus.cc:359] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.858347 22825 raft_consensus.cc:385] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.858717 22825 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.859630 22825 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.868811 22825 ts_tablet_manager.cc:1434] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:08:58.870260 22825 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:58.871358 23023 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.878796 22826 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:58.866665 22824 tablet_bootstrap.cc:654] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.879469 22826 ts_tablet_manager.cc:1403] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.093s	user 0.025s	sys 0.030s
I20260709 13:08:58.882794 22826 raft_consensus.cc:359] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.911010 22826 raft_consensus.cc:385] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.911446 22826 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.912528 22826 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.916594 22826 ts_tablet_manager.cc:1434] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.012s
I20260709 13:08:58.917894 22826 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:58.924896 22826 tablet_bootstrap.cc:654] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.926159 22825 tablet_bootstrap.cc:654] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.929116 23006 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:58.932885 23006 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:58.936046 23006 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:58.936331 22824 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:58.937103 22824 ts_tablet_manager.cc:1403] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.084s	user 0.011s	sys 0.004s
I20260709 13:08:58.937776 22826 tablet_bootstrap.cc:492] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:58.938300 22826 ts_tablet_manager.cc:1403] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.007s
I20260709 13:08:58.936939 22531 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.940555 22824 raft_consensus.cc:359] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.941347 22824 raft_consensus.cc:385] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.941677 22824 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.942721 23066 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.942562 22824 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.941541 22826 raft_consensus.cc:359] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.946341 22531 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.953449 22826 raft_consensus.cc:385] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.953908 22826 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.953402 23066 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.954952 22824 ts_tablet_manager.cc:1434] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:08:58.954687 22826 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.956005 22824 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:58.957274 22826 ts_tablet_manager.cc:1434] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:08:58.958382 22826 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:58.958730 23066 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:58.960263 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:58.962714 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:58.964272 22824 tablet_bootstrap.cc:654] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.965708 22675 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:58.966933 23066 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:58.964885 22826 tablet_bootstrap.cc:654] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:58.967276 22530 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 2, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:58.972775 22825 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:58.973412 22825 ts_tablet_manager.cc:1403] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.103s	user 0.010s	sys 0.011s
I20260709 13:08:58.976488 22825 raft_consensus.cc:359] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.977450 22825 raft_consensus.cc:385] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.977919 22825 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.982179 22824 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:58.982955 22824 ts_tablet_manager.cc:1403] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.004s
I20260709 13:08:58.978885 22825 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.988364 22825 ts_tablet_manager.cc:1434] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:08:58.989879 22825 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:58.986207 22824 raft_consensus.cc:359] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:58.991941 22824 raft_consensus.cc:385] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:58.992332 22824 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:58.993177 22824 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.004058 22824 ts_tablet_manager.cc:1434] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:08:59.004992 22825 tablet_bootstrap.cc:654] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.005345 22824 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:59.013386 22824 tablet_bootstrap.cc:654] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.017709 22942 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.022089 23069 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:08:59.032416 22601 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.049955 22826 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:59.050565 22826 ts_tablet_manager.cc:1403] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.092s	user 0.014s	sys 0.009s
I20260709 13:08:59.054487 22826 raft_consensus.cc:359] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.055822 22825 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.056031 22826 raft_consensus.cc:385] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.056470 22826 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.056612 22825 ts_tablet_manager.cc:1403] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.067s	user 0.015s	sys 0.006s
I20260709 13:08:59.057312 22826 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.059754 22826 ts_tablet_manager.cc:1434] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:08:59.059978 22825 raft_consensus.cc:359] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.060890 22825 raft_consensus.cc:385] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.061172 22825 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.061995 22825 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.065829 22826 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.065867 22825 ts_tablet_manager.cc:1434] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:08:59.067224 22825 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.073964 22826 tablet_bootstrap.cc:654] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.081923 22825 tablet_bootstrap.cc:654] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.088850 23023 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.089264 23069 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.089555 23023 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.089864 23069 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.092150 23069 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.092317 23023 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.094379 22824 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:59.094988 22824 ts_tablet_manager.cc:1403] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.090s	user 0.021s	sys 0.006s
I20260709 13:08:59.094826 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.095893 22635 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:59.098718 22749 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.099139 22824 raft_consensus.cc:359] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.099622 23023 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:59.099889 22824 raft_consensus.cc:385] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.100068 23023 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.100209 22824 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.100474 23023 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.100296 22825 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.101189 22825 ts_tablet_manager.cc:1403] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.003s
I20260709 13:08:59.101049 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.102450 22824 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.104840 22824 ts_tablet_manager.cc:1434] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:08:59.104760 22825 raft_consensus.cc:359] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.105593 22825 raft_consensus.cc:385] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.105937 22825 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.106871 22825 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.107877 22826 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
W20260709 13:08:59.108191 22750 leader_election.cc:343] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.108449 22826 ts_tablet_manager.cc:1403] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.007s
I20260709 13:08:59.108500 23069 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.108618 23023 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.109234 23069 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.111410 23069 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.111599 22826 raft_consensus.cc:359] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.111934 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:59.112358 22826 raft_consensus.cc:385] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.112525 22635 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.112668 22826 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.113037 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.115078 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.115171 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.115867 22783 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:08:59.114975 22826 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:59.117143 22601 leader_election.cc:343] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:59.118197 22600 leader_election.cc:343] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.122179 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.121140 22705 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.127286 23023 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.127977 23069 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.129572 22635 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:59.130003 22826 ts_tablet_manager.cc:1434] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.003s
I20260709 13:08:59.130085 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.131017 22749 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.131309 22826 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.131170 23069 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:59.131773 23069 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.132167 23069 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.132893 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.113698 22825 ts_tablet_manager.cc:1434] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
W20260709 13:08:59.134498 22601 leader_election.cc:343] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.135239 23023 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:59.135462 22825 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.135766 23023 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:59.136924 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:59.136674 23023 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:59.138696 22750 leader_election.cc:343] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.144907 22824 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:59.152835 22825 tablet_bootstrap.cc:654] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.155578 22826 tablet_bootstrap.cc:654] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.160991 23071 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.161610 23071 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.164264 23071 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.143929 23069 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.166144 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.167092 22635 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:59.167709 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:59.168431 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.169132 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:08:59.169608 22601 leader_election.cc:343] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.175132 22824 tablet_bootstrap.cc:654] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.169159 22706 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:59.175187 22949 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:59.176379 22949 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.176885 22949 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.173915 22530 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:59.193984 22635 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:59.195842 23026 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:08:59.197360 23069 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.201486 22824 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:59.202207 22824 ts_tablet_manager.cc:1403] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.058s	user 0.013s	sys 0.001s
I20260709 13:08:59.209430 23023 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.210057 23023 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.212841 22824 raft_consensus.cc:359] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.213256 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.214131 22634 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:59.216112 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.216949 23074 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:08:59.217340 23074 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.217674 23074 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.219937 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:08:59.220616 22782 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.223032 23023 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.226440 22824 raft_consensus.cc:385] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.227257 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.235205 22705 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:08:59.236826 22824 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.239076 22824 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.253245 22824 ts_tablet_manager.cc:1434] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 13:08:59.255007 22824 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:59.240540 23074 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.258875 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:59.264333 22705 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.262930 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:59.270200 22634 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.274832 23074 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.283134 22705 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:59.284992 22824 tablet_bootstrap.cc:654] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.285609 22750 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.286731 23074 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:59.287312 23074 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:59.288229 23074 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.301338 23023 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.311275 23023 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.313546 23023 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.322625 23006 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.323511 23006 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.314872 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.335314 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.338061 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:08:59.346522 22750 leader_election.cc:343] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:59.347697 22601 leader_election.cc:343] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:59.350664 22749 leader_election.cc:343] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.351219 22749 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:59.353531 23071 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.355479 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.347420 23006 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:08:59.360877 22600 leader_election.cc:343] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.346390 22634 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:59.367530 22600 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.368448 23006 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.364446 22949 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.373143 22531 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:59.378185 22949 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.376539 22782 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:08:59.379784 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:08:59.380463 22706 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.383781 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:59.384629 22825 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.385255 22825 ts_tablet_manager.cc:1403] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.250s	user 0.027s	sys 0.052s
I20260709 13:08:59.386305 23027 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.386876 23027 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.389091 23027 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.389225 23069 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:59.388993 22825 raft_consensus.cc:359] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.389784 23069 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:08:59.390502 22826 tablet_bootstrap.cc:492] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:59.390007 22825 raft_consensus.cc:385] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.390863 23069 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.391242 22825 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.391093 22826 ts_tablet_manager.cc:1403] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.260s	user 0.069s	sys 0.092s
I20260709 13:08:59.391017 22783 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:08:59.398097 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.398789 22845 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.398768 22825 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.399505 22845 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.400687 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.401921 22845 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.405164 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.404942 23027 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:08:59.406891 22600 leader_election.cc:343] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.412858 22825 ts_tablet_manager.cc:1434] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 13:08:59.414139 22825 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.414434 22705 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.419704 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.420861 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:59.423002 22601 leader_election.cc:343] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.423650 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.421010 23027 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.447445 22826 raft_consensus.cc:359] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.448355 22826 raft_consensus.cc:385] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.448734 22826 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.449455 22826 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.449960 22706 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:59.379261 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:08:59.450383 23006 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.451107 22634 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.451968 22826 ts_tablet_manager.cc:1434] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.002s
I20260709 13:08:59.453186 22826 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.451071 23006 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.426117 22845 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.465284 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:08:59.462483 22949 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.467494 22949 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.470279 22949 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:08:59.475384 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.477308 22710 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:08:59.478652 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:08:59.478652 22749 leader_election.cc:343] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:08:59.479799 22601 leader_election.cc:343] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.491567 22824 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:59.491951 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:08:59.492271 22824 ts_tablet_manager.cc:1403] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.238s	user 0.016s	sys 0.050s
I20260709 13:08:59.494853 23023 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:08:59.497382 23023 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:08:59.498385 23023 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:59.503196 22750 leader_election.cc:343] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.503690 22750 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:08:59.505605 23001 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.507483 22531 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:59.517397 22634 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:08:59.528927 22824 raft_consensus.cc:359] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.530210 22824 raft_consensus.cc:385] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.535957 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:08:59.540669 22600 leader_election.cc:343] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:08:59.541298 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.501554 23006 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.543004 22824 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.544660 23069 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.546887 22824 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.550208 22825 tablet_bootstrap.cc:654] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.560352 23065 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.561228 23065 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:08:59.562101 22750 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:59.564146 23065 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:08:59.626117 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:08:59.633775 22824 ts_tablet_manager.cc:1434] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.141s	user 0.009s	sys 0.007s
I20260709 13:08:59.647995 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:08:59.656350 22783 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:08:59.675860 22826 tablet_bootstrap.cc:654] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.679150 22824 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:08:59.682907 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:08:59.694370 22825 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.695811 22825 ts_tablet_manager.cc:1403] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.282s	user 0.021s	sys 0.004s
I20260709 13:08:59.698933 23065 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:08:59.766212 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:08:59.767398 22531 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:59.777676 22824 tablet_bootstrap.cc:654] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.784235 22949 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.788348 23065 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:08:59.789271 22825 raft_consensus.cc:359] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.796501 22825 raft_consensus.cc:385] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.796965 22825 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.790685 22826 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:59.797753 22825 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.798799 22826 ts_tablet_manager.cc:1403] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.346s	user 0.017s	sys 0.009s
I20260709 13:08:59.801409 22825 ts_tablet_manager.cc:1434] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.004s
I20260709 13:08:59.801774 22826 raft_consensus.cc:359] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.803025 22825 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.808470 23131 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.812098 22825 tablet_bootstrap.cc:654] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.816280 22826 raft_consensus.cc:385] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.816728 22826 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.817546 22949 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.819010 22826 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.821524 22826 ts_tablet_manager.cc:1434] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:08:59.822683 22826 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.829082 23001 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.835850 22826 tablet_bootstrap.cc:654] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.854646 22825 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.854866 22824 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:08:59.855376 22825 ts_tablet_manager.cc:1403] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.053s	user 0.028s	sys 0.014s
I20260709 13:08:59.858951 22825 raft_consensus.cc:359] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.860309 22825 raft_consensus.cc:385] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.860783 22825 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.861755 22825 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.864902 22825 ts_tablet_manager.cc:1434] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:08:59.866976 22826 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:59.867736 22826 ts_tablet_manager.cc:1403] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.003s
I20260709 13:08:59.870129 22825 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.871430 22826 raft_consensus.cc:359] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.872187 22826 raft_consensus.cc:385] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.872473 22826 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.873169 22826 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.875708 22826 ts_tablet_manager.cc:1434] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:08:59.876856 22826 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.880131 22825 tablet_bootstrap.cc:654] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.884331 22826 tablet_bootstrap.cc:654] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:08:59.894171 22825 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:08:59.894939 22825 ts_tablet_manager.cc:1403] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.001s
I20260709 13:08:59.905891 22826 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:08:59.906760 22826 ts_tablet_manager.cc:1403] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.004s
I20260709 13:08:59.911054 22826 raft_consensus.cc:359] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.912127 22826 raft_consensus.cc:385] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.912514 22826 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.913545 22826 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.915298 22825 raft_consensus.cc:359] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.916419 22825 raft_consensus.cc:385] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.916795 22825 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.917883 22825 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.921180 22825 ts_tablet_manager.cc:1434] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 13:08:59.923787 22826 ts_tablet_manager.cc:1434] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:08:59.855441 22824 ts_tablet_manager.cc:1403] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.189s	user 0.017s	sys 0.005s
I20260709 13:08:59.925096 22826 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:08:59.930101 22825 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:08:59.935364 22826 tablet_bootstrap.cc:654] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
W20260709 13:08:59.934967 22810 tablet.cc:2406] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:08:59.941906 22824 raft_consensus.cc:359] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.942593 22824 raft_consensus.cc:385] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:08:59.942919 22824 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:08:59.943491 22824 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:08:59.969385 23074 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:08:59.975203 22825 tablet_bootstrap.cc:654] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.008718 22949 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.009343 22949 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.013974 23131 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:00.015532 22949 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.018996 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:00.021126 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.021946 22635 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:00.023008 22824 ts_tablet_manager.cc:1434] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.098s	user 0.004s	sys 0.003s
W20260709 13:09:00.025094 22750 leader_election.cc:343] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.028986 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.035087 22949 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.035950 22949 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.036446 22949 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.046976 22824 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:00.059958 22826 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:00.060802 22826 ts_tablet_manager.cc:1403] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.136s	user 0.023s	sys 0.025s
I20260709 13:09:00.062333 23023 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.073510 23023 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.076506 23023 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.079310 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:00.082166 22750 leader_election.cc:343] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.086366 22826 raft_consensus.cc:359] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.087181 22826 raft_consensus.cc:385] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.087456 22826 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.087853 22824 tablet_bootstrap.cc:654] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.088454 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.089521 22635 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:00.090862 22826 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.098935 22826 ts_tablet_manager.cc:1434] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 13:09:00.100435 22826 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:00.101567 22749 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.103250 23074 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.104969 23074 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.105338 23074 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.106467 23023 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:00.115243 22825 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:00.115006 23074 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.115770 22825 ts_tablet_manager.cc:1403] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.186s	user 0.027s	sys 0.019s
I20260709 13:09:00.121833 22825 raft_consensus.cc:359] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.122972 22825 raft_consensus.cc:385] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.123278 22825 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.124478 22825 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.132227 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.133088 22632 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.133281 22826 tablet_bootstrap.cc:654] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.135293 23074 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.144383 22825 ts_tablet_manager.cc:1434] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.003s
I20260709 13:09:00.142472 22949 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.154835 22825 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:00.162449 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:00.171293 22750 leader_election.cc:343] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.190822 22709 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.193373 22824 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:00.194109 22824 ts_tablet_manager.cc:1403] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.153s	user 0.021s	sys 0.010s
I20260709 13:09:00.191447 23154 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.256307 22824 raft_consensus.cc:359] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.257164 22824 raft_consensus.cc:385] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.257427 22824 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.238322 23154 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.258841 22824 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.265892 22824 ts_tablet_manager.cc:1434] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.000s
I20260709 13:09:00.266821 22826 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:00.267248 22824 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:00.267381 22826 ts_tablet_manager.cc:1403] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.167s	user 0.017s	sys 0.007s
I20260709 13:09:00.288628 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.289469 22634 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:00.289754 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.290431 22633 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.292987 23154 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.294281 22949 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.295634 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:00.281905 22632 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:00.304981 23065 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.314231 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:00.310463 23074 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.326843 23065 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.337069 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.337811 22749 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:00.338861 22675 leader_election.cc:343] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.339099 23001 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.339321 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:00.340602 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:00.340934 22750 leader_election.cc:343] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.341301 22780 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:00.343161 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:00.344303 23134 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.344753 23134 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.345121 23134 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.337121 22826 raft_consensus.cc:359] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.353174 22826 raft_consensus.cc:385] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.353516 22826 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.358584 22826 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.335204 23074 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.333609 23065 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.366240 22825 tablet_bootstrap.cc:654] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.364701 22710 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:00.375828 23161 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.365458 23074 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.376402 23161 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.387344 23001 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:00.388474 23001 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.383236 23161 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.369141 23157 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.398214 22826 ts_tablet_manager.cc:1434] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.127s	user 0.021s	sys 0.021s
I20260709 13:09:00.397936 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:00.402158 23157 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.409543 23026 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.414374 23026 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.415654 23157 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.417860 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:00.419476 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.420430 22634 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:00.422405 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.423192 22632 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:00.427954 22824 tablet_bootstrap.cc:654] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.429039 22826 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:00.431348 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:00.434409 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:00.412415 22633 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:00.443544 23163 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.444046 23163 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.437741 23026 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.444412 23163 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.457988 23128 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.445043 23134 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.441658 23023 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.458664 23128 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.460289 23023 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.460763 23023 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.467414 23134 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.480770 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:00.482559 22601 leader_election.cc:343] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.485481 22825 tablet_bootstrap.cc:492] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:00.489902 22825 ts_tablet_manager.cc:1403] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.339s	user 0.037s	sys 0.029s
W20260709 13:09:00.490159 22750 leader_election.cc:343] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.487149 23074 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.491849 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.493921 22632 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:00.494292 23128 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.496065 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:00.496835 22780 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:09:00.496888 23023 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.498560 22600 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.494407 23163 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.499161 23023 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:00.551525 23163 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.533980 22530 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.552955 23128 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.553393 23128 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.558485 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.558996 23128 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.562336 22825 raft_consensus.cc:359] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.563149 22825 raft_consensus.cc:385] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.563495 22825 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.564603 22825 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.565802 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.566511 22632 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:00.567826 23164 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:09:00.505617 23074 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:00.568053 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 69641.8us
I20260709 13:09:00.571245 22825 ts_tablet_manager.cc:1434] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.081s	user 0.015s	sys 0.001s
I20260709 13:09:00.572549 22825 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:00.576315 23128 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.580235 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.581122 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:00.581627 22675 leader_election.cc:343] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.582149 22824 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:00.582820 22633 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.583025 22826 tablet_bootstrap.cc:654] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.583174 22824 ts_tablet_manager.cc:1403] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.316s	user 0.017s	sys 0.008s
I20260709 13:09:00.582932 22634 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.587555 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:00.589162 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:00.589545 23128 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.589802 22706 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.591858 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:00.593573 22601 leader_election.cc:343] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.594849 22633 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:00.598675 22825 tablet_bootstrap.cc:654] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.599068 22632 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:00.580305 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:00.596879 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:00.601433 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:00.602042 22783 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.602133 22779 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.603860 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:00.604727 23065 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:00.605093 23065 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:00.605158 22675 leader_election.cc:343] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.605438 23065 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.606225 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:09:00.607823 22750 leader_election.cc:343] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.622071 22826 tablet_bootstrap.cc:492] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:00.624761 22826 ts_tablet_manager.cc:1403] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.196s	user 0.017s	sys 0.000s
I20260709 13:09:00.614483 22706 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:00.617151 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:00.607167 22824 raft_consensus.cc:359] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.613095 23065 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.609427 23163 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.596907 22634 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:00.626873 23163 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:00.625917 22824 raft_consensus.cc:385] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.627948 22824 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.627835 23163 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.629060 22824 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.631307 22824 ts_tablet_manager.cc:1434] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.045s	user 0.007s	sys 0.000s
I20260709 13:09:00.628186 23065 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.625944 22781 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.636303 22824 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:00.618325 22783 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:00.637207 22825 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:00.637693 22825 ts_tablet_manager.cc:1403] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.002s
I20260709 13:09:00.638384 22600 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.640503 23128 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.617100 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.640697 22825 raft_consensus.cc:359] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.641429 23074 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.617348 22779 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:00.642009 23074 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:00.643186 23074 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.648030 22750 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:00.649418 23157 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.650017 23157 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:00.652664 23157 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.653308 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:00.654057 22634 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.641501 22825 raft_consensus.cc:385] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.662068 22634 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:00.664520 22825 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.666898 22825 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.670706 23128 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:00.671835 23128 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.672577 22825 ts_tablet_manager.cc:1434] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:09:00.675341 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:00.677088 23065 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.677670 23065 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:00.678663 23065 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.688144 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:00.692893 23157 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:00.693642 22825 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:00.693554 23157 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:00.688753 22824 tablet_bootstrap.cc:654] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.634506 22826 raft_consensus.cc:359] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.699007 22826 raft_consensus.cc:385] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.699340 22826 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.688674 22781 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
W20260709 13:09:00.703858 22675 leader_election.cc:343] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.700140 22826 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.706985 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:00.711265 22533 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:00.718762 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:09:00.732846 22675 leader_election.cc:343] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.756057 22826 ts_tablet_manager.cc:1434] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.000s
I20260709 13:09:00.697721 23157 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.779261 23065 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:09:00.784159 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51366.1us
I20260709 13:09:00.816708 22826 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:00.802088 22824 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:00.828523 22824 ts_tablet_manager.cc:1403] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.192s	user 0.013s	sys 0.014s
I20260709 13:09:00.811625 23128 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.830919 23128 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.832998 23160 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.833577 23160 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.835793 23128 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.838357 22824 raft_consensus.cc:359] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.839108 22824 raft_consensus.cc:385] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.839390 22824 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.784787 22825 tablet_bootstrap.cc:654] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.840063 22824 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.843068 22826 tablet_bootstrap.cc:654] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.843784 23160 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.848734 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:00.849737 22781 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:00.859908 22533 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:00.857846 22530 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:00.817269 23065 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:00.865034 23065 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.868466 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:00.869489 22632 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:00.871775 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:00.872213 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:00.878718 22825 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:00.879292 22824 ts_tablet_manager.cc:1434] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 13:09:00.879338 22825 ts_tablet_manager.cc:1403] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.186s	user 0.012s	sys 0.013s
I20260709 13:09:00.883199 22531 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.881232 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:00.889911 22531 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.891599 23134 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.892262 22531 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:00.887744 22601 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:00.898305 23069 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:00.899148 23134 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:00.888230 22826 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:00.900034 22826 ts_tablet_manager.cc:1403] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.084s	user 0.013s	sys 0.000s
I20260709 13:09:00.903616 22826 raft_consensus.cc:359] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.904592 22826 raft_consensus.cc:385] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.904974 22826 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.905802 22826 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.909382 23069 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:00.912609 22826 ts_tablet_manager.cc:1434] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:09:00.914978 22824 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:00.915448 22826 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
W20260709 13:09:00.917470 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:00.926877 23023 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:09:00.933048 22825 raft_consensus.cc:359] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.935835 22825 raft_consensus.cc:385] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:00.936261 22825 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:00.936986 22825 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.946483 22530 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:00.949328 22825 ts_tablet_manager.cc:1434] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.060s	user 0.003s	sys 0.005s
I20260709 13:09:00.940028 22826 tablet_bootstrap.cc:654] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:00.950536 22825 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:00.970669 23069 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.971414 23069 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:00.976145 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:00.978438 22600 leader_election.cc:343] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.983942 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:00.985562 22601 leader_election.cc:343] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:00.988642 22826 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:00.989260 22826 ts_tablet_manager.cc:1403] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.008s
I20260709 13:09:00.988513 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:00.990768 23067 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:00.985612 23069 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:00.993774 22824 tablet_bootstrap.cc:654] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.001772 22826 raft_consensus.cc:359] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.002988 22826 raft_consensus.cc:385] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.003377 22826 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.004582 22826 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.007717 22825 tablet_bootstrap.cc:654] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.008139 22826 ts_tablet_manager.cc:1434] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:09:01.009373 22826 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:01.015722 22710 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.041584 23023 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.044508 22826 tablet_bootstrap.cc:654] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.065989 22825 tablet_bootstrap.cc:492] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:01.066874 22825 ts_tablet_manager.cc:1403] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.117s	user 0.012s	sys 0.013s
I20260709 13:09:01.059325 22824 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:01.067636 22824 ts_tablet_manager.cc:1403] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.177s	user 0.017s	sys 0.001s
I20260709 13:09:01.070008 22825 raft_consensus.cc:359] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.070495 22824 raft_consensus.cc:359] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.071211 22824 raft_consensus.cc:385] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.071529 22824 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.072698 22826 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:01.072501 22824 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.073251 22826 ts_tablet_manager.cc:1403] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.009s
I20260709 13:09:01.070894 22825 raft_consensus.cc:385] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.073854 22825 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.074777 22825 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.075309 22824 ts_tablet_manager.cc:1434] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:09:01.077217 22824 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:01.081163 22826 raft_consensus.cc:359] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.081933 22826 raft_consensus.cc:385] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.082266 22826 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.082523 22825 ts_tablet_manager.cc:1434] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:01.083156 22826 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.084054 22825 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:01.085628 22826 ts_tablet_manager.cc:1434] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:09:01.086282 22824 tablet_bootstrap.cc:654] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.086752 22826 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:01.089996 23134 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.093943 22825 tablet_bootstrap.cc:654] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.094245 22826 tablet_bootstrap.cc:654] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.096161 23157 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.096800 23157 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.099820 23157 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.103147 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:01.103787 22824 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:01.104424 22824 ts_tablet_manager.cc:1403] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 13:09:01.104809 22634 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:01.108156 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:01.110858 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:01.112936 23157 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.115933 22825 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:01.116521 22825 ts_tablet_manager.cc:1403] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.006s
I20260709 13:09:01.118175 22824 raft_consensus.cc:359] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.119068 22824 raft_consensus.cc:385] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.119400 22824 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.120289 22824 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.120764 22825 raft_consensus.cc:359] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.121531 22825 raft_consensus.cc:385] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.121811 22825 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.123127 22824 ts_tablet_manager.cc:1434] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:09:01.122855 22825 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.124361 22824 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:01.125120 22825 ts_tablet_manager.cc:1434] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:09:01.126747 22825 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:01.133862 22824 tablet_bootstrap.cc:654] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.139659 22825 tablet_bootstrap.cc:654] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.145282 23157 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.145637 23134 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:01.151696 23023 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.152606 23023 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.155057 23023 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.159574 22825 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:01.160537 22825 ts_tablet_manager.cc:1403] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.034s	user 0.018s	sys 0.003s
I20260709 13:09:01.167517 22710 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.184716 22825 raft_consensus.cc:359] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.186139 22825 raft_consensus.cc:385] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.186666 22825 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.186549 23160 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.190673 22825 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.191380 23160 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.194272 22825 ts_tablet_manager.cc:1434] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:09:01.196043 22825 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:01.216054 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:01.226841 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:01.228101 22750 leader_election.cc:343] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.228147 22635 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:01.232105 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:01.194545 23160 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:01.222524 23067 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 13:09:01.253332 23023 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:01.216959 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:01.281008 22634 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:01.283542 23154 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.284098 23154 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.280817 23155 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.286396 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:01.305517 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:01.303850 23154 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.309857 22942 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:01.316995 23155 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.317521 23155 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.317883 23155 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.318869 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:01.322206 23160 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.322710 23160 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.323060 23160 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.318987 23134 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:01.342972 22749 leader_election.cc:343] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.343307 23162 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.343891 23162 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.353387 23155 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.360546 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:01.370088 22749 leader_election.cc:343] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.346405 23162 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.374105 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:01.381970 22634 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.388811 22826 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:01.390357 22826 ts_tablet_manager.cc:1403] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.304s	user 0.132s	sys 0.075s
I20260709 13:09:01.390511 22825 tablet_bootstrap.cc:700] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent opening tablet: real 0.114s	user 0.005s	sys 0.003s
I20260709 13:09:01.393983 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:01.394862 23156 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.395394 23156 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.400291 22825 tablet_bootstrap.cc:654] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.372900 23155 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.405923 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:01.420562 23155 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:09:01.431537 23160 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.433746 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:01.440062 23156 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.437577 22826 raft_consensus.cc:359] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.444370 22826 raft_consensus.cc:385] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.444793 22826 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.445384 22826 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.399253 23162 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.048s
I20260709 13:09:01.449769 22824 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:01.454066 22824 ts_tablet_manager.cc:1403] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.330s	user 0.058s	sys 0.085s
I20260709 13:09:01.456454 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:01.460552 22634 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:01.466523 23160 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:01.466841 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:01.469148 22826 ts_tablet_manager.cc:1434] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.003s
I20260709 13:09:01.489336 22824 raft_consensus.cc:359] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.490495 22824 raft_consensus.cc:385] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.491014 22824 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.492870 22824 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.504830 22826 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:01.506565 22635 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:01.476596 23155 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.509232 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:01.513305 22824 ts_tablet_manager.cc:1434] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.059s	user 0.010s	sys 0.000s
I20260709 13:09:01.515270 22824 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:01.518759 23074 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:01.528318 23074 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.528784 23074 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.516578 23155 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:01.537328 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:01.544077 22826 tablet_bootstrap.cc:654] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:01.544859 23155 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.550974 22825 tablet_bootstrap.cc:492] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:01.551640 22825 ts_tablet_manager.cc:1403] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.356s	user 0.028s	sys 0.014s
W20260709 13:09:01.554497 22675 leader_election.cc:343] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.556458 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:01.547950 22750 leader_election.cc:343] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.557516 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:01.559864 22750 leader_election.cc:343] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.560434 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:01.570427 23154 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 13:09:01.561857 23023 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.568744 23001 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:01.604784 22825 raft_consensus.cc:359] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.605983 22825 raft_consensus.cc:385] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.606383 22825 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.607950 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:01.609177 22629 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:01.609421 22825 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.613853 22825 ts_tablet_manager.cc:1434] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 13:09:01.562047 23161 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.037s
I20260709 13:09:01.573051 23163 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
W20260709 13:09:01.621285 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51499.7us
W20260709 13:09:01.627257 22750 leader_election.cc:343] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:01.631772 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57798.1us
W20260709 13:09:01.631948 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60770.3us
I20260709 13:09:01.659032 22825 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
W20260709 13:09:01.664386 23205 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:09:01.621399 23065 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
W20260709 13:09:01.674782 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47568.5us
W20260709 13:09:01.690461 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56488.3us
I20260709 13:09:01.713397 22824 tablet_bootstrap.cc:654] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:01.728829 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51400.8us
W20260709 13:09:01.743047 22750 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:01.744168 22750 leader_election.cc:343] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:01.752856 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58556us
W20260709 13:09:01.765827 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 116984us
W20260709 13:09:01.761220 23205 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:09:01.773552 22675 leader_election.cc:343] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:01.814458 22825 tablet_bootstrap.cc:654] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:01.821501 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51563.1us
I20260709 13:09:01.833714 22629 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:01.855477 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:01.856271 22826 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:01.856499 23211 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:01.856828 22826 ts_tablet_manager.cc:1403] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.364s	user 0.009s	sys 0.005s
I20260709 13:09:01.857079 23211 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:01.867219 22824 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:01.867831 22824 ts_tablet_manager.cc:1403] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.353s	user 0.023s	sys 0.007s
I20260709 13:09:01.889158 23074 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.888247 22826 raft_consensus.cc:359] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.890539 22826 raft_consensus.cc:385] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:01.890974 22826 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:01.894439 22826 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.909265 22826 ts_tablet_manager.cc:1434] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.003s
I20260709 13:09:01.945485 23074 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:01.971276 22826 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:01.875949 23211 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.966044 23156 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.975003 23065 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.993597 23065 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.009490 22825 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.010162 22825 ts_tablet_manager.cc:1403] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.353s	user 0.025s	sys 0.021s
I20260709 13:09:02.050597 22824 raft_consensus.cc:359] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.051543 22824 raft_consensus.cc:385] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.051965 22824 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.055403 22824 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:01.995841 23156 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.072336 22824 ts_tablet_manager.cc:1434] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.204s	user 0.013s	sys 0.000s
W20260709 13:09:02.080554 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:02.079641 22825 raft_consensus.cc:359] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.082298 22825 raft_consensus.cc:385] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.082706 22825 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.083391 22825 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.088426 22824 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:02.109392 23065 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:02.110560 23224 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:09:02.115211 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.125075 22825 ts_tablet_manager.cc:1434] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.115s	user 0.004s	sys 0.013s
I20260709 13:09:02.116176 22634 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:02.132139 22825 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.135638 23156 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:02.128607 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:02.149515 23208 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:02.166682 23208 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.166766 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:02.167227 23208 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.173187 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:02.174594 22629 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.186220 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.187608 22633 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:02.192075 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:02.192875 22782 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:02.205294 22826 tablet_bootstrap.cc:654] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:02.204769 22750 leader_election.cc:343] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.207605 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:02.214875 23154 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.215525 23154 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.217769 23154 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:02.222512 22825 tablet_bootstrap.cc:654] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.210408 23209 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.245039 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:02.247217 23231 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.245276 23209 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.249751 22629 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:02.253054 23209 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.254513 23230 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:02.264374 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:02.268185 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:02.271430 23207 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.272171 23207 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:02.285560 22824 tablet_bootstrap.cc:654] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:02.285794 22750 leader_election.cc:343] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.298156 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.301589 23231 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:02.302050 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:02.302886 22784 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:02.276849 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.305147 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.306767 23161 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:02.307415 23161 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.307904 23161 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:02.308100 22750 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:02.311913 22675 leader_election.cc:343] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.332760 22533 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:02.339802 22749 leader_election.cc:343] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.340368 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:02.353377 23207 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.360574 23230 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:02.344429 23208 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.387589 22825 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.388172 22825 ts_tablet_manager.cc:1403] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.256s	user 0.022s	sys 0.049s
I20260709 13:09:02.444397 23161 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.446977 22826 tablet_bootstrap.cc:492] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:02.447582 22826 ts_tablet_manager.cc:1403] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.484s	user 0.020s	sys 0.022s
I20260709 13:09:02.448114 22825 raft_consensus.cc:359] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.450855 22825 raft_consensus.cc:385] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.451143 22825 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.456934 22825 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.460893 23208 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.467224 22825 ts_tablet_manager.cc:1434] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 13:09:02.475301 22826 raft_consensus.cc:359] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.476235 22826 raft_consensus.cc:385] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.476583 22826 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.477824 22826 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.479216 22825 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.481503 22826 ts_tablet_manager.cc:1434] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:09:02.494975 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:02.496655 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:02.501235 22784 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.499446 23161 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.506861 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:02.507855 22630 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.516798 22824 tablet_bootstrap.cc:492] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:02.517618 22824 ts_tablet_manager.cc:1403] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.433s	user 0.019s	sys 0.011s
I20260709 13:09:02.526321 22784 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
W20260709 13:09:02.530903 22675 leader_election.cc:343] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.531008 22826 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:02.531057 22824 raft_consensus.cc:359] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.532124 22824 raft_consensus.cc:385] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.532403 22824 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.533121 22824 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.534348 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:02.535084 22784 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.536079 22704 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.539013 22825 tablet_bootstrap.cc:654] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.541378 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.542948 23208 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.543488 23208 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:02.544504 23208 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.545292 22824 ts_tablet_manager.cc:1434] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.003s
I20260709 13:09:02.546518 22824 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:02.536082 22630 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:02.550949 23134 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.560197 22784 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:02.561584 22825 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.561771 22826 tablet_bootstrap.cc:654] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.563958 22824 tablet_bootstrap.cc:654] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.572683 22825 ts_tablet_manager.cc:1403] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.102s	user 0.008s	sys 0.009s
I20260709 13:09:02.574426 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:02.576083 23208 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.576257 22825 raft_consensus.cc:359] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.577054 22825 raft_consensus.cc:385] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.577360 22825 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.576694 23208 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:02.578258 22825 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.581084 22825 ts_tablet_manager.cc:1434] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:02.578363 23208 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.582178 22825 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.585500 23160 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.591133 22825 tablet_bootstrap.cc:654] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.604672 22824 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:02.605341 22824 ts_tablet_manager.cc:1403] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.059s	user 0.010s	sys 0.009s
I20260709 13:09:02.608475 22826 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:02.608501 22824 raft_consensus.cc:359] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.609289 22826 ts_tablet_manager.cc:1403] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.091s	user 0.008s	sys 0.013s
I20260709 13:09:02.609477 22824 raft_consensus.cc:385] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.609791 22824 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.610736 22824 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.612334 22826 raft_consensus.cc:359] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.613051 22826 raft_consensus.cc:385] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.613373 22824 ts_tablet_manager.cc:1434] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:02.613492 22826 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.613142 22531 catalog_manager.cc:5697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.614583 22824 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:02.614283 22826 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.617697 22825 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.618328 22825 ts_tablet_manager.cc:1403] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.008s
I20260709 13:09:02.617699 22826 ts_tablet_manager.cc:1434] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:09:02.621307 22825 raft_consensus.cc:359] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.623303 22826 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:02.626303 22824 tablet_bootstrap.cc:654] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.623879 22825 raft_consensus.cc:385] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.628450 22825 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.629276 22825 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.631770 22825 ts_tablet_manager.cc:1434] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:09:02.636929 22826 tablet_bootstrap.cc:654] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.658715 23206 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.670104 23207 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.670998 23207 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.675236 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.676187 22632 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:02.677942 22749 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.679090 23232 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:02.679313 23207 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:02.679567 23232 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.680187 23232 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.680673 22531 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.682977 22531 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:02.696316 22825 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.702667 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:02.709179 22750 leader_election.cc:343] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.659366 23206 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.737409 22824 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:02.739598 23206 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:02.740373 22824 ts_tablet_manager.cc:1403] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.126s	user 0.008s	sys 0.011s
I20260709 13:09:02.744331 22825 tablet_bootstrap.cc:654] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.749488 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:02.749913 22826 tablet_bootstrap.cc:492] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:02.750469 22826 ts_tablet_manager.cc:1403] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.127s	user 0.019s	sys 0.016s
I20260709 13:09:02.751005 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.753703 22826 raft_consensus.cc:359] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.754482 22826 raft_consensus.cc:385] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.754812 22826 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.755505 22826 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:02.756163 22749 leader_election.cc:343] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.757177 22826 ts_tablet_manager.cc:1434] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:02.757409 23232 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:02.757795 22750 leader_election.cc:343] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.758350 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:02.759572 23208 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.760015 23154 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:02.760072 23208 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.762326 23208 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.764081 22824 raft_consensus.cc:359] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.758311 22826 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:02.769359 23232 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:02.770541 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:02.771265 22703 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.775970 22826 tablet_bootstrap.cc:654] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.780565 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:02.781379 22780 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:02.781304 22703 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:02.785403 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:02.786077 22630 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.788906 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.789479 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.796697 22824 raft_consensus.cc:385] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.767681 22825 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.797598 23219 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.765874 22533 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.798171 23219 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:02.799057 23219 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.789753 23208 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:02.800277 23208 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.800681 23208 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.805817 22826 tablet_bootstrap.cc:492] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:02.806420 22826 ts_tablet_manager.cc:1403] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.011s
I20260709 13:09:02.809499 22826 raft_consensus.cc:359] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.810277 22826 raft_consensus.cc:385] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.810567 22826 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.811304 22826 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.806430 22630 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:02.818468 22826 ts_tablet_manager.cc:1434] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:09:02.821238 22826 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:02.821853 23208 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.824194 23208 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.797115 22824 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.797739 22825 ts_tablet_manager.cc:1403] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.102s	user 0.011s	sys 0.002s
I20260709 13:09:02.825315 22824 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.827878 22824 ts_tablet_manager.cc:1434] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.086s	user 0.006s	sys 0.005s
I20260709 13:09:02.827977 22825 raft_consensus.cc:359] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.828856 22825 raft_consensus.cc:385] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.829063 22824 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:02.829201 22825 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.768553 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:02.831156 22826 tablet_bootstrap.cc:654] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.838052 22824 tablet_bootstrap.cc:654] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.840633 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:02.841367 22780 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.841413 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:02.852504 22675 leader_election.cc:343] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:02.854710 22675 leader_election.cc:343] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:02.853947 22825 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.861410 22825 ts_tablet_manager.cc:1434] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:09:02.862185 23154 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:02.876459 22825 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.891251 22826 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:02.892271 22826 ts_tablet_manager.cc:1403] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.071s	user 0.026s	sys 0.025s
I20260709 13:09:02.888655 22530 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:02.899191 22825 tablet_bootstrap.cc:654] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.904014 22824 tablet_bootstrap.cc:492] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:02.904652 22824 ts_tablet_manager.cc:1403] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.002s
I20260709 13:09:02.884639 22780 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:02.913769 22824 raft_consensus.cc:359] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.914481 22824 raft_consensus.cc:385] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.914791 22824 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.915503 22824 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.918002 22824 ts_tablet_manager.cc:1434] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:09:02.918507 22826 raft_consensus.cc:359] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.919325 22826 raft_consensus.cc:385] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.920761 22826 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.920337 22824 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:02.921551 22826 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.924162 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:02.925305 23208 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:02.925849 23208 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:02.926728 23208 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.927316 22824 tablet_bootstrap.cc:654] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:02.933290 22825 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:02.933800 22825 ts_tablet_manager.cc:1403] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.007s
I20260709 13:09:02.936733 22825 raft_consensus.cc:359] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.937508 22825 raft_consensus.cc:385] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:02.937772 22825 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:02.938498 22825 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.945004 22703 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.945529 22825 ts_tablet_manager.cc:1434] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:09:02.949790 22826 ts_tablet_manager.cc:1434] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:09:02.952719 22826 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:02.958088 23160 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:09:02.977250 23067 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.977826 23067 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.978641 22825 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:02.982951 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:02.983839 22784 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:09:02.985692 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:02.986443 22704 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:09:02.987673 22600 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:02.989507 23203 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.990197 23203 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:02.990640 22942 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:02.991110 22942 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.991433 22942 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:02.993078 23203 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:02.994968 23209 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.995383 23231 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:02.995512 23209 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.007095 23209 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.007157 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:02.989344 23067 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.009545 22782 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
I20260709 13:09:03.008828 23203 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.013432 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:03.016131 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:03.013545 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:03.011765 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:03.047466 23231 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.048046 23231 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.034402 23217 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:03.038381 22942 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:03.051936 22601 leader_election.cc:343] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:03.050303 23217 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.054678 23217 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.052186 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:03.056375 22749 leader_election.cc:343] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:03.057405 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:03.057569 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:03.058084 22706 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.054283 22826 tablet_bootstrap.cc:654] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.012220 23203 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.068257 23208 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:03.076165 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:03.077610 23231 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:03.077316 22942 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.084785 23203 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.088105 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:03.097770 22750 leader_election.cc:343] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:03.101616 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:03.107033 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:03.107591 22784 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:03.107788 22782 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.109593 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:03.111366 22632 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.112026 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:03.113324 23231 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:03.116107 22531 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:03.136497 22942 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:03.138321 22706 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:03.144203 23208 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:03.144719 22824 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:03.145244 22824 ts_tablet_manager.cc:1403] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.225s	user 0.016s	sys 0.018s
I20260709 13:09:03.147074 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:03.153700 23217 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.160089 23217 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.161180 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:03.161118 22824 raft_consensus.cc:359] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.161952 22824 raft_consensus.cc:385] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.162274 22824 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.161962 22780 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.161247 22942 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:03.163424 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:03.165928 22942 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:03.167165 22601 leader_election.cc:343] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:03.166894 22942 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.164556 23134 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:03.175052 22824 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.173243 22782 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:03.222548 23203 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.226325 23203 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.228611 23203 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:03.229296 22824 ts_tablet_manager.cc:1434] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.084s	user 0.012s	sys 0.011s
I20260709 13:09:03.173261 22780 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
W20260709 13:09:03.248879 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:03.251263 22824 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:03.254410 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:03.263489 23069 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:03.264101 23069 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:03.265062 23069 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.271512 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:03.272531 22776 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 1: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:03.276566 22825 tablet_bootstrap.cc:654] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.278301 23231 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:03.314833 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60597.6us
I20260709 13:09:03.320981 22826 tablet_bootstrap.cc:492] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:03.321597 22826 ts_tablet_manager.cc:1403] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.369s	user 0.017s	sys 0.025s
I20260709 13:09:03.325562 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:03.332329 23160 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:09:03.349860 22533 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:03.477881 23231 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:09:03.501953 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98960us
W20260709 13:09:03.559883 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74842.5us
I20260709 13:09:03.561719 22826 raft_consensus.cc:359] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.562587 22826 raft_consensus.cc:385] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:03.562940 22826 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:03.563673 22826 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.644907 22826 ts_tablet_manager.cc:1434] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.323s	user 0.011s	sys 0.031s
W20260709 13:09:03.665486 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66730.5us
I20260709 13:09:03.665800 22826 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:03.652961 23203 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:03.693887 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67123.1us
I20260709 13:09:03.677121 23203 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:03.773382 22826 tablet_bootstrap.cc:654] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.775161 23203 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:03.787962 23282 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:09:03.815968 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:03.816838 22778 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 0.
W20260709 13:09:03.819487 23278 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:09:03.827346 22824 tablet_bootstrap.cc:654] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:03.848682 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:03.852515 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 181986us
I20260709 13:09:03.899456 22825 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:03.900156 22825 ts_tablet_manager.cc:1403] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.937s	user 0.026s	sys 0.021s
I20260709 13:09:03.910189 23275 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:03.919884 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:03.881191 23291 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:09:03.923671 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53713.2us
I20260709 13:09:03.932883 22824 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:03.933470 22824 ts_tablet_manager.cc:1403] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.684s	user 0.030s	sys 0.014s
I20260709 13:09:03.937593 22826 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:03.938167 22826 ts_tablet_manager.cc:1403] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.275s	user 0.026s	sys 0.013s
I20260709 13:09:03.931375 23272 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:03.931421 23291 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:03.962895 23134 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.013761 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79582.4us
I20260709 13:09:03.973429 23291 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:03.997377 23284 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:09:04.032076 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:04.008811 23134 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.039727 22710 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:04.040796 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45050.6us
W20260709 13:09:04.045241 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 114665us
I20260709 13:09:04.054503 22825 raft_consensus.cc:359] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.055419 22825 raft_consensus.cc:385] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.055711 22825 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.057989 22826 raft_consensus.cc:359] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.058795 22826 raft_consensus.cc:385] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.059059 22826 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.059588 22826 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.063807 23134 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:04.056501 22825 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.104090 22826 ts_tablet_manager.cc:1434] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.166s	user 0.015s	sys 0.014s
I20260709 13:09:04.124918 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:04.137719 22824 raft_consensus.cc:359] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.139237 22824 raft_consensus.cc:385] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.139773 22824 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.142112 22824 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.146873 22825 ts_tablet_manager.cc:1434] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.246s	user 0.006s	sys 0.007s
I20260709 13:09:04.153700 22824 ts_tablet_manager.cc:1434] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.220s	user 0.009s	sys 0.003s
I20260709 13:09:04.179204 22825 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Bootstrap starting.
I20260709 13:09:04.186424 23279 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 13:09:04.225656 22824 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:04.273203 23286 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:04.274004 23286 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.288971 23291 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:04.304152 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 144519us
I20260709 13:09:04.288259 23275 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.306746 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 238244us
I20260709 13:09:04.307303 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:04.326996 22826 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:04.363312 22826 tablet_bootstrap.cc:654] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
W20260709 13:09:04.422035 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 104205us
I20260709 13:09:04.316115 23272 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:04.316992 23275 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:04.546586 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63863.9us
W20260709 13:09:04.546999 22601 leader_election.cc:343] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.482697 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:04.558056 23291 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:04.559518 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:04.571259 22707 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.572113 23293 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:09:04.562563 23219 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.575742 23219 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.578123 23219 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:04.562341 22601 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:04.583158 22825 tablet_bootstrap.cc:700] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent opening tablet: real 0.103s	user 0.005s	sys 0.003s
I20260709 13:09:04.484835 22530 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:04.484138 23286 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:04.599118 22825 tablet_bootstrap.cc:654] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.673942 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:04.687716 23317 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:09:04.706501 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112222us
I20260709 13:09:04.717638 23275 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:04.718504 23326 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.719350 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:04.720057 22778 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:04.722747 22824 tablet_bootstrap.cc:654] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:04.725244 22707 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:04.719151 23326 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.760751 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:04.765313 22749 leader_election.cc:343] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.775117 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:04.779948 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:04.766311 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:04.784318 23299 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:04.760659 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:04.790189 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 188649us
I20260709 13:09:04.795040 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:04.809897 23295 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 13:09:04.786118 23160 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:04.789810 23326 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:04.831126 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:04.836843 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:04.848664 22778 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:04.886229 22826 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:04.887161 22826 ts_tablet_manager.cc:1403] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.560s	user 0.016s	sys 0.006s
W20260709 13:09:04.893234 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76347.7us
I20260709 13:09:04.895372 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:09:04.896718 22601 leader_election.cc:343] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:04.957934 22825 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: No bootstrap required, opened a new log
I20260709 13:09:04.958633 22825 ts_tablet_manager.cc:1403] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent bootstrapping tablet: real 0.795s	user 0.021s	sys 0.038s
I20260709 13:09:04.964046 22824 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:04.964617 22824 ts_tablet_manager.cc:1403] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.786s	user 0.020s	sys 0.022s
I20260709 13:09:04.962963 23275 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:04.973909 23275 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:04.975224 23275 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.988291 22824 raft_consensus.cc:359] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.989202 22824 raft_consensus.cc:385] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.989494 22824 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.966118 22826 raft_consensus.cc:359] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.993009 22826 raft_consensus.cc:385] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.993345 22826 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.991881 22825 raft_consensus.cc:359] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.995054 22825 raft_consensus.cc:385] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:04.995361 22825 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Initialized, Role: FOLLOWER
I20260709 13:09:04.996043 22825 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:04.994134 22826 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.000118 22826 ts_tablet_manager.cc:1434] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.113s	user 0.007s	sys 0.000s
I20260709 13:09:04.990229 22824 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.044056 22825 ts_tablet_manager.cc:1434] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent starting tablet: real 0.085s	user 0.007s	sys 0.000s
I20260709 13:09:05.046878 22824 ts_tablet_manager.cc:1434] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.002s
W20260709 13:09:05.090260 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 274218us
W20260709 13:09:05.102126 22750 leader_election.cc:343] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:09:05.104665 22750 leader_election.cc:343] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:05.105373 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.116606 22824 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Bootstrap starting.
I20260709 13:09:05.133474 23219 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:05.172855 23308 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:05.173547 23134 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:05.173516 23308 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.174299 23134 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.197760 23134 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:05.198050 23308 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:05.252444 23337 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.253219 23337 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.261722 23337 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:05.280007 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 191655us
I20260709 13:09:05.285317 23342 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:05.288616 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.286026 23342 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.313340 22826 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:05.314110 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:05.332051 22600 leader_election.cc:343] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:05.332629 22600 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.351039 23295 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.355687 23295 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.353294 22826 tablet_bootstrap.cc:654] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.378170 23342 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:05.383602 23347 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:05.361032 23281 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:09:05.395337 23203 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:05.398048 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.352910 23067 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:05.399344 23067 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.402151 23067 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:05.360157 23295 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:05.394555 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62523.8us
I20260709 13:09:05.456228 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:05.467971 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 104095us
I20260709 13:09:05.471813 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.471990 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 97) took 1133 ms. Trace:
I20260709 13:09:05.472656 22631 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:05.474253 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.475404 23349 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:05.479631 22824 tablet_bootstrap.cc:654] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.472538 22530 rpcz_store.cc:276] 0709 13:09:04.339033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.346739 (+  7706us) service_pool.cc:224] Handling call
0709 13:09:05.471875 (+1125136us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:04.719001 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:04.719644 (+   643us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:04.719688 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:09:04.719705 (+    17us) tablet.cc:662] Decoding operations
0709 13:09:04.720096 (+   391us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:04.720163 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:04.720196 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:04.720474 (+   278us) tablet.cc:701] Row locks acquired
0709 13:09:04.720496 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:09:04.720700 (+   204us) write_op.cc:270] Start()
0709 13:09:04.720811 (+   111us) write_op.cc:276] Timestamp: P: 1783602544720644 usec, L: 0
0709 13:09:04.720828 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:09:04.721550 (+   722us) log.cc:844] Serialized 537 byte log entry
0709 13:09:05.299314 (+577764us) op_driver.cc:464] REPLICATION: finished
0709 13:09:05.324384 (+ 25070us) write_op.cc:301] APPLY: starting
0709 13:09:05.324470 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:05.324585 (+   115us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:05.324709 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:05.324999 (+   290us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:05.325175 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:05.325227 (+    52us) write_op.cc:312] APPLY: finished
0709 13:09:05.325483 (+   256us) log.cc:844] Serialized 22 byte log entry
0709 13:09:05.326346 (+   863us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:05.326437 (+    91us) write_op.cc:454] Released schema lock
0709 13:09:05.326477 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:05.326870 (+   393us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:05.326899 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3051,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":3973,"apply.run_cpu_time_us":1837,"apply.run_wall_time_us":2506,"num_ops":1,"prepare.queue_time_us":118444,"prepare.run_cpu_time_us":74955,"prepare.run_wall_time_us":165996,"raft.queue_time_us":327586,"raft.run_cpu_time_us":3970,"raft.run_wall_time_us":43435,"replication_time_us":560002,"thread_start_us":518670,"threads_started":4,"wal-append.queue_time_us":190583}]]}
I20260709 13:09:05.482534 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:05.464596 22826 tablet_bootstrap.cc:492] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:05.483443 22826 ts_tablet_manager.cc:1403] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.170s	user 0.014s	sys 0.001s
I20260709 13:09:05.483750 23349 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:05.484175 23349 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.484541 23349 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:09:05.487619 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 134369us
I20260709 13:09:05.524513 22533 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.495965 23349 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.527916 23349 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:05.500414 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:05.529460 22749 leader_election.cc:343] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:05.529923 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.514072 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:05.488900 22750 leader_election.cc:343] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:05.532537 23337 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:09:05.532660 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43814.4us
W20260709 13:09:05.492635 22581 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:09:05.502455 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:05.496513 23308 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:05.536283 23308 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.537139 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:05.537770 22628 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.538574 23308 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:05.540720 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.542428 22778 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:05.486423 22826 raft_consensus.cc:359] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.547322 22826 raft_consensus.cc:385] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.547649 22826 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.548413 22826 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.552685 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:05.560205 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.564074 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:05.566498 22780 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.573958 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.578601 23275 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:05.578999 23276 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:05.582263 23346 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.582916 23346 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.589898 22826 ts_tablet_manager.cc:1434] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.000s
I20260709 13:09:05.598330 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.609030 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.610597 23346 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:05.612377 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.614127 22826 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:05.635871 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.636502 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.637495 22626 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 0.
I20260709 13:09:05.637805 23346 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:05.659005 22705 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.662672 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.672824 22600 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.683275 22600 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.688632 23293 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:05.695634 22628 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:05.703353 22749 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.704181 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:05.705363 23309 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:05.705802 23309 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.706770 23309 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.707885 23280 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:05.734088 22780 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:05.745051 23309 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.747484 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.748368 23232 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:05.756448 22824 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: No bootstrap required, opened a new log
I20260709 13:09:05.757076 22824 ts_tablet_manager.cc:1403] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent bootstrapping tablet: real 0.641s	user 0.045s	sys 0.011s
I20260709 13:09:05.760133 22824 raft_consensus.cc:359] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.760883 22824 raft_consensus.cc:385] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.761190 22824 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.763427 23342 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:05.774214 23208 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:05.774950 23208 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:09:05.775326 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 71239.4us
I20260709 13:09:05.776062 23208 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.797698 22826 tablet_bootstrap.cc:654] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.812043 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:05.813231 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:05.814008 22629 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.761893 22824 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.823401 22824 ts_tablet_manager.cc:1434] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.002s
I20260709 13:09:05.823933 23309 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:05.824790 22629 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:05.839490 22826 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
W20260709 13:09:05.841149 22750 leader_election.cc:343] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:09:05.842711 22826 ts_tablet_manager.cc:1403] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.239s	user 0.018s	sys 0.002s
I20260709 13:09:05.845922 22826 raft_consensus.cc:359] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.849637 22826 raft_consensus.cc:385] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.850004 22826 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.850731 22826 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.846666 22749 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:05.835572 22533 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:05.854974 23276 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:05.855551 23276 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:05.856426 23276 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.858137 22826 ts_tablet_manager.cc:1434] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:09:05.859357 22826 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Bootstrap starting.
I20260709 13:09:05.869994 22826 tablet_bootstrap.cc:654] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Neither blocks nor log segments found. Creating new log.
I20260709 13:09:05.885876 22826 tablet_bootstrap.cc:492] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: No bootstrap required, opened a new log
I20260709 13:09:05.886448 22826 ts_tablet_manager.cc:1403] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.003s
I20260709 13:09:05.886221 22530 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.889605 22826 raft_consensus.cc:359] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.890383 22826 raft_consensus.cc:385] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:09:05.890731 22826 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Initialized, Role: FOLLOWER
I20260709 13:09:05.891515 22826 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.893517 22826 ts_tablet_manager.cc:1434] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:09:05.974960 23208 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.975499 23208 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.977506 23208 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:05.978957 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:05.979012 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.980428 22629 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 1: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:05.982060 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:05.982932 23208 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:05.989108 23208 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:09:05.995303 23208 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.995795 23208 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:05.998533 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:05.999869 22629 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:05.999127 23208 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:06.000662 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:06.001362 22778 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 0.
I20260709 13:09:06.003016 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:06.003767 23295 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:09:06.004148 23295 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.004475 23295 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.012692 23295 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:06.018607 23295 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:06.019260 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:06.019964 22629 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.020570 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:06.021206 22780 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:09:06.029322 22780 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:06.032106 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:06.032936 23295 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:09:06.037894 23295 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:06.039121 22629 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:06.040496 23295 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:06.062760 22530 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:06.250728 23276 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:06.275261 23219 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:06.296618 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:09:06.331363 23369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:09:06.334676 23370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:06.339264 23289 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:06.344632 21526 server_base.cc:1061] running on GCE node
W20260709 13:09:06.346248 23372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:09:06.348374 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:09:06.348745 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:09:06.348994 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602546348976 us; error 0 us; skew 500 ppm
I20260709 13:09:06.349895 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:09:06.358920 21526 webserver.cc:533] Webserver started at http://127.21.5.132:34779/ using document root <none> and password file <none>
I20260709 13:09:06.366578 23295 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:06.369272 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:09:06.369855 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:09:06.370478 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:09:06.372532 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-3-root/instance:
uuid: "6a31ac6451ff4f73bee634487d84d001"
format_stamp: "Formatted at 2026-07-09 13:09:06 on dist-test-slave-h0rz"
I20260709 13:09:06.411829 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.032s	user 0.005s	sys 0.002s
I20260709 13:09:06.422989 23381 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:06.425053 21526 fs_manager.cc:730] Time spent opening block manager: real 0.009s	user 0.005s	sys 0.002s
I20260709 13:09:06.425655 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
uuid: "6a31ac6451ff4f73bee634487d84d001"
format_stamp: "Formatted at 2026-07-09 13:09:06 on dist-test-slave-h0rz"
I20260709 13:09:06.426177 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:09:06.453872 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:09:06.456096 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:09:06.458858 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:09:06.461985 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:09:06.462275 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:09:06.462574 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:09:06.462872 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:09:06.501505 23378 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:09:06.537598 23208 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:09:06.698176 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.132:36877
I20260709 13:09:06.698369 23445 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.132:36877 every 8 connection(s)
I20260709 13:09:06.706369 21526 auto_leader_rebalancer-test.cc:384] add a tserver: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:06.726225 23446 heartbeater.cc:344] Connected to a master server at 127.21.5.190:44025
I20260709 13:09:06.726794 23446 heartbeater.cc:461] Registering TS with master...
I20260709 13:09:06.727706 23446 heartbeater.cc:507] Master 127.21.5.190:44025 requested a full tablet report, sending...
I20260709 13:09:06.730391 22533 ts_manager.cc:194] Registered new tserver with Master: 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:06.732136 22533 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43414
I20260709 13:09:07.837327 23295 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.882828 23295 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.887526 23208 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.888833 23208 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.903098 23208 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.903510 23295 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.903883 23208 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.911720 23208 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.913813 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:07.914526 23208 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.915402 23452 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.914140 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:07.916235 23452 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.920964 23289 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:07.923990 23464 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.922287 23289 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.926069 23208 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.927189 23208 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.933465 23208 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.935005 23453 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:07.935900 23453 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.945778 23461 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:07.946651 23461 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.938936 23280 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:07.938942 23289 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.949389 23280 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.952572 23464 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.940526 23452 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.954074 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:07.999035 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.000566 23458 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.001598 23458 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.954766 23462 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.003129 23462 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:07.976475 23208 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.975577 23313 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:07.997167 23453 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:07.951601 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.006419 23466 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:08.012147 22656 debug-util.cc:398] Leaking SignalData structure 0x7b080014db20 after lost signal to thread 21529
I20260709 13:09:08.013361 23466 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.014719 23453 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.016022 23453 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:09:08.016433 23208 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.019022 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.019971 23466 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.022704 23453 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.007334 23461 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.073338 23208 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.066598 23295 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.081743 23378 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.086421 23378 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.090545 23458 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.091534 23470 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.092180 23470 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.095278 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.095522 23446 heartbeater.cc:499] Master 127.21.5.190:44025 was elected leader, sending a full tablet report...
I20260709 13:09:08.096616 23471 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.097519 23471 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.098079 23464 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.078799 23313 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.100576 23280 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.104048 22533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 116) took 1170 ms. Trace:
I20260709 13:09:08.104745 22533 rpcz_store.cc:276] 0709 13:09:06.933649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:06.933839 (+   190us) service_pool.cc:224] Handling call
0709 13:09:08.103739 (+1169900us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:09:08.106165 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.112672 23458 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.107314 23464 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.113564 23458 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.113829 23208 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.113729 23464 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.112246 23462 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.114694 23208 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.116182 23458 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.116842 23458 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.106047 23295 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.117738 23458 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.109742 23280 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.118404 23452 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.116742 23464 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.112838 23473 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.120986 23452 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.118893 23280 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.117069 23208 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.126482 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.127028 23313 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.127713 23313 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.129463 23462 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.130182 23462 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.128288 23313 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.133584 23462 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.134021 23313 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.135808 23378 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.136080 23452 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:08.137475 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08000c5900 after lost signal to thread 22567
I20260709 13:09:08.138506 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:08.139254 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2040 after lost signal to thread 22659
I20260709 13:09:08.138448 23378 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.140388 23378 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.142136 23455 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.142840 23378 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.142947 23455 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.148551 23455 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:08.153777 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002bece0 after lost signal to thread 22734
I20260709 13:09:08.157224 23458 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.160684 23470 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.107332 23469 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.162343 23469 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:08.163913 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9200 after lost signal to thread 22808
I20260709 13:09:08.163668 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.165894 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.166391 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.168637 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.126360 23468 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.167281 23473 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.169270 23455 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.177114 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.177580 23470 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.178022 23462 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.178447 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.178707 23462 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.173022 23468 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.178148 23470 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.182844 23468 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.177047 23464 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.206234 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.209002 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.209728 22626 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:08.180440 23455 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.212380 23455 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.212632 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.165902 23452 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.214525 23452 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.187450 23462 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.200960 23471 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.201557 23208 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:08.196718 23378 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.217216 23208 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.217437 23378 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.219358 23208 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.202064 23280 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.202452 23295 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:08.201749 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4b00 after lost signal to thread 23445
I20260709 13:09:08.176457 23458 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.206046 23464 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } }
I20260709 13:09:08.204351 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.220629 23458 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.222417 23464 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.209851 23451 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.222949 23458 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.219409 23378 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.185998 23469 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.235417 23470 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.185268 23473 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.213666 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.204216 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.223308 23451 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.254155 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.227247 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.228506 23452 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.229275 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.230934 23471 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.231691 23458 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.267326 23458 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.233999 23472 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.246003 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.247174 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.254155 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.257141 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.257606 23451 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.258605 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.261960 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.275194 22631 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:08.265092 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.266235 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.270411 23458 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.271333 23472 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.280609 23472 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.281484 23471 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.283752 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.284533 23473 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.284694 23458 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.282099 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.285187 23473 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.285465 23458 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.287006 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.288041 22702 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:08.288399 23473 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.288457 23471 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.289686 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.291865 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.289975 23458 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.292585 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.293201 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.294039 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.294957 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.295627 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.297451 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.300536 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.302834 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.303479 23458 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.304101 23471 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.304324 23458 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.304765 23471 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.307111 23458 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.307176 23471 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.308476 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.317207 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.315011 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.323261 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:08.324317 23451 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:08.324806 23451 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.325181 23451 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.324324 23471 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.327517 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.327430 23471 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.329128 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.329180 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.329746 23471 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.331560 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.332747 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.333530 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.332727 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.334993 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.335242 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.336488 23454 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.336927 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.337136 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.337158 23454 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.341516 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.343127 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.344262 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.345830 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.346951 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.349351 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.350080 23454 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.352334 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.354558 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.355167 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:08.356204 23469 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:08.356706 23469 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.357131 23469 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.356204 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.366983 23474 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.367901 23474 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.370891 23474 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.372228 23469 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.376121 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.385480 23469 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.381791 23451 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.395929 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.396557 23451 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.387876 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:08.397043 23459 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.398064 22601 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.398954 23459 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.399353 22601 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.400430 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:08.400550 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.401507 23467 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:08.401786 22601 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.402588 23467 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:08.403059 23467 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.403147 22601 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.403319 23454 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.404110 23454 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.404485 22601 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.405326 23454 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.405877 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.406738 23454 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.400473 23459 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.411100 22601 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.412168 23459 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.412431 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.413815 23454 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.417269 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.418079 23467 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.421875 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.422936 23454 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.423810 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:08.432655 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:08.433554 23467 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:08.434753 22600 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.436223 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.436457 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.436484 23459 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.441965 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.442921 23472 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.444775 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.445725 23472 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.448150 23472 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.448103 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.449180 23458 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.448889 23472 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.449734 22749 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.450584 23458 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.451381 23472 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.455255 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.456066 22749 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.457475 23458 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.459602 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.460443 23458 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:08.460881 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.466640 23458 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.468155 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.468998 23474 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.469583 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.470249 22702 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:08.470752 23458 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.462744 22629 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:08.471904 23458 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:08.473165 23474 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:08.473520 23474 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:08.474532 23474 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.475797 23474 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.489956 23458 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.491047 23458 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.495870 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.499563 22749 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.501701 22706 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:08.502350 23473 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.504338 23471 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:08.506108 23471 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:08.506592 23471 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:08.507503 23471 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.508885 23471 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.510715 23313 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:08.511432 23313 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.513799 23313 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.519529 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:08.520062 23458 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:08.531939 22706 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:08.532171 22779 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:08.534056 23459 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:08.535406 23454 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:08.547127 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.548877 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.550405 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.551918 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.555245 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.560434 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.568841 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.570217 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.571465 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.573493 22780 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:08.571291 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.574893 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.570592 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.575462 22780 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:08.568037 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.579696 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.580222 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.574546 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.582221 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.582808 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.583559 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:08.584256 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.585588 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:08.589942 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.590127 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.591490 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.592643 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.593847 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.595033 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.596206 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.596851 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.597800 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.598171 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.598464 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.590085 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:08.611769 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.613094 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.613291 22749 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.613520 23377 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.613955 23452 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.586198 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.619323 23474 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.615798 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.620891 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.621040 22675 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.622088 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.625238 23474 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.625304 23464 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.627138 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.626981 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.629142 22749 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.629050 23466 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.631263 23455 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.632651 23474 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.634794 22749 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.634570 23454 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.637908 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.637846 22749 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.623605 23377 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.623800 23452 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.645051 23466 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.650542 22749 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.648207 22780 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:08.649080 23474 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.652667 23474 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.657060 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.657894 23474 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.661643 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.662477 23474 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.663674 22749 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.664340 23474 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.665433 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.666095 23474 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.628616 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.668574 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:08.668279 22706 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.669395 23474 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.671402 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.673367 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.674253 23452 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.675367 23464 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.676086 23377 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.683229 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.684496 22675 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.686259 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.688808 23464 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:08.693085 23377 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.693409 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:08.697302 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.695540 22628 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:08.701488 23464 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.699424 23377 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.698648 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.706274 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.707058 23464 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.707424 23377 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.712939 22779 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.719854 22706 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:08.720569 22675 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.721957 23467 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:08.720418 22533 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.723577 23464 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.726070 22675 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.729140 23377 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.728358 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.736011 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:08.737879 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.739646 22675 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:08.741698 23208 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.735407 23452 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.746101 23469 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:08.746503 23455 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:08.772440 23469 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:08.773581 23469 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.755560 22775 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:08.780112 23467 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:08.740855 22777 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:08.801918 22628 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:08.804195 23458 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:09:08.809830 23458 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:08.810336 23458 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:08.811578 23458 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.810596 23459 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.817965 22706 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.820387 23458 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:08.850135 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:08.852145 23469 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:08.853245 23469 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:09:08.871412 22598 consensus_peers.cc:597] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c015ab1d14c443258576403d9d39c485. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:09:08.872489 22598 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1d580d080ca644b2af67c9e48cbd3b78. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:08.872596 22777 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.854560 23469 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:08.929919 22531 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:08.953682 22530 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.960765 22530 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.974813 22708 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.001078 22628 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:09.001600 22779 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:09.006750 23486 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.008538 23469 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:09.035015 22673 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9e3855678a8148a09ffe99d7624c9b4f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:09.036578 23486 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.046191 22779 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.054174 22631 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.070128 22708 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.074265 22530 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:09.094313 22673 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d92b65b2cbd546e18ddac5ce049ea91c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:09.102236 22779 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:09.120820 23469 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.127547 22631 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:09.129734 23469 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.144495 23499 ts_tablet_manager.cc:933] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:09.162258 23378 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.192160 23503 ts_tablet_manager.cc:933] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:09.203862 22630 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.223871 22530 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:09.226897 23499 tablet_copy_client.cc:323] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:09.226594 22777 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:09.272249 23503 tablet_copy_client.cc:323] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:09.284566 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet c015ab1d14c443258576403d9d39c485 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:09.285418 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet c015ab1d14c443258576403d9d39c485 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-c015ab1d14c443258576403d9d39c485
I20260709 13:09:09.295531 22644 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 1d580d080ca644b2af67c9e48cbd3b78 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:09.296376 22644 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 1d580d080ca644b2af67c9e48cbd3b78 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-1d580d080ca644b2af67c9e48cbd3b78
I20260709 13:09:09.314500 22644 tablet_copy_source_session.cc:215] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:09.313786 22645 tablet_copy_source_session.cc:215] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:09.323787 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d580d080ca644b2af67c9e48cbd3b78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.326894 23499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c015ab1d14c443258576403d9d39c485. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.347934 22780 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:09.357102 23469 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.372944 22779 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:09.382472 23499 tablet_copy_client.cc:806] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:09.385134 23378 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.411211 23499 tablet_copy_client.cc:670] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:09.421420 23503 tablet_copy_client.cc:806] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:09.422287 23503 tablet_copy_client.cc:670] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:09.423452 23499 tablet_copy_client.cc:538] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:09.434361 22630 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.435525 23503 tablet_copy_client.cc:538] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:09.448225 22630 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:09.453567 23503 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:09.460295 23486 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.465674 22630 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:09.479096 23499 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:09.504084 23503 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:09.505398 23503 tablet_bootstrap.cc:492] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:09.506243 23503 ts_tablet_manager.cc:1403] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.025s
I20260709 13:09:09.507553 22630 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:09.512420 23469 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:09.518260 23503 raft_consensus.cc:359] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.519688 23503 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:09.520689 23503 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.546276 23503 ts_tablet_manager.cc:1434] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 13:09:09.558303 23499 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:09.559561 23514 ts_tablet_manager.cc:933] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:09.571004 23499 tablet_bootstrap.cc:492] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:09.571985 23499 ts_tablet_manager.cc:1403] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.093s	user 0.019s	sys 0.007s
I20260709 13:09:09.577781 23499 raft_consensus.cc:359] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.581768 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-1d580d080ca644b2af67c9e48cbd3b78 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:09.586078 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-1d580d080ca644b2af67c9e48cbd3b78 on tablet 1d580d080ca644b2af67c9e48cbd3b78 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:09.583892 23514 tablet_copy_client.cc:323] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:09.590804 23499 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:09.591787 23499 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.594802 23499 ts_tablet_manager.cc:1434] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.006s
I20260709 13:09:09.614560 23503 ts_tablet_manager.cc:933] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:09.616472 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-c015ab1d14c443258576403d9d39c485 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:09.617069 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-c015ab1d14c443258576403d9d39c485 on tablet c015ab1d14c443258576403d9d39c485 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:09.623379 22720 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet d92b65b2cbd546e18ddac5ce049ea91c from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:09.624215 22720 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet d92b65b2cbd546e18ddac5ce049ea91c from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-d92b65b2cbd546e18ddac5ce049ea91c
I20260709 13:09:09.634578 22720 tablet_copy_source_session.cc:215] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:09.650378 23514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d92b65b2cbd546e18ddac5ce049ea91c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.650871 23503 tablet_copy_client.cc:323] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:09.688271 23514 tablet_copy_client.cc:806] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:09.689855 22720 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet 9e3855678a8148a09ffe99d7624c9b4f from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:09.695029 22720 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet 9e3855678a8148a09ffe99d7624c9b4f from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-9e3855678a8148a09ffe99d7624c9b4f
I20260709 13:09:09.704659 22720 tablet_copy_source_session.cc:215] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:09.705538 23514 tablet_copy_client.cc:670] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:09.722451 23503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e3855678a8148a09ffe99d7624c9b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:09.726091 23514 tablet_copy_client.cc:538] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:09:09.729138 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44686.7us
I20260709 13:09:09.745690 23514 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:09.757360 23420 raft_consensus.cc:1217] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:09.770546 23503 tablet_copy_client.cc:806] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:09.771281 23503 tablet_copy_client.cc:670] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:09.783917 23503 tablet_copy_client.cc:538] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:09.804031 23514 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:09.805321 23514 tablet_bootstrap.cc:492] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:09.806257 23514 ts_tablet_manager.cc:1403] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.007s
I20260709 13:09:09.809949 23514 raft_consensus.cc:359] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.811094 23514 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:09.811915 23503 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:09.811923 23514 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.825421 23514 ts_tablet_manager.cc:1434] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:09:09.829777 22720 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-d92b65b2cbd546e18ddac5ce049ea91c received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:09.830349 22720 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-d92b65b2cbd546e18ddac5ce049ea91c on tablet d92b65b2cbd546e18ddac5ce049ea91c with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:09.866119 23503 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:09.872901 23503 tablet_bootstrap.cc:492] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:09.873811 23503 ts_tablet_manager.cc:1403] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.017s
I20260709 13:09:09.877420 23503 raft_consensus.cc:359] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.878356 23503 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:09.879160 23503 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:09.884953 23503 ts_tablet_manager.cc:1434] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:09:09.910732 22720 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-9e3855678a8148a09ffe99d7624c9b4f received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:09.911268 22720 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-9e3855678a8148a09ffe99d7624c9b4f on tablet 9e3855678a8148a09ffe99d7624c9b4f with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:09.955973 23420 raft_consensus.cc:1217] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:10.043776 23420 raft_consensus.cc:1217] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:10.052217 23420 raft_consensus.cc:1217] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:10.265473 23467 raft_consensus.cc:1064] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:10.269203 23467 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:10.300330 22780 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.302220 23420 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.304198 23459 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.306368 23459 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:09:10.311483 22706 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.313438 23467 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:10.329681 23486 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:10.330346 23486 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:10.332604 23486 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:10.334761 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:10.335646 22635 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:10.349424 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:10.350384 22779 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:10.366295 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:10.372781 23455 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:10.375177 23455 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:10.375818 23455 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:10.394078 23529 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:09:10.398231 23455 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:10.400825 23467 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.403476 23455 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:10.407202 23420 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.416085 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:10.416976 22635 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:10.419744 22780 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.423931 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:10.430140 22530 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:10.435730 22706 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.441818 22779 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:10.493886 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:10.495306 23455 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:10.495951 23455 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:10.496905 23455 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:10.508011 23469 raft_consensus.cc:1064] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:10.518225 23533 raft_consensus.cc:1064] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:10.518740 23469 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:10.522230 23533 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:10.559319 23420 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.563465 23531 raft_consensus.cc:1064] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:10.563679 22780 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.567358 23531 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:10.569283 23486 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.573127 22630 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.589608 23378 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.613806 23533 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.629595 22777 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.631397 22629 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.652513 23540 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:09:10.692797 23469 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.690238 22629 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:10.690902 23517 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.747427 22781 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:10.749105 23421 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.749233 23342 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:09:10.753432 23542 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:09:10.762516 23517 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:10.763274 23459 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.765990 23517 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:10.772369 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:10.773296 22778 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:10.775175 22600 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:10.794312 23342 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:10.810073 23342 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.773485 22781 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.839433 23421 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:10.848299 23455 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.852509 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:10.846580 22531 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:10.820924 23342 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:10.930352 23554 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:10.946105 22784 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.954949 22780 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:10.954746 23342 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:10.957386 23531 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:09:10.976738 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 119179us
I20260709 13:09:10.978932 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1d580d080ca644b2af67c9e48cbd3b78 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:10.981976 22635 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:10.986918 22533 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.001868 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:11.003100 22779 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:11.009882 23517 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:11.012462 23342 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:11.016044 23421 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:11.020547 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 1d580d080ca644b2af67c9e48cbd3b78 with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:11.021371 23531 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:11.028908 23419 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.031203 23421 raft_consensus.cc:2955] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:11.034019 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93613.9us
I20260709 13:09:11.039407 22702 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:11.047652 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:11.048281 22533 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:11.079911 22779 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:11.088294 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 88425.1us
I20260709 13:09:11.100520 22600 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:11.101768 23549 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:11.102346 23549 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:11.103330 23549 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:11.109364 22702 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:11.129781 23562 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:11.165315 22781 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:11.197781 22628 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:11.227273 23418 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.235018 23469 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.236047 23342 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.242151 23553 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:11.243140 23553 consensus_queue.cc:1243] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:09:11.243677 23553 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:11.244046 23553 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:11.243942 23421 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.246264 23566 tablet_replica.cc:333] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:11.247790 23566 raft_consensus.cc:2243] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:11.248821 23566 raft_consensus.cc:2272] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:11.245783 23553 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:11.252559 23419 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.249648 23455 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:11.263221 23565 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:09:11.280131 22533 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:11.287451 23420 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.288151 23553 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:11.230085 23561 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.293885 22777 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.298401 22777 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.306094 23566 ts_tablet_manager.cc:1916] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:11.313591 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9e3855678a8148a09ffe99d7624c9b4f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:11.314810 22635 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.324728 22533 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.356133 23540 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.363557 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet 9e3855678a8148a09ffe99d7624c9b4f with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:11.359820 22532 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:11.443557 22779 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:11.459898 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77144us
I20260709 13:09:11.460958 22533 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.467973 22780 raft_consensus.cc:2955] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.536093 22779 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:11.555573 22704 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:09:11.556054 22704 pending_rounds.cc:85] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 1
I20260709 13:09:11.556349 22704 pending_rounds.cc:107] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:09:11.556649 22704 raft_consensus.cc:2889] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:11.563294 23549 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:09:11.573161 23549 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:11.582151 23573 tablet_replica.cc:333] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:11.588680 23573 raft_consensus.cc:2243] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:11.589671 23573 raft_consensus.cc:2272] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:11.611377 23573 ts_tablet_manager.cc:1916] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:11.555660 22701 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.643651 22710 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:11.671444 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 130802us
I20260709 13:09:11.756824 23421 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.763274 23554 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.776548 23486 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.781940 23420 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.803881 22779 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.825311 22630 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:11.832672 23554 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.868582 22784 raft_consensus.cc:2955] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.887609 22531 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.886723 23420 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.898021 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d92b65b2cbd546e18ddac5ce049ea91c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:11.897003 22701 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:11.906440 23549 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.918195 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet d92b65b2cbd546e18ddac5ce049ea91c with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:11.909409 23566 ts_tablet_manager.cc:1929] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:11.924959 23551 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.932255 23547 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:11.947252 22709 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.952644 23420 raft_consensus.cc:2955] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:11.962822 23566 log.cc:1199] T 1d580d080ca644b2af67c9e48cbd3b78 P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/1d580d080ca644b2af67c9e48cbd3b78
I20260709 13:09:11.968112 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 1d580d080ca644b2af67c9e48cbd3b78 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:11.979840 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c015ab1d14c443258576403d9d39c485 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:11.989509 22531 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:12.008991 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet c015ab1d14c443258576403d9d39c485 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:12.026129 23580 tablet_replica.cc:333] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:12.027395 23580 raft_consensus.cc:2243] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:12.027709 23573 ts_tablet_manager.cc:1929] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:12.028227 23573 log.cc:1199] T 9e3855678a8148a09ffe99d7624c9b4f P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/9e3855678a8148a09ffe99d7624c9b4f
I20260709 13:09:12.028254 23580 raft_consensus.cc:2272] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:12.042055 23573 tablet_replica.cc:333] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:12.042838 23580 ts_tablet_manager.cc:1916] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:12.038673 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet 9e3855678a8148a09ffe99d7624c9b4f (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:12.069561 23580 ts_tablet_manager.cc:1929] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:12.070715 23580 log.cc:1199] T c015ab1d14c443258576403d9d39c485 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/c015ab1d14c443258576403d9d39c485
I20260709 13:09:12.072499 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet c015ab1d14c443258576403d9d39c485 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:12.077805 23573 raft_consensus.cc:2243] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:12.078791 23573 raft_consensus.cc:2272] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:12.083084 23573 ts_tablet_manager.cc:1916] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:12.105580 23573 ts_tablet_manager.cc:1929] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:12.106065 23573 log.cc:1199] T d92b65b2cbd546e18ddac5ce049ea91c P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/d92b65b2cbd546e18ddac5ce049ea91c
I20260709 13:09:12.107507 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet d92b65b2cbd546e18ddac5ce049ea91c (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:13.100893 22630 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.108199 22783 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:13.108852 22709 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:13.110149 23549 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:13.111757 23551 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:13.113152 22598 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 99c51f61e4f74c1593ebf98255a83b4c. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:13.133002 23551 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.134842 22709 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.137524 22783 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.160729 22530 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.162599 22630 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.230453 22783 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.232173 23551 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:13.238988 22598 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 26e6130713d74c0ea9492f5151053c42. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:13.241921 22710 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.245095 23342 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:13.259244 23342 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.264132 22775 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.290078 22532 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.294109 22710 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.320441 22775 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.342808 22710 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.344515 22630 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.344805 23458 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:13.346392 23555 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:13.374385 22630 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.371893 23458 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:13.420615 22747 consensus_peers.cc:597] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed3a2e4658ea473ba5c2b68313aeaf3b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:13.444690 22701 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.451485 22630 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:13.459544 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59888.1us
I20260709 13:09:13.441205 22532 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.504227 22710 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.505702 23549 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:13.508543 22783 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:13.511365 23551 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:13.530121 22598 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a79a6f618f4944b1a11b9c1df766ae7f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:13.536612 23342 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.542158 22775 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.551909 23609 ts_tablet_manager.cc:933] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:13.576431 23609 tablet_copy_client.cc:323] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:13.579553 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 99c51f61e4f74c1593ebf98255a83b4c from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:13.580219 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 99c51f61e4f74c1593ebf98255a83b4c from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-99c51f61e4f74c1593ebf98255a83b4c
I20260709 13:09:13.586064 22532 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:13.592630 22645 tablet_copy_source_session.cc:215] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:13.607297 22710 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:13.613960 23609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99c51f61e4f74c1593ebf98255a83b4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:13.658234 23609 tablet_copy_client.cc:806] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:13.659032 23609 tablet_copy_client.cc:670] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:13.669102 23609 tablet_copy_client.cc:538] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:13.679662 23609 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:13.700059 23609 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:13.701184 23609 tablet_bootstrap.cc:492] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:13.701959 23609 ts_tablet_manager.cc:1403] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 13:09:13.705623 23609 raft_consensus.cc:359] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.706844 23609 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:13.707572 23609 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.748344 23609 ts_tablet_manager.cc:1434] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.046s	user 0.025s	sys 0.016s
I20260709 13:09:13.750288 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-99c51f61e4f74c1593ebf98255a83b4c received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:13.750833 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-99c51f61e4f74c1593ebf98255a83b4c on tablet 99c51f61e4f74c1593ebf98255a83b4c with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:13.758319 23609 ts_tablet_manager.cc:933] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:13.783942 23609 tablet_copy_client.cc:323] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:13.786053 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 26e6130713d74c0ea9492f5151053c42 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:13.786728 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 26e6130713d74c0ea9492f5151053c42 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-26e6130713d74c0ea9492f5151053c42
I20260709 13:09:13.797878 22645 tablet_copy_source_session.cc:215] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:13.803220 23609 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e6130713d74c0ea9492f5151053c42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:13.838287 23609 tablet_copy_client.cc:806] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:13.839082 23609 tablet_copy_client.cc:670] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:13.848044 23609 tablet_copy_client.cc:538] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:13.862306 23609 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:13.892665 23615 ts_tablet_manager.cc:933] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:13.899629 23609 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:13.900805 23609 tablet_bootstrap.cc:492] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:13.901686 23609 ts_tablet_manager.cc:1403] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.015s
I20260709 13:09:13.905355 23609 raft_consensus.cc:359] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.906368 23609 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:13.907167 23609 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:13.909268 23615 tablet_copy_client.cc:323] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:13.914253 23609 ts_tablet_manager.cc:1434] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:09:13.918567 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-26e6130713d74c0ea9492f5151053c42 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:13.919070 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-26e6130713d74c0ea9492f5151053c42 on tablet 26e6130713d74c0ea9492f5151053c42 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:13.931806 23619 ts_tablet_manager.cc:933] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:13.941687 23619 tablet_copy_client.cc:323] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:13.943578 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet a79a6f618f4944b1a11b9c1df766ae7f from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:13.944253 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet a79a6f618f4944b1a11b9c1df766ae7f from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-a79a6f618f4944b1a11b9c1df766ae7f
I20260709 13:09:13.966849 22645 tablet_copy_source_session.cc:215] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:13.970492 23619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a79a6f618f4944b1a11b9c1df766ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:13.978721 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet ed3a2e4658ea473ba5c2b68313aeaf3b from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:13.979466 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet ed3a2e4658ea473ba5c2b68313aeaf3b from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-ed3a2e4658ea473ba5c2b68313aeaf3b
I20260709 13:09:13.988369 22794 tablet_copy_source_session.cc:215] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:13.993105 23615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed3a2e4658ea473ba5c2b68313aeaf3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:14.000756 23619 tablet_copy_client.cc:806] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:14.001529 23619 tablet_copy_client.cc:670] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:14.017078 23421 raft_consensus.cc:1217] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:09:14.019367 23615 tablet_copy_client.cc:806] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:14.020102 23615 tablet_copy_client.cc:670] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:14.024581 23619 tablet_copy_client.cc:538] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:14.032022 23615 tablet_copy_client.cc:538] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:14.044227 23615 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:14.055166 23619 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:14.145737 23615 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:14.147215 23615 tablet_bootstrap.cc:492] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:14.148263 23615 ts_tablet_manager.cc:1403] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.104s	user 0.043s	sys 0.042s
I20260709 13:09:14.152338 23615 raft_consensus.cc:359] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:14.153355 23615 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:14.162091 23615 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:14.167428 23619 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:14.168601 23619 tablet_bootstrap.cc:492] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:14.173722 23615 ts_tablet_manager.cc:1434] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:09:14.178403 23619 ts_tablet_manager.cc:1403] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.124s	user 0.022s	sys 0.006s
I20260709 13:09:14.181479 23619 raft_consensus.cc:359] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:14.182189 23619 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:14.182904 23619 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:14.185124 23619 ts_tablet_manager.cc:1434] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:09:14.185746 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-ed3a2e4658ea473ba5c2b68313aeaf3b received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:14.186299 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-ed3a2e4658ea473ba5c2b68313aeaf3b on tablet ed3a2e4658ea473ba5c2b68313aeaf3b with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:14.187415 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-a79a6f618f4944b1a11b9c1df766ae7f received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:14.187896 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-a79a6f618f4944b1a11b9c1df766ae7f on tablet a79a6f618f4944b1a11b9c1df766ae7f with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:14.205957 23421 raft_consensus.cc:1217] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:14.360260 23421 raft_consensus.cc:1217] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:14.371575 23421 raft_consensus.cc:1217] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:14.451315 23342 raft_consensus.cc:1064] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:14.455590 23342 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:14.509232 23421 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:14.509513 22783 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:14.512404 22710 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:14.515818 23551 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:14.518395 23607 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.521503 23605 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.578212 22710 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.575210 23342 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.583400 22783 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.625278 22532 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.641609 23421 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.713101 23607 raft_consensus.cc:1064] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:14.713822 22631 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:14.716218 23607 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:14.727988 23421 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:09:14.729014 22783 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:09:14.730944 23605 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:14.732429 23342 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:14.741353 23549 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.744745 23420 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.748572 22783 raft_consensus.cc:2955] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.776930 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 99c51f61e4f74c1593ebf98255a83b4c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:14.787657 22532 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.793560 22710 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.795641 23605 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.798722 23599 raft_consensus.cc:1064] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:14.803732 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 99c51f61e4f74c1593ebf98255a83b4c with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:14.802062 23599 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:14.816840 22775 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.824011 23419 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.835744 23342 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:14.837587 23342 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:14.840785 23607 raft_consensus.cc:1064] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:14.851817 23607 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:14.883441 23551 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.885257 22783 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.899019 22775 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.900681 22701 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.902077 23607 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.902945 23421 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:14.903607 23342 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:14.918574 23342 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.923113 22710 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.927793 23632 tablet_replica.cc:333] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:14.929272 23632 raft_consensus.cc:2243] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:14.930174 23632 raft_consensus.cc:2272] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:14.941681 22530 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:14.950847 23635 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:09:14.950480 23342 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.952440 22704 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.985677 23419 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:14.996899 23420 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:15.000173 23555 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:14.982362 23632 ts_tablet_manager.cc:1916] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:15.022203 22630 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:15.028083 22530 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.040498 23418 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.070396 22630 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:15.077981 23420 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.083214 23458 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:15.088068 23607 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:15.098111 22704 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.100848 23607 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:15.107699 23632 ts_tablet_manager.cc:1929] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:09:15.108295 23632 log.cc:1199] T 99c51f61e4f74c1593ebf98255a83b4c P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/99c51f61e4f74c1593ebf98255a83b4c
W20260709 13:09:15.111096 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59403.3us
I20260709 13:09:15.111810 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 99c51f61e4f74c1593ebf98255a83b4c (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:15.115891 22701 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:15.134764 23605 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.134121 23640 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
W20260709 13:09:15.175774 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 46284.4us
W20260709 13:09:15.191195 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51677.3us
I20260709 13:09:15.194845 22784 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.190675 22708 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.214722 23420 raft_consensus.cc:2955] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.289790 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 26e6130713d74c0ea9492f5151053c42 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:15.290724 23641 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.303710 22532 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.320012 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 26e6130713d74c0ea9492f5151053c42 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:15.348424 23655 tablet_replica.cc:333] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:15.349696 23655 raft_consensus.cc:2243] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.350409 23655 raft_consensus.cc:2272] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.347635 22706 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.359311 23655 ts_tablet_manager.cc:1916] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:15.386251 23655 ts_tablet_manager.cc:1929] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:15.385329 22635 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.387005 23655 log.cc:1199] T 26e6130713d74c0ea9492f5151053c42 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/26e6130713d74c0ea9492f5151053c42
I20260709 13:09:15.402555 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 26e6130713d74c0ea9492f5151053c42 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:15.399375 22630 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:15.406726 23420 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.418702 23421 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.431576 23656 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:15.423879 22531 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.444382 23605 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.459852 23421 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.484256 22706 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.486703 22784 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:15.488551 23656 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:15.492578 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a79a6f618f4944b1a11b9c1df766ae7f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:15.496198 22706 raft_consensus.cc:2955] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.498253 22710 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.498435 23418 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:15.501096 23644 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:15.508553 23644 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:15.512253 23644 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.513310 22531 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.515751 23420 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.524756 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ed3a2e4658ea473ba5c2b68313aeaf3b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:15.527699 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet a79a6f618f4944b1a11b9c1df766ae7f with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:15.528754 23655 tablet_replica.cc:333] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:15.533674 23655 raft_consensus.cc:2243] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.534480 23655 raft_consensus.cc:2272] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.550184 23655 ts_tablet_manager.cc:1916] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:15.551952 22710 raft_consensus.cc:2955] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:15.576890 22530 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.582901 23655 ts_tablet_manager.cc:1929] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:15.583453 23655 log.cc:1199] T a79a6f618f4944b1a11b9c1df766ae7f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/a79a6f618f4944b1a11b9c1df766ae7f
I20260709 13:09:15.585942 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet a79a6f618f4944b1a11b9c1df766ae7f (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:15.594156 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet ed3a2e4658ea473ba5c2b68313aeaf3b with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:15.600127 23657 tablet_replica.cc:333] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:15.601320 23657 raft_consensus.cc:2243] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:15.602196 23657 raft_consensus.cc:2272] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:15.605707 23657 ts_tablet_manager.cc:1916] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:15.625406 23657 ts_tablet_manager.cc:1929] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:15.625880 23657 log.cc:1199] T ed3a2e4658ea473ba5c2b68313aeaf3b P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/ed3a2e4658ea473ba5c2b68313aeaf3b
I20260709 13:09:15.627736 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet ed3a2e4658ea473ba5c2b68313aeaf3b (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:16.675922 22630 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:16.683008 22775 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.683431 22710 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.685211 23658 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.687254 23549 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:16.688357 22598 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91be8db7516b4bca939f439f045b40eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.700855 23549 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.706122 22702 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.723824 22702 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:16.724773 22784 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.728382 22530 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:16.742172 22673 consensus_peers.cc:597] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 17042e7fd6cb40cfb21cfdfb5121a4bc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.747949 22784 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.750109 22630 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.751806 23646 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:16.753382 23603 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:16.782938 23646 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.785629 22630 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.801923 22784 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.835475 22710 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:16.836365 22533 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:16.854244 22673 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc8b169704cc4157ad021d774f83419e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.856113 22784 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.858662 23671 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:16.859769 22635 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.862180 23671 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.881846 22628 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.875921 23671 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.884756 22784 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.917672 22784 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:16.920145 22530 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:16.959249 22747 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7235e142d8ec45adb46a8c389a2962fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:16.961361 22628 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.962384 22710 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:16.963544 23662 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.964609 23640 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:16.975021 23641 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.976908 22710 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.983523 22628 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:16.999922 22533 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:17.194994 23682 ts_tablet_manager.cc:933] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:17.197273 23682 tablet_copy_client.cc:323] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:17.199611 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 91be8db7516b4bca939f439f045b40eb from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:17.200254 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 91be8db7516b4bca939f439f045b40eb from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-91be8db7516b4bca939f439f045b40eb
I20260709 13:09:17.210559 22645 tablet_copy_source_session.cc:215] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:17.217099 23682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91be8db7516b4bca939f439f045b40eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.237967 23682 tablet_copy_client.cc:806] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:17.238761 23682 tablet_copy_client.cc:670] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:17.244796 23682 tablet_copy_client.cc:538] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:17.255429 23682 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:17.277998 23682 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.278982 23682 tablet_bootstrap.cc:492] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:17.279662 23682 ts_tablet_manager.cc:1403] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.013s
I20260709 13:09:17.282416 23682 raft_consensus.cc:359] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.283416 23682 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:17.284155 23682 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.294763 23682 ts_tablet_manager.cc:1434] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.005s
I20260709 13:09:17.297048 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-91be8db7516b4bca939f439f045b40eb received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:17.297544 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-91be8db7516b4bca939f439f045b40eb on tablet 91be8db7516b4bca939f439f045b40eb with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:17.340425 23682 ts_tablet_manager.cc:933] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:17.342372 23682 tablet_copy_client.cc:323] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:17.344237 22720 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet cc8b169704cc4157ad021d774f83419e from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:17.344856 22720 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet cc8b169704cc4157ad021d774f83419e from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-cc8b169704cc4157ad021d774f83419e
I20260709 13:09:17.353946 22720 tablet_copy_source_session.cc:215] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:17.358062 23682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc8b169704cc4157ad021d774f83419e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.362635 23687 ts_tablet_manager.cc:933] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:17.384167 23687 tablet_copy_client.cc:323] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:17.387090 22720 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:17.387755 22720 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet 17042e7fd6cb40cfb21cfdfb5121a4bc from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-17042e7fd6cb40cfb21cfdfb5121a4bc
I20260709 13:09:17.404690 23682 tablet_copy_client.cc:806] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:17.405596 23682 tablet_copy_client.cc:670] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:17.416522 23682 tablet_copy_client.cc:538] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:17.422227 22720 tablet_copy_source_session.cc:215] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:17.428614 23687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.448405 23682 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:17.450269 23687 tablet_copy_client.cc:806] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:17.451061 23687 tablet_copy_client.cc:670] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:17.501634 23687 tablet_copy_client.cc:538] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:17.517661 23687 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:17.518061 23682 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.519405 23682 tablet_bootstrap.cc:492] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:17.520228 23682 ts_tablet_manager.cc:1403] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.072s	user 0.020s	sys 0.020s
I20260709 13:09:17.524003 23682 raft_consensus.cc:359] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.524978 23682 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:17.525763 23682 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.530951 23682 ts_tablet_manager.cc:1434] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 13:09:17.532846 22720 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-cc8b169704cc4157ad021d774f83419e received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:17.533308 22720 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-cc8b169704cc4157ad021d774f83419e on tablet cc8b169704cc4157ad021d774f83419e with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:17.561558 23687 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.562809 23687 tablet_bootstrap.cc:492] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:17.563661 23687 ts_tablet_manager.cc:1403] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.025s
I20260709 13:09:17.567030 23687 raft_consensus.cc:359] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.567958 23687 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:17.568630 23687 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.571314 23687 ts_tablet_manager.cc:1434] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:09:17.574018 23682 ts_tablet_manager.cc:933] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:17.574168 22720 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-17042e7fd6cb40cfb21cfdfb5121a4bc received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:17.574728 22720 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-17042e7fd6cb40cfb21cfdfb5121a4bc on tablet 17042e7fd6cb40cfb21cfdfb5121a4bc with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:17.577960 23682 tablet_copy_client.cc:323] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:17.580623 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet 7235e142d8ec45adb46a8c389a2962fa from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:17.581260 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet 7235e142d8ec45adb46a8c389a2962fa from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-7235e142d8ec45adb46a8c389a2962fa
I20260709 13:09:17.594290 22794 tablet_copy_source_session.cc:215] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:17.597663 23682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7235e142d8ec45adb46a8c389a2962fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:17.613691 23682 tablet_copy_client.cc:806] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:17.614467 23682 tablet_copy_client.cc:670] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:17.621332 23682 tablet_copy_client.cc:538] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:17.630801 23682 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:17.656287 23682 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:17.657147 23682 tablet_bootstrap.cc:492] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:17.657791 23682 ts_tablet_manager.cc:1403] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.010s
I20260709 13:09:17.660219 23682 raft_consensus.cc:359] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.661019 23682 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:17.661576 23682 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:17.663935 23682 ts_tablet_manager.cc:1434] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:09:17.667034 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-7235e142d8ec45adb46a8c389a2962fa received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:17.667552 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-7235e142d8ec45adb46a8c389a2962fa on tablet 7235e142d8ec45adb46a8c389a2962fa with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:17.788585 23421 raft_consensus.cc:1217] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:17.821918 23420 raft_consensus.cc:1217] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:17.826038 23421 raft_consensus.cc:1217] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:17.976256 23421 raft_consensus.cc:1217] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:18.188920 23681 raft_consensus.cc:1064] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:18.192714 23681 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.250998 23675 raft_consensus.cc:1064] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:18.257776 22710 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.261132 23421 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.261785 22783 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.254182 23675 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.268934 23549 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.271530 23697 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.274015 23660 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:09:18.316184 23421 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.318673 23671 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.322597 22784 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.317508 23697 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.326411 22710 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.337337 22775 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.337620 22628 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.342515 23695 raft_consensus.cc:1064] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:18.346246 23695 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.341156 23675 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:09:18.359833 23421 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.369341 23646 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.380160 22533 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.468250 23421 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.470896 23646 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:18.472878 23690 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.503530 22631 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.506979 22775 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.506575 22635 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.519263 23420 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.521102 23703 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.523909 23703 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.543566 22779 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.558318 23675 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.571811 22635 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.574148 22779 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.574880 23420 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.598762 23679 raft_consensus.cc:1064] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:18.602349 23679 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.616979 22635 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.617779 22530 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.623636 22530 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.626535 23420 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.629405 22779 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.631240 23681 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:18.632468 23697 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:18.653218 23681 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.660758 23420 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.665585 22779 raft_consensus.cc:2955] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.669826 23421 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.672617 23680 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.680621 22533 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.694343 22706 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.697726 22708 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:18.705183 23420 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.705348 22635 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.706885 23703 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:18.710786 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 91be8db7516b4bca939f439f045b40eb with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:18.718554 23703 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:09:18.727744 23719 tablet_replica.cc:333] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:18.726699 23675 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.729084 23420 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.732978 22635 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:18.734218 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67261.6us
I20260709 13:09:18.738039 22628 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.739058 23646 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:18.740525 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cc8b169704cc4157ad021d774f83419e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:18.749140 23703 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.751391 22783 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:18.751001 23421 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.752744 23719 raft_consensus.cc:2243] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:18.761539 22706 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:18.771911 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:18.772509 23420 raft_consensus.cc:2955] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.778788 23719 raft_consensus.cc:2272] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:18.785213 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91be8db7516b4bca939f439f045b40eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:18.776157 22533 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:18.786429 23641 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.786718 23690 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:18.793352 23719 ts_tablet_manager.cc:1916] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:18.804467 22530 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.808019 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet cc8b169704cc4157ad021d774f83419e with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:18.815112 23716 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.820050 22775 raft_consensus.cc:2955] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:18.852360 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet 17042e7fd6cb40cfb21cfdfb5121a4bc with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:18.907568 23720 tablet_replica.cc:333] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:18.911654 23722 tablet_replica.cc:333] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:18.935716 23722 raft_consensus.cc:2243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:18.936743 23722 raft_consensus.cc:2272] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:18.942519 23722 ts_tablet_manager.cc:1916] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:18.973831 23720 raft_consensus.cc:2243] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:18.978996 23720 raft_consensus.cc:2272] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:19.002041 23720 ts_tablet_manager.cc:1916] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:19.017864 23717 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.036574 23719 ts_tablet_manager.cc:1929] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:19.037451 23719 log.cc:1199] T 91be8db7516b4bca939f439f045b40eb P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/91be8db7516b4bca939f439f045b40eb
I20260709 13:09:19.037079 22635 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.039736 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 91be8db7516b4bca939f439f045b40eb (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:19.043272 22702 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.052155 23722 ts_tablet_manager.cc:1929] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:19.058763 23722 log.cc:1199] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/17042e7fd6cb40cfb21cfdfb5121a4bc
I20260709 13:09:19.062904 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet 17042e7fd6cb40cfb21cfdfb5121a4bc (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:19.084697 22532 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.097349 23420 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.107256 23720 ts_tablet_manager.cc:1929] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:19.107942 23720 log.cc:1199] T cc8b169704cc4157ad021d774f83419e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/cc8b169704cc4157ad021d774f83419e
I20260709 13:09:19.112643 22783 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:19.116397 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet cc8b169704cc4157ad021d774f83419e (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:19.122380 23421 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:19.122367 22635 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:19.124148 23717 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:19.124977 23680 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:19.131765 23680 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.134680 22635 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.150426 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7235e142d8ec45adb46a8c389a2962fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:19.147465 23420 raft_consensus.cc:2955] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.162535 22532 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:19.178814 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 7235e142d8ec45adb46a8c389a2962fa with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:19.182924 23719 tablet_replica.cc:333] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:19.198400 23719 raft_consensus.cc:2243] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:19.199441 23719 raft_consensus.cc:2272] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:19.203157 23719 ts_tablet_manager.cc:1916] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:19.222594 23719 ts_tablet_manager.cc:1929] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:19.223196 23719 log.cc:1199] T 7235e142d8ec45adb46a8c389a2962fa P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/7235e142d8ec45adb46a8c389a2962fa
I20260709 13:09:19.224797 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 7235e142d8ec45adb46a8c389a2962fa (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:20.273372 22702 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:20.292201 22673 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6d9cbf659974b358ba6d364ca9ca431. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.294339 22775 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.294663 22628 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.295948 23727 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.297317 23695 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.305176 23671 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.311177 22628 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.311834 22775 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.322698 22783 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:20.327701 22533 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:20.348076 22702 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:20.349409 22747 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11f8cc3feaf345e397b685036c912e13. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.350093 22628 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.353835 23680 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:09:20.355034 23679 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:20.368206 23679 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.377681 22628 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.379617 22708 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.392364 22532 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:20.393618 22783 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:20.402861 22747 consensus_peers.cc:597] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fee10603c0c042fd8e28683daeb86a99. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.403270 22628 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.404896 22708 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.405896 23641 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.408531 23641 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.420526 23680 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.424808 22708 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.428278 22635 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.446990 22530 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.460371 22708 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:20.472131 22673 consensus_peers.cc:597] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec9764b0b7dd4b67a5b50a652203c60a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:20.472193 22783 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.472826 22628 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:20.474182 23727 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:20.475355 23740 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:20.486886 23727 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.490154 22635 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.491338 22783 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.514478 22533 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:20.853688 23745 ts_tablet_manager.cc:933] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:20.856439 23746 ts_tablet_manager.cc:933] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:20.865512 23746 tablet_copy_client.cc:323] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:20.868036 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet 11f8cc3feaf345e397b685036c912e13 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:20.868700 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet 11f8cc3feaf345e397b685036c912e13 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-11f8cc3feaf345e397b685036c912e13
I20260709 13:09:20.879674 22794 tablet_copy_source_session.cc:215] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:20.884461 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11f8cc3feaf345e397b685036c912e13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:20.890903 23745 tablet_copy_client.cc:323] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:20.894117 22720 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet e6d9cbf659974b358ba6d364ca9ca431 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:20.894826 22720 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet e6d9cbf659974b358ba6d364ca9ca431 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-e6d9cbf659974b358ba6d364ca9ca431
I20260709 13:09:20.904819 22720 tablet_copy_source_session.cc:215] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:20.907267 23746 tablet_copy_client.cc:806] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:20.908202 23746 tablet_copy_client.cc:670] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:20.908735 23745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6d9cbf659974b358ba6d364ca9ca431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:20.917151 23746 tablet_copy_client.cc:538] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:20.929342 23745 tablet_copy_client.cc:806] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:20.929991 23746 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:20.930228 23745 tablet_copy_client.cc:670] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:20.936179 23745 tablet_copy_client.cc:538] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:20.951987 23745 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:20.969914 23746 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.971165 23746 tablet_bootstrap.cc:492] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:20.972013 23746 ts_tablet_manager.cc:1403] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.042s	user 0.025s	sys 0.008s
I20260709 13:09:20.975970 23746 raft_consensus.cc:359] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:20.977030 23746 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:20.978596 23746 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:20.987243 23745 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:20.988747 23745 tablet_bootstrap.cc:492] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:20.989691 23745 ts_tablet_manager.cc:1403] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.038s	user 0.027s	sys 0.005s
I20260709 13:09:20.992821 23746 ts_tablet_manager.cc:1434] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 13:09:20.996345 23745 raft_consensus.cc:359] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:20.998983 23745 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:20.999805 23745 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:21.002568 23745 ts_tablet_manager.cc:1434] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.000s
I20260709 13:09:21.012730 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-11f8cc3feaf345e397b685036c912e13 received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:21.013456 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-11f8cc3feaf345e397b685036c912e13 on tablet 11f8cc3feaf345e397b685036c912e13 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:21.028992 23746 ts_tablet_manager.cc:933] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:21.039342 23750 ts_tablet_manager.cc:933] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:21.044337 23746 tablet_copy_client.cc:323] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:21.046976 23750 tablet_copy_client.cc:323] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:21.049785 22719 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet ec9764b0b7dd4b67a5b50a652203c60a from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:21.050385 22720 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-e6d9cbf659974b358ba6d364ca9ca431 received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:21.050985 22720 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-e6d9cbf659974b358ba6d364ca9ca431 on tablet e6d9cbf659974b358ba6d364ca9ca431 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:21.052327 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet fee10603c0c042fd8e28683daeb86a99 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:21.052953 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet fee10603c0c042fd8e28683daeb86a99 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-fee10603c0c042fd8e28683daeb86a99
I20260709 13:09:21.058295 22719 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet ec9764b0b7dd4b67a5b50a652203c60a from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-ec9764b0b7dd4b67a5b50a652203c60a
I20260709 13:09:21.068077 22794 tablet_copy_source_session.cc:215] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:21.069007 22719 tablet_copy_source_session.cc:215] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:21.072638 23750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fee10603c0c042fd8e28683daeb86a99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.073586 23746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9764b0b7dd4b67a5b50a652203c60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:21.096434 23750 tablet_copy_client.cc:806] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:21.097304 23750 tablet_copy_client.cc:670] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:21.103564 23746 tablet_copy_client.cc:806] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:21.104585 23746 tablet_copy_client.cc:670] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:21.117265 23750 tablet_copy_client.cc:538] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:21.123054 23746 tablet_copy_client.cc:538] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:21.133787 23750 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:21.168145 23750 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:21.169482 23750 tablet_bootstrap.cc:492] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:21.170531 23750 ts_tablet_manager.cc:1403] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.009s
I20260709 13:09:21.174033 23750 raft_consensus.cc:359] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:21.175144 23750 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:21.176071 23750 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:21.177690 23746 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:21.182659 23750 ts_tablet_manager.cc:1434] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:09:21.185329 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-fee10603c0c042fd8e28683daeb86a99 received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:21.185868 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-fee10603c0c042fd8e28683daeb86a99 on tablet fee10603c0c042fd8e28683daeb86a99 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:21.205255 23746 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:21.206413 23746 tablet_bootstrap.cc:492] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:21.207497 23746 ts_tablet_manager.cc:1403] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.013s
I20260709 13:09:21.211675 23746 raft_consensus.cc:359] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:21.212602 23746 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:21.213328 23746 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:21.216063 23746 ts_tablet_manager.cc:1434] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:09:21.218166 22719 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-ec9764b0b7dd4b67a5b50a652203c60a received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:21.218585 22719 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-ec9764b0b7dd4b67a5b50a652203c60a on tablet ec9764b0b7dd4b67a5b50a652203c60a with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:21.223347 23421 raft_consensus.cc:1217] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:21.291416 23421 raft_consensus.cc:1217] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:21.455320 23421 raft_consensus.cc:1217] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:21.475870 23421 raft_consensus.cc:1217] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:21.720381 23695 raft_consensus.cc:1064] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:21.723457 23695 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:21.733592 23421 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.735013 22783 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.736864 22635 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.736943 23758 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:21.739341 23758 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:21.741348 23758 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:21.760486 23758 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.764626 22783 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.774989 22628 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.787202 23421 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.791101 22533 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.825289 23679 raft_consensus.cc:1064] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:21.828135 23679 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:21.844523 22702 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:21.848672 23420 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.850589 23680 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:21.850948 22706 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.856611 22635 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:21.852625 23641 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:09:21.858536 23727 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:21.860492 23420 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:21.868534 23671 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.875615 23758 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:21.882043 22628 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.882964 23421 raft_consensus.cc:2955] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.904279 22631 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:21.909978 23641 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:21.924902 23758 raft_consensus.cc:1064] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:21.929870 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e6d9cbf659974b358ba6d364ca9ca431 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:21.928539 23758 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:21.949911 22531 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.964499 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet e6d9cbf659974b358ba6d364ca9ca431 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:21.975565 23641 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.982436 23420 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:21.992532 23773 tablet_replica.cc:333] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:21.993031 22628 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.021669 23418 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.028486 23671 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:22.029016 23769 raft_consensus.cc:1064] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:22.034029 22775 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.033553 23769 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:22.039462 23671 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:22.040437 22630 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.055017 23758 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:22.092576 23420 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.094048 23773 raft_consensus.cc:2243] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:22.094399 23641 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:22.095257 22630 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.095484 23773 raft_consensus.cc:2272] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:22.099852 23773 ts_tablet_manager.cc:1916] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:22.107359 23641 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:22.098455 22531 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.147490 23671 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.164367 22702 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:22.164660 22710 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.163249 22783 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:22.171533 23773 ts_tablet_manager.cc:1929] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:22.178496 23773 log.cc:1199] T e6d9cbf659974b358ba6d364ca9ca431 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/e6d9cbf659974b358ba6d364ca9ca431
I20260709 13:09:22.176985 23420 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.179746 22779 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.181371 23641 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:22.176156 22530 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:22.190850 23418 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.194025 22630 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.198432 23769 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:22.199572 23779 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:22.203660 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet e6d9cbf659974b358ba6d364ca9ca431 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:22.211164 23641 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.226279 23680 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.230042 22635 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.254544 22628 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.260988 23420 raft_consensus.cc:2955] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.266031 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 11f8cc3feaf345e397b685036c912e13 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:22.279548 22531 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.288215 22631 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.299770 23418 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.305784 22530 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.324163 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 11f8cc3feaf345e397b685036c912e13 with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:22.341213 23784 tablet_replica.cc:333] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:22.348817 22706 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:22.363948 23784 raft_consensus.cc:2243] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:22.376362 23784 raft_consensus.cc:2272] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:22.376075 22710 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.364956 22783 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:22.381088 23784 ts_tablet_manager.cc:1916] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:22.385251 23420 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.387091 23680 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:22.387820 23418 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.389402 23671 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:22.404208 23421 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.402472 23680 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.416114 23671 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.423221 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fee10603c0c042fd8e28683daeb86a99 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:22.431524 22628 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.428234 23418 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.445094 22533 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:22.451066 22706 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:22.443888 23695 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:22.455662 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec9764b0b7dd4b67a5b50a652203c60a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:22.461438 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet fee10603c0c042fd8e28683daeb86a99 with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:22.466163 23768 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:22.470268 22631 raft_consensus.cc:2955] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.473039 22530 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.483036 23784 ts_tablet_manager.cc:1929] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:22.483601 23784 log.cc:1199] T 11f8cc3feaf345e397b685036c912e13 P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/11f8cc3feaf345e397b685036c912e13
I20260709 13:09:22.485497 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 11f8cc3feaf345e397b685036c912e13 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:22.488695 22706 raft_consensus.cc:2955] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:22.493477 23785 tablet_replica.cc:333] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:22.497694 23785 raft_consensus.cc:2243] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:22.498562 23785 raft_consensus.cc:2272] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:22.502076 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet ec9764b0b7dd4b67a5b50a652203c60a with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:22.503494 23773 tablet_replica.cc:333] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:22.504642 23773 raft_consensus.cc:2243] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:22.505451 23785 ts_tablet_manager.cc:1916] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:22.505482 23773 raft_consensus.cc:2272] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:22.518540 23773 ts_tablet_manager.cc:1916] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:22.528877 23785 ts_tablet_manager.cc:1929] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:22.529565 23785 log.cc:1199] T fee10603c0c042fd8e28683daeb86a99 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/fee10603c0c042fd8e28683daeb86a99
I20260709 13:09:22.531477 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet fee10603c0c042fd8e28683daeb86a99 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:22.540122 23773 ts_tablet_manager.cc:1929] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:22.540802 23773 log.cc:1199] T ec9764b0b7dd4b67a5b50a652203c60a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/ec9764b0b7dd4b67a5b50a652203c60a
I20260709 13:09:22.542438 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet ec9764b0b7dd4b67a5b50a652203c60a (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:23.585572 22779 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:23.593595 22631 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:23.593920 22706 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:23.594309 22747 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 887cfff73fab4d52b2e39f5b6089133a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.595579 23787 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.596578 23790 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.607740 23790 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.609787 22631 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.611512 22706 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.625898 22631 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:23.629081 22530 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.666886 22779 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:23.668223 22598 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 160d972ad1454f9db922aa360a0e7436. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.668768 23724 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:23.669456 22706 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:23.671566 23725 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.693065 23724 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.694548 22779 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.727213 22706 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.728811 22708 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:23.743438 22779 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:23.743925 22673 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 66c163cd962d4a568a04fc251b913b43. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.744278 22631 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:23.736750 22532 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.745635 23800 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.746695 23671 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.765106 23794 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.767081 22779 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.767992 22631 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.788610 22775 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:23.791889 22533 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:23.798831 22747 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 22e83136ddd24d71ad143a6f5fc28dd9. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:23.800203 22706 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:23.800246 22631 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:23.801882 23769 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:23.802963 23787 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:23.828107 23787 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.832302 22631 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.832595 22706 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:23.861327 22532 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:24.139115 23809 ts_tablet_manager.cc:933] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:24.142318 23809 tablet_copy_client.cc:323] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:24.144311 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet 887cfff73fab4d52b2e39f5b6089133a from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:24.144929 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet 887cfff73fab4d52b2e39f5b6089133a from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-887cfff73fab4d52b2e39f5b6089133a
I20260709 13:09:24.150398 23811 ts_tablet_manager.cc:933] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:24.154341 22794 tablet_copy_source_session.cc:215] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:24.157514 23811 tablet_copy_client.cc:323] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:24.158490 23809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887cfff73fab4d52b2e39f5b6089133a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:24.160560 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 160d972ad1454f9db922aa360a0e7436 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:24.161302 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 160d972ad1454f9db922aa360a0e7436 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-160d972ad1454f9db922aa360a0e7436
I20260709 13:09:24.177400 22645 tablet_copy_source_session.cc:215] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:24.179360 23809 tablet_copy_client.cc:806] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:24.180214 23809 tablet_copy_client.cc:670] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:24.184372 23811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 160d972ad1454f9db922aa360a0e7436. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:24.188153 23809 tablet_copy_client.cc:538] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:24.202665 23809 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:24.206041 23811 tablet_copy_client.cc:806] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:24.206940 23811 tablet_copy_client.cc:670] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:24.212733 23811 tablet_copy_client.cc:538] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:24.228055 23813 ts_tablet_manager.cc:933] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:24.231786 23811 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:24.267508 23813 tablet_copy_client.cc:323] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:24.280042 22719 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet 66c163cd962d4a568a04fc251b913b43 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:24.280709 22719 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet 66c163cd962d4a568a04fc251b913b43 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-66c163cd962d4a568a04fc251b913b43
I20260709 13:09:24.291307 23809 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:24.292845 23809 tablet_bootstrap.cc:492] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:24.302794 23809 ts_tablet_manager.cc:1403] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.100s	user 0.024s	sys 0.008s
I20260709 13:09:24.304477 22719 tablet_copy_source_session.cc:215] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:24.306483 23809 raft_consensus.cc:359] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.309214 23813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66c163cd962d4a568a04fc251b913b43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:24.307533 23811 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:24.318873 23811 tablet_bootstrap.cc:492] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:24.319975 23811 ts_tablet_manager.cc:1403] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.088s	user 0.017s	sys 0.022s
I20260709 13:09:24.317160 23809 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:24.321285 23809 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.326260 23811 raft_consensus.cc:359] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.329331 23811 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:24.328877 23809 ts_tablet_manager.cc:1434] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.002s
I20260709 13:09:24.330173 23811 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.336184 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-887cfff73fab4d52b2e39f5b6089133a received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:24.336728 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-887cfff73fab4d52b2e39f5b6089133a on tablet 887cfff73fab4d52b2e39f5b6089133a with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:24.346437 23813 tablet_copy_client.cc:806] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:24.347293 23813 tablet_copy_client.cc:670] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:24.355190 23811 ts_tablet_manager.cc:1434] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 13:09:24.355988 23813 tablet_copy_client.cc:538] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:24.357544 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-160d972ad1454f9db922aa360a0e7436 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:24.358079 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-160d972ad1454f9db922aa360a0e7436 on tablet 160d972ad1454f9db922aa360a0e7436 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:24.371279 23813 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:24.402094 23811 ts_tablet_manager.cc:933] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:24.413802 23811 tablet_copy_client.cc:323] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:24.416144 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet 22e83136ddd24d71ad143a6f5fc28dd9 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:24.416783 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet 22e83136ddd24d71ad143a6f5fc28dd9 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-22e83136ddd24d71ad143a6f5fc28dd9
I20260709 13:09:24.425074 22794 tablet_copy_source_session.cc:215] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:24.428678 23811 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22e83136ddd24d71ad143a6f5fc28dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:24.429086 23813 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:24.430223 23813 tablet_bootstrap.cc:492] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:24.431074 23813 ts_tablet_manager.cc:1403] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.026s
I20260709 13:09:24.435036 23813 raft_consensus.cc:359] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.435971 23813 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:24.436694 23813 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.439666 23813 ts_tablet_manager.cc:1434] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:09:24.443081 22719 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-66c163cd962d4a568a04fc251b913b43 received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:24.443567 22719 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-66c163cd962d4a568a04fc251b913b43 on tablet 66c163cd962d4a568a04fc251b913b43 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:24.450009 23811 tablet_copy_client.cc:806] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:24.452889 23811 tablet_copy_client.cc:670] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:24.480232 23811 tablet_copy_client.cc:538] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:24.489850 23811 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:24.510535 23811 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:24.511691 23811 tablet_bootstrap.cc:492] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:24.512523 23811 ts_tablet_manager.cc:1403] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.004s
I20260709 13:09:24.515759 23811 raft_consensus.cc:359] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.516645 23811 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:24.517308 23811 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:24.522326 23811 ts_tablet_manager.cc:1434] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:09:24.524657 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-22e83136ddd24d71ad143a6f5fc28dd9 received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:24.525183 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-22e83136ddd24d71ad143a6f5fc28dd9 on tablet 22e83136ddd24d71ad143a6f5fc28dd9 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:24.593858 23420 raft_consensus.cc:1217] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:24.668771 23420 raft_consensus.cc:1217] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:24.761911 23420 raft_consensus.cc:1217] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:25.023306 23420 raft_consensus.cc:1217] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:25.188735 23788 raft_consensus.cc:1064] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:25.191403 23788 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.199635 23420 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.199832 22775 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.200523 22710 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.201479 23725 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:25.202559 23788 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:25.203428 23822 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:25.208472 23794 raft_consensus.cc:1064] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:25.227200 23794 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.245066 23788 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.250252 22706 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.265461 23418 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.271250 22533 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:25.275876 23420 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.284308 22775 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.288118 22779 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.295706 23800 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.290166 22635 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.297942 23828 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:09:25.310246 23800 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.317631 23787 raft_consensus.cc:1064] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:25.336715 23787 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.381763 23800 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.386770 22783 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.410796 22635 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.488202 23769 raft_consensus.cc:1064] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:25.494738 23769 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.528136 22628 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.534765 23769 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.554059 23420 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.582499 23419 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.586289 22706 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.593016 22631 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.596684 22702 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.600564 23418 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:25.601382 23837 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.604206 23790 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.605448 23835 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:25.606448 22532 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.608194 23769 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:25.615411 23835 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:25.642583 23790 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.645841 23418 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.647547 22710 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.674468 22530 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.685690 22779 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.687778 23794 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:25.700660 23800 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.704195 22783 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.771853 22631 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.772763 23418 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.786659 23838 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.790966 23828 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:25.796864 22628 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.801780 23418 raft_consensus.cc:2955] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.802820 22702 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.819962 22779 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:25.827877 22633 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.830081 23420 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.832263 23837 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:25.853976 22532 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.859218 23787 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:25.875770 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 64105.5us
W20260709 13:09:25.876669 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58948.5us
I20260709 13:09:25.828161 23419 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.880023 22635 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.879391 23420 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.899250 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 22e83136ddd24d71ad143a6f5fc28dd9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:25.903379 23421 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.914250 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 66c163cd962d4a568a04fc251b913b43 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:25.917397 22779 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:25.922684 23725 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:25.946142 23787 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:25.952006 23851 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:09:25.957813 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet 66c163cd962d4a568a04fc251b913b43 with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:25.957864 22532 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.969815 23853 tablet_replica.cc:333] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:25.971169 23853 raft_consensus.cc:2243] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:25.972240 23853 raft_consensus.cc:2272] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:25.977020 23822 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.979099 22779 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.978354 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 22e83136ddd24d71ad143a6f5fc28dd9 with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:25.982806 22532 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:25.984961 22633 raft_consensus.cc:2955] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:25.996402 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54872.3us
I20260709 13:09:26.013545 23854 tablet_replica.cc:333] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:26.013903 22779 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:26.006114 23853 ts_tablet_manager.cc:1916] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:26.020983 23420 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:26.023007 23843 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:26.025313 23854 raft_consensus.cc:2243] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:26.030447 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 160d972ad1454f9db922aa360a0e7436 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:26.031560 23836 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.034188 23421 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.041100 23854 raft_consensus.cc:2272] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:26.054900 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 887cfff73fab4d52b2e39f5b6089133a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:26.056294 23854 ts_tablet_manager.cc:1916] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:26.067457 23419 raft_consensus.cc:2955] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.070415 22530 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:26.076959 22702 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.096530 22530 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.118870 22708 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:26.132216 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet 887cfff73fab4d52b2e39f5b6089133a with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:26.134171 23853 ts_tablet_manager.cc:1929] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:26.135455 23853 log.cc:1199] T 66c163cd962d4a568a04fc251b913b43 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/66c163cd962d4a568a04fc251b913b43
I20260709 13:09:26.137532 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet 66c163cd962d4a568a04fc251b913b43 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:26.137552 23853 tablet_replica.cc:333] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:26.139529 23853 raft_consensus.cc:2243] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:26.149427 23853 raft_consensus.cc:2272] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:26.166805 23836 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:26.174460 23854 ts_tablet_manager.cc:1929] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:26.175417 23854 log.cc:1199] T 22e83136ddd24d71ad143a6f5fc28dd9 P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/22e83136ddd24d71ad143a6f5fc28dd9
I20260709 13:09:26.176250 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 160d972ad1454f9db922aa360a0e7436 with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:26.177577 23854 tablet_replica.cc:333] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:26.177630 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 22e83136ddd24d71ad143a6f5fc28dd9 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:26.181489 23854 raft_consensus.cc:2243] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:26.183769 23854 raft_consensus.cc:2272] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:26.185683 23853 ts_tablet_manager.cc:1916] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:26.188491 23854 ts_tablet_manager.cc:1916] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:26.203504 22704 raft_consensus.cc:2955] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:26.206333 23853 ts_tablet_manager.cc:1929] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:26.207093 23853 log.cc:1199] T 887cfff73fab4d52b2e39f5b6089133a P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/887cfff73fab4d52b2e39f5b6089133a
I20260709 13:09:26.209168 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet 887cfff73fab4d52b2e39f5b6089133a (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:26.216919 23854 ts_tablet_manager.cc:1929] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:26.217512 23854 log.cc:1199] T 160d972ad1454f9db922aa360a0e7436 P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/160d972ad1454f9db922aa360a0e7436
I20260709 13:09:26.224140 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 160d972ad1454f9db922aa360a0e7436 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:27.252919 22628 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:27.281337 22779 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:09:27.281471 22598 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 308af34f5c0144e185e0af82342c7f92. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.282950 23788 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.283886 22704 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:27.285262 23788 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:27.290792 23855 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.292702 22779 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.295169 22704 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.311105 22779 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:27.312723 22533 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.319032 22708 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:27.319063 22747 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9ac0029002284e64a5fde523ffaf77fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.320242 22631 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:27.321125 23837 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:27.323930 23837 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:27.336056 23861 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.337491 22702 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.370641 22633 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:27.388867 22631 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.392400 22779 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:27.393090 22702 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:27.395629 23849 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:27.403378 23788 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:27.407552 22598 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4f6afadc577c4fd08e4345a82ae50e5f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.419099 22531 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.420074 23788 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.444567 22702 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.454310 22779 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.456333 22778 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:27.468348 22531 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.483649 22702 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:27.484257 22747 consensus_peers.cc:597] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ffde8dbdd2d54723860d280e3e916e60. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:27.485423 23862 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:27.499444 22631 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:27.501361 23866 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:27.503482 23861 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.505544 22708 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.551352 22628 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:27.557925 22533 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:27.607620 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:27.694060 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:27.694932 22631 raft_consensus.cc:606] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:27.697598 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:27.698165 22631 raft_consensus.cc:606] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:27.701179 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:27.701792 22631 raft_consensus.cc:606] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:27.718566 22779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:27.719306 22779 raft_consensus.cc:606] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:27.722174 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:27.722826 22631 raft_consensus.cc:606] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:27.725972 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c015ab1d14c443258576403d9d39c485"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:27.726765 22631 raft_consensus.cc:606] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:27.757063 22708 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:27.757763 22708 raft_consensus.cc:606] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:27.765271 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:09:27.767572 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:27.788642 23788 raft_consensus.cc:993] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:27.789796 23788 raft_consensus.cc:1081] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:27.798919 23421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c015ab1d14c443258576403d9d39c485"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:27.799654 23421 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:27.800015 23421 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.807737 23421 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:27.810525 23421 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:27.847810 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:27.849910 22628 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:27.850500 22628 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:27.857188 22628 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:27.863298 22628 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.884562 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:27.885902 22702 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:27.954190 23884 ts_tablet_manager.cc:933] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.010044 23886 ts_tablet_manager.cc:933] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:28.010138 23882 raft_consensus.cc:993] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.024931 22628 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.036970 22702 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.052035 23885 ts_tablet_manager.cc:933] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:28.065042 23889 ts_tablet_manager.cc:933] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.076313 23882 raft_consensus.cc:1081] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.065095 23849 raft_consensus.cc:993] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:09:28.133239 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67097.1us
I20260709 13:09:28.136981 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:28.147051 23421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:28.148072 23421 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.148667 23421 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.153344 23886 tablet_copy_client.cc:323] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:28.156001 23788 raft_consensus.cc:1081] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:28.163070 23885 tablet_copy_client.cc:323] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:28.187701 22702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:09:28.188501 22702 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.189010 22702 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.190521 23800 raft_consensus.cc:993] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.191219 23800 raft_consensus.cc:1081] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.195756 23419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46262
I20260709 13:09:28.196635 23419 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.197113 23419 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.215190 23889 tablet_copy_client.cc:323] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:28.222857 23788 raft_consensus.cc:993] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.223632 23788 raft_consensus.cc:1081] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.229456 23421 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:28.262583 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132089us
I20260709 13:09:28.279968 23884 tablet_copy_client.cc:323] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.131:39075
I20260709 13:09:28.299408 23882 raft_consensus.cc:993] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.300284 23882 raft_consensus.cc:1081] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:28.294219 22702 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:28.311606 23421 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:28.326432 23421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:28.327183 23421 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.327585 23421 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.333298 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 308af34f5c0144e185e0af82342c7f92 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:28.334065 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 308af34f5c0144e185e0af82342c7f92 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-308af34f5c0144e185e0af82342c7f92
I20260709 13:09:28.336087 23420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:28.337649 23420 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.338109 23420 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.339231 22644 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 4f6afadc577c4fd08e4345a82ae50e5f from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:28.340004 22644 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 4f6afadc577c4fd08e4345a82ae50e5f from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-4f6afadc577c4fd08e4345a82ae50e5f
I20260709 13:09:28.348371 23421 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.352075 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:28.352428 23421 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:28.352793 22635 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.353197 22635 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:28.355139 22644 tablet_copy_source_session.cc:215] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:28.354108 22635 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.357434 22635 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.357851 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:28.358497 22631 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.358842 22631 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:28.359095 23901 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.359719 23885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4f6afadc577c4fd08e4345a82ae50e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:28.368844 22631 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.372579 22631 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.431253 23419 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.459321 23901 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:28.542434 22645 tablet_copy_source_session.cc:215] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:28.639472 22702 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.659483 22635 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.669840 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:28.671411 23902 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.675405 23886 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 308af34f5c0144e185e0af82342c7f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:28.670714 23903 raft_consensus.cc:993] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:28.695490 23902 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:28.697072 23902 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.677672 23903 raft_consensus.cc:1081] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:28.711753 22635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:28.726177 22635 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:28.726518 23885 tablet_copy_client.cc:806] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:28.726974 22635 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.727322 23885 tablet_copy_client.cc:670] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:28.723584 23886 tablet_copy_client.cc:806] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:28.728164 23886 tablet_copy_client.cc:670] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:28.733675 23886 tablet_copy_client.cc:538] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:28.745461 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:28.746282 22632 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.746672 22632 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:28.747987 22632 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:28.749269 22632 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.751443 23886 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:28.759102 22635 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:28.764957 22633 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:28.765228 23420 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.767645 23420 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.770076 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:28.770843 22776 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.760317 22631 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.771021 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:28.771344 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:28.771752 22777 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.772053 22783 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.774595 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:28.775942 23902 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.776752 23902 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:28.777925 23902 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.776397 23419 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.547688 23901 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.786798 22635 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:28.787804 23837 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:28.789055 23837 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.789047 22783 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.789438 23837 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:28.790422 23837 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:28.792696 23837 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.796175 23417 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:28.779852 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:28.802868 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:28.804453 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:28.805202 23907 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:28.805194 22702 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.803566 22631 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.807906 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:28.809603 22631 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:28.811762 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:28.812490 22710 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.807255 23800 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:09:28.819681 23894 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:28.819722 22776 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:28.820271 23894 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:28.821264 23894 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.822418 23894 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.827966 22631 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.829155 22631 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:28.830211 22776 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:28.837625 22702 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.848819 22631 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.839350 22632 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:28.871104 23907 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:28.872515 23907 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.877081 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:28.880764 22777 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.884780 22775 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:28.896214 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:28.902400 23912 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:09:28.921548 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 88648.6us
I20260709 13:09:28.913290 23912 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:28.934080 22705 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:28.939891 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:28.943295 22793 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet 9ac0029002284e64a5fde523ffaf77fb from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:28.944161 22793 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet 9ac0029002284e64a5fde523ffaf77fb from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-9ac0029002284e64a5fde523ffaf77fb
I20260709 13:09:28.951182 23912 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:28.972137 22704 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.974489 22710 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:28.975459 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:28.978907 22706 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:28.987118 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:28.989540 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49867.2us
I20260709 13:09:28.999662 23916 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:29.020025 23916 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:29.021407 23916 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:29.053557 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:29.056059 22793 tablet_copy_source_session.cc:215] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:29.057392 22794 tablet_copy_service.cc:140] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Received BeginTabletCopySession request for tablet ffde8dbdd2d54723860d280e3e916e60 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:42226)
I20260709 13:09:29.058049 22794 tablet_copy_service.cc:161] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Beginning new tablet copy session on tablet ffde8dbdd2d54723860d280e3e916e60 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:42226: session id = 6a31ac6451ff4f73bee634487d84d001-ffde8dbdd2d54723860d280e3e916e60
I20260709 13:09:29.065243 22781 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:29.050436 22706 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:29.077270 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:29.078881 23788 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:29.079406 23788 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:29.080332 23788 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:29.080977 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 44549.5us
I20260709 13:09:29.084138 23885 tablet_copy_client.cc:538] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:29.091843 23418 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:29.106262 23884 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ac0029002284e64a5fde523ffaf77fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:29.098392 23886 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:29.119568 23886 tablet_bootstrap.cc:492] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:29.121992 23886 ts_tablet_manager.cc:1403] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.371s	user 0.035s	sys 0.018s
I20260709 13:09:29.120826 22707 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.125957 23907 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:29.171262 22794 tablet_copy_source_session.cc:215] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:29.171998 22531 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:29.174082 22531 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:29.176468 22531 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:29.180759 22531 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:29.184123 23889 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffde8dbdd2d54723860d280e3e916e60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:29.183409 22531 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:29.188257 23886 raft_consensus.cc:359] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:29.215011 23886 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:29.215816 23886 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:29.228816 22708 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.232649 23907 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:09:29.239322 23885 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:29.250221 23886 ts_tablet_manager.cc:1434] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.128s	user 0.009s	sys 0.001s
I20260709 13:09:29.258145 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-308af34f5c0144e185e0af82342c7f92 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:29.258867 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-308af34f5c0144e185e0af82342c7f92 on tablet 308af34f5c0144e185e0af82342c7f92 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:29.285552 22533 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:29.288937 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:29.322887 23884 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46274 (request call id 130) took 1446 ms. Trace:
I20260709 13:09:29.323473 23884 rpcz_store.cc:276] 0709 13:09:27.876457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.876806 (+   349us) service_pool.cc:224] Handling call
0709 13:09:27.954864 (+ 78058us) ts_tablet_manager.cc:934] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
0709 13:09:29.085225 (+1130361us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:29.271463 (+186238us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:29.271634 (+   171us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:29.322047 (+ 50413us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:29.322376 (+   329us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:29.322731 (+   355us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:29.322789 (+    58us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":48893,"tablet-copy.queue_time_us":76008,"thread_start_us":307302,"threads_started":2}
I20260709 13:09:29.325165 23884 tablet_copy_client.cc:806] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:29.325819 23884 tablet_copy_client.cc:670] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:29.334143 22708 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.349370 22631 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.366132 22632 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.521457 22781 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:29.338356 23901 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.175s
W20260709 13:09:29.564603 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 142466us
I20260709 13:09:29.527160 23882 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:09:29.576220 22530 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:29.583413 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800039680 after lost signal to thread 21529
W20260709 13:09:29.590685 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800196340 after lost signal to thread 22567
W20260709 13:09:29.592346 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4720 after lost signal to thread 22659
W20260709 13:09:29.593725 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800291980 after lost signal to thread 22734
I20260709 13:09:29.592645 23889 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46274 (request call id 132) took 1624 ms. Trace:
W20260709 13:09:29.618870 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080019ec00 after lost signal to thread 22808
W20260709 13:09:29.621351 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800346ba0 after lost signal to thread 23445
W20260709 13:09:29.622008 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800344320 after lost signal to thread 23864
W20260709 13:09:29.622397 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800346dc0 after lost signal to thread 23867
W20260709 13:09:29.622720 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08003441a0 after lost signal to thread 23870
I20260709 13:09:29.633352 22706 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.651098 22706 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:29.659479 22631 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:29.660362 22632 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.656411 23912 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:09:29.682719 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 58872.4us
I20260709 13:09:29.697162 23800 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:29.697507 22779 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.696995 23926 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:09:29.700017 23415 raft_consensus.cc:1217] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 13:09:29.700619 23855 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:09:29.700934 22779 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.709479 23927 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:09:29.719255 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45145.8us
I20260709 13:09:29.609381 23889 rpcz_store.cc:276] 0709 13:09:27.968325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.973113 (+  4788us) service_pool.cc:224] Handling call
0709 13:09:28.065682 (+ 92569us) ts_tablet_manager.cc:934] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
0709 13:09:29.179032 (+1113350us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:29.225053 (+ 46021us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:29.225214 (+   161us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:29.591666 (+366452us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:29.591987 (+   321us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:29.592480 (+   493us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:29.592572 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":68886,"tablet-copy.queue_time_us":90747,"thread_start_us":91556,"threads_started":2}
I20260709 13:09:29.726852 22635 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:29.698690 23912 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:09:29.733991 22633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:29.736260 22633 raft_consensus.cc:606] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:29.736688 22633 raft_consensus.cc:614] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:29.736073 23889 tablet_copy_client.cc:806] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:29.763155 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:29.756187 23889 tablet_copy_client.cc:670] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:29.778872 23926 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:09:29.814795 22779 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:09:29.915055 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 48877.6us
I20260709 13:09:29.916206 23926 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:29.920976 23884 tablet_copy_client.cc:538] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:09:29.927186 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 53200.4us
I20260709 13:09:29.932646 23912 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:09:29.966495 23938 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:09:30.002163 22635 raft_consensus.cc:606] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:30.002710 22635 raft_consensus.cc:614] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:30.065879 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:30.067058 22627 raft_consensus.cc:606] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:30.067798 22627 raft_consensus.cc:614] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:30.069417 23889 tablet_copy_client.cc:538] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:30.093356 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:30.094178 22632 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:30.109642 23885 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:30.118475 23935 raft_consensus.cc:1064] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
W20260709 13:09:30.170794 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 314530us
I20260709 13:09:30.154577 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c015ab1d14c443258576403d9d39c485"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:30.184458 23885 tablet_bootstrap.cc:492] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:30.192302 23885 ts_tablet_manager.cc:1403] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.953s	user 0.032s	sys 0.025s
I20260709 13:09:30.193748 23883 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
W20260709 13:09:30.280891 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 109583us
W20260709 13:09:30.306746 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:30.347875 23884 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
W20260709 13:09:30.372308 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53350.8us
W20260709 13:09:30.509886 23957 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:09:30.571729 23959 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:09:30.599772 22627 raft_consensus.cc:606] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:30.600351 22627 raft_consensus.cc:614] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:30.604069 22702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca809087a6724923ab06ace347425d31"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:30.604918 22702 raft_consensus.cc:606] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:30.620757 22784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:30.621537 22784 raft_consensus.cc:606] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:30.644002 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:09:30.633380 23935 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:30.660728 23885 raft_consensus.cc:359] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:30.679657 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:30.670192 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 195) took 1327 ms. Trace:
I20260709 13:09:30.679402 23885 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:30.650059 23889 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:30.699756 22530 rpcz_store.cc:276] 0709 13:09:29.342503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.342760 (+   257us) service_pool.cc:224] Handling call
0709 13:09:30.670059 (+1327299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:29.923628 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:29.932181 (+  8553us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:29.935369 (+  3188us) write_op.cc:435] Acquired schema lock
0709 13:09:29.935852 (+   483us) tablet.cc:662] Decoding operations
0709 13:09:29.960013 (+ 24161us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:29.961077 (+  1064us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:29.961466 (+   389us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:29.966161 (+  4695us) tablet.cc:701] Row locks acquired
0709 13:09:29.966186 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:09:29.966349 (+   163us) write_op.cc:270] Start()
0709 13:09:29.966506 (+   157us) write_op.cc:276] Timestamp: P: 1783602569966303 usec, L: 0
0709 13:09:29.966529 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:09:30.028442 (+ 61913us) log.cc:844] Serialized 538 byte log entry
0709 13:09:30.147175 (+118733us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed8e28 00007f180a913581 00007f180a912d58 00007f180a9179f1 00007f180a9179a9 00007f180a917939 00007f180a917901 00007f180a916b2d 000000000038f354 000000000038f199
0709 13:09:30.545930 (+398755us) op_driver.cc:464] REPLICATION: finished
0709 13:09:30.572692 (+ 26762us) write_op.cc:301] APPLY: starting
0709 13:09:30.572816 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:30.572925 (+   109us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:30.572954 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:30.573306 (+   352us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:30.573526 (+   220us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:30.573593 (+    67us) write_op.cc:312] APPLY: finished
0709 13:09:30.573853 (+   260us) log.cc:844] Serialized 23 byte log entry
0709 13:09:30.574094 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:30.574211 (+   117us) write_op.cc:454] Released schema lock
0709 13:09:30.574269 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:30.574795 (+   526us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:30.574837 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":3328,"child_traces":[["op",{"apply.queue_time_us":3734,"apply.run_cpu_time_us":1966,"apply.run_wall_time_us":1965,"num_ops":1,"prepare.queue_time_us":315036,"prepare.run_cpu_time_us":6155,"prepare.run_wall_time_us":223710,"replication_time_us":566282,"spinlock_wait_cycles":48710528}]]}
W20260709 13:09:30.562120 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:30.747382 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 130322us
W20260709 13:09:30.779418 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 169720us
I20260709 13:09:30.703491 23885 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:30.932505 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 146250us
W20260709 13:09:30.949604 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 139135us
I20260709 13:09:30.995893 23788 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:09:30.958840 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 105242us
I20260709 13:09:30.993395 23889 tablet_bootstrap.cc:700] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Time spent opening tablet: real 0.151s	user 0.002s	sys 0.006s
W20260709 13:09:31.056815 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87253.9us
I20260709 13:09:31.024542 23896 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:09:31.057998 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48365.9us
I20260709 13:09:31.100024 23885 ts_tablet_manager.cc:1434] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.907s	user 0.013s	sys 0.008s
W20260709 13:09:31.100611 23885 ts_tablet_manager.cc:1467] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Tablet startup took 907ms
W20260709 13:09:31.112330 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 147410us
W20260709 13:09:31.128126 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70288.5us
I20260709 13:09:31.168956 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
W20260709 13:09:31.203745 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51448.3us
W20260709 13:09:31.203828 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 50641.9us
W20260709 13:09:31.204684 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52259us
W20260709 13:09:31.208556 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49427.3us
I20260709 13:09:31.203080 22706 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:31.211508 22706 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:31.101046 23885 ts_tablet_manager.cc:1469] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Trace:
0709 13:09:27.937395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.937523 (+   128us) service_pool.cc:224] Handling call
0709 13:09:28.052702 (+115179us) ts_tablet_manager.cc:934] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
0709 13:09:28.357911 (+305209us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:28.703886 (+345975us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:28.704099 (+   213us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:28.725269 (+ 21170us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:28.725632 (+   363us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:28.726235 (+   603us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:28.726321 (+    86us) inbound_call.cc:177] Queueing success response
0709 13:09:29.238899 (+512578us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:09:30.192653 (+953754us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:09:30.376599 (+183946us) tablet_replica.cc:261] Starting instrumentation
0709 13:09:30.534995 (+158396us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":36833,"mutex_wait_us":21346,"raft.queue_time_us":61833,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":144,"tablet-copy.queue_time_us":45876,"tablet-download-pool.queue_time_us":250988,"tablet-download-pool.run_cpu_time_us":2055,"tablet-download-pool.run_wall_time_us":95491,"thread_start_us":45373,"threads_started":2}
I20260709 13:09:31.226307 23800 raft_consensus.cc:993] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:31.227880 23933 raft_consensus.cc:1081] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:31.245535 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:31.249521 22627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca809087a6724923ab06ace347425d31"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:31.250252 22627 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:31.250803 22627 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.253551 23971 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:09:31.255028 23418 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:31.265017 23964 raft_consensus.cc:993] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:31.265758 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-4f6afadc577c4fd08e4345a82ae50e5f received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:31.266273 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-4f6afadc577c4fd08e4345a82ae50e5f on tablet 4f6afadc577c4fd08e4345a82ae50e5f with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:31.269325 23881 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:09:31.283738 23964 raft_consensus.cc:1081] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
W20260709 13:09:31.289433 23984 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:09:31.329682 22706 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.338531 23416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46274
I20260709 13:09:31.339377 23416 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:31.339818 23416 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.346843 22706 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:31.352432 22702 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:31.372120 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:31.374025 22632 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:31.374346 22632 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:31.354179 23416 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:31.424131 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44375.9us
I20260709 13:09:31.431080 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:31.432004 22783 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.449309 22632 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.450408 23935 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:31.362735 22627 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.459595 23416 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:31.466440 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 109384us
I20260709 13:09:31.475572 22783 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:31.478552 23866 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:31.475904 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:31.479974 22709 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.479319 23866 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:31.489821 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:31.490564 22783 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:31.490928 22783 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:31.491736 22783 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:31.493182 22783 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.493315 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:31.503280 22775 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:31.515241 22783 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:31.516292 22628 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:31.502491 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:31.522640 22632 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.527196 22777 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:31.536003 23892 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:31.561124 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:31.562028 22704 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:31.562424 22704 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:31.561239 23417 raft_consensus.cc:1217] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:31.538321 23384 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:31.559520 23988 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:09:31.564749 22704 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.574864 22632 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:31.575194 22704 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.594841 23969 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:31.595808 23969 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:31.597184 23969 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:31.562984 23881 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:09:31.622473 22709 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
W20260709 13:09:31.627121 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66452.2us
W20260709 13:09:31.585892 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52815.3us
I20260709 13:09:31.626514 22627 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:31.627277 23892 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:31.642349 23892 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.653993 23884 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:31.659834 23884 tablet_bootstrap.cc:492] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:31.667604 23884 ts_tablet_manager.cc:1403] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 1.320s	user 0.034s	sys 0.030s
I20260709 13:09:31.668056 23866 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:31.684640 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:31.696406 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:31.700374 22704 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:31.705401 23889 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:31.719012 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:31.719794 22783 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:31.725478 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:31.735498 22750 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:31.737708 22783 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:31.731979 23889 tablet_bootstrap.cc:492] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:31.748752 23849 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:31.779930 22709 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:31.776453 23849 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:31.788084 23849 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:31.803689 23982 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:31.775489 23889 ts_tablet_manager.cc:1403] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 1.168s	user 0.027s	sys 0.026s
I20260709 13:09:31.861721 22531 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:31.905043 23889 raft_consensus.cc:359] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:31.906100 23889 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:31.720631 23966 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:31.910897 23884 raft_consensus.cc:359] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:31.911903 23884 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:31.912645 23884 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:31.756824 23415 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:31.975152 23884 ts_tablet_manager.cc:1434] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.307s	user 0.006s	sys 0.004s
I20260709 13:09:31.943642 23889 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:31.979856 23889 ts_tablet_manager.cc:1434] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.142s	user 0.008s	sys 0.000s
W20260709 13:09:31.989270 23995 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:32.005961 22533 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:32.006846 22794 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-9ac0029002284e64a5fde523ffaf77fb received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:32.017904 22794 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-9ac0029002284e64a5fde523ffaf77fb on tablet 9ac0029002284e64a5fde523ffaf77fb with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:32.016381 22779 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.019773 22793 tablet_copy_service.cc:342] P a0e9f7dc2a2e4bdb9287d910e4b491f7: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-ffde8dbdd2d54723860d280e3e916e60 received from {username='slave'} at 127.0.0.1:42226
I20260709 13:09:32.020287 22793 tablet_copy_service.cc:434] P a0e9f7dc2a2e4bdb9287d910e4b491f7: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-ffde8dbdd2d54723860d280e3e916e60 on tablet ffde8dbdd2d54723860d280e3e916e60 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:32.011965 22532 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:32.068989 23418 raft_consensus.cc:1217] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:32.089131 22530 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.124135 23985 raft_consensus.cc:1064] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:32.133322 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:32.144251 23985 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:32.173956 22782 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.176071 23896 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:32.181667 22709 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.183861 23935 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:32.186937 23415 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.192597 23896 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:09:32.225910 22704 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:32.236200 22704 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:32.240302 23997 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:09:32.235375 23896 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:09:32.244527 22782 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:32.248023 23927 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
W20260709 13:09:32.333619 24007 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:09:32.343418 23966 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.357371 22783 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:32.369573 23985 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:32.372507 22783 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.375486 22782 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:32.375331 23421 raft_consensus.cc:1217] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 13:09:32.380378 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 44559.9us
I20260709 13:09:32.382292 22632 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:32.408473 23978 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:32.409865 23800 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:32.422254 22632 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:32.444698 22704 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:32.466959 23421 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:32.463991 22702 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.489500 23415 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.472208 23966 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:09:32.499071 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53621.8us
I20260709 13:09:32.501664 23849 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:32.524539 22531 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:32.574208 23881 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.599036 22709 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.614512 23421 raft_consensus.cc:2955] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:32.721788 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 65224.1us
I20260709 13:09:32.769472 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 308af34f5c0144e185e0af82342c7f92 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:32.816458 23887 raft_consensus.cc:1064] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:32.797874 22531 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 4 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:32.820744 23887 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:32.844583 23415 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.848723 22710 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.852890 23887 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:09:32.856212 22635 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:32.856603 23980 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:32.863690 23418 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:32.867882 23935 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:32.877451 23896 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.883423 23418 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.894234 22710 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:32.896284 23935 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:32.904253 22779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:32.905452 22779 raft_consensus.cc:606] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:32.909792 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 308af34f5c0144e185e0af82342c7f92 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:32.939476 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:32.941586 22635 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:32.943517 22634 raft_consensus.cc:606] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:32.953195 23866 raft_consensus.cc:1064] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:32.956624 23866 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:32.971743 24029 tablet_replica.cc:333] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:32.973201 22704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:32.976475 22704 raft_consensus.cc:606] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:32.980161 22710 raft_consensus.cc:2955] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.981308 22704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:32.981967 22704 raft_consensus.cc:606] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:09:32.983294 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52776.5us
I20260709 13:09:32.983842 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4f6afadc577c4fd08e4345a82ae50e5f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:32.986217 23936 raft_consensus.cc:993] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:32.986907 23861 raft_consensus.cc:1081] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:32.989557 22709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47418
I20260709 13:09:32.990186 22709 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:32.990676 22709 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:32.990876 24024 raft_consensus.cc:993] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:32.991400 24023 raft_consensus.cc:1081] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:32.990049 23861 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.997450 23418 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:32.999193 22709 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.000490 22532 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.001473 22709 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:33.011456 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:33.012167 22634 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:33.012588 22634 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.017977 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:33.018800 22635 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.020026 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 4f6afadc577c4fd08e4345a82ae50e5f with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50366
I20260709 13:09:33.023698 22634 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.028682 22634 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:33.033332 22702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:33.034056 22702 raft_consensus.cc:606] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:33.036192 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:33.037752 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:33.034053 22635 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:33.038455 22702 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:33.038904 22702 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:33.040007 22702 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.040584 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:33.041217 22702 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.041399 22779 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.044658 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:33.049501 22702 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.048406 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:33.052507 24024 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:33.053401 24024 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:33.054525 24024 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.065205 22779 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.068921 23936 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:33.067180 22533 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.076967 23415 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:33.079119 22702 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:33.079051 23936 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:33.080857 23980 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:33.087718 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:33.088757 23935 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:33.089504 23935 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:33.090447 23935 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.117224 23890 raft_consensus.cc:993] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:33.118029 23950 raft_consensus.cc:1081] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:33.126410 22628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:33.127095 22628 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:33.127466 22628 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.135136 22531 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.136579 22628 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.138799 22781 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:33.139117 22634 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.140774 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:33.141392 22781 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:33.141701 22781 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:33.143154 22781 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:33.144546 22781 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.147636 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:33.148429 22704 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:33.148806 22704 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:33.151714 22704 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.153086 22704 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.152572 22709 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.155740 22628 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:33.156761 24020 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.163065 22704 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.164439 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:33.148527 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:33.167258 23415 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.165319 23935 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:33.167308 22704 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.172746 23935 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:33.173841 23935 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.182982 22781 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:33.210440 22634 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:33.215176 23861 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:33.223260 23415 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.205883 22532 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.249001 22533 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.253971 24029 raft_consensus.cc:2243] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:33.255332 24029 raft_consensus.cc:2272] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:33.251894 22634 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.270116 23896 raft_consensus.cc:993] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:33.270509 24029 ts_tablet_manager.cc:1916] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:33.270783 23896 raft_consensus.cc:1081] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:33.256428 23415 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.272718 23417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:33.282909 23417 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:33.283370 23417 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:33.291558 24029 ts_tablet_manager.cc:1929] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:09:33.292143 24029 log.cc:1199] T 308af34f5c0144e185e0af82342c7f92 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/308af34f5c0144e185e0af82342c7f92
I20260709 13:09:33.294739 24029 tablet_replica.cc:333] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:09:33.296025 24029 raft_consensus.cc:2243] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:33.296868 24029 raft_consensus.cc:2272] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:33.301017 24029 ts_tablet_manager.cc:1916] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:33.306056 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 308af34f5c0144e185e0af82342c7f92 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:33.309188 23417 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.313035 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:33.313696 22634 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:33.314038 22634 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:33.315021 22634 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.316512 22634 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:33.322988 22634 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:33.323524 24029 ts_tablet_manager.cc:1929] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:33.324082 24029 log.cc:1199] T 4f6afadc577c4fd08e4345a82ae50e5f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/4f6afadc577c4fd08e4345a82ae50e5f
I20260709 13:09:33.324257 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:33.325435 24010 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:33.326005 24010 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:33.325901 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 4f6afadc577c4fd08e4345a82ae50e5f (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:33.326607 22532 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.327229 24010 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.323819 23417 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:33.344094 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:33.344951 22782 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:33.353313 22782 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:33.398336 22532 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:33.423048 22781 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.443132 23417 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:33.446589 24022 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:33.456766 23861 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.459551 23418 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.469198 22702 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:33.471124 23961 raft_consensus.cc:993] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:33.471691 23961 raft_consensus.cc:1081] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:33.475461 23866 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:33.475471 22628 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:33.504752 23950 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:33.508730 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ffde8dbdd2d54723860d280e3e916e60 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:33.530685 22702 raft_consensus.cc:2955] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.532408 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:33.534267 22634 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:33.534801 22634 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.546679 22634 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.550076 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:33.550829 23418 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.548914 22634 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:33.566850 23418 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.568480 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:33.570237 23935 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:33.570952 23935 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:33.574460 23935 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.575203 22532 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 3 to 4, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:33.593755 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:33.594511 22709 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:33.594936 22709 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:33.595834 22709 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.597414 22709 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:33.600636 22779 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:33.603003 22702 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:33.605995 22779 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:33.611477 23896 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:33.618098 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet ffde8dbdd2d54723860d280e3e916e60 with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45654
I20260709 13:09:33.619032 24041 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:33.633473 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52776.5us
I20260709 13:09:33.635667 24046 tablet_replica.cc:333] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:09:33.652944 22532 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:33.655143 22781 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:33.661819 24046 raft_consensus.cc:2243] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:33.662842 24046 raft_consensus.cc:2272] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:33.664407 23418 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:33.666765 22628 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:09:33.677645 23800 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:33.679567 22709 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:33.684976 23417 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:33.702263 24022 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:33.704524 23896 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:09:33.704551 24022 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:09:33.718312 24046 ts_tablet_manager.cc:1916] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:33.717243 23866 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.721637 23417 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.729462 22709 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:33.733342 22628 raft_consensus.cc:2955] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.767768 22779 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:33.771096 23997 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:33.781178 24041 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:33.840178 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9ac0029002284e64a5fde523ffaf77fb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:09:33.825810 22530 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 3 to 4, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 1 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:33.883724 24046 ts_tablet_manager.cc:1929] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:33.887475 24046 log.cc:1199] T ffde8dbdd2d54723860d280e3e916e60 P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/ffde8dbdd2d54723860d280e3e916e60
I20260709 13:09:33.891778 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet ffde8dbdd2d54723860d280e3e916e60 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
W20260709 13:09:33.926172 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62084.7us
I20260709 13:09:33.928475 22632 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:33.932824 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 9ac0029002284e64a5fde523ffaf77fb with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:47402
I20260709 13:09:33.942232 24010 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:34.015800 24060 tablet_replica.cc:333] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:09:34.017096 24060 raft_consensus.cc:2243] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:09:34.017972 24060 raft_consensus.cc:2272] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:09:34.022415 24060 ts_tablet_manager.cc:1916] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:09:34.045945 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:34.048349 24060 ts_tablet_manager.cc:1929] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:09:34.049005 24060 log.cc:1199] T 9ac0029002284e64a5fde523ffaf77fb P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/9ac0029002284e64a5fde523ffaf77fb
I20260709 13:09:34.067508 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 9ac0029002284e64a5fde523ffaf77fb (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:09:34.099335 23417 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:34.101779 23935 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:34.122690 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:34.124958 22710 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:34.128605 23935 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:34.211202 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:34.211860 22632 raft_consensus.cc:606] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:34.214967 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:34.215624 22632 raft_consensus.cc:606] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:34.218433 22783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:34.219175 22783 raft_consensus.cc:606] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:34.222321 22710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:34.223067 22710 raft_consensus.cc:606] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:34.225945 22710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:34.226606 22710 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:34.234860 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:34.235543 22632 raft_consensus.cc:606] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:34.240811 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:09:34.242955 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:34.251995 23861 raft_consensus.cc:993] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:34.252637 23861 raft_consensus.cc:1081] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:34.255541 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:34.256150 22634 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.256549 22634 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.265498 22634 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.268186 22634 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:34.268709 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:34.269383 22783 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:34.269810 22783 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:34.270687 22783 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.270816 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:34.271499 22710 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.271736 22783 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.278921 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:34.280148 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:34.280948 24041 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:34.281431 24041 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:34.281554 22710 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:34.282415 24041 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.296927 22532 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:34.307941 23950 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:34.308626 23961 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:34.310549 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:34.312100 22634 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.312697 22634 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:34.326184 22634 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.328336 22634 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:34.329564 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:34.331585 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:34.332386 22710 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:34.332842 22710 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:34.333710 22710 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.335125 22710 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:34.330451 22783 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:34.346535 22710 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:34.348940 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:34.349967 24041 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:34.350762 24041 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:34.351707 24041 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.358759 23935 raft_consensus.cc:993] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:34.359303 23935 raft_consensus.cc:1081] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:34.361191 23417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:09:34.361867 23417 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.366770 22783 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:34.362291 23417 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.396277 23896 raft_consensus.cc:993] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:34.403088 23896 raft_consensus.cc:1081] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:34.406598 22783 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:09:34.416201 22783 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.416632 22783 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.432631 23417 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.445927 23417 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:34.449776 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:34.450500 22632 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:34.450968 22632 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:34.451768 22632 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.453190 22632 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.471954 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:34.472721 22709 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:34.479091 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57372.3us
I20260709 13:09:34.480625 22709 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:34.481055 22632 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:34.483278 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:34.485255 24010 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:34.485860 24010 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:34.486853 24010 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.492086 22532 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:34.490433 22783 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.495960 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:34.496618 22635 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:34.497000 22635 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:34.496708 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:34.497907 22709 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.498063 22635 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.499907 22635 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.496996 22783 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:34.513887 22709 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:34.515369 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:34.515499 22635 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:34.516360 24022 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:34.516911 24022 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:34.518023 24022 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.543740 22532 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.576542 23961 raft_consensus.cc:993] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:34.577226 23950 raft_consensus.cc:1081] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:09:34.579418 23418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46262
I20260709 13:09:34.580150 23418 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.580535 23418 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.596772 23418 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.600445 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:34.600800 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:34.601485 22710 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:34.602293 23418 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:34.602221 22710 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:34.611143 22710 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.601533 22783 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.631220 22710 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.636823 22783 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:34.638147 22782 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:34.641093 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:34.643072 24067 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:34.644346 24067 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:34.646274 24067 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:34.664337 22710 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:34.667912 22530 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.672993 22531 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:34.704315 24066 raft_consensus.cc:993] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:34.704921 24041 raft_consensus.cc:1081] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:34.706697 22710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:09:34.707357 22710 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:34.707758 22710 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.717934 22710 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.721343 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:34.721578 22710 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:34.722092 22635 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:34.722483 22635 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:34.722752 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:34.723444 22783 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.723471 22635 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.724970 22635 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:34.733539 22635 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:34.741043 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:34.742137 24024 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:34.742875 24024 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:34.744233 24024 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:34.747393 22710 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:34.748838 24066 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:34.755836 22782 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:34.760739 23896 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:34.767002 22783 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:34.773778 22709 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:34.775941 24066 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:34.795018 22781 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:34.797048 24041 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:34.853174 22531 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:34.895859 22702 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:34.897411 24067 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:34.930936 22632 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:34.932758 24057 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:35.076248 22702 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:35.083640 22783 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:35.084170 22632 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:35.085493 23950 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:35.087693 24024 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:35.102510 22673 consensus_peers.cc:597] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a937bd6b17c04d5199f368b1d595cf8b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.103986 22710 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:35.106419 23866 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:35.116834 23950 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.128669 22782 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.157840 22632 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.160442 22635 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:35.174644 24022 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:35.192692 22635 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:35.195467 22629 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:35.197484 23961 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:35.218521 22710 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:09:35.219803 22598 consensus_peers.cc:597] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f33fee9d64384b499e55f8bc58339f69. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.220515 23896 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:35.220561 22781 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:09:35.231532 24066 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:35.235790 22531 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.236421 22781 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:35.242195 22702 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:35.244468 22782 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:35.246213 23950 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:35.246984 24010 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:35.248139 24057 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:35.244068 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:35.348333 23896 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:09:35.380836 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 128141us
I20260709 13:09:35.404475 22781 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.446192 22709 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.463804 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:35.467873 22629 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:35.526841 22782 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:35.528988 24066 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:35.529342 22531 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index -1 to 2, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.540485 22709 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:35.561441 23896 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:35.567806 24095 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.573053 22782 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.579453 22702 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.608888 22783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:35.614140 22783 raft_consensus.cc:606] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:09:35.584765 22598 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53c258bc80ce4d88bac804cb01a34b32. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.619484 22783 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:35.620157 22783 raft_consensus.cc:606] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:35.624508 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:35.625244 22634 raft_consensus.cc:606] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:35.629081 22710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:35.629830 22710 raft_consensus.cc:606] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:35.632578 22531 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.633692 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca809087a6724923ab06ace347425d31"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:35.641280 22634 raft_consensus.cc:606] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:35.642544 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:35.644513 23861 raft_consensus.cc:993] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:35.645051 23861 raft_consensus.cc:1081] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:35.644573 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:35.648676 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:35.649220 22634 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:35.649576 22634 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:35.629016 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 62800.3us
I20260709 13:09:35.663978 23935 raft_consensus.cc:993] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:35.664508 23935 raft_consensus.cc:1081] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:35.666371 22702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca809087a6724923ab06ace347425d31"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:09:35.670166 22634 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.671542 22702 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:35.674822 22702 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:35.676410 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:35.677047 23418 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.682904 22702 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.685220 22702 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:35.674330 22634 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:35.674140 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:35.687336 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:35.687747 22782 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:35.687969 22783 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:35.688146 22782 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:35.685320 23418 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:35.689072 22782 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.690412 22782 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.697896 22783 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:35.698719 22781 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:35.699476 22782 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:35.700261 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:35.700960 22634 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:35.700901 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:35.701452 22634 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:35.702339 23896 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:35.702291 22634 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.702967 23896 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:35.703560 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:35.703867 23896 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.704744 24083 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:35.706110 24083 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:35.707089 24083 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.710985 24066 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:35.713091 22634 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:35.726791 24103 ts_tablet_manager.cc:933] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:35.737884 22531 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.740679 24103 tablet_copy_client.cc:323] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:09:35.740885 22634 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:35.742692 24104 ts_tablet_manager.cc:933] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:35.742761 22702 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:35.755725 24104 tablet_copy_client.cc:323] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:35.758311 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet f33fee9d64384b499e55f8bc58339f69 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:35.759047 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet f33fee9d64384b499e55f8bc58339f69 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-f33fee9d64384b499e55f8bc58339f69
W20260709 13:09:35.770658 22673 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f4adfa257c34bb88d79bda9d39c5146. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:35.776083 22783 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:35.786556 22645 tablet_copy_source_session.cc:215] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:35.799813 22531 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:35.807804 24104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f33fee9d64384b499e55f8bc58339f69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:35.824190 24083 raft_consensus.cc:993] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:35.825196 22719 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet a937bd6b17c04d5199f368b1d595cf8b from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:35.825877 22719 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet a937bd6b17c04d5199f368b1d595cf8b from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-a937bd6b17c04d5199f368b1d595cf8b
I20260709 13:09:35.826263 22629 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:35.827855 24083 raft_consensus.cc:1081] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:35.835976 23896 raft_consensus.cc:993] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:35.836562 23896 raft_consensus.cc:1081] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:35.837540 22719 tablet_copy_source_session.cc:215] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:35.840883 24098 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:35.853163 24104 tablet_copy_client.cc:806] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:35.853936 24104 tablet_copy_client.cc:670] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:35.845407 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:09:35.855986 22782 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:35.856387 22782 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.860827 24104 tablet_copy_client.cc:538] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:35.865725 22782 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.870432 22782 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:35.870432 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:35.871459 22629 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:35.871785 22629 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:35.872709 22632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:35.873462 22632 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:35.872586 22629 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.873900 22632 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.880129 24103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a937bd6b17c04d5199f368b1d595cf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:35.886421 22632 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.886370 23961 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:09:35.890252 22632 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:35.893354 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:35.890441 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:35.901207 23418 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.894068 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:35.903903 22710 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:35.904474 22710 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:35.904690 24104 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:35.905344 22710 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.906131 22783 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.910555 22710 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.896317 22629 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:35.921669 22710 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:35.922402 23418 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:35.923580 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:35.925858 24066 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:35.927201 24066 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:35.928763 22629 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:35.930676 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:35.931926 24108 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:35.932603 24108 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:35.933565 24108 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:35.928512 24066 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:35.953290 22783 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:35.956147 24111 ts_tablet_manager.cc:933] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:09:35.973958 24103 tablet_copy_client.cc:806] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:35.979269 24083 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.981186 22632 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:35.979310 22783 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:09:36.004505 24103 tablet_copy_client.cc:670] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:36.009531 24103 tablet_copy_client.cc:538] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:35.990134 22531 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.020288 23861 raft_consensus.cc:993] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:36.020848 23861 raft_consensus.cc:1081] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:36.037292 24111 tablet_copy_client.cc:323] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:09:36.039604 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 53c258bc80ce4d88bac804cb01a34b32 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:09:36.040223 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 53c258bc80ce4d88bac804cb01a34b32 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-53c258bc80ce4d88bac804cb01a34b32
I20260709 13:09:36.044135 22530 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.069788 22629 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:36.070551 22629 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:36.070957 22629 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:36.052731 22533 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 3, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.082203 24104 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:36.083374 24104 tablet_bootstrap.cc:492] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:36.084326 24104 ts_tablet_manager.cc:1403] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.180s	user 0.032s	sys 0.023s
I20260709 13:09:36.087743 24104 raft_consensus.cc:359] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:36.088722 24104 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:36.089425 24104 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:36.091297 22629 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:36.082540 22645 tablet_copy_source_session.cc:215] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:36.098191 22632 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:36.098889 24111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53c258bc80ce4d88bac804cb01a34b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:36.099135 23417 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:36.100703 24104 ts_tablet_manager.cc:1434] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:09:36.107244 24083 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:09:36.108590 23935 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:09:36.114037 23417 raft_consensus.cc:1217] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:36.128718 24103 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:36.159775 22629 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:36.160653 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:36.161430 22783 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:36.161788 22783 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:36.162600 22783 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:36.164264 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:36.164966 23418 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:36.107015 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-f33fee9d64384b499e55f8bc58339f69 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:36.166033 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-f33fee9d64384b499e55f8bc58339f69 on tablet f33fee9d64384b499e55f8bc58339f69 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:36.183274 23418 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:36.185195 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:36.203151 22783 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:36.212349 24096 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:36.213274 24096 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:36.248020 22783 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:36.250836 22779 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:36.266906 23935 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:36.311606 24111 tablet_copy_client.cc:806] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:36.315016 24111 tablet_copy_client.cc:670] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:36.236512 24096 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:36.330477 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 80948.1us
I20260709 13:09:36.375265 22783 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:36.402293 24104 ts_tablet_manager.cc:933] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:36.432093 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 84412.8us
I20260709 13:09:36.467244 22779 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:36.474066 23896 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:36.483047 22709 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:36.485057 24122 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:36.519383 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141449us
I20260709 13:09:36.541553 22532 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.549130 22629 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:36.573514 24111 tablet_copy_client.cc:538] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:36.578266 24133 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:09:36.549269 24104 tablet_copy_client.cc:323] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
W20260709 13:09:36.602538 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66536.6us
I20260709 13:09:36.655069 24111 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:36.664810 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:36.592494 23936 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:09:36.687958 23417 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:36.692229 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 144408us
I20260709 13:09:36.706763 23936 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:09:36.719694 22719 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet 2f4adfa257c34bb88d79bda9d39c5146 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:09:36.720438 22719 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet 2f4adfa257c34bb88d79bda9d39c5146 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-2f4adfa257c34bb88d79bda9d39c5146
I20260709 13:09:36.730381 24136 raft_consensus.cc:1064] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
W20260709 13:09:36.774600 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52113.1us
I20260709 13:09:36.766557 22719 tablet_copy_source_session.cc:215] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:36.768672 24136 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:36.839315 24103 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:36.862176 24103 tablet_bootstrap.cc:492] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:36.865931 24103 ts_tablet_manager.cc:1403] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.738s	user 0.028s	sys 0.011s
W20260709 13:09:37.147706 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 200210us
W20260709 13:09:37.150091 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 111692us
W20260709 13:09:37.183375 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 232607us
W20260709 13:09:37.193516 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 150919us
W20260709 13:09:37.210075 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 432106us
I20260709 13:09:37.262427 22780 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:37.274595 23896 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:37.247464 24103 raft_consensus.cc:359] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:37.288836 24103 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:37.291677 22782 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:37.296535 24103 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:37.302129 24126 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:37.315673 22707 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:09:37.345567 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44580.7us
W20260709 13:09:37.352536 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 50985.9us
I20260709 13:09:37.360421 24103 ts_tablet_manager.cc:1434] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.494s	user 0.009s	sys 0.002s
W20260709 13:09:37.363390 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 155174us
W20260709 13:09:37.366024 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 185740us
I20260709 13:09:37.393793 24104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f4adfa257c34bb88d79bda9d39c5146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:37.372431 23421 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:37.374413 23416 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:37.397068 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 208171us
W20260709 13:09:37.390128 24103 ts_tablet_manager.cc:1467] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Tablet startup took 523ms
W20260709 13:09:37.428860 24103 ts_tablet_manager.cc:1469] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Trace:
0709 13:09:35.705812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.718795 (+ 12983us) service_pool.cc:224] Handling call
0709 13:09:35.727374 (+  8579us) ts_tablet_manager.cc:934] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
0709 13:09:35.878588 (+151214us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:35.902289 (+ 23701us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:35.902471 (+   182us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:35.972968 (+ 70497us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:35.973307 (+   339us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:35.973651 (+   344us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:35.973704 (+    53us) inbound_call.cc:177] Queueing success response
0709 13:09:36.128261 (+154557us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:09:36.866352 (+738091us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:09:36.918701 (+ 52349us) tablet_replica.cc:261] Starting instrumentation
0709 13:09:36.919443 (+   742us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":312,"raft.queue_time_us":51569,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":161,"tablet-copy.queue_time_us":7230,"tablet-download-pool.queue_time_us":148,"tablet-download-pool.run_cpu_time_us":1502,"tablet-download-pool.run_wall_time_us":4079,"thread_start_us":21194,"threads_started":2}
I20260709 13:09:37.435665 24111 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:37.436866 24111 tablet_bootstrap.cc:492] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:37.437674 24111 ts_tablet_manager.cc:1403] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 0.783s	user 0.026s	sys 0.006s
W20260709 13:09:37.451252 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 72352.6us
I20260709 13:09:37.479400 24132 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:09:37.559639 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 160589us
W20260709 13:09:37.639129 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 161337us
I20260709 13:09:37.651690 22719 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-a937bd6b17c04d5199f368b1d595cf8b received from {username='slave'} at 127.0.0.1:49444
I20260709 13:09:37.658648 22719 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-a937bd6b17c04d5199f368b1d595cf8b on tablet a937bd6b17c04d5199f368b1d595cf8b with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:37.662137 24163 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:37.662999 24163 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:37.665518 24163 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:37.706715 24166 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:38.109892 24123 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.111012 24123 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.114763 24123 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.121330 24162 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.122257 24162 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.125375 24162 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.131734 24168 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:09:38.132951 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 480148us
W20260709 13:09:38.135078 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 623251us
I20260709 13:09:38.151914 24175 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.153818 24066 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.154520 24066 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.157847 24066 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.158895 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.159754 24140 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.160863 24140 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.165793 24140 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.178891 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:38.184185 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.068s	sys 0.117s
W20260709 13:09:38.184741 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.068s	sys 0.117s
I20260709 13:09:38.184346 24175 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:38.185112 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 783090us
I20260709 13:09:38.187319 24175 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.190738 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.193261 24162 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.193940 24162 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.196352 24162 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.201037 24104 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46262 (request call id 271) took 1805 ms. Trace:
I20260709 13:09:38.201742 24104 rpcz_store.cc:276] 0709 13:09:36.395614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.401521 (+  5907us) service_pool.cc:224] Handling call
0709 13:09:36.402753 (+  1232us) ts_tablet_manager.cc:934] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
0709 13:09:37.392439 (+989686us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:38.180285 (+787846us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:38.180446 (+   161us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:38.197907 (+ 17461us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:38.200476 (+  2569us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:38.200880 (+   404us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:38.200957 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":42088,"tablet-copy.queue_time_us":164,"thread_start_us":94057,"threads_started":1}
I20260709 13:09:38.203670 24104 tablet_copy_client.cc:806] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:38.204380 24104 tablet_copy_client.cc:670] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:38.210330 24176 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.211055 24176 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.212996 24134 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.213773 24176 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.213547 24134 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.216326 24162 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.217324 24162 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.219065 24178 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.219708 24162 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.219878 24178 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.222451 24178 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.223692 24134 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.293078 24111 raft_consensus.cc:359] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.294190 24111 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:38.294996 24111 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.297837 24111 ts_tablet_manager.cc:1434] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.860s	user 0.019s	sys 0.009s
W20260709 13:09:38.298156 24111 ts_tablet_manager.cc:1467] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Tablet startup took 860ms
I20260709 13:09:38.348037 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:38.351567 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:38.360260 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.364966 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.370318 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.375566 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.378896 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.379973 22775 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:38.382987 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.398772 24098 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.404105 23961 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.400413 24098 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.410823 24083 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.411761 24083 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.413267 24098 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.413980 24108 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.415398 24123 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.415958 24024 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.416273 24123 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.416684 24024 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.410358 24067 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.405277 23961 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.424124 24133 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.423626 24067 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.406586 24152 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.432291 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 132) took 1624 ms. Trace:
I20260709 13:09:38.433022 22530 rpcz_store.cc:276] 0709 13:09:36.805504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.805787 (+   283us) service_pool.cc:224] Handling call
0709 13:09:38.426558 (+1620771us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":673249,"reported_tablets":1}
I20260709 13:09:38.424118 24162 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.425007 24160 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.434825 24171 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.422988 24172 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.420486 24066 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.437669 24148 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.423960 23896 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.425875 24099 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.428046 24133 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.423867 24163 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.420181 24083 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.445902 24148 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.443243 24162 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.420734 24108 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.444612 24066 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.425699 24141 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.448683 24163 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.443329 24024 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.434651 24152 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.439832 24171 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.441262 24160 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.469106 24160 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.464051 23936 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.451052 24099 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.481498 24148 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.455938 23896 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.487387 24133 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.462213 24141 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:38.298502 24111 ts_tablet_manager.cc:1469] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Trace:
0709 13:09:35.926807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.926985 (+   178us) service_pool.cc:224] Handling call
0709 13:09:35.969921 (+ 42936us) ts_tablet_manager.cc:934] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
0709 13:09:36.097417 (+127496us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:36.109854 (+ 12437us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:36.109995 (+   141us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:36.310299 (+200304us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:36.310684 (+   385us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:36.311179 (+   495us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:36.311275 (+    96us) inbound_call.cc:177] Queueing success response
0709 13:09:36.548358 (+237083us) spinlock_profiling.cc:243] Waited 937 us on lock 0x7b180031c060. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803c94ed8 00007f1803defde2 00007f180be54d23 00007f180be5455d 00007f180bef03d8 00007f180bf21f4e 00007f180bf06fdd
0709 13:09:36.634722 (+ 86364us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:09:37.437978 (+803256us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:09:37.448577 (+ 10599us) tablet_replica.cc:261] Starting instrumentation
0709 13:09:37.449280 (+   703us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":316,"mutex_wait_us":202399,"raft.queue_time_us":160,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"spinlock_wait_cycles":2154240,"tablet-copy.queue_time_us":28480,"tablet-download-pool.queue_time_us":5330,"tablet-download-pool.run_cpu_time_us":2011,"tablet-download-pool.run_wall_time_us":33971,"thread_start_us":131576,"threads_started":2}
I20260709 13:09:38.470491 24172 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.477993 23936 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.482323 24108 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:38.496558 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4cc0 after lost signal to thread 21529
I20260709 13:09:38.490984 24099 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.504712 24182 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.514701 23961 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.517016 24162 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.496665 24143 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.521337 24066 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.525856 23896 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.526443 24163 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.501582 24067 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.542062 24171 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:38.542531 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141257us
I20260709 13:09:38.544204 24163 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.513919 24083 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.550218 24083 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.550829 23936 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.553498 24083 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.556636 24141 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.565325 24172 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.519104 24143 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.513412 24099 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.545509 24163 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.571171 24143 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.571421 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.572436 24024 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.506103 24182 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.575587 24099 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.577039 23936 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.583673 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.585536 24156 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.584575 22705 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:38.586504 24156 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.589370 24067 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.590201 24067 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.587946 24083 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.593029 24083 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.595683 24180 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.598418 24010 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:38.599388 24010 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.600241 24067 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.586907 24024 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.604003 24024 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.607375 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.597563 24083 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.592163 24133 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.618825 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.596776 24180 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.628962 24010 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.631108 24126 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.643059 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.646384 24099 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.644605 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.652572 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.655555 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.631863 24172 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.662254 24172 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.659452 24133 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.667032 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 111) took 1012 ms. Trace:
I20260709 13:09:38.675128 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.675161 23418 raft_consensus.cc:1217] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:09:38.675791 22707 rpcz_store.cc:276] 0709 13:09:37.645640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.433406 (+787766us) service_pool.cc:224] Handling call
0709 13:09:38.433861 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:38.487031 (+ 53170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.487067 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:38.487132 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:38.487193 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.658212 (+171019us) spinlock_profiling.cc:243] Waited 50 ms on lock 0x7b54004100d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:38.658262 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.658404 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":114892672}
W20260709 13:09:38.679392 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52161us
I20260709 13:09:38.685631 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.649302 23936 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.646507 24126 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.693060 24182 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.702296 22533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 241) took 2146 ms. Trace:
I20260709 13:09:38.702473 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.703246 22780 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:38.702934 22533 rpcz_store.cc:276] 0709 13:09:36.555772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.556877 (+  1105us) service_pool.cc:224] Handling call
0709 13:09:36.607908 (+ 51031us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5400002490. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db90b2 00007f180cd32a89 00007f180cccada6 00007f180cc9c088 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc
0709 13:09:38.702071 (+2094163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159740,"reported_tablets":1,"spinlock_wait_cycles":26929024}
I20260709 13:09:38.695302 22532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 228) took 2217 ms. Trace:
I20260709 13:09:38.699647 24123 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.691704 24152 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.661268 24156 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.716168 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.755378 24133 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.731353 24141 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.779825 24141 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:38.767253 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002ad220 after lost signal to thread 22567
I20260709 13:09:38.818878 24183 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.825708 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.826205 24180 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.706238 22532 rpcz_store.cc:276] 0709 13:09:36.477601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.477808 (+   207us) service_pool.cc:224] Handling call
0709 13:09:38.695040 (+2217232us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:36.589645 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:36.669333 (+ 79688us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:36.670597 (+  1264us) write_op.cc:435] Acquired schema lock
0709 13:09:36.672337 (+  1740us) tablet.cc:662] Decoding operations
0709 13:09:36.676800 (+  4463us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:36.676862 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:36.676899 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:36.677093 (+   194us) tablet.cc:701] Row locks acquired
0709 13:09:36.677117 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:09:36.677287 (+   170us) write_op.cc:270] Start()
0709 13:09:36.677410 (+   123us) write_op.cc:276] Timestamp: P: 1783602576677236 usec, L: 0
0709 13:09:36.677439 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:09:36.709318 (+ 31879us) log.cc:844] Serialized 533 byte log entry
0709 13:09:36.728912 (+ 19594us) op_driver.cc:464] REPLICATION: finished
0709 13:09:36.841834 (+112922us) write_op.cc:301] APPLY: starting
0709 13:09:36.842187 (+   353us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:36.842309 (+   122us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:36.918205 (+ 75896us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:36.918570 (+   365us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:36.918831 (+   261us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:36.918908 (+    77us) write_op.cc:312] APPLY: finished
0709 13:09:36.919214 (+   306us) log.cc:844] Serialized 23 byte log entry
0709 13:09:36.919414 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:36.919507 (+    93us) write_op.cc:454] Released schema lock
0709 13:09:36.919558 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:36.919944 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:36.919978 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":936743,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":105849,"apply.run_cpu_time_us":2448,"apply.run_wall_time_us":78102,"num_ops":1,"prepare.queue_time_us":655,"prepare.run_cpu_time_us":3983,"prepare.run_wall_time_us":121536,"replication_time_us":51319,"thread_start_us":53382,"threads_started":1}]]}
I20260709 13:09:38.824555 24133 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.824066 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.836302 24141 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.838274 24184 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.839562 24172 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.811822 24126 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.812800 24163 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.847626 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:38.849143 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4b00 after lost signal to thread 22659
W20260709 13:09:38.849587 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08001bdfc0 after lost signal to thread 22660
I20260709 13:09:38.831107 24182 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.851271 24182 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.854089 24182 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.855454 24083 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.855780 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.856307 24083 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.857955 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.858749 24099 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:38.859513 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08001c8900 after lost signal to thread 22734
I20260709 13:09:38.859445 24099 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:38.861056 22656 debug-util.cc:398] Leaking SignalData structure 0x7b080024e340 after lost signal to thread 22808
I20260709 13:09:38.862013 24099 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.862529 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.836290 24133 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.864698 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:38.865969 24133 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:38.867223 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08003662e0 after lost signal to thread 23445
W20260709 13:09:38.867585 22656 debug-util.cc:398] Leaking SignalData structure 0x7b0800366300 after lost signal to thread 23446
W20260709 13:09:38.869184 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 208499us
I20260709 13:09:38.820052 24183 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:38.839432 24184 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.872841 24185 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:38.877794 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.878384 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:38.877188 22656 debug-util.cc:398] Leaking SignalData structure 0x7b0800366520 after lost signal to thread 24102
I20260709 13:09:38.877930 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.884145 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.844502 24123 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.885447 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.832877 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.884508 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.872884 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.884948 24141 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.898020 24141 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.898216 24123 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.889707 24143 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.900391 24143 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.900887 24123 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.901026 24141 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.886861 24010 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.887627 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.905228 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.905481 24024 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.889205 23419 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:38.906066 22710 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:38.906157 24024 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:38.906864 24083 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.903182 24010 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.912811 24010 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.887064 24126 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.913520 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:38.882993 24152 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.913848 24126 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.908643 24180 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:38.920356 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.920357 24180 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.921610 24126 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.890228 23936 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.921872 24024 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:38.918694 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.919075 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:38.923098 24180 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:38.878948 24133 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:38.929893 24143 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:38.878002 24185 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.942919 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:38.929697 24152 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:38.928205 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:38.904218 24163 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.007221 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.007817 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:39.010540 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08003666a0 after lost signal to thread 24129
I20260709 13:09:39.010216 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:39.019222 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91113.9us
W20260709 13:09:39.027701 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 114442us
I20260709 13:09:39.011087 24024 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:39.035543 24189 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:09:39.036345 24139 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.037264 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:09:39.022497 24190 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:09:39.013381 24156 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.039230 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.023717 24126 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.046844 24180 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.023947 24141 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.065587 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.069829 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.071660 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.071977 24184 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.049206 24172 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.078584 24172 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.050904 24133 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.081338 24172 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.067162 24139 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.083647 24133 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.084038 24139 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.084604 24185 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.096213 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.097636 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.052995 24126 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.088716 24136 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.102592 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:39.102475 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 62008us
I20260709 13:09:39.053072 24180 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.109979 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.110914 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:39.117141 23950 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.117489 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.095273 24024 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.124969 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.054035 24163 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.108734 24126 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.112300 24180 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.102547 24167 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.118541 24182 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:39.128551 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.122606 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.089813 24141 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.114473 24139 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.138216 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.112859 24156 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.117869 23950 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.162501 23950 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.144755 24173 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.144773 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.146133 24024 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.166846 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1430) took 1480 ms. Trace:
I20260709 13:09:39.167532 22779 rpcz_store.cc:276] 0709 13:09:37.657468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.450779 (+793311us) service_pool.cc:224] Handling call
0709 13:09:38.451635 (+   856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:38.500362 (+ 48727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.500402 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:38.582966 (+ 82564us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.814694 (+231728us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:38.814731 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.814824 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.137049 (+322225us) raft_consensus.cc:1685] finished
0709 13:09:39.137080 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.137265 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6450,"raft.queue_time_us":24857,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":988,"thread_start_us":220885,"threads_started":2,"wal-append.queue_time_us":205886}
I20260709 13:09:39.146893 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.170128 24024 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.170583 24024 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.171063 24024 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.147194 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.175295 24167 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:39.149626 24191 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 13:09:39.150314 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.174455 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.184836 24156 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.185679 24174 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.186031 24139 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.187428 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.188139 22704 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:39.188392 24139 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.155303 24152 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.192195 24183 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:39.178279 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 69660.2us
I20260709 13:09:39.192507 24141 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.140965 24182 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:39.193534 24192 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:09:39.193874 24182 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.195539 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.196318 23414 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:39.197773 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.198482 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.199246 23414 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:39.201725 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.203081 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.204387 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.205796 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.206780 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.208763 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.208897 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.209720 24099 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:39.209837 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.210161 24099 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.210585 24099 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:39.211175 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.212915 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.214481 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.214777 24156 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.215175 24156 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.215559 24156 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.216890 24024 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.132138 24136 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.218833 23936 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:39.156836 24163 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.157970 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.219587 23936 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.222513 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.227150 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.227895 22632 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:09:39.230082 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108634us
I20260709 13:09:39.236621 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.239048 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.240396 24104 tablet_copy_client.cc:538] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:39.250824 24138 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.251580 24138 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.252220 24123 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.252673 23950 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.252907 24123 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.253397 23950 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.253862 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.263441 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.233855 24136 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.263664 24067 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.270238 24067 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:39.270566 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.270843 24067 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.272441 22704 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:39.273722 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.274967 24176 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.278368 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.279207 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.279794 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.279973 22629 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:39.280432 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.281528 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.282091 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.282408 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.278441 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.282824 22632 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:39.280587 22633 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:39.279258 22631 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:39.301011 24166 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:39.310914 24163 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:39.312484 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.317886 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.311210 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.317166 24182 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.301051 24185 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.348483 24185 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.349215 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.350008 22635 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:39.339255 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.347117 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.356405 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.320387 22779 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:39.328603 23950 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.339916 24138 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.315651 24163 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:39.376991 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.362243 24185 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.381844 24123 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.375128 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.388006 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.394946 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.396145 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.396263 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.362694 24156 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.401276 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.401999 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.405436 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.405671 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.406183 22635 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:39.407284 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:39.407634 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.408388 22634 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:39.411573 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:39.402010 23411 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.422019 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.333699 24126 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:39.431483 24163 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.432252 24182 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:39.432492 24126 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:39.433089 24163 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.400166 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.415062 24067 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.435616 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.436782 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:39.476049 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.476974 24067 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:39.477267 24185 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.438493 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.393440 24193 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.443207 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:39.478924 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.479458 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.480237 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.480708 23421 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:39.481508 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.444104 22709 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:39.483801 24156 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.485711 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.444433 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.445838 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.487619 24185 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.488255 23416 raft_consensus.cc:1217] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:09:39.488178 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.489226 24185 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.443516 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.444905 24126 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.486334 24136 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:39.495369 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.456112 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.496300 23419 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:39.496644 22629 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:39.456010 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.457598 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-53c258bc80ce4d88bac804cb01a34b32 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:09:39.459439 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.503014 24156 raft_consensus.cc:1064] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:39.503201 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-53c258bc80ce4d88bac804cb01a34b32 on tablet 53c258bc80ce4d88bac804cb01a34b32 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.504002 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:39.505556 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.506366 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.460551 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.506500 24156 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.507571 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.460801 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.461431 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.391970 24024 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.471065 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.474210 24180 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.511863 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.512945 24152 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.513345 22778 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:39.514029 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.485883 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.515004 24152 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.515499 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.504077 23411 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:39.517199 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.517554 23412 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:39.518136 24133 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:39.513402 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.518769 24133 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:39.449831 24104 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:09:39.519820 24133 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.521014 24180 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.500495 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.525529 24197 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.528756 24197 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.533838 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.535185 24136 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:39.535845 24136 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:39.524681 24099 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.545269 24138 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:39.548656 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.531411 24197 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:39.552196 24136 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.554785 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.566548 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:39.568215 24167 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.570137 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.566489 24136 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.572192 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.574404 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.570993 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.586321 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.586488 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.587234 24167 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.591717 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.592192 24099 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.593075 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:39.594696 24126 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.605192 23412 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:39.643317 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.650317 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.660458 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.661895 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.662912 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.663748 22779 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:39.664852 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.672297 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.615275 24141 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.689487 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:39.701658 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 79515.5us
I20260709 13:09:39.703430 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.714041 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.716126 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.721268 22703 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:39.722497 24162 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.723230 24162 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.725673 24162 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:39.717193 24167 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.735184 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.738044 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.743227 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.747411 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:39.750542 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127788us
I20260709 13:09:39.712504 24139 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:39.751871 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:39.753371 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.756075 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.718391 24066 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.760066 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.762929 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.764307 22532 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:39.766957 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:39.767597 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.769671 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.772107 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.773860 22631 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:39.778332 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:39.781646 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.774118 24128 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.784085 24128 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.784518 24128 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.786772 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.787747 24066 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.788290 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.789927 24174 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.751590 24156 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:09:39.792450 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.799284 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.795786 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.801677 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.808816 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.804458 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.792165 24139 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:39.803601 24162 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.811133 24183 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.812619 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.813431 24183 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.803453 24184 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.816354 24024 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.816843 24024 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.817269 24024 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.817765 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.817153 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.818601 24198 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:39.819062 24198 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.819470 24198 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.820983 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:39.819708 24183 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.824167 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.825165 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:09:39.825560 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55987.3us
I20260709 13:09:39.826270 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.826541 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.829891 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:39.832057 24139 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.831890 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.846714 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.833113 24141 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:39.832988 24128 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.833629 24066 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.834852 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.828749 24156 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:39.848760 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.827267 24133 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.828547 24099 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:39.848975 24141 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:39.851013 24179 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.852074 24170 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.854346 24128 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.854672 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.855082 24203 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.878290 24141 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.879987 24141 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:39.843883 24083 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.885178 24083 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.885435 24194 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.885689 24083 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.886094 24194 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.886369 24024 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.888989 24162 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:39.809619 24152 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:09:39.870739 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:39.890309 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55558.5us
I20260709 13:09:39.890900 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:39.891113 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.870658 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:39.873023 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.870982 24174 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.859975 22782 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:39.877300 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.864867 24133 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.895331 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:39.871146 24197 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:39.889596 24162 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:39.895774 22706 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:39.897828 23419 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:39.857417 24156 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.871701 24099 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.903815 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.874502 23950 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:39.900625 24024 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:39.929203 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.911406 24133 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:39.882079 24196 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.938213 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:39.943419 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.923249 24156 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.922264 24010 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.925920 24099 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:39.925638 24182 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.926285 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:39.973418 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.986348 24199 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:39.986970 24199 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:39.987382 24199 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:39.989250 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69304.4us
I20260709 13:09:39.990101 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:39.936120 24197 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:39.993083 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:39.993783 22631 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
W20260709 13:09:39.998675 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55304.8us
W20260709 13:09:39.999246 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 96073.1us
I20260709 13:09:39.947965 24183 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:39.934379 23950 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:39.941823 24196 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:40.040459 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.001073 24184 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.022558 24196 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.024983 24138 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.024677 24193 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.041658 24152 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.044785 24166 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.042557 24210 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:40.081646 24210 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:40.072418 24193 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.110944 24210 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.145783 24134 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:40.146356 24134 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:40.147419 24134 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.128476 24162 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:40.156749 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 143056us
I20260709 13:09:40.156762 24134 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.165892 23950 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.184999 22775 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
W20260709 13:09:40.195132 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 181985us
W20260709 13:09:40.192328 24206 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 13:09:40.212199 24193 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.196117 24067 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.191075 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:40.235805 22632 raft_consensus.cc:606] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:40.263652 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 270613us
I20260709 13:09:40.265250 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:40.268091 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.272975 24226 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:40.280097 24199 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.286106 24099 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.289394 24099 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:40.295065 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 252831us
I20260709 13:09:40.269744 24152 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:40.296744 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70117.3us
I20260709 13:09:40.307426 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:40.308386 24010 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:40.308900 24010 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:40.309801 24010 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.314474 22633 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.317911 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.310562 24156 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.325654 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.327441 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:09:40.329907 24215 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:40.330596 24215 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.329334 22629 raft_consensus.cc:606] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:40.331300 24198 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.006s
W20260709 13:09:40.337276 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.313990 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:40.340233 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:40.347069 24195 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.337060 24083 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.351416 22775 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.353466 24223 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.337152 24180 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.355866 24182 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.369735 24126 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:40.364334 24024 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:40.376008 24024 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:40.376461 24024 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.377323 24083 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:40.377100 24180 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.379935 24195 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.381186 24180 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.381745 24182 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.384845 24182 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.334281 24198 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:40.388859 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 74976.2us
I20260709 13:09:40.407959 23414 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:09:40.391927 24223 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.427189 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:40.428308 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:40.432116 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.434245 24166 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:40.434985 24166 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:40.436414 24166 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.439831 24156 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.442802 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.435061 24099 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.445847 24221 raft_consensus.cc:993] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:40.438279 24166 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.449174 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.452072 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:40.446862 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 105751us
I20260709 13:09:40.452826 24214 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.459517 24024 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.418920 24194 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.455262 24104 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:40.453467 24211 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:09:40.450306 24134 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:09:40.459954 24167 raft_consensus.cc:1064] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:40.468467 24198 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.471469 24223 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.481061 24196 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.470942 22710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:40.395704 24195 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:40.453678 24133 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.001s
I20260709 13:09:40.399438 24199 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:40.368558 24210 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:40.491503 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 130651us
I20260709 13:09:40.494226 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.497108 22701 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.499994 24225 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.487401 24083 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.507025 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.511049 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.511111 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.512298 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:40.496687 24024 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:40.516901 24134 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:40.496376 24215 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:40.517594 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:40.486542 24221 raft_consensus.cc:1081] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:09:40.520313 23936 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.521520 22710 raft_consensus.cc:606] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:40.531478 24104 tablet_bootstrap.cc:492] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:09:40.531289 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:40.490649 22533 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 2 to 3, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:40.533478 22632 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:40.536542 24128 raft_consensus.cc:993] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:40.535272 24134 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:40.552198 24134 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.532397 24198 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.539131 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.545894 24141 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:40.547423 24104 ts_tablet_manager.cc:1403] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 1.098s	user 0.026s	sys 0.031s
I20260709 13:09:40.555326 24099 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.553627 24134 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.534083 22628 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:40.542775 23936 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.561081 23936 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.558324 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.533787 22626 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:40.571663 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.569587 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.573818 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.575073 24099 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.575312 22775 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:40.572368 23417 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.575995 22775 raft_consensus.cc:606] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:40.578177 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:09:40.578933 22782 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:40.579298 22782 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.579140 24141 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:40.573755 22632 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:40.581651 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.578253 24185 raft_consensus.cc:1081] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:09:40.585280 24099 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.594492 24104 raft_consensus.cc:359] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:40.595669 24167 raft_consensus.cc:1068] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Unable to promote non-voter 6a31ac6451ff4f73bee634487d84d001: Service unavailable: leader transfer in progress
I20260709 13:09:40.583078 24141 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:40.595707 24104 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:09:40.597149 24141 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.597389 24104 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.574468 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.601864 24163 raft_consensus.cc:1064] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:40.601790 24228 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:09:40.602392 24163 raft_consensus.cc:1068] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Unable to promote non-voter 6a31ac6451ff4f73bee634487d84d001: Service unavailable: leader transfer in progress
I20260709 13:09:40.603087 24228 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:40.603438 24228 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.603791 24228 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.605211 24233 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:40.605628 24233 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:09:40.585839 24232 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:09:40.606593 24233 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.608266 24233 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.530305 24133 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.592274 24199 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.582451 24198 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.598575 22710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:09:40.590708 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.609476 22749 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.613466 22710 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:40.613885 22710 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.591588 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.615976 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.617746 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.620882 22782 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.622727 24215 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.624545 23936 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:40.622465 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.626497 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.629199 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.630733 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.632784 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.625032 23936 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.622225 24104 ts_tablet_manager.cc:1434] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.002s
I20260709 13:09:40.633442 22632 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:40.636960 22632 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:40.637217 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.637955 22632 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.633169 24184 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:40.630800 24210 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.634671 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.632431 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:09:40.633945 22782 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.632537 24194 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:40.632709 24215 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.632899 24199 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.636716 23936 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.636529 24141 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.651000 22706 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.651755 22709 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.651865 24198 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.652324 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:40.658041 22701 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:40.648110 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:40.656631 22632 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.656667 22704 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.659235 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.663785 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:40.665998 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.666428 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:40.666428 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.668393 22633 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:09:40.669880 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.672564 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.677124 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.677474 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.678699 22635 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.679293 22628 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:09:40.679917 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.680333 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.681914 22703 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.683234 24196 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:40.649843 24194 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.684196 24194 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.693154 24194 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.694937 22784 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.697546 24194 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.699126 24228 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.708384 22629 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:40.701210 24133 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.714854 22775 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:40.714881 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.701984 24228 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:40.645823 24184 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:40.716322 24184 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.714416 22710 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.723305 24099 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.724615 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:40.725351 22633 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.724815 22710 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:40.727087 22706 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.727975 22707 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.728812 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.731809 22707 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.732595 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.733366 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.731885 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.736287 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.737159 22629 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.740922 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.742976 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:40.743798 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:40.745368 22704 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.753459 22629 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:40.753147 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.753155 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.756758 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.757174 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:40.747970 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.761415 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.755537 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:40.765807 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.776338 24083 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.778241 22705 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.780447 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.781630 22532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 243) took 1099 ms. Trace:
I20260709 13:09:40.766016 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.766918 24194 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:09:40.767387 24235 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:09:40.761518 24213 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:40.771288 23417 raft_consensus.cc:1217] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:09:40.742983 24067 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:09:40.781631 24083 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.785889 24223 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:40.794378 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.794713 24223 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:40.795876 24223 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.792691 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.791307 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:40.785880 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:40.785745 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.783005 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:40.782771 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.784724 24194 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:40.801954 24083 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.782076 22532 rpcz_store.cc:276] 0709 13:09:39.682666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.683072 (+   406us) service_pool.cc:224] Handling call
0709 13:09:39.709845 (+ 26773us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:39.751590 (+ 41745us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b5400002490. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db9640 00007f180cc90b76 00007f180cd9ac3e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:40.781560 (+1029970us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:40.107512 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:40.147466 (+ 39954us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:40.147527 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:09:40.147549 (+    22us) tablet.cc:662] Decoding operations
0709 13:09:40.147994 (+   445us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:40.148094 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:40.148129 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:40.149848 (+  1719us) tablet.cc:701] Row locks acquired
0709 13:09:40.149885 (+    37us) write_op.cc:260] PREPARE: finished
0709 13:09:40.150107 (+   222us) write_op.cc:270] Start()
0709 13:09:40.150305 (+   198us) write_op.cc:276] Timestamp: P: 1783602580150036 usec, L: 0
0709 13:09:40.150351 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:09:40.205368 (+ 55017us) log.cc:844] Serialized 533 byte log entry
0709 13:09:40.695173 (+489805us) op_driver.cc:464] REPLICATION: finished
0709 13:09:40.770556 (+ 75383us) write_op.cc:301] APPLY: starting
0709 13:09:40.770695 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:40.770840 (+   145us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:40.770987 (+   147us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:40.771349 (+   362us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:40.771600 (+   251us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:40.771669 (+    69us) write_op.cc:312] APPLY: finished
0709 13:09:40.772015 (+   346us) log.cc:844] Serialized 23 byte log entry
0709 13:09:40.772223 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:40.772318 (+    95us) write_op.cc:454] Released schema lock
0709 13:09:40.772380 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:40.772846 (+   466us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:40.772878 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":63855360,"child_traces":[["op",{"apply.queue_time_us":75009,"apply.run_cpu_time_us":2290,"apply.run_wall_time_us":2296,"num_ops":1,"prepare.queue_time_us":300898,"prepare.run_cpu_time_us":37694,"prepare.run_wall_time_us":344338,"raft.queue_time_us":159585,"raft.run_cpu_time_us":25239,"raft.run_wall_time_us":63305,"replication_time_us":544637,"thread_start_us":458580,"threads_started":4,"wal-append.queue_time_us":313285}]]}
I20260709 13:09:40.793929 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:40.807610 22633 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:40.787848 24213 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:40.788456 24210 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:40.810362 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:40.810228 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.810094 24083 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:40.810227 24210 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:40.802963 24184 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.810051 22703 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:40.833179 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.834559 24225 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:40.820713 24233 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:40.829578 24024 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.840034 22781 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:40.842595 24229 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.809376 24194 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:40.849416 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.853055 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:40.834502 24211 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:09:40.844914 24233 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:40.854336 24024 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.893893 24233 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.854079 24213 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:40.888986 24199 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:40.940505 24199 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:40.887560 24210 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:40.948452 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 95471.5us
I20260709 13:09:40.952942 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.959196 22632 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:40.960901 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:40.962109 24182 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:40.962141 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.963676 22632 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:40.964062 22632 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:40.965778 22632 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:40.968571 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:40.969276 22782 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.970422 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:40.977398 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.984532 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:40.985812 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.989423 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.981963 24010 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:09:40.992019 22632 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:40.941607 24199 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.000849 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.002810 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:40.991251 24214 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:40.992599 24133 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:40.969146 24067 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:09:40.996049 24024 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:41.019194 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 134481us
I20260709 13:09:40.991510 24226 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:09:41.007776 24234 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:41.023741 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.019277 22626 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:09:41.056710 22782 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:41.027573 24133 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.049957 24184 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:41.044715 22703 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:41.071285 24196 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:41.071983 24196 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.080329 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.103648 22632 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:41.122206 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.074424 24196 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:41.046068 24241 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.084110 24226 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:41.076328 22627 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:09:41.140319 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 172349us
I20260709 13:09:41.144407 22780 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:41.130492 24225 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.139272 24248 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.138911 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:41.153733 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.156608 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.131606 23936 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.155256 22719 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-2f4adfa257c34bb88d79bda9d39c5146 received from {username='slave'} at 127.0.0.1:49444
W20260709 13:09:41.165139 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 134924us
I20260709 13:09:41.148588 24226 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.159238 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.160948 22703 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:09:41.172595 22775 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:41.155413 24182 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:09:41.174963 22719 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-2f4adfa257c34bb88d79bda9d39c5146 on tablet 2f4adfa257c34bb88d79bda9d39c5146 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:41.170164 24183 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.188335 23936 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:41.160965 24255 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:09:41.193202 24255 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.193781 24255 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.199676 24213 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.200223 24213 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:41.201197 24213 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.208233 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:41.209102 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:41.177410 24156 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:41.194126 24226 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:41.224303 24225 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:41.211146 24250 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:09:41.225618 24198 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.225919 24250 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.226006 24198 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.226528 24250 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.226729 24243 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.227007 24010 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.227185 24243 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:41.227501 24010 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:41.227259 24198 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.228278 24243 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.228498 24010 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.229249 24198 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.229710 24243 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.230453 24215 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.231958 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:41.230571 24010 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.230930 24215 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.218322 24255 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.239276 24255 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.246978 24228 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.247568 24228 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.183769 24254 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:41.213393 24213 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:41.250465 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93953.6us
I20260709 13:09:41.252368 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:41.257360 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:41.258482 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.248389 24248 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:41.263288 24248 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.264432 24248 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.272320 22632 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:09:41.272215 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:41.247413 24215 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:41.274355 22775 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:09:41.278438 22630 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:41.279213 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:41.270900 24134 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.287225 24196 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:09:41.293068 24195 raft_consensus.cc:993] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:09:41.293855 24249 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:09:41.293812 24233 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:41.258461 24226 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:09:41.227603 24250 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.311595 24250 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.311690 24195 raft_consensus.cc:1081] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:09:41.312553 24133 consensus_queue.cc:1175] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 3 status { last_received { term: 3 index: 2 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:41.316753 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.321662 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.323647 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.296331 24215 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.328509 24233 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.328886 24223 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:41.326998 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.336061 24246 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.385s	user 0.007s	sys 0.000s
W20260709 13:09:41.337776 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.343070 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.262410 24228 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.318493 24252 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:09:41.349581 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.348624 24195 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:41.356096 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:41.358158 24256 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:09:41.364338 24141 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:41.330734 24241 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:09:41.365024 24141 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.367172 22628 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.367369 24141 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:41.368484 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:41.369306 22782 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:41.369647 24234 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.370082 24234 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.370246 24194 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.370692 24194 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.371596 24194 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.372893 24194 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.373677 24210 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.385782 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:41.328289 24258 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.389582 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.387447 24234 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.390323 22633 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: [1.3-1.3]
I20260709 13:09:41.399536 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:41.399438 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 136) took 1178 ms. Trace:
I20260709 13:09:41.401849 22530 rpcz_store.cc:276] 0709 13:09:40.221546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.221697 (+   151us) service_pool.cc:224] Handling call
0709 13:09:40.419875 (+198178us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:41.399371 (+979496us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40909,"reported_tablets":1,"spinlock_wait_cycles":380681600}
I20260709 13:09:41.415311 24229 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.415829 24229 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.391129 23419 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.391712 24234 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.426935 22634 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:41.431936 24133 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.439630 24184 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:41.428048 22533 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:41.420035 24253 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:41.416662 24229 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.449637 22628 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:41.458783 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:41.443949 22533 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:41.449733 22782 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:41.468484 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:41.484346 22710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47418
I20260709 13:09:41.485172 22710 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:41.485594 22710 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.465996 22533 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:41.448998 24228 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.512501 22633 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:09:41.496400 22707 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:09:41.499327 24242 consensus_queue.cc:1175] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1f99ad34cc414233929e0a92f8c3d8db" responder_term: 2 status { last_received { term: 1 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: [1.3-1.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:41.516240 22710 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.501695 24252 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.492718 24229 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.465837 24163 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:41.488636 24213 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:09:41.517198 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f33fee9d64384b499e55f8bc58339f69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:41.489073 24258 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:09:41.520424 24214 consensus_queue.cc:1175] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 2 status { last_received { term: 2 index: 3 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: [1.3-1.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:41.527592 23419 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:09:41.529862 22702 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:09:41.532367 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45161.8us
I20260709 13:09:41.536716 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:41.521836 23414 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Advancing to term 3
I20260709 13:09:41.542210 22710 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:41.543040 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:41.557912 22778 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:41.558377 22778 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.559437 22778 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.560384 22633 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.560587 22778 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:41.564584 22701 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:41.570070 22778 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:41.547194 23419 pending_rounds.cc:85] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 2
I20260709 13:09:41.466121 24253 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.585649 24255 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:41.562409 22703 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:41.533875 24223 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:09:41.608244 22633 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:09:41.619071 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68723.1us
I20260709 13:09:41.581903 23419 pending_rounds.cc:107] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:09:41.545461 24252 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.624415 24225 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:41.682654 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:41.633347 23419 raft_consensus.cc:2917] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
W20260709 13:09:41.696811 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.708168 22634 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:09:41.708554 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 122476us
I20260709 13:09:41.707072 22626 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:41.683655 24225 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:41.705937 23419 raft_consensus.cc:2889] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:41.700791 24223 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:09:41.738117 24254 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:41.717509 24067 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:09:41.742669 22704 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 3. (term mismatch)
I20260709 13:09:41.743160 22704 pending_rounds.cc:85] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:09:41.743525 22704 pending_rounds.cc:107] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.3
I20260709 13:09:41.743841 22704 raft_consensus.cc:2917] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted by new leader
I20260709 13:09:41.744247 22704 raft_consensus.cc:2889] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:41.767257 24253 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:41.787247 24266 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:09:41.787760 24225 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.798653 23414 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:41.807487 24252 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:41.808573 24228 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:09:41.853094 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.859802 24228 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:41.872272 24228 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.880340 24228 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:41.886484 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.891870 22628 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:41.891999 22630 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:41.895435 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:41.898638 23419 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:41.900429 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:41.903286 24255 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:41.903689 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:41.903892 24255 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:41.904701 23936 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:41.905177 23936 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:41.905553 23936 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.905503 24255 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.921315 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:41.922075 22701 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:41.925658 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:41.926774 24195 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.927361 24195 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:41.929682 24195 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:41.934041 23419 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:41.935245 22709 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:41.936892 24225 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:41.941115 24233 raft_consensus.cc:1064] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
W20260709 13:09:41.941635 24233 raft_consensus.cc:1068] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Unable to promote non-voter 6a31ac6451ff4f73bee634487d84d001: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:41.942546 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:41.943403 22630 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:41.941817 22531 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:41.953538 22531 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:41.955334 24139 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:41.966508 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:41.967568 22628 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:09:41.971992 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48851.4us
I20260709 13:09:41.970772 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:41.975281 22630 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:41.980760 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:41.955964 24139 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:41.983582 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:41.989037 22775 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:41.996523 24254 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:41.997135 24254 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.997555 24254 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.007112 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.007150 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.008769 22780 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:42.009842 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:42.007833 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.011571 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.011202 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:42.034339 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.035354 22707 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:41.986357 24083 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:42.038056 24252 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:42.053792 24252 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.054319 24252 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.029313 24242 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:09:42.061393 23936 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.015648 24260 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:42.083853 24251 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:09:42.100946 24156 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:09:42.088353 24083 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:42.029012 24248 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:09:42.090054 24182 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:42.141376 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 97706.8us
W20260709 13:09:42.141124 24277 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:09:42.162052 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56540.3us
I20260709 13:09:42.136139 24182 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:42.179896 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108874us
I20260709 13:09:42.168602 24083 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.185910 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:42.186515 24139 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.180647 24268 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.176281 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:42.202426 24287 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:42.217080 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.194933 24268 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.221938 22627 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:42.182039 23936 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.263342 24228 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:42.264285 24228 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:42.265481 24269 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.266109 24269 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:42.267287 24228 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.268453 24269 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.276546 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.242873 23412 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:42.251394 24254 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.516074 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.516959 24163 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.517592 24163 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.518217 24166 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.518718 24166 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.519088 24166 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.520015 24260 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:42.522589 24163 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.523249 24252 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.525681 24252 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.531291 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.533654 23412 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:42.479094 22782 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:42.481185 22531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 137) took 1032 ms. Trace:
I20260709 13:09:42.536729 22531 rpcz_store.cc:276] 0709 13:09:41.448438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.448693 (+   255us) service_pool.cc:224] Handling call
0709 13:09:42.481100 (+1032407us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:41.984212 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:42.004591 (+ 20379us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:42.004632 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:09:42.004652 (+    20us) tablet.cc:662] Decoding operations
0709 13:09:42.005249 (+   597us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:42.015129 (+  9880us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:42.015171 (+    42us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:09:42.065594 (+ 50423us) tablet.cc:701] Row locks acquired
0709 13:09:42.065616 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:09:42.065793 (+   177us) write_op.cc:270] Start()
0709 13:09:42.065939 (+   146us) write_op.cc:276] Timestamp: P: 1783602582065737 usec, L: 0
0709 13:09:42.065965 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:09:42.066708 (+   743us) log.cc:844] Serialized 913 byte log entry
0709 13:09:42.097722 (+ 31014us) op_driver.cc:464] REPLICATION: finished
0709 13:09:42.198834 (+101112us) write_op.cc:301] APPLY: starting
0709 13:09:42.198912 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:42.199040 (+   128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:42.199131 (+    91us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:42.199485 (+   354us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:42.199653 (+   168us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:09:42.199715 (+    62us) write_op.cc:312] APPLY: finished
0709 13:09:42.199921 (+   206us) log.cc:844] Serialized 29 byte log entry
0709 13:09:42.200067 (+   146us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:42.200165 (+    98us) write_op.cc:454] Released schema lock
0709 13:09:42.200206 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:42.200566 (+   360us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:42.200594 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":57442,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":88578,"apply.run_cpu_time_us":1690,"apply.run_wall_time_us":1691,"num_ops":2,"prepare.queue_time_us":143,"prepare.run_cpu_time_us":3879,"prepare.run_wall_time_us":86299,"replication_time_us":17447,"thread_start_us":65827,"threads_started":1}]]}
I20260709 13:09:42.483942 22533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 233) took 1098 ms. Trace:
I20260709 13:09:42.484647 23411 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:42.489369 24268 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.490653 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.542435 23414 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:42.544239 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:42.545114 24268 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:42.545493 24268 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.545881 24268 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:42.412160 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 142188us
I20260709 13:09:42.550453 24210 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.551096 24210 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.553239 24210 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.503837 24133 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.510716 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.556519 24268 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.556352 24133 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:42.669903 24133 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.669991 24268 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.565231 22533 rpcz_store.cc:276] 0709 13:09:41.385201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.393341 (+  8140us) service_pool.cc:224] Handling call
0709 13:09:42.483844 (+1090503us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:41.543574 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:41.676790 (+133216us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:41.679684 (+  2894us) write_op.cc:435] Acquired schema lock
0709 13:09:41.700474 (+ 20790us) tablet.cc:662] Decoding operations
0709 13:09:41.771165 (+ 70691us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:41.771286 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:41.771323 (+    37us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:09:41.771704 (+   381us) tablet.cc:701] Row locks acquired
0709 13:09:41.771737 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:09:41.771923 (+   186us) write_op.cc:270] Start()
0709 13:09:41.772062 (+   139us) write_op.cc:276] Timestamp: P: 1783602581771869 usec, L: 0
0709 13:09:41.772087 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:09:41.772876 (+   789us) log.cc:844] Serialized 1422 byte log entry
0709 13:09:41.900024 (+127148us) op_driver.cc:464] REPLICATION: finished
0709 13:09:42.019360 (+119336us) write_op.cc:301] APPLY: starting
0709 13:09:42.036495 (+ 17135us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:42.084048 (+ 47553us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:42.113909 (+ 29861us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:42.191415 (+ 77506us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:42.197204 (+  5789us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:09:42.197264 (+    60us) write_op.cc:312] APPLY: finished
0709 13:09:42.197544 (+   280us) log.cc:844] Serialized 35 byte log entry
0709 13:09:42.197780 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:42.197881 (+   101us) write_op.cc:454] Released schema lock
0709 13:09:42.197931 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:42.198341 (+   410us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:42.198374 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":225488,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":104950,"apply.run_cpu_time_us":3753,"apply.run_wall_time_us":193031,"num_ops":3,"prepare.queue_time_us":30710,"prepare.run_cpu_time_us":4851,"prepare.run_wall_time_us":229634,"replication_time_us":127752,"thread_start_us":112434,"threads_started":2}]]}
I20260709 13:09:42.586180 22530 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 4, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.588162 24233 raft_consensus.cc:1064] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:42.671769 22530 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:42.673353 24233 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.588315 24254 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.591588 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:42.675364 22629 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:42.592749 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:42.588837 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f33fee9d64384b499e55f8bc58339f69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:42.677323 22519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88547.7us
I20260709 13:09:42.588195 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:42.491235 23419 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.010s
I20260709 13:09:42.680578 23419 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:42.595144 24182 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.534301 24166 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.684897 24166 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.605768 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.608017 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:42.686018 22631 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.611263 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.686847 22701 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.625928 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:42.687764 22702 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.628134 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.691134 22708 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.630008 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:42.630031 22703 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.630695 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:42.692128 22710 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:42.692442 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61922.8us
I20260709 13:09:42.631507 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.644675 24194 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.650192 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:42.695010 22709 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:42.695282 22782 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.695062 24194 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.696861 24285 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.697505 24285 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.649945 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.698809 23417 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:42.651401 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:42.700117 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:42.700687 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49379.8us
I20260709 13:09:42.700894 22635 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:42.703511 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:42.705555 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:42.706426 23416 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.706712 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:42.707412 24268 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.708029 24268 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.708439 24268 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.708714 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:42.709556 22709 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.710242 24285 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.711869 24133 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:42.718262 24182 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:42.719072 24182 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:42.720084 24182 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.721094 24300 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.657024 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.667645 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:42.721740 24300 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:42.722349 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54813.3us
I20260709 13:09:42.722525 24141 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:42.723095 24141 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.723475 24141 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.724313 24300 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.724520 24269 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.725109 24269 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.667902 24179 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.726949 24179 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.727358 24269 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.705484 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:42.670511 22704 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:42.730278 22634 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.731480 24179 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.731526 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:42.732638 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.735312 24253 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:42.735795 24253 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.736197 24253 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.739010 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:42.747563 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.748358 22702 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:42.740434 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 77574.2us
I20260709 13:09:42.748986 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:42.752323 22631 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:42.750530 23421 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:42.750802 22782 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:42.754908 22710 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:42.755527 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:42.739373 23416 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
W20260709 13:09:42.756384 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53812.1us
I20260709 13:09:42.757004 23385 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:42.756389 24199 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:42.764276 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.765390 22704 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.759208 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.768683 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.770407 22709 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.771567 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:42.772776 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:42.764209 24199 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.773360 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.773845 24199 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.774195 24285 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:42.774103 22703 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:42.759737 24166 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:42.753803 24254 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:09:42.781213 24179 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:42.781620 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.781805 24179 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:42.782284 22780 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:42.783021 24166 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:42.783439 24269 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.783970 24269 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.784009 24166 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.785111 24225 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.785668 24225 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.786314 24269 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.790859 24139 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.791452 24139 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.791801 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:42.793303 22634 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.793812 24139 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.794255 24307 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.796943 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:42.797184 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48307.6us
I20260709 13:09:42.797648 22710 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.796434 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:42.781271 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:42.799265 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.772089 24300 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:42.788168 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.800380 22780 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:42.788450 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.760432 24260 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:42.760661 24268 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.794835 24307 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.802202 24307 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.801270 22775 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:42.814630 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.807093 24234 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.815565 24234 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:42.816013 24234 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.816097 24268 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:42.803128 24156 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:09:42.815582 22703 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.818315 24156 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.818934 24156 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.820259 24233 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:42.820611 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:42.820349 24258 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.821026 24233 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:42.821486 22702 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.823333 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.780330 24253 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.828790 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.842231 24268 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.843097 24268 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.828879 24260 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:42.846774 24194 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.847748 24260 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.854128 22784 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.857743 22710 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:42.858990 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.860105 22702 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.866263 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.866880 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:42.867705 22630 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:42.869293 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:42.870654 23419 raft_consensus.cc:2368] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:42.856887 24268 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.871780 24083 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.872221 24083 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.872632 24083 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.873158 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.873901 22703 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:42.856482 24167 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.874922 24167 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.807596 24300 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.877389 24300 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.890846 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.891398 24248 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.891637 24241 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.833098 24225 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.862516 24199 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:42.879596 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.900213 24156 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.901911 22631 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.902980 24253 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:42.822013 24233 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.895689 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:42.908550 22710 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:42.898020 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:42.830651 24141 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:42.911435 22780 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:42.914487 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:42.892620 24269 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:42.892462 24163 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:42.912282 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:42.927103 22635 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.928730 22633 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:42.897917 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.893258 24195 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:09:42.906294 24083 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.916971 22702 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:42.804036 24179 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.877420 24167 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:42.943395 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:42.893515 24248 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.948181 22631 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:42.917472 24269 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.926674 24163 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:42.958146 22702 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:42.935990 24133 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.975098 24133 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.978719 24163 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.898365 24241 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:42.984134 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:42.944314 23414 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:42.943889 23412 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:42.970822 24194 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:42.992065 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.992513 24194 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:42.993634 24170 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:42.994079 24170 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:42.994496 24170 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:42.985015 24248 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:42.998510 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 116561us
I20260709 13:09:43.001194 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:43.001904 22780 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:43.005295 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:43.007603 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:43.008496 22782 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:43.010133 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:43.015206 24316 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.016165 24194 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:43.020869 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:42.966434 24269 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:42.999365 24133 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:43.024127 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:43.029870 23421 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:43.031369 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:43.020138 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.022854 24139 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:43.049600 24139 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:43.050177 24139 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.016094 24316 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.027238 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:43.042727 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:43.092795 22635 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:43.095716 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:43.097302 24067 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:43.054549 22626 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:43.051352 24228 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:09:43.073354 24300 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:43.073068 23412 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:43.066282 24223 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:09:43.124897 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:43.116690 24312 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:43.110854 24313 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:43.118583 24199 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.126933 24199 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:43.127363 24199 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.127759 24199 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:43.127475 24269 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.128417 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:43.129163 23421 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:43.129969 24269 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.130349 24234 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:43.131876 22784 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:43.112617 24316 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:43.141916 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:43.126439 24313 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:43.123272 24141 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.157711 24133 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:43.158231 24133 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.158632 24133 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:43.160991 24163 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.063088 24307 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:43.165477 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 205799us
I20260709 13:09:43.181226 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:43.182475 22635 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:43.171517 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:43.190204 24228 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:43.195634 24228 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:43.196058 24228 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.187264 24316 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:09:43.199317 24313 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.164891 24163 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.210686 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:43.208607 24067 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:09:43.209163 24309 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.088s	user 0.000s	sys 0.002s
I20260709 13:09:43.305315 22749 leader_election.cc:400] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:43.307883 22749 leader_election.cc:403] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:09:43.309508 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88161.2us
W20260709 13:09:43.328428 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 304401us
I20260709 13:09:43.315991 24267 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:43.345844 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:43.350569 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:43.357872 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:43.369894 24319 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.372727 22710 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:43.381817 22775 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
W20260709 13:09:43.390692 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 363058us
I20260709 13:09:43.393242 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:43.395727 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:43.397570 22709 raft_consensus.cc:2368] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3.
I20260709 13:09:43.398713 22710 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:43.374426 24319 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.403079 24167 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:43.404017 24319 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.403744 24167 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.406021 24167 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.385231 24253 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:43.407596 24253 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.411902 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.416566 24253 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.417570 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:43.420661 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:43.372080 24184 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.436578 22782 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:43.437873 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:43.452894 24241 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.456430 22702 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:43.470050 22598 leader_election.cc:400] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:43.470520 22598 leader_election.cc:403] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:43.471947 24269 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:43.473975 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:43.474915 22782 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:43.477952 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58073.4us
I20260709 13:09:43.487665 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:43.460515 24213 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:43.494841 24083 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:43.495322 24324 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:09:43.509073 24307 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:43.371408 24067 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.510077 23419 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:09:43.510213 24170 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.515s	user 0.013s	sys 0.001s
I20260709 13:09:43.511021 24199 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.514204 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:43.523857 24312 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:43.526589 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:43.527652 23421 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:43.554200 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:43.555306 22710 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.530936 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:43.556157 22628 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:43.531141 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.531865 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.469035 24184 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:43.551355 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63775.3us
I20260709 13:09:43.543759 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:43.551144 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:43.561830 22709 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:43.561878 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:43.536996 24258 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.704s	user 0.013s	sys 0.001s
I20260709 13:09:43.563155 24233 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:43.567233 24194 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:43.568177 24199 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:43.582449 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.583465 22628 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:43.559450 23416 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.583927 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:43.559595 24312 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:43.574574 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.576622 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:43.559791 24253 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:09:43.589252 22601 leader_election.cc:400] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3.
I20260709 13:09:43.518414 24139 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.589188 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:43.594928 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:43.582155 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.550951 24228 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.564666 24234 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:43.557368 24170 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:43.865687 22629 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:43.872303 23421 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:43.832144 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 243041us
I20260709 13:09:43.868665 22710 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:43.838248 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 240570us
I20260709 13:09:43.581426 24307 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:09:43.876398 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 271010us
I20260709 13:09:43.872342 22709 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:43.866475 24195 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.883482 24195 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:43.585907 24258 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:43.617538 24254 raft_consensus.cc:1064] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
W20260709 13:09:43.695529 24133 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.536s	user 0.013s	sys 0.000s
I20260709 13:09:43.790581 24327 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:43.898795 22782 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:43.898936 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:43.855204 24184 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:43.744334 24267 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:43.613209 24194 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:43.903173 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:43.591686 24268 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:43.885844 24195 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:43.896826 24307 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:43.891091 24337 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:43.996068 24337 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.000427 24139 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:43.933697 24194 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:44.003456 24194 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.004595 24194 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:43.903337 24179 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:09:43.948487 22782 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:43.902256 22627 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
W20260709 13:09:44.010356 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107095us
W20260709 13:09:44.011174 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.012634 23421 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:44.014605 23421 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:09:44.014978 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:44.015723 22778 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:44.016183 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:44.017279 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:44.018954 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:44.007786 24228 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:44.028784 22702 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:43.959807 24329 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:43.960317 24327 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:43.996085 23416 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:43.915439 24133 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.042382 24170 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:44.143934 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 95025.2us
I20260709 13:09:44.045943 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:44.404744 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:44.149294 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 102943us
W20260709 13:09:44.144443 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73260.5us
I20260709 13:09:44.055378 22781 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:44.409713 22709 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:44.407572 22777 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:44.420478 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:44.042713 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:44.421972 22633 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
W20260709 13:09:44.431481 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 412573us
I20260709 13:09:44.474027 22701 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:44.475664 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:44.405122 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:44.484231 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 462084us
I20260709 13:09:44.046350 24337 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:44.039547 24329 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.408245 24266 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.485342 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:44.468909 24163 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.150836 24317 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:44.043792 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:44.424152 24289 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.423933 24338 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.521350 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:44.530920 22709 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:44.534261 22780 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.001s
I20260709 13:09:44.509931 24329 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:44.534799 22780 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:44.043978 24307 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:44.535152 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 98828.8us
I20260709 13:09:44.487103 22776 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:44.541718 24156 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:44.556592 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 71392.8us
W20260709 13:09:44.542328 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 66796.6us
I20260709 13:09:44.515326 24266 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.519486 24163 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.653275 24223 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:44.555388 24312 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.964s	user 0.008s	sys 0.006s
I20260709 13:09:44.670260 24312 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.670146 24223 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.536351 24228 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:44.550004 24338 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.557062 24156 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.538066 24289 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.591622 24337 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.675491 24337 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.556536 24199 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.676553 24199 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.589454 24269 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:44.592978 24342 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:44.689584 24342 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.690088 24342 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:44.584818 24313 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.692050 24156 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.692466 24289 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.636075 24317 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:44.693495 24337 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:44.693857 24199 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.694898 24338 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:44.675034 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:44.698259 24338 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:44.698450 24223 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:44.698840 24338 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.700040 24266 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:44.694118 24317 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.702457 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:44.706053 22776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:44.707098 22776 raft_consensus.cc:606] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:44.691533 24313 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.608861 22777 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:44.717984 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:44.727236 24133 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:44.716871 24260 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.732272 24260 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.733475 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 245) took 2787 ms. Trace:
I20260709 13:09:44.736238 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1514) took 1194 ms. Trace:
I20260709 13:09:44.736954 24133 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:44.737416 24133 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.737788 24133 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:44.736685 22701 rpcz_store.cc:276] 0709 13:09:43.541587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.547585 (+  5998us) service_pool.cc:224] Handling call
0709 13:09:43.548000 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:44.474600 (+926600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.735929 (+261329us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b54003d13d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:44.736138 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129179392}
I20260709 13:09:44.740072 22532 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 140) took 1869 ms. Trace:
I20260709 13:09:44.740504 22532 rpcz_store.cc:276] 0709 13:09:42.870938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.871117 (+   179us) service_pool.cc:224] Handling call
0709 13:09:44.740005 (+1868888us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214612,"reported_tablets":1}
I20260709 13:09:44.740945 24260 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:44.744607 24333 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.066s	user 0.006s	sys 0.018s
I20260709 13:09:44.712157 24338 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:44.748410 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 220367us
I20260709 13:09:44.749382 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:44.756531 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:44.557039 24327 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.775334 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:44.779269 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1624) took 1565 ms. Trace:
I20260709 13:09:44.779865 22635 rpcz_store.cc:276] 0709 13:09:43.213716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.213863 (+   147us) service_pool.cc:224] Handling call
0709 13:09:43.214276 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:43.214755 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:43.214800 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:43.446417 (+231617us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b2c00000c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:09:43.451498 (+  5081us) log.cc:844] Serialized 26 byte log entry
0709 13:09:43.578206 (+126708us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:09:43.578251 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:43.578320 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:44.743984 (+1165664us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:44.778906 (+ 34922us) raft_consensus.cc:1685] finished
0709 13:09:44.778940 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:44.779158 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8238,"raft.queue_time_us":147,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":74262,"spinlock_wait_cycles":75068544,"thread_start_us":126182,"threads_started":1,"wal-append.queue_time_us":226787}
W20260709 13:09:44.785590 24194 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.780s	user 0.011s	sys 0.000s
I20260709 13:09:44.733906 22530 rpcz_store.cc:276] 0709 13:09:41.946153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.946525 (+   372us) service_pool.cc:224] Handling call
0709 13:09:41.998887 (+ 52362us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:44.733395 (+2734508us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:42.843532 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:42.844246 (+   714us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:42.844293 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:09:42.844315 (+    22us) tablet.cc:662] Decoding operations
0709 13:09:42.844808 (+   493us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:42.844904 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:42.844937 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:09:42.845317 (+   380us) tablet.cc:701] Row locks acquired
0709 13:09:42.845337 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:09:42.845510 (+   173us) write_op.cc:270] Start()
0709 13:09:42.845647 (+   137us) write_op.cc:276] Timestamp: P: 1783602582845461 usec, L: 0
0709 13:09:42.845671 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:09:42.846550 (+   879us) log.cc:844] Serialized 918 byte log entry
0709 13:09:42.951309 (+104759us) op_driver.cc:464] REPLICATION: finished
0709 13:09:43.323933 (+372624us) write_op.cc:301] APPLY: starting
0709 13:09:43.334349 (+ 10416us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:43.350851 (+ 16502us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:43.386225 (+ 35374us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:43.485984 (+ 99759us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:43.520803 (+ 34819us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:09:43.520857 (+    54us) write_op.cc:312] APPLY: finished
0709 13:09:43.521113 (+   256us) log.cc:844] Serialized 29 byte log entry
0709 13:09:43.521318 (+   205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:43.521428 (+   110us) write_op.cc:454] Released schema lock
0709 13:09:43.521491 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:43.521876 (+   385us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:43.521905 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":82942848,"child_traces":[["op",{"apply.queue_time_us":349466,"apply.run_cpu_time_us":3423,"apply.run_wall_time_us":205651,"num_ops":2,"prepare.queue_time_us":168200,"prepare.run_cpu_time_us":3497,"prepare.run_wall_time_us":18527,"replication_time_us":105476,"thread_start_us":344677,"threads_started":2}]]}
I20260709 13:09:44.787967 24313 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.788866 24313 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.789634 24313 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.789944 24354 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.801798 24139 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:44.802947 24139 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.793661 22776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:44.803481 24139 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.803865 22776 raft_consensus.cc:606] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:44.804059 24346 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:44.804713 24346 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:44.805927 24139 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.813251 23415 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:44.814949 22776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:44.817360 22750 leader_election.cc:400] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
W20260709 13:09:44.820789 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60490.8us
I20260709 13:09:44.791368 24354 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.834550 24326 raft_consensus.cc:1064] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:44.837005 24228 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:44.837417 24341 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:09:44.837765 24228 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.840806 24225 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.844511 24253 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:44.845341 24253 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.801757 24342 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:44.849722 24342 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:44.857705 24010 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:44.858700 24010 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.886514 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:44.887624 22707 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:44.884997 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:44.888902 22709 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:44.841670 24225 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:44.928524 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:44.930105 24325 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:44.949923 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.944084 24325 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:44.931090 24260 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.805800 24346 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.014683 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.027959 23417 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:45.015297 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.015753 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:44.980712 24361 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:45.018306 24325 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.029665 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:44.972223 24269 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:44.980062 24354 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.020663 24228 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.023955 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:44.968948 24317 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:44.968361 24133 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.028621 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:45.051895 24364 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.052515 24364 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.057664 24313 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.043102 24346 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:45.045925 24354 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.028786 24260 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.087018 24225 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.108836 24182 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:45.109386 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.109501 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 408) took 1073 ms. Trace:
I20260709 13:09:45.044885 24170 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.207304 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1398) took 1189 ms. Trace:
I20260709 13:09:45.049865 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 319) took 2262 ms. Trace:
I20260709 13:09:45.053475 24253 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.090013 24269 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.143718 24357 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:45.090660 24368 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.100514 24285 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.049913 24369 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.037719 24329 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.051038 24317 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.044185 24361 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.207769 24170 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.207881 22779 rpcz_store.cc:276] 0709 13:09:44.018332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.018480 (+   148us) service_pool.cc:224] Handling call
0709 13:09:44.018901 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:44.035598 (+ 16697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:44.035631 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:44.829228 (+793597us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:09:44.829787 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:09:44.979020 (+149233us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:09:44.979066 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.979152 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:45.114569 (+135417us) raft_consensus.cc:1685] finished
0709 13:09:45.114594 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.207192 (+ 92598us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26478,"raft.queue_time_us":18509,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":94543,"spinlock_wait_cycles":147122048,"thread_start_us":148755,"threads_started":1,"wal-append.queue_time_us":221408}
I20260709 13:09:45.313525 24170 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.258850 24182 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.318830 24364 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.319752 24359 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:45.320242 24359 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.320609 24359 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:45.206634 23420 rpcz_store.cc:276] 0709 13:09:44.035789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.036004 (+   215us) service_pool.cc:224] Handling call
0709 13:09:44.424608 (+388604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:44.525561 (+100953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:44.525600 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:44.538767 (+ 13167us) log.cc:844] Serialized 26 byte log entry
0709 13:09:44.822449 (+283682us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:44.822494 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.920290 (+ 97796us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:45.109191 (+188901us) raft_consensus.cc:1685] finished
0709 13:09:45.109220 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.109397 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9378,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":30762,"spinlock_wait_cycles":41728,"thread_start_us":242617,"threads_started":2,"wal-append.queue_time_us":546089}
I20260709 13:09:45.109846 24360 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.240403 24225 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:45.286233 24363 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.313817 24368 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.153861 24354 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.377174 24253 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.382754 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.385061 24067 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.397128 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.435672 22630 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:45.205591 24228 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:45.398866 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.436904 22633 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:45.438429 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.440099 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.440877 22633 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:45.442029 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:45.402561 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.017s	sys 0.017s
I20260709 13:09:45.407864 22634 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:09:45.444173 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.020s	sys 0.017s
I20260709 13:09:45.315025 23414 rpcz_store.cc:276] 0709 13:09:42.787651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.788729 (+  1078us) service_pool.cc:224] Handling call
0709 13:09:42.911595 (+122866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.985547 (+ 73952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.047399 (+2061852us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54007c01d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:45.049723 (+  2324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271707520}
I20260709 13:09:45.445182 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.445793 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:45.427771 22628 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:45.342078 24225 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.329442 24285 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.460178 24353 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.461113 24353 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.463853 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:45.464759 22631 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:45.456265 24228 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.469480 24368 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.486071 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.452979 24182 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.455178 24225 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:45.471349 24364 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:45.448417 24253 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.449839 24067 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.497987 24361 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.420076 24360 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.499672 23420 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:45.463406 24363 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.334561 24369 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.500555 24228 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.520605 22631 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:45.525532 24359 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.525561 24067 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.527814 24369 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.523449 24260 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.510357 24253 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.336083 24357 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.536509 24317 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.537550 24365 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.521046 24355 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:45.538466 24365 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:45.545264 24363 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.546063 24363 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.548623 24253 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:45.549458 24253 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.556226 24253 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.533459 24364 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:45.552726 24329 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.565310 24353 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.566073 24353 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.566677 24353 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:45.567184 24327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.027s	sys 0.043s
W20260709 13:09:45.567880 24327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.027s	sys 0.043s
I20260709 13:09:45.569218 24353 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:45.572487 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 161053us
I20260709 13:09:45.579171 24313 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.580282 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.579784 24313 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.581683 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.582724 24313 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.583277 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.584077 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.584383 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.585186 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:45.585433 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.586037 22633 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:45.587041 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.562440 24354 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.554545 24363 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:45.591441 24367 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.249s	user 0.000s	sys 0.002s
I20260709 13:09:45.562222 24355 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.591656 24285 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.595746 24359 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.599764 24357 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.578454 24384 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.592549 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.602797 24364 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.604238 24384 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.603514 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:45.613843 24285 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:45.614442 24285 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.614939 24285 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:45.638892 24384 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.639392 24363 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.602816 24383 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.644313 24383 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.649887 24355 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.640520 24365 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.640591 24360 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.653201 23419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 341) took 1644 ms. Trace:
I20260709 13:09:45.661720 24225 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.659824 24312 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.648756 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.666115 23419 rpcz_store.cc:276] 0709 13:09:44.009309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.012287 (+  2978us) service_pool.cc:224] Handling call
0709 13:09:45.650692 (+1638405us) spinlock_profiling.cc:243] Waited 7.54 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:45.650867 (+   175us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.651080 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.651117 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:45.652835 (+  1718us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:45.652884 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.652970 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.653127 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17347328}
W20260709 13:09:45.675362 24346 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.008s
I20260709 13:09:45.690949 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.692142 23419 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:45.682435 23416 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:45.689957 24383 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.695960 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.696700 22629 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
W20260709 13:09:45.704619 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.706704 24225 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.714829 24285 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.717917 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:45.720227 22629 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:45.717319 24285 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.731145 22629 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:45.733219 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:45.734557 24285 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:45.733256 24133 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.735437 24285 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:45.736475 24285 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.749522 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:45.750134 24391 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:45.754029 24383 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:45.754587 24383 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.753978 24391 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.750728 24364 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:45.761214 24307 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:45.750291 24363 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.761595 24307 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:45.761866 24363 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.762478 24307 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.763691 24307 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:45.764379 24363 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:45.755020 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.000s	sys 0.049s
W20260709 13:09:45.765666 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.000s	sys 0.049s
W20260709 13:09:45.766012 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 736418us
W20260709 13:09:45.760267 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1380) took 2231 ms (client timeout 2542 ms). Trace:
I20260709 13:09:45.768883 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:45.755002 24383 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:45.769727 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.047s	sys 0.057s
W20260709 13:09:45.770099 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.047s	sys 0.057s
W20260709 13:09:45.770378 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 754700us
I20260709 13:09:45.771579 24163 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.773032 24364 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:45.773433 24364 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.773823 24364 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:45.777690 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:45.774505 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 328) took 1763 ms. Trace:
I20260709 13:09:45.778776 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.779489 23412 rpcz_store.cc:276] 0709 13:09:44.011276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.016320 (+  5044us) service_pool.cc:224] Handling call
0709 13:09:45.773573 (+1757253us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8e9c 00007f1803db8a5b 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:45.773825 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.774071 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.774110 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:45.774167 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:45.774213 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.774283 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.774430 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295413632}
I20260709 13:09:45.781554 24358 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:45.782199 24358 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:45.783290 24358 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:09:45.785323 22627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1382) took 2255 ms (client timeout 1571 ms). Trace:
W20260709 13:09:45.785821 22627 rpcz_store.cc:269] 0709 13:09:43.529610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.536911 (+  7301us) service_pool.cc:224] Handling call
0709 13:09:45.785101 (+2248190us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b54003d3954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:45.785248 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44323200}
I20260709 13:09:45.787107 24364 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.787937 24141 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:45.788539 24141 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.790939 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:45.791294 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.791698 23419 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:09:45.768532 22626 rpcz_store.cc:269] 0709 13:09:43.529014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.538092 (+  9078us) service_pool.cc:224] Handling call
0709 13:09:45.759999 (+2221907us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54002f5154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:45.760171 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4966985088}
I20260709 13:09:45.798280 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.800459 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:45.801168 24141 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.802211 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.802693 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.803050 23415 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:45.806228 23419 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:45.807682 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:45.808701 24363 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:45.809295 24363 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:45.810271 24363 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.814333 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 330) took 1789 ms. Trace:
I20260709 13:09:45.814410 24364 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.814908 23421 rpcz_store.cc:276] 0709 13:09:44.024488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.024972 (+   484us) service_pool.cc:224] Handling call
0709 13:09:45.813445 (+1788473us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:45.813707 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.813935 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.813965 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:45.814016 (+    51us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:45.814052 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.814126 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.814261 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29814016}
I20260709 13:09:45.815026 24383 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.825995 24391 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.831036 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:45.832255 24255 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:45.853794 24255 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.854655 24383 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:45.856616 24255 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.832885 23936 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:45.857949 23936 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.860615 23936 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.833312 24334 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.861976 24334 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.864771 24334 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.834410 24348 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:09:45.832302 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.865617 24348 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:45.841497 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46083.5us
I20260709 13:09:45.867736 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.868283 24348 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.868639 24156 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:45.869004 24348 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.869298 24156 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:45.870663 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.870363 24156 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:45.872040 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.844715 24010 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:45.873731 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:45.869561 24348 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:45.875242 22749 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.879678 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:45.880066 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:45.880664 22632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1396) took 2300 ms (client timeout 2519 ms). Trace:
W20260709 13:09:45.881089 22632 rpcz_store.cc:269] 0709 13:09:43.580566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.583594 (+  3028us) service_pool.cc:224] Handling call
0709 13:09:45.879532 (+2295938us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:09:45.880584 (+  1052us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3701888}
I20260709 13:09:45.881456 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:45.894950 24288 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:45.884755 23936 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:09:45.882952 24255 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:45.887787 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.897485 24255 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.897969 24255 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:45.849124 23419 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:45.899605 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.903193 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.903597 22701 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:45.890772 24348 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:45.901402 24318 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:45.904856 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:45.904824 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.368s	user 0.063s	sys 0.199s
W20260709 13:09:45.929112 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.392s	user 0.064s	sys 0.200s
I20260709 13:09:45.931314 22707 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:45.932552 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:45.934124 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:45.935590 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.934305 22708 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:45.931315 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.937793 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:45.938514 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.882670 24334 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:45.938540 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.951572 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.951491 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.954807 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:45.958535 22702 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:45.939800 24348 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:45.968274 22704 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:45.968966 22749 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:45.959954 23936 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:45.948835 24334 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.956025 24363 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:45.967756 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:46.004029 22701 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:45.937359 24288 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:45.978760 24258 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:45.950023 24170 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:46.002885 23936 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.006402 23936 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:45.999816 22707 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.009547 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:45.959331 24307 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.009459 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.014478 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.015080 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.015352 22780 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:46.015709 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:45.999532 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.016901 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.017815 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.023044 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.023809 22784 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:46.024902 22779 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:46.018595 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.018357 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.016801 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.026266 22777 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:46.026779 22631 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.019032 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.021564 22702 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:46.018326 24010 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.020475 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.005394 24334 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.009866 24182 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.028782 24010 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.005621 24288 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.021243 24170 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:46.022301 24307 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.025705 24261 raft_consensus.cc:993] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:46.016551 24363 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.051136 24223 raft_consensus.cc:1081] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:46.041770 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.052162 24307 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.052503 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:46.043377 24288 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.052906 24351 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.053493 24351 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.053912 24351 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.055943 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:46.058243 24010 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.061034 24010 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.061673 24010 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.062201 24010 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.027711 24258 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:46.038527 24182 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.054051 24255 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.100795 23936 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.103413 24391 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.104086 24391 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.072063 24392 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.110921 23936 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.099126 24182 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.074599 24285 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:09:46.139133 24010 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.139488 23385 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.140226 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.140822 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.141085 22633 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:46.141758 22675 connection.cc:256] RPC call timeout handler was delayed by -1.078564101999973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15710000000000002s for a response
I20260709 13:09:46.142879 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.114199 24392 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.143664 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:46.142330 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86485.1us
I20260709 13:09:46.144268 22630 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:46.122455 22631 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:46.145676 23385 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:46.147825 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.152523 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.147307 23936 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.149142 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.159765 23936 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.160965 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.153515 24223 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.155278 24156 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:46.156075 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.157416 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.115646 24258 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.167454 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:46.166491 22600 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.211s (SENT) [suppressed 3 similar messages]
I20260709 13:09:46.171655 24255 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.172914 24182 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.175410 24334 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.176013 24288 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.136847 24170 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.177929 24334 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.178036 24288 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.149003 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.181496 22779 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:46.162061 22777 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:46.162221 24010 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.164674 24363 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.164940 24253 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.183858 24253 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:46.184485 24253 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.165162 24248 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:46.142992 24233 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.164878 24223 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.150971 24381 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.154103 24392 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.181628 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.190299 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.183451 23936 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.192682 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.188437 24233 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.193392 22777 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:46.194916 24233 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.195416 24223 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.176045 24182 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.210805 22779 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:46.211408 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.215533 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.216749 22701 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:46.220373 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.222744 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.224267 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45654 (request call id 83) took 1251 ms. Trace:
I20260709 13:09:46.225363 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.225986 22780 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.227694 22635 rpcz_store.cc:276] 0709 13:09:44.970331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.408913 (+438582us) service_pool.cc:224] Handling call
0709 13:09:46.221665 (+812752us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540006d6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee3735 00007f1806edb9fa 00007f180bea34c9 00007f180541fc54 00007f180541fbe2 00007f180541fb11 00007f180541fa8c 00007f180541ed42 00007f180521848c 00007f18052178c6
0709 13:09:46.221856 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385917696}
I20260709 13:09:46.231168 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.162039 24391 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.191463 24363 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.242882 22777 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.247704 22780 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:46.220813 24156 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.251588 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.258895 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.224256 24248 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:46.239019 24358 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.266832 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.270517 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.272507 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.273209 22777 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:46.274435 22775 raft_consensus.cc:2410] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:09:46.276597 24182 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:46.277230 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 110770us
I20260709 13:09:46.283484 24182 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.249743 24253 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.262255 24248 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.251310 24351 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.270138 24225 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:46.250907 24363 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:46.276597 24223 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.304433 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.305676 22635 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:46.309465 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.311832 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.315730 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.320081 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:46.321273 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 90283.9us
I20260709 13:09:46.321544 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.326327 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.327809 22775 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.328126 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.328248 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.328827 22635 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.331643 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.335543 22779 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:46.339182 24363 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.340883 24408 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.341521 24408 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.342149 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.343330 22635 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:46.344158 24408 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.344756 24408 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:46.345299 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.344774 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.346580 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.349071 24381 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:46.350136 22635 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:46.351737 24392 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.354307 24392 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:46.354498 22674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.318s	user 0.010s	sys 0.050s
W20260709 13:09:46.356235 22674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.320s	user 0.011s	sys 0.050s
I20260709 13:09:46.345237 24408 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.361299 24408 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:46.365356 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet f33fee9d64384b499e55f8bc58339f69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:46.377616 22628 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:46.381286 22628 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.382539 22628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:46.383423 22628 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:46.383976 22628 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.393872 22628 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.397899 22628 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:46.425930 22782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 1568) took 2849 ms (client timeout 2211 ms). Trace:
W20260709 13:09:46.426544 22782 rpcz_store.cc:269] 0709 13:09:43.576180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.576334 (+   154us) service_pool.cc:224] Handling call
0709 13:09:46.425636 (+2849302us) spinlock_profiling.cc:243] Waited 306 ms on lock 0x7b5400132754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:46.425808 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":703432576}
I20260709 13:09:46.431082 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.449457 24340 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:46.450253 24340 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:09:46.450912 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.454367 24340 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.459281 24195 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:09:46.461346 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1807799000000614s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
I20260709 13:09:46.462303 24195 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:46.463855 22627 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.465241 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:46.473034 24266 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:46.475121 22635 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:46.477598 24340 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.478484 24340 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.481032 24340 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.481632 24340 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.487651 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.488942 24335 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.489588 24335 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.473856 24266 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.492547 24348 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:46.493773 24348 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.494268 24266 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.496352 24266 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:46.462807 24195 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:46.496749 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.499155 24351 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.494519 24348 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 13:09:46.500212 22676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.464s	user 0.016s	sys 0.018s
W20260709 13:09:46.508113 22676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.472s	user 0.017s	sys 0.019s
I20260709 13:09:46.501663 22706 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:46.482092 24340 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.512712 24340 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.503995 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.518934 24335 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.519299 22531 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:46.540633 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.541543 22778 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
W20260709 13:09:46.520257 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.244s	user 0.010s	sys 0.018s
I20260709 13:09:46.521867 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:46.522490 22748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.267s	user 0.000s	sys 0.009s
W20260709 13:09:46.547821 22748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.000s	sys 0.010s
W20260709 13:09:46.548435 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.496709 24266 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:09:46.546761 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.281s	user 0.011s	sys 0.018s
W20260709 13:09:46.549803 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84685us
I20260709 13:09:46.550836 24266 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.555572 24266 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.557420 23419 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.559109 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.560746 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.562254 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.563131 23415 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:09:46.572371 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.572800 24340 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.573278 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.573446 24340 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.575078 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.575543 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.576016 24340 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.576555 24340 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.577042 24340 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.577909 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.578131 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.578975 22627 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:46.579861 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.581569 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.582439 24287 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.583029 24287 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.583652 24267 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:46.583943 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:46.585127 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.585223 24287 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.585858 24287 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.586197 24287 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.586220 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.586784 24335 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:46.587252 24335 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.587642 24335 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:46.589087 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.589747 22775 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:46.590752 24287 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.591099 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.301307 24223 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.594462 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.595268 22783 raft_consensus.cc:2368] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:46.595561 24340 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.597357 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.598026 22783 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:46.599318 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.297814 24248 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.600237 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.600473 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.601282 22782 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:46.600890 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.286415 24182 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.603564 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:46.603915 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53229.4us
I20260709 13:09:46.604420 22782 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:46.604857 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.586931 24348 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.606153 24182 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:46.609052 22783 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:46.609350 24223 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.607466 24335 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.610634 22778 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:46.641088 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.636813 24287 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.631145 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 412) took 1154 ms. Trace:
I20260709 13:09:46.650269 22775 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:46.647959 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.652060 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.652709 24287 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.612877 23417 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.641789 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:46.640017 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.654922 23421 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:46.650154 23420 rpcz_store.cc:276] 0709 13:09:45.458029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.458197 (+   168us) service_pool.cc:224] Handling call
0709 13:09:46.612496 (+1154299us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b54004e9654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:46.612650 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28851584}
I20260709 13:09:46.643586 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.650575 22600 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.657794 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.660557 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.658906 24166 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:46.650338 24170 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:46.661458 24266 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:46.664317 24266 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.664791 24266 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:09:46.666147 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.006s	sys 0.004s
I20260709 13:09:46.661891 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:46.666985 22627 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.667694 22630 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:46.668040 22784 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:46.638206 24335 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:46.666518 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.006s	sys 0.004s
I20260709 13:09:46.662040 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.634752 24223 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:09:46.672957 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.675873 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.683137 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.662721 24248 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.685794 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.669415 24287 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:09:46.677039 23421 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:46.679651 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.691864 22781 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:46.681027 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.690030 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.694752 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:46.612627 24267 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 13:09:46.614516 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.684451 24405 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.690269 22600 leader_election.cc:400] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:46.700814 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.698946 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.687889 24166 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.688460 24336 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.703215 22781 raft_consensus.cc:2376] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3. Re-sending same reply.
I20260709 13:09:46.700459 24392 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:46.701192 24414 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:46.702775 22600 leader_election.cc:403] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:09:46.702592 24405 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.706231 24381 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.718971 24414 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:46.710501 24336 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.727496 24336 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.727919 24414 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.729166 23411 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:46.729930 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 407) took 2704 ms. Trace:
I20260709 13:09:46.721079 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.730980 22777 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.690062 23415 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:46.725785 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:46.740660 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73388.9us
I20260709 13:09:46.724581 24405 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.734321 24381 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.742343 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.744997 24381 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.745143 24170 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:46.745663 24170 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.746028 24170 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.746066 22627 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:46.747041 24248 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.747752 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.743229 24163 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.748912 24163 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.749128 24337 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:46.749336 24163 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.752172 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.730279 23411 rpcz_store.cc:276] 0709 13:09:44.025548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.029656 (+  4108us) service_pool.cc:224] Handling call
0709 13:09:46.728721 (+2699065us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:46.728962 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:46.729651 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:46.729868 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23963904}
I20260709 13:09:46.753535 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.754318 22777 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:46.749759 24337 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:46.757370 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:46.736079 24345 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:46.754259 22775 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.758222 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 164) took 1178 ms. Trace:
I20260709 13:09:46.761045 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.764878 22634 rpcz_store.cc:276] 0709 13:09:45.579401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.582728 (+  3327us) service_pool.cc:224] Handling call
0709 13:09:46.757238 (+1174510us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:09:46.758139 (+   901us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9955584}
I20260709 13:09:46.760293 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.780900 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.761607 22777 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:46.787714 22779 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:46.797344 24351 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.797899 24351 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.758051 24337 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.800282 24351 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:46.762073 23414 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:46.801596 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 411) took 1344 ms. Trace:
I20260709 13:09:46.780032 24166 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.771903 24266 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.791887 24254 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 2, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.809249 24067 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:46.809903 24067 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:46.810351 22779 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:09:46.813354 24266 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:46.814671 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:46.801941 23414 rpcz_store.cc:276] 0709 13:09:45.457641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.465754 (+  8113us) service_pool.cc:224] Handling call
0709 13:09:46.761737 (+1295983us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8e9c 00007f1803db8a5b 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:46.761903 (+   166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:46.801268 (+ 39365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:46.801526 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12822272}
I20260709 13:09:46.797039 24391 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:46.815939 24391 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.786824 22782 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:46.816423 24391 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:46.816560 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:46.817416 23412 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:46.788542 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.812709 24195 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.827178 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.829635 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:46.797024 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.835827 24345 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.789664 24253 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.760388 24381 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.840078 24195 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.835868 24067 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.834343 24363 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:46.844640 24170 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.855930 24170 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.846139 24336 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.857997 24336 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.784862 24405 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.861019 24405 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.834790 24406 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:09:46.819186 24392 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:46.758900 24163 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.864704 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.867818 22782 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.852046 22775 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:46.865515 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.871078 22777 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:46.850530 24335 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:46.878821 24381 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.886448 22782 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:46.862236 24363 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:46.891597 24363 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:46.872907 24392 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.893217 24163 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.880649 24335 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:46.895783 24335 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.898425 24392 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.915114 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.915853 22782 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:46.923200 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91235us
I20260709 13:09:46.924276 22782 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:46.925033 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.914605 24335 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.929808 22777 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:46.929805 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.931314 24355 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:46.932099 24355 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.932646 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:46.929642 24392 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:46.933924 24407 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:46.934587 24392 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:46.934460 24355 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.935623 24408 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.935777 24392 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:46.936211 24408 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.938340 24408 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:46.939913 22780 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:46.941059 24214 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.941649 24214 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.942431 24316 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:46.943038 24316 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:46.949286 24316 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:46.962576 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.011s	sys 0.012s
W20260709 13:09:46.963064 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.012s	sys 0.012s
W20260709 13:09:46.963725 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 818088us
W20260709 13:09:46.964928 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 633327us
I20260709 13:09:46.967569 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:46.931383 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:46.972376 22775 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:46.978966 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.980432 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.971367 24360 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:46.977924 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.984402 22775 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:46.985769 22708 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:46.986729 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:46.989518 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:46.990324 22709 raft_consensus.cc:2368] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:46.999043 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:47.000264 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.001928 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.002940 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:46.971740 22776 raft_consensus.cc:606] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:46.983915 24381 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.017256 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.018883 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:47.024639 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.029490 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:47.010759 24360 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:47.033674 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.011149 24248 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:46.974957 24425 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:09:47.043795 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.046084 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.044360 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.046774 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.023278 24358 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:47.054731 24343 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:47.035633 24316 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:09:47.056370 24343 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.056617 24214 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.057608 24343 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.025947 24318 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:47.070842 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.078855 23421 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.080682 22783 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.081280 22784 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.081844 22779 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:47.082037 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.042915 24360 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:47.083228 22775 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.083431 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.068743 24214 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.083983 22782 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
W20260709 13:09:47.083979 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 97369.2us
I20260709 13:09:47.040978 24248 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:47.085130 23421 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:47.025738 24407 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:09:47.045058 24392 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:47.084951 24318 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.087481 22775 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:47.088383 24426 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:47.090444 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 350) took 1291 ms. Trace:
I20260709 13:09:47.091979 24408 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.021771 24381 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.092896 24288 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:47.128125 22701 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.111455 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.112921 22708 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.113747 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.131464 22702 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.109975 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:47.127470 24408 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.106276 24248 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.135567 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.135852 24408 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.137324 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.110116 24133 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:47.139040 24381 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:47.114518 24163 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:47.114885 23412 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:09:47.140138 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.140949 24163 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.141456 24163 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.115868 24166 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:47.117898 24363 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.097190 24383 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:47.094691 24360 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:47.126448 24407 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.127014 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.157213 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.123081 23417 rpcz_store.cc:276] 0709 13:09:45.798582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.798757 (+   175us) service_pool.cc:224] Handling call
0709 13:09:46.612667 (+813910us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:46.654071 (+ 41404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.090222 (+436151us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54004e9650. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:47.090379 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1871125760}
I20260709 13:09:47.143193 22634 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.143095 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.159204 22709 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:47.139391 24133 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.130461 22708 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.166291 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.139029 23411 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:47.118798 24336 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:09:47.094045 24391 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:47.108767 24351 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:47.182912 24351 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.185609 22709 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:47.185966 22702 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:47.168233 24316 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:09:47.190121 23412 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.142184 24392 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:47.197307 24392 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.152518 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.145651 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.200351 24363 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.119402 24214 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.164175 24383 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.204102 24383 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.173023 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.163889 24266 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:09:47.205384 22704 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:47.191660 24391 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.163419 24407 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.207110 24358 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:47.208004 24358 consensus_queue.cc:1243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 6a31ac6451ff4f73bee634487d84d001 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:09:47.208534 24358 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.208369 24407 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.209750 24407 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.210421 24316 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:47.210811 24316 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.211666 24316 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.212931 24316 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:47.216986 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:47.217906 24248 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.208895 24358 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.219419 24307 raft_consensus.cc:2764] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:09:47.249022 22634 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:09:47.249450 22634 pending_rounds.cc:85] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 5
I20260709 13:09:47.249706 22634 pending_rounds.cc:107] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:09:47.249972 22634 raft_consensus.cc:2889] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:47.251065 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.250363 23414 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEARNER]: Advancing to term 4
I20260709 13:09:47.254192 24408 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.175860 22708 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.264828 22708 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.190397 22703 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:47.266698 23414 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEARNER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:47.204294 24214 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:47.267488 24248 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.223965 24383 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:47.268639 24383 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.268977 24383 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:47.269989 24326 raft_consensus.cc:1068] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Unable to promote non-voter 6a31ac6451ff4f73bee634487d84d001: Service unavailable: leader transfer in progress
I20260709 13:09:47.269800 24383 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.224504 24268 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:47.156477 24318 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.229219 23419 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:09:47.230273 23417 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.171232 24163 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.178354 24133 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:47.235004 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 125659us
I20260709 13:09:47.290772 24248 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:09:47.242038 24358 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.191145 24351 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:47.244536 24345 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.245456 23416 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:09:47.256261 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.256148 24223 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:47.223111 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:47.258898 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:47.312563 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:47.313813 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:47.260043 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.260851 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.277372 22632 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.280083 22635 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.317909 22531 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 234) took 4412 ms. Trace:
W20260709 13:09:47.271016 24343 consensus_queue.cc:1175] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a31ac6451ff4f73bee634487d84d001" responder_term: 2 status { last_received { term: 2 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:47.283160 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.277354 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.289951 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:47.322080 23417 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:47.295295 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 178485us
I20260709 13:09:47.241665 24253 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:47.299640 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.243397 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:47.307309 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.258697 22630 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:47.318728 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:47.324127 22626 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:47.323897 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.289410 24268 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.325941 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:47.309719 24345 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.306901 24167 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:09:47.310905 24223 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:47.357983 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.318354 22531 rpcz_store.cc:276] 0709 13:09:42.899425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.906773 (+  7348us) service_pool.cc:224] Handling call
0709 13:09:43.496785 (+590012us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:47.311467 (+3814682us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:46.542519 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:46.543665 (+  1146us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:46.543701 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:09:46.543719 (+    18us) tablet.cc:662] Decoding operations
0709 13:09:46.544076 (+   357us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:46.544249 (+   173us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:46.544284 (+    35us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:09:46.544481 (+   197us) tablet.cc:701] Row locks acquired
0709 13:09:46.544498 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:09:46.544673 (+   175us) write_op.cc:270] Start()
0709 13:09:46.544795 (+   122us) write_op.cc:276] Timestamp: P: 1783602586544599 usec, L: 0
0709 13:09:46.544817 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:46.545589 (+   772us) log.cc:844] Serialized 538 byte log entry
0709 13:09:46.762196 (+216607us) op_driver.cc:464] REPLICATION: finished
0709 13:09:46.793835 (+ 31639us) write_op.cc:301] APPLY: starting
0709 13:09:46.793968 (+   133us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:46.794129 (+   161us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:46.794295 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:46.794663 (+   368us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:46.794906 (+   243us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:09:46.794990 (+    84us) write_op.cc:312] APPLY: finished
0709 13:09:46.795562 (+   572us) log.cc:844] Serialized 23 byte log entry
0709 13:09:46.795788 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:46.795887 (+    99us) write_op.cc:454] Released schema lock
0709 13:09:46.795926 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:46.796328 (+   402us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:46.796365 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12302,"reported_tablets":6,"spinlock_wait_cycles":469620608,"child_traces":[["op",{"apply.queue_time_us":31325,"apply.run_cpu_time_us":2325,"apply.run_wall_time_us":2525,"num_ops":1,"prepare.queue_time_us":2922,"prepare.run_cpu_time_us":12533,"prepare.run_wall_time_us":91922,"raft.queue_time_us":100625,"raft.run_cpu_time_us":15569,"raft.run_wall_time_us":31617,"replication_time_us":217228,"thread_start_us":165593,"threads_started":4,"wal-append.queue_time_us":113760}]]}
I20260709 13:09:47.348547 24363 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:47.354809 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.358950 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:47.347743 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080014b9e0 after lost signal to thread 21526
I20260709 13:09:47.364624 22628 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:47.327129 24253 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:47.322131 23415 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:47.332650 24358 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.350294 23414 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:47.350478 24351 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:47.376237 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:47.367525 22805 debug-util.cc:398] Leaking SignalData structure 0x7b0800094f20 after lost signal to thread 21529
I20260709 13:09:47.372845 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:47.389448 23412 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:47.390168 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:37296 (request call id 9) took 2575 ms. Trace:
I20260709 13:09:47.360991 23411 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:09:47.380295 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54486.6us
I20260709 13:09:47.374758 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.376751 24349 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:09:47.382308 24253 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.328143 24352 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:09:47.384493 24300 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:47.421419 23420 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.366508 24363 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.428213 22601 leader_election.cc:400] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4.
I20260709 13:09:47.431548 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.423801 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:47.419859 24325 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:47.418417 24166 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:47.427263 24253 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:09:47.436980 22805 debug-util.cc:398] Leaking SignalData structure 0x7b08001ca9c0 after lost signal to thread 22567
I20260709 13:09:47.467695 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.469527 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.463217 22706 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:47.465408 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.475093 24166 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:47.475951 24166 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.443007 24325 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:47.480199 24325 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:47.478183 24166 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.504294 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.506208 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.511855 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:47.463898 24336 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.420200 24345 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.522948 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.398065 22776 rpcz_store.cc:276] 0709 13:09:44.814483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.814731 (+   248us) service_pool.cc:224] Handling call
0709 13:09:47.267482 (+2452751us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b54000c9f20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5a520 00007f1806ed7652 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6
0709 13:09:47.389978 (+122496us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54003250d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:47.390072 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4850225792}
I20260709 13:09:47.443902 24167 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:09:47.525985 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:47.554000 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.556766 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.560669 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.467466 24318 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:47.525401 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.538316 24268 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:47.466187 24360 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.427429 24223 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.543731 24445 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:47.544759 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:47.549006 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.550477 22635 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:47.551750 22704 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:09:47.551841 24356 consensus_queue.cc:1175] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ebb78bf12f41baae84669c1effc423" responder_term: 3 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:47.551780 24345 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.572293 24392 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:47.601732 23417 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:47.573418 24351 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:09:47.578570 22805 debug-util.cc:398] Leaking SignalData structure 0x7b0800120960 after lost signal to thread 22659
I20260709 13:09:47.575711 23412 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.613797 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.577390 24339 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:47.558403 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.588063 22783 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:47.615374 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1418) took 1584 ms. Trace:
I20260709 13:09:47.588876 22780 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:09:47.617672 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 193987us
I20260709 13:09:47.593734 22628 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20260709 13:09:47.593911 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.574405 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:47.588778 24360 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:47.583791 24363 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:47.624958 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66624.7us
I20260709 13:09:47.602911 24300 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.581923 24195 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
W20260709 13:09:47.615342 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 183897us
W20260709 13:09:47.616381 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080032f740 after lost signal to thread 22660
I20260709 13:09:47.686335 24300 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:47.688309 22805 debug-util.cc:398] Leaking SignalData structure 0x7b0800392320 after lost signal to thread 22734
W20260709 13:09:47.692739 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.695578 22708 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:47.643468 24352 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:47.696875 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.696892 24352 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.698755 24352 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.686280 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.701150 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.701942 24170 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:47.704644 22601 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.618041 22629 rpcz_store.cc:276] 0709 13:09:46.031370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:46.031505 (+   135us) service_pool.cc:224] Handling call
0709 13:09:47.097702 (+1066197us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:47.097963 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:47.098216 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:09:47.291504 (+193288us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.591456 (+299952us) spinlock_profiling.cc:243] Waited 41.6 ms on lock 0x7b1800393220. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:47.615034 (+ 23578us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:47.615097 (+    63us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:09:47.615141 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.615208 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.615317 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":192704,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":10810,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5468,"spinlock_wait_cycles":101426816,"thread_start_us":477464,"threads_started":3,"wal-append.queue_time_us":175470}
I20260709 13:09:47.600025 24318 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.720528 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.614074 24167 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:47.702431 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.698274 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.736056 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:47.741430 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.742966 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.744738 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.643060 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:47.714968 24170 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:47.717674 24432 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:47.724000 22776 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:47.737406 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080032f3c0 after lost signal to thread 22808
I20260709 13:09:47.763319 24268 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:47.727150 24067 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:09:47.764465 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.771314 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.618589 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:47.774382 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 49878.1us
I20260709 13:09:47.740322 24167 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.777519 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.777330 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.745600 24349 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:09:47.753461 24248 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:09:47.789439 22706 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:47.790475 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.752731 24325 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.793449 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.795483 22780 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:47.601083 24445 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.787283 24408 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:47.799976 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.803602 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.803104 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.800998 24248 consensus_queue.cc:1243] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 6a31ac6451ff4f73bee634487d84d001 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:09:47.804656 24456 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:47.805612 23412 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.806018 24167 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.807300 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:47.807842 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080039c1a0 after lost signal to thread 23445
I20260709 13:09:47.808543 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:47.808938 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080009fcc0 after lost signal to thread 24323
I20260709 13:09:47.809185 22708 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:47.810122 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.811859 24450 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:47.813690 24349 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.814074 24349 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.814944 24349 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.815315 24167 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:47.816308 24349 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:47.818535 24401 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.401s	user 0.003s	sys 0.000s
I20260709 13:09:47.797485 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.824647 23418 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:47.825322 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.825639 22673 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:47.822464 24435 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.828511 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.831041 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.831449 22629 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:47.833369 22708 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:47.820693 24408 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.831943 24435 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.834194 24170 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:47.834904 24170 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.835081 24435 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.835839 24170 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.836592 24435 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.840833 24408 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:47.841847 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.828451 24248 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:47.843581 24248 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.844647 22710 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:47.827955 24351 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:09:47.844568 24248 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.845338 24456 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:47.850219 22626 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:47.850706 22626 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:47.851578 22626 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:47.852805 22626 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:47.826539 22703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:47.854352 22703 raft_consensus.cc:606] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:47.857640 22710 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:47.859560 22710 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.862859 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:47.833773 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 215269us
I20260709 13:09:47.870654 22780 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:47.871770 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:09:47.875978 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.791100 24166 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:09:47.875303 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.803275 24392 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:09:47.885093 24456 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:47.881151 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:47.888695 24166 consensus_queue.cc:1243] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:09:47.891819 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:47.895700 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.895980 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:47.896867 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:47.897781 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.898435 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:47.898907 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.899314 22705 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:47.900053 23385 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.900135 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.902302 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.907806 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:47.911953 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:47.913612 22701 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:47.914175 22701 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:47.881517 24336 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:47.921872 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:47.922590 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.925616 22775 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:47.922878 22709 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:09:47.919625 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:47.923558 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:47.886366 24365 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:09:47.932333 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.938817 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:47.886896 24450 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:09:47.922416 24268 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:47.923100 24287 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:47.957181 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57232.7us
W20260709 13:09:47.955237 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86882.3us
I20260709 13:09:47.932911 24318 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:47.965131 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:47.965189 22633 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:47.949993 22701 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:47.967381 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:47.973649 22701 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:47.982824 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:47.984607 24325 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:47.936508 24254 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:09:47.991693 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.992622 24287 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:47.993474 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.994602 24266 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:47.994984 22781 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:47.997440 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.997804 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.999312 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:47.999891 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:47.983263 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:48.001521 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:48.002506 22781 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:47.945930 24365 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:47.962509 24451 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:09:48.004495 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:48.005541 22778 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:48.002746 22710 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:47.996121 24266 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:47.938552 24363 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:09:48.013321 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:48.013788 22775 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:48.044947 22777 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:48.045827 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:47.978942 24455 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:48.016852 22781 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:48.050758 22782 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:48.016945 24383 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:47.938715 24228 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:09:48.008469 24432 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.023664 24163 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:48.001310 24166 raft_consensus.cc:2764] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:09:47.958351 24268 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:48.033447 22701 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:09:48.033814 24358 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.652s	user 0.009s	sys 0.000s
I20260709 13:09:48.043642 24325 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:48.015105 24254 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:48.016494 22784 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:48.045921 24225 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:48.046881 22533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 232) took 5242 ms. Trace:
W20260709 13:09:48.048252 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.079555 22533 rpcz_store.cc:276] 0709 13:09:42.804990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.805123 (+   133us) service_pool.cc:224] Handling call
0709 13:09:42.860527 (+ 55404us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:09:48.046804 (+5186277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63430,"reported_tablets":5,"spinlock_wait_cycles":6466944}
I20260709 13:09:48.081239 22777 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:09:48.084208 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:48.048004 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:48.054028 22626 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:48.059648 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:48.087795 22781 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:48.060712 22630 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:48.063298 22778 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:48.066495 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.011606 24287 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.069440 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.104418 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.035472 24445 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:48.094216 22629 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:09:48.105710 22629 pending_rounds.cc:85] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 1
W20260709 13:09:48.092732 24307 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:09:48.105985 22629 pending_rounds.cc:107] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:09:48.106859 22629 raft_consensus.cc:2889] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:48.096694 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:48.096837 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 90649.4us
I20260709 13:09:48.084908 24455 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:48.050192 24351 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:09:48.073397 24268 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:48.086766 24225 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.119390 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.101668 24414 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:48.126715 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.126245 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:48.130056 24408 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:48.127511 24445 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:48.131494 24445 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:48.131943 24445 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:48.133368 24225 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.127756 24358 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:48.135358 24358 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.135409 24414 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.136541 24358 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.105541 24254 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:09:48.137979 24254 consensus_queue.cc:1243] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:48.123134 24248 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.100577 24365 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.141674 24455 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:48.142489 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.147276 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.150583 22635 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:48.153048 22626 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:48.159272 22626 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:48.159642 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:48.162401 22626 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.167760 22626 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:48.146937 24225 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:48.156903 24253 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.657s	user 0.014s	sys 0.000s
W20260709 13:09:48.182035 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 276870us
I20260709 13:09:48.183785 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.184530 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:48.185282 22701 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:48.185875 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:48.182300 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.193411 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:48.170135 24405 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:48.169667 24214 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:09:48.164199 24408 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:09:48.191294 24435 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.197144 24456 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:48.198444 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.205379 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:48.207259 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:48.207427 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 95859.7us
I20260709 13:09:48.210327 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.212711 24456 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:48.204107 24405 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:48.213888 24456 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:48.214388 24405 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.194092 24352 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:09:48.211616 24254 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:48.211817 24195 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.218860 24195 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.221185 24195 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:48.232486 22701 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:48.233165 22701 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.234324 22701 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.239367 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.240681 22706 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:48.241755 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.249192 22626 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:09:48.249993 22626 pending_rounds.cc:85] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 2
I20260709 13:09:48.250316 22626 pending_rounds.cc:107] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:09:48.254065 24254 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.208096 24339 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.254997 24228 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.264225 24356 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:48.264792 24356 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:48.265673 24356 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.267314 24356 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:48.271342 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.250659 22626 raft_consensus.cc:2889] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:48.274140 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:48.288311 22701 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:48.291929 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:09:48.292919 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.293503 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:48.294173 22635 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:48.296300 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:48.261972 24445 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:48.319303 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77635.6us
W20260709 13:09:48.322575 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.326400 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.348337 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.350803 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.351765 22635 pending_rounds.cc:85] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 5
I20260709 13:09:48.352217 22635 pending_rounds.cc:107] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:09:48.360504 22701 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:48.352607 22635 raft_consensus.cc:2889] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:48.293047 24477 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:48.378185 24287 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.320094 24414 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:48.346145 24258 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:09:48.358222 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.361984 24470 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:09:48.283664 24214 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 13:09:48.346642 24300 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:48.363421 24254 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:09:48.406898 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.363338 24349 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:48.422457 24163 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:48.423081 22633 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:09:48.420922 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68867.5us
I20260709 13:09:48.450148 22633 pending_rounds.cc:85] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 1
W20260709 13:09:48.380157 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 169113us
I20260709 13:09:48.450841 22633 pending_rounds.cc:107] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:09:48.376170 24463 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:48.451248 22633 raft_consensus.cc:2889] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:48.444347 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 61001.6us
I20260709 13:09:48.385658 24477 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.463934 24477 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.465485 24477 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:48.468394 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 120292us
I20260709 13:09:48.478168 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.480274 24435 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:48.480952 24435 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:48.440874 24300 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:48.445848 24254 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.491446 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.491850 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:48.509146 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:48.513680 22750 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:48.516438 22781 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:48.516927 22782 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:48.518797 24258 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:48.519286 24258 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:48.519687 24258 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:48.519994 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:48.530097 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.484540 24445 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:48.500574 24300 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:48.495538 24466 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.511740 24254 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:48.511372 24261 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:48.533077 24475 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 13:09:48.482118 24435 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:48.580435 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89071.8us
I20260709 13:09:48.530179 24195 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.582371 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.555305 24214 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.555542 24358 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.586114 24358 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.588764 24479 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.548144 24486 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:09:48.537285 23414 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:09:48.590785 24167 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:48.590466 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:48.592386 24479 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.559405 24483 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:48.594969 24483 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.596112 24483 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.597713 24483 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:48.599157 24448 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.599202 24487 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:48.599629 24479 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:48.601713 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:48.603291 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.533597 24408 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:09:48.612375 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:48.599961 24448 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.610116 23419 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:48.599995 24487 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.588824 24358 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:48.633378 24358 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:48.608777 24492 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.609400 24496 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.638106 24479 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:48.561133 24466 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:48.685727 24493 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:48.686012 24448 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:48.687471 24480 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:09:48.620396 24498 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:09:48.689085 24258 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.692889 24214 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:48.693379 24214 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.694262 24214 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.696257 24214 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:48.698843 24300 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.664440 24496 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.703323 24496 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:48.704558 24487 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:48.706267 24223 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:48.706444 24258 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:48.707976 24484 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:09:48.708416 24300 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:48.714278 24383 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.645s	user 0.009s	sys 0.003s
W20260709 13:09:48.707778 24268 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.001s
I20260709 13:09:48.716840 24268 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.715284 24254 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.788120 22631 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:48.791628 22701 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:48.884229 24254 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:48.799213 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.726675 24465 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.586s	user 0.000s	sys 0.014s
I20260709 13:09:48.889715 24318 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.703364 24466 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:48.725567 24473 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.461s	user 0.003s	sys 0.010s
W20260709 13:09:48.689126 24482 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:09:48.684341 24479 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:48.952348 24479 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:48.952679 24477 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:09:48.727815 24463 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:48.720198 24268 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:48.853786 23411 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:48.855446 24468 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.592s	user 0.003s	sys 0.000s
I20260709 13:09:48.869153 24356 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.873389 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:48.877120 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:48.877640 24253 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:48.878815 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.970439 24496 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.971560 24496 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.954356 24479 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.942561 24325 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:48.975375 24253 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.976974 24325 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.978231 24248 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.979372 24248 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:48.981494 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.927352 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.984946 24253 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:48.935940 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.927470 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:48.985984 23414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46262 (request call id 363) took 1254 ms (client timeout 1660 ms). Trace:
I20260709 13:09:48.986163 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:48.926604 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 161) took 1051 ms. Trace:
I20260709 13:09:48.927434 24449 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.987833 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:48.988597 24449 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:48.950734 24448 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:48.926792 24478 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:09:48.963526 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 324678us
W20260709 13:09:49.001732 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.978209 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:48.981613 24491 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.003s	sys 0.000s
I20260709 13:09:48.890792 24318 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:48.946851 24254 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.031738 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:49.032342 22635 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.020s	sys 0.010s
W20260709 13:09:49.032862 22635 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.020s	sys 0.010s
I20260709 13:09:49.032948 22702 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:48.998473 24248 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:49.045651 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.048017 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:48.986533 23414 rpcz_store.cc:269] 0709 13:09:47.731799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.731954 (+   155us) service_pool.cc:224] Handling call
0709 13:09:47.869666 (+137712us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:09:48.985457 (+1115791us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54000db454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:48.985699 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677898240}
W20260709 13:09:49.044265 24365 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.015s	sys 0.011s
I20260709 13:09:49.059366 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:49.062155 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 103801us
W20260709 13:09:49.063400 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.066676 24254 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.068967 24474 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.068845 24356 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:49.070044 24254 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.071657 22710 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:49.071767 24486 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:49.072729 24486 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:49.080251 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 175) took 1795 ms (client timeout 1656 ms). Trace:
W20260709 13:09:49.080883 22631 rpcz_store.cc:269] 0709 13:09:47.284833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.313594 (+ 28761us) service_pool.cc:224] Handling call
0709 13:09:49.079961 (+1766367us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540006c2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:49.080131 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3349452032}
I20260709 13:09:49.087674 22627 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:49.090919 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 159) took 1270 ms. Trace:
I20260709 13:09:48.987439 22780 rpcz_store.cc:276] 0709 13:09:47.874724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.876808 (+  2084us) service_pool.cc:224] Handling call
0709 13:09:47.877291 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.877796 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.877842 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.879137 (+  1295us) log.cc:844] Serialized 26 byte log entry
0709 13:09:48.051731 (+172594us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:48.051797 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:48.051901 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:48.725964 (+674063us) spinlock_profiling.cc:243] Waited 42 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:48.925562 (+199598us) raft_consensus.cc:1685] finished
0709 13:09:48.925617 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:48.926218 (+   601us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1070,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":5547,"spinlock_wait_cycles":96570624,"thread_start_us":133608,"threads_started":1,"wal-append.queue_time_us":236142}
W20260709 13:09:49.086833 24365 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.016s	sys 0.012s
I20260709 13:09:49.058403 24448 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:49.112821 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1710) took 1208 ms. Trace:
I20260709 13:09:49.113574 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 362) took 1391 ms. Trace:
I20260709 13:09:49.114019 22632 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:49.114864 23415 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEARNER]: Advancing to term 5
W20260709 13:09:49.115494 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 62126.7us
I20260709 13:09:49.116828 24486 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.117832 24318 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.118510 24449 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.119151 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.089965 23421 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:49.120919 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1445) took 1133 ms. Trace:
W20260709 13:09:49.121703 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.098816 24496 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.137887 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.140414 22706 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:49.143309 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:49.144433 22631 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:49.144876 24490 raft_consensus.cc:993] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:49.144992 24391 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:09:49.147100 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.123147 23420 rpcz_store.cc:276] 0709 13:09:47.721677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.721995 (+   318us) service_pool.cc:224] Handling call
0709 13:09:47.909614 (+187619us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:47.909809 (+   195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:48.987566 (+1077757us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800345c40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ed5bfb 00007f1806ee064b
0709 13:09:48.987921 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:48.987967 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:48.989143 (+  1176us) log.cc:844] Serialized 26 byte log entry
0709 13:09:49.022758 (+ 33615us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:49.022817 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.035062 (+ 12245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:49.112966 (+ 77904us) raft_consensus.cc:1685] finished
0709 13:09:49.112994 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.113224 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2854,"raft.queue_time_us":38549,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":5978,"spinlock_wait_cycles":40852096,"thread_start_us":86873,"threads_started":2,"wal-append.queue_time_us":63068}
I20260709 13:09:49.148527 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.091504 22776 rpcz_store.cc:276] 0709 13:09:47.820103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.820292 (+   189us) service_pool.cc:224] Handling call
0709 13:09:47.823171 (+  2879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.832797 (+  9626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.856664 (+ 23867us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.892636 (+ 35972us) log.cc:844] Serialized 26 byte log entry
0709 13:09:48.093114 (+200478us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:48.096251 (+  3137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:48.103176 (+  6925us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:48.730701 (+627525us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:49.090187 (+359486us) raft_consensus.cc:1685] finished
0709 13:09:49.090241 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.090662 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1677,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12916,"spinlock_wait_cycles":36586368,"thread_start_us":139896,"threads_started":1,"wal-append.queue_time_us":228836}
I20260709 13:09:49.152557 24325 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.061057 24478 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.156287 24478 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.161535 24481 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:49.162127 24481 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:49.172716 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:49.070109 24474 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.174602 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:49.176054 24474 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.178700 24448 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.178047 22530 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.179330 24343 raft_consensus.cc:993] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:49.179965 23411 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:49.176069 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.132982 22629 rpcz_store.cc:276] 0709 13:09:47.986930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.987115 (+   185us) service_pool.cc:224] Handling call
0709 13:09:47.987436 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:48.107480 (+120044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.120416 (+1012936us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002f5154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:49.120728 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244674432}
I20260709 13:09:49.178303 24479 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:49.185221 24356 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.188611 24258 raft_consensus.cc:1081] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:09:49.189359 22780 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:49.196802 24170 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:49.197047 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:49.197863 24170 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:49.180823 22530 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.198824 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:49.199461 22630 raft_consensus.cc:2376] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2. Re-sending same reply.
I20260709 13:09:49.200002 22530 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.201349 24318 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:49.202327 24318 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:49.205197 24287 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.206207 24287 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.211227 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.219388 24481 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:49.221217 24481 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:49.189622 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:49.123096 22628 rpcz_store.cc:276] 0709 13:09:47.903916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.904052 (+   136us) service_pool.cc:224] Handling call
0709 13:09:47.904355 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.920424 (+ 16069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.920457 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.933544 (+ 13087us) log.cc:844] Serialized 26 byte log entry
0709 13:09:48.623047 (+689503us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18004030c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:09:49.031953 (+408906us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:09:49.032056 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.032194 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:49.112170 (+ 79976us) raft_consensus.cc:1685] finished
0709 13:09:49.112196 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.112573 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16689,"raft.queue_time_us":18248,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":28200,"spinlock_wait_cycles":17084800,"thread_start_us":326759,"threads_started":1,"wal-append.queue_time_us":1102586}
I20260709 13:09:49.228718 23420 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:49.202350 22530 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 3, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:49.240891 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55880.1us
W20260709 13:09:49.239748 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61390.9us
I20260709 13:09:49.257649 24225 raft_consensus.cc:1081] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
W20260709 13:09:49.260987 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123394us
I20260709 13:09:49.263437 24503 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:49.263865 24318 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.187178 24486 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:09:49.271616 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:49.287309 22702 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:09:49.287776 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 98237.5us
I20260709 13:09:49.270695 24170 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.291013 22631 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:49.283653 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.272933 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 186) took 1832 ms. Trace:
I20260709 13:09:49.302019 22530 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:49.309455 24166 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 13:09:49.309616 24490 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:49.312500 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66466.4us
I20260709 13:09:49.318256 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.323765 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1574) took 1032 ms. Trace:
I20260709 13:09:49.310493 24432 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.303414 22627 rpcz_store.cc:276] 0709 13:09:47.440359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.445226 (+  4867us) service_pool.cc:224] Handling call
0709 13:09:49.082746 (+1637520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:49.088355 (+  5609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.268609 (+180254us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b540006c2d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:49.270576 (+  1967us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3633518464}
I20260709 13:09:49.303920 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.328070 22776 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:49.309533 24223 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:49.332724 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:49.333019 24223 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.333588 24223 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:49.322257 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:49.355119 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.358577 24214 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.662s	user 0.005s	sys 0.012s
I20260709 13:09:49.359730 24195 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:49.360867 24195 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.367652 24195 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.330878 22630 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:49.381662 23415 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:09:49.382169 23415 pending_rounds.cc:85] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 3
I20260709 13:09:49.342736 24325 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:49.393147 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 90340us
I20260709 13:09:49.355245 24445 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:49.407796 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:09:49.326718 24432 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.409185 24435 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:49.411036 24432 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.412086 24325 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.416287 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1441) took 1508 ms. Trace:
I20260709 13:09:49.396847 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.397746 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1718) took 1229 ms. Trace:
I20260709 13:09:49.382463 23415 pending_rounds.cc:107] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:09:49.398597 24287 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.399097 24466 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:49.400573 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.400971 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:49.404834 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142192us
I20260709 13:09:49.407229 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.426265 22780 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:49.324365 22703 rpcz_store.cc:276] 0709 13:09:48.291126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.295352 (+  4226us) service_pool.cc:224] Handling call
0709 13:09:48.396095 (+100743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:48.510702 (+114607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:48.516650 (+  5948us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:48.600727 (+ 84077us) log.cc:844] Serialized 26 byte log entry
0709 13:09:49.081679 (+480952us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:49.191525 (+109846us) log.cc:844] Serialized 14 byte log entry
0709 13:09:49.220981 (+ 29456us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b18000b57e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:49.262931 (+ 41950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.263038 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:49.323046 (+ 60008us) raft_consensus.cc:1685] finished
0709 13:09:49.323269 (+   223us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.323521 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105938,"prepare.queue_time_us":222115,"raft.queue_time_us":96954,"raft.run_cpu_time_us":416,"raft.run_wall_time_us":23149,"spinlock_wait_cycles":24270080,"thread_start_us":512114,"threads_started":2,"wal-append.queue_time_us":429678}
I20260709 13:09:49.397660 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:49.418843 23415 raft_consensus.cc:2889] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:49.416786 22632 rpcz_store.cc:276] 0709 13:09:47.907760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.907965 (+   205us) service_pool.cc:224] Handling call
0709 13:09:49.113582 (+1205617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:49.131111 (+ 17529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.415972 (+284861us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003d2f50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:49.416196 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2771237760}
I20260709 13:09:49.430227 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.417905 22635 rpcz_store.cc:276] 0709 13:09:48.168270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.351223 (+182953us) service_pool.cc:224] Handling call
0709 13:09:48.351549 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:49.046768 (+695219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.050031 (+  3263us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:49.164607 (+114576us) log.cc:844] Serialized 52 byte log entry
0709 13:09:49.284272 (+119665us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:49.284331 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.284463 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:49.397425 (+112962us) raft_consensus.cc:1685] finished
0709 13:09:49.397451 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:49.397679 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45171,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":98,"thread_start_us":770732,"threads_started":2,"wal-append.queue_time_us":169462}
W20260709 13:09:49.438657 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.443377 22781 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:49.449406 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.429973 24356 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:49.451195 24356 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:49.454072 24356 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.455731 22631 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:09:49.457060 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.423910 24325 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:49.459227 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116135us
I20260709 13:09:49.461015 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:49.466827 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.469404 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:49.470443 22627 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:49.472004 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:49.475210 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:49.453850 22633 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.476122 22702 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:49.480744 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:49.482016 22629 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:49.476809 24483 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:49.478538 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:49.481179 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.428210 24365 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.495875 22780 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:09:49.490763 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.478299 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:49.492489 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:49.492828 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.480604 24466 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 13:09:49.497349 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50562.9us
W20260709 13:09:49.502681 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.509130 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.504204 22782 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:49.504217 24337 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:49.505553 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.506124 22701 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:09:49.506251 24253 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:49.519750 24253 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.520255 24253 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:49.520987 24365 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.522826 22783 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:49.524189 24365 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.527299 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:49.528368 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 50214.1us
I20260709 13:09:49.529665 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:49.530925 24339 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:49.531844 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:49.510277 22634 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:49.502022 24166 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:49.517970 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:49.519003 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.533095 22634 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:49.532522 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43540.2us
I20260709 13:09:49.506835 24483 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:49.548738 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 46230.2us
I20260709 13:09:49.508714 24223 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.567081 24223 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.571899 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.545540 24337 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.543823 24445 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:49.577848 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 78644.2us
I20260709 13:09:49.575456 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:49.596606 22781 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:49.574915 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:49.595108 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.538308 24166 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.619205 22777 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:49.596249 24445 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.606099 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:49.561983 24170 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:09:49.636468 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.641486 23414 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:49.644585 24358 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:49.645367 24358 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:49.647023 24167 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:49.646859 24358 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:49.648746 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53742.8us
I20260709 13:09:49.649611 24509 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:49.648806 24527 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:09:49.668681 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:49.675884 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103373us
I20260709 13:09:49.635586 24445 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:49.694779 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:49.696130 24343 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:49.713996 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:09:49.720616 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 82152.7us
I20260709 13:09:49.718601 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:49.728195 24528 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 13:09:49.718999 24352 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:49.716605 24167 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:49.720160 24464 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.183s
I20260709 13:09:49.729161 24483 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.713438 24339 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:49.757012 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.757514 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:49.763805 22634 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.744912 24352 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.738387 24170 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:49.720942 24391 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:09:49.799854 24449 raft_consensus.cc:1064] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:49.800321 24266 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:09:49.800570 24449 raft_consensus.cc:1068] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Unable to promote non-voter 6a31ac6451ff4f73bee634487d84d001: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:49.800990 24266 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:49.802088 24266 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:49.807193 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64853us
I20260709 13:09:49.808259 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:49.740257 24166 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.811244 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:49.766839 24336 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:49.817514 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.772630 24337 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.821491 24337 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:09:49.820691 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.825659 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:49.828521 24300 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:49.826409 24365 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:49.831221 24300 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:49.831758 24300 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.832499 24300 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:49.811625 24339 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.834244 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:49.837352 24339 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:49.837419 24268 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:49.837962 24268 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.838330 24268 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:49.843281 24167 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.770730 24352 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:49.859059 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.868718 24343 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:09:49.847644 24534 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:09:49.769863 24483 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:49.883402 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 163576us
I20260709 13:09:49.884367 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.817401 24336 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:49.902806 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.871209 24166 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:09:49.911403 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 89288.8us
I20260709 13:09:49.896956 24336 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:49.895915 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:49.932359 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.933997 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.926841 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.937809 22777 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
W20260709 13:09:49.945715 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61475.1us
I20260709 13:09:49.945637 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.912357 23411 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:49.948089 22783 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:49.951277 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.955641 24266 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:49.956502 24266 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:49.958968 24539 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:49.959681 24539 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:49.960181 24539 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:49.904351 24253 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.966936 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:49.976215 22779 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:49.937752 24365 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:09:49.980690 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:49.981431 24365 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:49.981882 24365 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:49.982043 24535 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 13:09:49.982263 24365 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:49.983743 24167 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:49.984203 24167 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:49.985049 24167 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:49.986356 24167 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:49.986532 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:49.987224 24253 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:49.963340 24266 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:49.995247 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 104258us
W20260709 13:09:49.998857 24481 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.777s	user 0.011s	sys 0.009s
I20260709 13:09:49.999925 24166 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:50.021055 22710 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:09:50.021670 22710 pending_rounds.cc:85] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 3
I20260709 13:09:50.021971 22710 pending_rounds.cc:107] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:09:50.022264 22710 raft_consensus.cc:2889] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:50.023250 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:50.050052 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.018443 24166 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:50.053754 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:50.024106 24481 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:09:50.041627 24214 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:50.061920 22706 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:09:50.065322 22705 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:50.057714 24166 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:50.092212 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:50.099314 24539 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.106859 22706 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:50.110762 22707 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:50.103966 22750 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.544s (SENT)
W20260709 13:09:50.115649 24546 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:09:50.116061 24499 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:50.116854 24499 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.121683 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:50.129464 22705 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:09:50.129464 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:50.129288 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:50.132318 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:50.135134 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.136090 22707 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:50.136858 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.137701 22707 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:50.138640 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.139351 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:50.140003 24545 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:09:50.140179 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.140472 24486 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:50.141100 24486 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.142099 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.144049 22706 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 13:09:50.147558 24544 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:09:50.135142 24561 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.151757 24561 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.155454 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:50.175788 22628 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:50.144075 22708 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:09:50.154871 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.165323 22634 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:50.166709 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64339.4us
I20260709 13:09:50.122860 24556 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.154945 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:50.153702 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:50.185815 24539 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:50.155606 24543 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:09:50.195139 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.222735 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:50.206764 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53152.1us
I20260709 13:09:50.224423 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:50.225502 22706 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:50.227746 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:50.158144 24214 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.258953 24214 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.159723 24499 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:50.222803 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:50.232225 24551 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.160215 24462 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:09:50.233296 24541 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:09:50.234601 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:50.235929 22632 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:50.236147 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:50.222246 24166 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.273401 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:50.239547 24503 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.163913 24550 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:50.240865 24532 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:09:50.243954 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:50.285172 22628 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:50.245252 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 155099us
I20260709 13:09:50.246930 24405 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:50.229610 24556 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.247910 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:50.252048 23414 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:50.226864 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:50.268599 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.270813 22708 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:50.271891 22631 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:50.266069 24462 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:50.239324 24325 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:50.212419 24358 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:50.294498 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46699.5us
I20260709 13:09:50.295897 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:50.208269 24266 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:50.300174 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 99202.8us
I20260709 13:09:50.308534 22633 raft_consensus.cc:2410] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:09:50.310149 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.314771 22627 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:50.314888 24486 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.300459 24561 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:50.325871 24445 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.611s	user 0.001s	sys 0.012s
I20260709 13:09:50.291633 24336 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.329030 24336 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:50.329893 22626 raft_consensus.cc:2368] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:50.306847 24325 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:50.330689 24325 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:50.331341 24481 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:50.323365 24557 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.323577 24351 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:50.340584 22631 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:50.345433 24462 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.346072 24166 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.347404 22708 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:50.341185 24358 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.305186 24503 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.316987 24555 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.364638 24575 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:50.300871 24550 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.367782 24358 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.368996 24550 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.369426 24503 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.326390 24445 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.320236 24214 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.374547 22705 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:50.315464 24551 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.376636 24555 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:50.340708 24300 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.507s	user 0.014s	sys 0.000s
I20260709 13:09:50.310797 24365 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.340031 24268 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.409153 24474 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:09:50.384517 24557 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.407200 24481 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.415938 24557 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.417019 24481 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.417856 24551 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:50.418648 24300 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.419592 24365 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:50.406800 24560 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.293s	user 0.000s	sys 0.004s
I20260709 13:09:50.421635 24300 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:50.424777 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 115494us
I20260709 13:09:50.375593 24462 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:50.391482 24556 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.394066 24339 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
W20260709 13:09:50.437177 24352 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.003s
I20260709 13:09:50.446128 24462 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:50.431643 24445 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:50.446936 24268 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:50.447757 24462 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.451001 24462 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:50.452070 24325 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.442015 24352 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:50.479975 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51282.7us
I20260709 13:09:50.482210 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:50.486305 24195 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.493932 24195 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.495625 24352 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:50.619194 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 126716us
I20260709 13:09:50.492903 24325 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:50.613694 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112588us
I20260709 13:09:50.625397 24576 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:50.650847 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.651687 24580 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:50.494730 24572 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 13:09:50.655841 24195 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:50.701125 24580 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:50.704273 24580 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:50.811388 24579 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:50.812633 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 139000us
I20260709 13:09:50.813716 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:50.899541 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:50.899551 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 196031us
I20260709 13:09:50.905428 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:50.908360 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94773.2us
I20260709 13:09:50.912644 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:50.913612 22778 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:09:50.812327 24579 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:50.907629 24499 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:50.938344 24499 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:50.943795 24572 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:50.957274 24499 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:51.292861 23385 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.000s	sys 0.011s
W20260709 13:09:51.291230 22750 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.624s (SENT)
W20260709 13:09:51.294411 23385 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.012s
I20260709 13:09:51.301199 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:51.301754 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 642855us
I20260709 13:09:50.963485 24572 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.288136 24325 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.328532 22778 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:51.338160 24585 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:51.338409 24583 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:09:51.368068 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 458369us
W20260709 13:09:51.369784 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.371204 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:51.374632 24585 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:51.375550 24585 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:51.376343 24585 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:51.391991 24581 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.392926 24581 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.396844 24581 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.391973 24588 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.398823 24588 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.354652 24583 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.324200 24325 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.409691 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:51.412353 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:51.413210 22775 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:51.406558 22628 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.113s	user 0.024s	sys 0.011s
I20260709 13:09:51.417104 22628 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:51.416617 24588 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.435433 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 212) took 1192 ms. Trace:
I20260709 13:09:51.436090 22628 rpcz_store.cc:276] 0709 13:09:50.243342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.243523 (+   181us) service_pool.cc:224] Handling call
0709 13:09:51.417814 (+1174291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:51.442414 22775 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:51.454524 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:51.455210 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.449458 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.457746 22781 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:51.449091 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.723s (SENT) [suppressed 17 similar messages]
I20260709 13:09:51.462252 24499 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:51.470010 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.785s (SENT)
I20260709 13:09:51.471838 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.460795 24586 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:51.471055 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.532s (SENT)
I20260709 13:09:51.472077 22776 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:51.473336 24586 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:51.474244 22750 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.797s (SENT)
I20260709 13:09:51.474936 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.449743 24579 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.461009 24253 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:51.477283 24253 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:51.481315 24325 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.484011 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.468214 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.471920 24580 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:51.486951 24580 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.488390 22776 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:09:51.487252 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.489480 24580 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.490139 24580 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:51.452816 24583 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.493942 24583 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:51.492792 24586 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.503569 22781 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:09:51.504937 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:51.478507 24253 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.507999 24585 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.510291 24325 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:51.511093 24325 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:51.511322 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.959s (SENT)
I20260709 13:09:51.511621 24325 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:51.517701 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.802s (SENT)
I20260709 13:09:51.518316 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.518551 24598 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:51.519618 24598 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.520262 24590 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:51.520409 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 1738) took 1401 ms. Trace:
I20260709 13:09:51.523135 22633 rpcz_store.cc:276] 0709 13:09:50.118804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.122753 (+  3949us) service_pool.cc:224] Handling call
0709 13:09:51.520130 (+1397377us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400127854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:51.520337 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399531904}
W20260709 13:09:51.526129 22750 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.848s (SENT)
I20260709 13:09:51.526762 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.532953 24325 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.540657 24325 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.540542 24590 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:51.542304 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1675) took 1303 ms. Trace:
I20260709 13:09:51.542454 24598 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.542927 22782 rpcz_store.cc:276] 0709 13:09:50.239419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.239613 (+   194us) service_pool.cc:224] Handling call
0709 13:09:50.239983 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:50.352920 (+112937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.352963 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:50.353012 (+    49us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:09:50.353043 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.541965 (+1188922us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003282d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:51.542045 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.542227 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259168000}
I20260709 13:09:51.547305 24601 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.548044 24483 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.548081 24601 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.548882 24483 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:51.549600 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1606) took 1555 ms. Trace:
I20260709 13:09:51.550866 24601 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.551456 24483 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.552563 24483 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.553164 24483 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.550046 22709 rpcz_store.cc:276] 0709 13:09:49.993845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.994074 (+   229us) service_pool.cc:224] Handling call
0709 13:09:49.994504 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.996653 (+  2149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.996685 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.034603 (+ 37918us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.111140 (+ 76537us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:09:50.111185 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.111264 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.331436 (+220172us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:51.549166 (+1217730us) raft_consensus.cc:1685] finished
0709 13:09:51.549195 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.549495 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87924992,"thread_start_us":45812,"threads_started":1,"wal-append.queue_time_us":66174,"wal-append.run_cpu_time_us":11411,"wal-append.run_wall_time_us":1323817}
I20260709 13:09:51.555753 24483 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.559165 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:51.560746 23411 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:09:51.564317 24590 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.567636 24590 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.570601 24590 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.576231 24483 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:51.577317 24483 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:51.577714 24483 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:51.578568 24483 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:51.579317 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:51.579874 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.580207 24483 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:51.584757 24601 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.589121 24601 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.586220 24598 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:51.598493 24598 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.602351 24601 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.608909 24479 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.609931 24479 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.612097 24604 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.612823 24604 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.613137 24479 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.615291 24604 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.615535 24598 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.626593 24601 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.627020 24604 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.627665 24601 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.627803 24604 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.629832 24601 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.630283 24604 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:51.626965 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.632290 24604 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.633411 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.633498 22704 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:51.634173 22709 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.632856 24604 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.635643 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.636220 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.636777 24604 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.636895 22708 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:51.636476 22709 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:51.639815 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.647825 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.649278 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.649978 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.650022 22704 raft_consensus.cc:2376] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2. Re-sending same reply.
I20260709 13:09:51.651314 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.651468 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.651912 22704 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:51.652621 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.652897 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.653494 22706 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:51.653530 22704 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.654381 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.657424 24604 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:51.660065 24604 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:51.661365 24598 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.662042 24598 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.662241 24604 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:51.665220 24603 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.666031 24603 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.668988 24603 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.669725 24598 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.670324 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.671123 22704 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.671212 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.672295 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.672524 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.673085 22704 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.673897 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.674099 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.674889 22704 raft_consensus.cc:2376] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2. Re-sending same reply.
I20260709 13:09:51.675174 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.675938 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.675523 22706 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:51.676505 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.677150 22709 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.677196 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.678215 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.764000 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 205) took 2200 ms. Trace:
I20260709 13:09:51.766212 24577 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:51.840130 24300 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.840555 24268 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.840940 24300 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.841269 24268 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.772125 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.678224 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.843423 22706 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:09:51.844296 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72305.4us
I20260709 13:09:51.845503 22706 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: [2.4-2.4]
I20260709 13:09:51.843532 24609 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.846464 24356 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:51.846421 24609 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.700206 24605 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.847151 24356 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.847918 24605 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.848671 24432 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:51.849042 24609 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.849633 24609 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.850317 24605 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.850183 24609 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.851379 24300 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.851239 24432 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:51.852350 24609 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.816295 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.858026 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.859486 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:51.860852 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:51.817786 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.862178 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:51.863263 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.819128 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.869859 24268 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.734064 24607 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.879992 24607 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.738055 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.889947 22702 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:51.764701 24608 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:51.825030 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.47114e+06us
I20260709 13:09:51.825166 24562 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.708976 24580 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:51.710146 24603 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.719177 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:51.721249 24590 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:51.794726 22708 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:09:51.772680 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 1740) took 1653 ms (client timeout 1865 ms). Trace:
I20260709 13:09:51.750017 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:51.713325 24586 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:09:51.915124 24490 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.109s	user 0.011s	sys 0.007s
W20260709 13:09:51.935842 24490 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.130s	user 0.012s	sys 0.007s
I20260709 13:09:51.770011 24325 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:51.843384 24351 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:51.852360 22706 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:51.915771 24474 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:51.813860 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.947698 24608 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:51.948819 23417 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:51.949787 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.855165 24356 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.794055 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.861505 23412 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:51.951663 24474 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.955751 24474 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.956813 24555 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.953336 23414 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:51.870935 24499 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:51.791499 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.951896 22747 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:51.678479 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.709949 24606 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.894958 22709 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 2: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:51.960752 24606 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.896555 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:51.908632 24607 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.909920 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:09:51.915045 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 78361.9us
W20260709 13:09:51.937252 22626 rpcz_store.cc:269] 0709 13:09:50.119692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.119836 (+   144us) service_pool.cc:224] Handling call
0709 13:09:51.772367 (+1652531us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540022e054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:51.772588 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304111488}
I20260709 13:09:51.963984 22704 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:51.940400 22706 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:51.839426 22629 rpcz_store.cc:276] 0709 13:09:49.563730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.563954 (+   224us) service_pool.cc:224] Handling call
0709 13:09:49.565360 (+  1406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.600846 (+ 35486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.600878 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.614708 (+ 13830us) log.cc:844] Serialized 26 byte log entry
0709 13:09:49.709351 (+ 94643us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b1800394ae0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:09:49.905963 (+196612us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:50.013530 (+107567us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.056774 (+ 43244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.056884 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.462016 (+405132us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b1800394ae0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:51.763619 (+1301603us) raft_consensus.cc:1685] finished
0709 13:09:51.763656 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.763868 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25234,"prepare.queue_time_us":15910,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":158250,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":8362,"spinlock_wait_cycles":45138432,"thread_start_us":318653,"threads_started":3,"wal-append.queue_time_us":339346,"wal-append.run_cpu_time_us":2530,"wal-append.run_wall_time_us":1010522}
I20260709 13:09:51.967471 23416 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:51.967274 22706 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.936909 24343 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.941105 24580 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:51.941314 24195 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:51.968375 24343 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:51.968521 24195 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:51.941594 24598 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:51.942786 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 244228us
I20260709 13:09:51.943426 24582 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:09:51.946103 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 196207us
I20260709 13:09:51.972934 24499 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:51.973932 22750 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.156s (SENT)
I20260709 13:09:51.940016 24603 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:51.947201 24536 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 2.109s	user 0.006s	sys 0.000s
I20260709 13:09:51.940275 24562 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:51.939687 22783 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:51.963455 24606 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:51.980937 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.982340 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:51.984126 24603 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:51.973574 24499 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:51.986183 23412 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:51.974335 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.975844 22747 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:51.988840 22747 leader_election.cc:403] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:09:51.977530 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81066.8us
I20260709 13:09:51.979230 22783 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:51.979965 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.980047 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.991299 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:51.992043 22781 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:51.980194 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:51.989089 22778 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:51.990033 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:51.996762 22702 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:09:51.997866 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.998929 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.000444 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.005775 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:51.977949 24609 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:51.975970 24590 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.005925 22783 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.012567 22783 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.008551 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.011089 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.003741 24580 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:52.007659 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.015731 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.013934 22702 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:52.018720 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.018431 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.019826 22702 raft_consensus.cc:2410] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:52.020635 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:52.021301 22778 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:52.021677 22782 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:52.022553 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.023195 22780 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:52.024066 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.024834 22708 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:52.027350 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:52.028513 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54628.2us
I20260709 13:09:52.028627 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:52.030197 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:52.032553 24325 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:52.033041 24325 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.033456 24325 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:52.036439 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.027971 24607 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.038273 22778 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:52.041237 24562 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.044909 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.045573 22778 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:52.049750 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.052155 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.054283 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.055789 22778 raft_consensus.cc:2410] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 13:09:52.043692 24603 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.071074 24603 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:52.071650 24603 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.062953 24582 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.073702 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.074923 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.075408 22710 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.075645 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.075639 22779 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:52.077353 22750 leader_election.cc:400] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:09:52.077703 22750 leader_election.cc:403] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:09:52.019131 24590 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.077627 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.079444 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.053023 24339 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.081137 22708 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:52.038574 24607 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:52.103359 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:52.104161 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.105644 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.106897 22710 raft_consensus.cc:2410] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:52.108124 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.108834 22710 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 1.
I20260709 13:09:52.111227 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.111860 24607 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.111994 22708 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.112057 24325 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.114384 24590 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.079316 24582 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.115165 24325 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.115461 24582 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.126820 24582 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.133206 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.134195 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.134774 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.135516 22782 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:52.135861 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.136446 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.140460 24343 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.142025 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.143469 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.134802 24499 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.144727 22783 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:52.151100 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.152760 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.153535 22779 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:52.154632 22701 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:52.155658 24474 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:52.164309 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.164565 24195 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.166956 24195 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.163480 24603 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.173331 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:52.175722 24603 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:52.177980 24564 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.830s	user 0.000s	sys 0.029s
W20260709 13:09:52.178952 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 101738us
I20260709 13:09:52.183388 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.190233 22782 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:09:52.194281 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.196776 22779 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:52.202656 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.203403 22703 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:52.203864 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:52.205353 24325 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:52.205780 24325 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:52.206176 24325 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.212939 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.216367 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.217999 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.221355 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:52.234748 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.233143 22703 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:09:52.236405 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:52.222604 24339 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.222802 24590 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.223165 24603 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:52.232318 24580 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:52.241117 24580 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.241271 24607 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:52.241611 24580 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:52.243700 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:52.244704 24607 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.246061 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.246955 24607 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.247387 24607 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.247718 24607 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.239727 24590 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.251441 24590 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.257674 24474 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:09:52.259590 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 1671) took 2065 ms (client timeout 1919 ms). Trace:
I20260709 13:09:52.259979 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:52.260872 24343 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:52.219360 24583 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:09:52.239562 24603 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.286041 23412 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:09:52.275141 22783 rpcz_store.cc:269] 0709 13:09:50.193987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.194155 (+   168us) service_pool.cc:224] Handling call
0709 13:09:52.259342 (+2065187us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b54003250d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:52.259472 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3945336576}
I20260709 13:09:52.277840 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.276088 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.278352 24609 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:09:52.288522 22703 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.289937 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.284862 24607 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.320914 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1741) took 2200 ms. Trace:
I20260709 13:09:52.321509 22627 rpcz_store.cc:276] 0709 13:09:50.120102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.121449 (+  1347us) service_pool.cc:224] Handling call
0709 13:09:50.155809 (+ 34360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:50.315426 (+159617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.320551 (+2005125us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54003d2f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:52.320830 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331972352}
I20260709 13:09:52.322685 24343 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:52.287657 24603 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:52.294399 24609 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:52.293625 24479 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.323206 24343 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.322098 24582 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.323195 24580 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.327463 24609 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:52.329074 24479 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.333979 24343 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:52.359817 24609 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.361311 24609 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.338560 24479 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.369706 23412 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:09:52.370640 24580 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.374887 24585 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.442744 24590 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.435870 24325 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.444921 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 191) took 2330 ms. Trace:
W20260709 13:09:52.448302 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.450083 23418 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:52.448344 22707 rpcz_store.cc:276] 0709 13:09:50.114124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.142184 (+ 28060us) service_pool.cc:224] Handling call
0709 13:09:50.142523 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:50.145429 (+  2906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.145466 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.146073 (+   607us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.272988 (+126915us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:50.290093 (+ 17105us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.290353 (+   260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.290436 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.583973 (+1293537us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b18003145c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:52.177564 (+593591us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b24000fc5a8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:09:52.444614 (+267050us) raft_consensus.cc:1685] finished
0709 13:09:52.444636 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:52.444806 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20524,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":7086,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":284,"spinlock_wait_cycles":47774848,"thread_start_us":126457,"threads_started":1,"wal-append.queue_time_us":201259}
I20260709 13:09:52.456758 24603 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.447710 24585 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:52.470184 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:52.477077 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 143642us
I20260709 13:09:52.475414 24603 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.478235 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:52.472923 24585 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.482053 24585 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:52.486086 22674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.162s	user 0.008s	sys 0.058s
I20260709 13:09:52.496598 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:52.498051 22674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.174s	user 0.009s	sys 0.059s
I20260709 13:09:52.508015 24626 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.509152 24626 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.509757 24626 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.510708 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.511693 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 180) took 1107 ms. Trace:
I20260709 13:09:52.512228 22776 rpcz_store.cc:276] 0709 13:09:51.404630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.407559 (+  2929us) service_pool.cc:224] Handling call
0709 13:09:52.511462 (+1103903us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b5400186fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:52.511613 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138686720}
I20260709 13:09:52.491108 24607 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.519336 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.523388 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.520465 24627 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:52.513819 24590 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:52.549759 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.550952 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:52.537014 22706 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:52.565828 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.566897 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.564184 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.568794 22701 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:52.569823 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:52.571481 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:52.574035 24405 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.574656 24405 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.539434 24627 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.575592 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.576920 24351 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:52.537959 24603 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.577742 24603 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.577767 24432 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:52.578336 24603 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.578549 24432 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:52.579727 24432 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:52.580865 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.577025 24405 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.529700 24583 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:52.570492 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:52.581722 24583 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:52.577564 24351 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.582895 24607 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.582793 24583 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.583470 24607 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.584654 24351 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.585726 24607 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.546061 24587 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:09:52.589236 24587 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:52.589694 24587 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:52.591044 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.591678 24170 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:52.591827 22627 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.592062 24351 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:52.592339 24170 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.584697 24583 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:52.529213 24343 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.595535 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.596236 22626 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.596694 24343 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.597307 24450 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.597589 24325 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.546955 24582 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.597856 24450 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.594835 22629 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:09:52.598789 24449 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:52.600698 24449 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.601009 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.601613 22626 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.601877 24463 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:52.602523 24463 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.603030 24449 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:52.603209 22705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 190) took 2489 ms (client timeout 1698 ms). Trace:
W20260709 13:09:52.603675 22705 rpcz_store.cc:269] 0709 13:09:50.113923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.134212 (+ 20289us) service_pool.cc:224] Handling call
0709 13:09:52.602953 (+2468741us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001ea354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:52.603116 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347315072}
I20260709 13:09:52.605420 24463 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:52.606881 22710 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:52.609596 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.610085 24572 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.614055 22633 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
W20260709 13:09:52.610962 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.609660 24450 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:52.606906 22627 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:52.617820 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.539805 24585 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:52.618605 24585 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.618769 22626 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:52.619133 24585 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:52.621201 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:52.622206 22731 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7d80 after lost signal to thread 21529
I20260709 13:09:52.623109 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:52.623945 22731 debug-util.cc:398] Leaking SignalData structure 0x7b08000be3a0 after lost signal to thread 22567
W20260709 13:09:52.626240 22731 debug-util.cc:398] Leaking SignalData structure 0x7b08003535e0 after lost signal to thread 22659
W20260709 13:09:52.627795 22731 debug-util.cc:398] Leaking SignalData structure 0x7b080011fa80 after lost signal to thread 22734
W20260709 13:09:52.628311 22731 debug-util.cc:398] Leaking SignalData structure 0x7b08001df200 after lost signal to thread 22735
I20260709 13:09:52.628908 22701 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.633227 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.635725 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.637315 24490 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:52.637789 24490 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.638286 24490 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.640050 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.640887 22633 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.644754 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.638293 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.575682 24627 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:52.647689 22633 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:52.646685 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.677187 22626 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.651594 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.678676 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.652899 24603 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.657362 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.687093 22708 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.592690 24351 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:52.657006 24490 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.660036 24587 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.709733 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.660573 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.663081 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.664461 22706 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:09:52.646512 24580 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:09:52.634555 22731 debug-util.cc:398] Leaking SignalData structure 0x7b08002b9f60 after lost signal to thread 22808
I20260709 13:09:52.684437 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.693194 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.711123 22706 pending_rounds.cc:85] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:09:52.740110 22706 pending_rounds.cc:107] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:09:52.740486 22706 raft_consensus.cc:2889] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:52.741451 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1673) took 1089 ms. Trace:
I20260709 13:09:52.527606 24629 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.710840 24582 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.714123 24603 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.691763 24195 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:52.718858 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 801851us
I20260709 13:09:52.746229 22705 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:52.719144 24559 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:09:52.720211 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 752204us
I20260709 13:09:52.720479 24562 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:52.721503 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.725553 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 141) took 5733 ms. Trace:
I20260709 13:09:52.732419 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.741814 22706 rpcz_store.cc:276] 0709 13:09:51.651689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.846546 (+194857us) service_pool.cc:224] Handling call
0709 13:09:51.852228 (+  5682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:52.443707 (+591479us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b180012d440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806f18646 00007f1806e5258e 00007f1806ed3f9e 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4
0709 13:09:52.536729 (+ 93022us) spinlock_profiling.cc:243] Waited 91.7 ms on lock 0x7b58006509d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4
0709 13:09:52.741066 (+204337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.741367 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300340352}
I20260709 13:09:52.714260 24490 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.691339 24590 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:52.753217 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.756283 22706 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:52.765631 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 214798us
I20260709 13:09:52.728964 24580 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.769124 22626 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.749979 24170 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.780618 22633 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:09:52.749912 24195 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.778496 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.796855 22708 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.804121 22706 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:52.799201 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.805783 22708 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.776396 24580 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:52.754277 24582 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:52.808284 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 206932us
I20260709 13:09:52.746922 24629 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.784468 22703 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:52.807214 22731 debug-util.cc:398] Leaking SignalData structure 0x7b080032d980 after lost signal to thread 23445
I20260709 13:09:52.804576 24195 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.823032 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.810539 24587 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.825309 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:52.826362 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.827430 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.844385 22631 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.828461 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.829181 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.830593 24603 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:52.828524 24490 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:09:52.833482 22708 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.820209 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:52.835583 22731 debug-util.cc:398] Leaking SignalData structure 0x7b080033faa0 after lost signal to thread 23446
I20260709 13:09:52.833530 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.824160 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:52.850685 24604 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:52.858319 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:52.867828 24343 raft_consensus.cc:2764] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:09:52.841800 24479 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:52.879637 24626 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.879842 22707 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:52.880988 24479 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:52.882000 24582 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:52.864203 24604 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.883087 24604 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.893503 24608 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:52.900871 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 166352us
I20260709 13:09:52.836154 24629 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:52.856281 24474 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:09:52.750952 22530 rpcz_store.cc:276] 0709 13:09:46.991704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:46.991985 (+   281us) service_pool.cc:224] Handling call
0709 13:09:52.725436 (+5733451us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:49.578343 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:49.765437 (+187094us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:49.772261 (+  6824us) write_op.cc:435] Acquired schema lock
0709 13:09:49.812193 (+ 39932us) tablet.cc:662] Decoding operations
0709 13:09:49.839676 (+ 27483us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:49.839807 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:49.839839 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:09:49.840579 (+   740us) tablet.cc:701] Row locks acquired
0709 13:09:49.840628 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:09:49.840839 (+   211us) write_op.cc:270] Start()
0709 13:09:49.841019 (+   180us) write_op.cc:276] Timestamp: P: 1783602589840773 usec, L: 0
0709 13:09:49.841056 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:09:49.841951 (+   895us) log.cc:844] Serialized 2050 byte log entry
0709 13:09:50.325075 (+483124us) op_driver.cc:464] REPLICATION: finished
0709 13:09:50.492732 (+167657us) write_op.cc:301] APPLY: starting
0709 13:09:50.495829 (+  3097us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:50.496103 (+   274us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:50.614703 (+118600us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:50.649021 (+ 34318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:50.653209 (+  4188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:09:50.653301 (+    92us) write_op.cc:312] APPLY: finished
0709 13:09:50.653715 (+   414us) log.cc:844] Serialized 47 byte log entry
0709 13:09:50.653961 (+   246us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:50.654089 (+   128us) write_op.cc:454] Released schema lock
0709 13:09:50.654160 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:52.384792 (+1730632us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b18001ea0c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:52.433555 (+ 48763us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:52.434741 (+  1186us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1028429,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":157694,"apply.run_cpu_time_us":5955,"apply.run_wall_time_us":165867,"num_ops":5,"prepare.queue_time_us":126284,"prepare.run_cpu_time_us":24816,"prepare.run_wall_time_us":357985,"raft.queue_time_us":183113,"raft.run_cpu_time_us":21958,"raft.run_wall_time_us":110541,"replication_time_us":483814,"spinlock_wait_cycles":4387328,"thread_start_us":445289,"threads_started":4,"wal-append.queue_time_us":101277,"wal-append.run_cpu_time_us":71456,"wal-append.run_wall_time_us":2532867}]]}
W20260709 13:09:52.914376 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.904945 24474 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:52.841220 24351 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.930255 22631 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:52.908470 24604 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:52.937160 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 162441us
I20260709 13:09:52.929303 24474 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:52.945977 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.953917 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:52.948200 24356 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:52.954973 24356 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.957414 24356 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:52.958210 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 211642us
W20260709 13:09:52.958789 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 207537us
I20260709 13:09:52.964349 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:52.966931 24562 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:52.967530 24562 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:52.967737 24170 raft_consensus.cc:1064] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:09:52.968750 24562 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:52.969384 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.970148 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:52.970302 22778 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:52.966343 24539 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 13:09:52.965945 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.971650 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:52.972216 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.972985 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.978121 22775 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:52.947983 24558 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:52.972586 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.979259 24558 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:52.960254 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.972487 22635 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:52.960886 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:52.982535 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:52.982523 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:52.974223 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:53.007613 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 157823us
W20260709 13:09:53.008431 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 159468us
W20260709 13:09:53.012920 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51692.6us
I20260709 13:09:53.013623 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:53.014406 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:53.038105 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.014418 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:53.039840 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:53.041489 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:53.041992 22635 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
W20260709 13:09:53.043973 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.044625 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.045013 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.045466 22632 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:09:53.046124 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.047331 22633 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:53.049162 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:53.049626 22777 pending_rounds.cc:85] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 1
I20260709 13:09:53.050004 22777 pending_rounds.cc:107] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:09:53.050529 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:53.051362 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:53.052129 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:53.045778 22702 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:09:53.054692 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.057477 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:53.057632 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.058413 22631 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:53.059365 22779 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:53.050271 22777 raft_consensus.cc:2889] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:53.012115 22600 connection.cc:256] RPC call timeout handler was delayed by -1.0763891219999095s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:09:53.063009 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:52.976035 24170 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.067189 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.063503 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:53.068430 22775 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:53.063854 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.069973 22707 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:53.074484 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.981886 24464 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:09:53.111840 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:53.081400 22704 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:53.085523 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:53.086441 22600 connection.cc:256] RPC call timeout handler was delayed by -1.0761390769999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1869s for a response
I20260709 13:09:53.087188 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:53.113353 22631 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:53.088007 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:53.090327 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:53.052572 24356 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:53.093220 22709 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:09:53.095661 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.117923 22628 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:53.096843 22630 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:09:53.098043 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.119252 22634 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:09:53.099643 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.120781 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.121677 22707 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:53.057231 24539 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:09:53.132861 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 91634.6us
I20260709 13:09:53.131274 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.134418 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.107327 24580 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:53.111053 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.114781 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.146636 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.147742 22703 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:53.113210 22600 connection.cc:256] RPC call timeout handler was delayed by -1.0320549369999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19190000000000002s for a response
I20260709 13:09:53.090983 22708 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 5. Ops: []
I20260709 13:09:53.130126 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:53.122576 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:53.136619 22749 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:09:53.136763 22628 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:53.131883 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.151154 22705 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:53.155001 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.100283 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.130264 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:53.141580 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.157395 22633 raft_consensus.cc:2410] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:09:53.129824 22533 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 4 to 5, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:53.163676 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.062214 24582 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.075s
W20260709 13:09:53.167644 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45153.8us
I20260709 13:09:53.112794 24490 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:53.115633 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:53.170866 22632 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:09:53.169070 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:53.173763 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.131368 22627 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:53.174775 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.170511 22703 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
W20260709 13:09:53.187026 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56969.6us
I20260709 13:09:53.134438 24449 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:53.202865 24558 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:53.203011 24449 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.204196 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.205288 24582 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:53.206185 24449 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:53.189589 22632 pending_rounds.cc:85] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 5
I20260709 13:09:53.216138 22632 pending_rounds.cc:107] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:09:53.209969 24629 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.209338 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:53.189051 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:53.229562 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60565.1us
I20260709 13:09:53.175369 22533 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index 3 to 5, term changed from 3 to 4, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to <none>, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:53.177870 24557 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:53.196267 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.199571 24641 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:53.216605 22632 raft_consensus.cc:2889] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:53.230566 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:53.153911 24603 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:09:53.181639 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.245232 24629 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:53.251231 24604 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:53.252743 24586 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:53.253453 24586 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:53.162211 24356 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:53.242081 22630 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:53.231124 22533 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:09:53.286940 22630 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:53.257485 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.263804 22706 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:53.268973 22709 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:09:53.288046 22533 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.206401 24582 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:09:53.278265 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 226173us
I20260709 13:09:53.289995 22533 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:53.179312 24640 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:53.215852 22776 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:53.308525 22533 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 4 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:53.312989 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.233013 24479 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.233634 24405 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:53.178009 24539 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:09:53.234171 24557 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.329762 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:53.323558 22533 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:53.351081 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.359278 22533 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:53.306113 24582 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:53.370272 22533 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:09:53.376686 22630 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.310086 24556 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:09:53.384495 22533 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 4, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:53.355186 24405 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.370508 24557 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:53.390991 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:53.403025 22709 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:53.403406 22709 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:53.331019 24555 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:09:53.404403 22709 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.405570 24583 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:53.406047 22709 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:53.407451 24582 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.395573 24586 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:53.410791 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.402434 24405 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:53.415520 24586 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:53.392477 22630 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:53.297070 24365 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:09:53.395526 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.420545 22705 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:53.393041 22533 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 4, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:53.322546 24644 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:53.307108 24195 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.379107 24640 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:53.372846 24474 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.368569 24558 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:09:53.389127 24432 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:53.391235 24539 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:09:53.428610 24555 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:09:53.350723 24479 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.432910 22533 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:53.436316 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.437107 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 288677us
W20260709 13:09:53.475838 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.489405 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48256.8us
W20260709 13:09:53.502590 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.500831 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.504428 22533 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:09:53.501816 22778 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:53.497056 24365 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:09:53.514595 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.515789 22633 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:53.519475 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.520665 22634 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:53.527652 22632 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:09:53.511137 24432 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.521230 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.522061 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.534015 22631 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:53.502105 24555 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:53.529618 24474 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.534458 24405 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.539630 24555 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.551141 22628 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:53.554658 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.555959 22628 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:53.557407 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:53.561375 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.570777 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 340297us
I20260709 13:09:53.506933 24490 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:53.570945 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.576526 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.577422 22628 raft_consensus.cc:2376] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4. Re-sending same reply.
I20260709 13:09:53.486828 24356 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:53.572593 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 287060us
I20260709 13:09:53.572589 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.581835 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.582566 22630 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:53.565587 24580 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.570001 24603 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:09:53.672338 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.677242 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.734270 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.737391 24432 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:53.717959 24585 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.098s	user 0.013s	sys 0.007s
I20260709 13:09:53.711292 22709 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:53.712462 24365 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:09:53.715706 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.718354 22629 raft_consensus.cc:1217] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Deduplicated request from leader. Original: 2.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 13:09:53.744184 22629 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:09:53.721462 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.694375 24641 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:53.626814 24640 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.745301 22627 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:09:53.570585 24474 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:53.571630 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:09:53.689299 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 399263us
W20260709 13:09:53.750458 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 248599us
I20260709 13:09:53.773842 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.725917 24629 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.778151 22630 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
W20260709 13:09:53.779026 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:53.783810 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 212263us
I20260709 13:09:53.787671 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.788628 22635 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:53.741832 24645 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:09:53.745671 24365 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:53.744845 24627 consensus_meta.cc:287] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.097s	user 0.008s	sys 0.012s
I20260709 13:09:53.850421 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.864138 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.857442 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:53.794656 24648 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:09:53.819118 24587 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.972s	user 0.008s	sys 0.005s
I20260709 13:09:53.795244 24580 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:53.789710 24405 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.752507 24585 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:53.892936 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 170011us
I20260709 13:09:53.933313 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.819000 24653 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:53.948099 24580 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:53.948782 24580 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:53.948614 24653 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.950215 24656 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:53.950896 24656 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:53.951321 24653 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.724592 24490 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.791497 24629 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.867242 24604 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:53.942212 22627 raft_consensus.cc:2410] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:09:53.957093 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:53.921767 24195 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.777635 24474 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.943495 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:53.936091 24479 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:53.991693 24658 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:54.006604 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.006657 22780 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:54.007920 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.010887 22630 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:53.884481 24627 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:53.961782 24551 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:09:54.012228 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.013684 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.013895 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.020555 22627 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.025615 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.026408 23420 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:09:54.027631 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:53.939644 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 153131us
I20260709 13:09:53.957093 24604 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.048897 24604 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:54.049501 24583 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.643s	user 0.008s	sys 0.003s
I20260709 13:09:54.049749 24268 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:54.049989 24583 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:54.050226 24268 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:54.033478 22776 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.683s	user 0.012s	sys 0.001s
I20260709 13:09:54.034508 24585 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:53.964969 24365 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.036048 24474 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:54.038156 24629 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.038971 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.046030 24558 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:54.048995 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 1692) took 1075 ms. Trace:
I20260709 13:09:54.052204 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.053022 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.011286 24587 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:54.031984 22701 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:54.056497 24559 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:54.056875 24463 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:54.061479 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1696) took 1048 ms. Trace:
I20260709 13:09:54.107620 22780 rpcz_store.cc:276] 0709 13:09:53.012637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.012790 (+   153us) service_pool.cc:224] Handling call
0709 13:09:53.067214 (+ 54424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:54.012245 (+945031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.061200 (+ 48955us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b5400341cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:54.061407 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159887232}
W20260709 13:09:54.064049 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 328964us
I20260709 13:09:54.071105 24638 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.072494 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.073571 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:54.076918 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.086110 22627 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:54.114630 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.115155 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.115442 22634 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:53.940196 24603 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:54.075232 24663 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:09:54.096736 22778 rpcz_store.cc:276] 0709 13:09:52.973347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.973871 (+   524us) service_pool.cc:224] Handling call
0709 13:09:54.048824 (+1074953us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54000cc454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:54.048930 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183684992}
I20260709 13:09:54.030388 24655 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.111835 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.182225 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.012533 24658 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.116997 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.183840 22707 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:54.183339 22627 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:54.096310 24558 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.118978 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:09:54.192163 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.189239 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.192320 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.187903 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:54.195762 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:54.198127 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 444340us
W20260709 13:09:54.032284 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.193835 24629 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:54.179886 24662 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:09:54.203316 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.204524 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:54.205569 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.107038 24559 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.085047 22776 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:09:54.190326 24644 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.761s	user 0.003s	sys 0.011s
W20260709 13:09:54.218413 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:54.208348 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 200149us
I20260709 13:09:54.211287 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.239796 22627 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:54.208045 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.018714 24641 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.229048 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:54.196714 24555 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:09:54.245302 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.252143 22707 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3.
I20260709 13:09:54.248063 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:54.202651 24590 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.259377 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.259752 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.253500 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:54.260502 22631 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:54.262140 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:54.262462 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.263521 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:54.107402 24463 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.112345 24587 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:54.265875 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 59389.3us
I20260709 13:09:54.277136 22631 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:54.277630 22631 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.278055 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.115238 24268 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.277792 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:54.267256 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.289017 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.290059 23421 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:54.290694 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.293422 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:54.295310 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.248322 24585 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.248054 24644 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:54.126725 24638 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.278501 22631 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.299965 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.300921 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.248090 24673 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:54.307883 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.252758 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 1685) took 1289 ms. Trace:
W20260709 13:09:54.309337 22635 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.520s	user 0.012s	sys 0.001s
I20260709 13:09:54.313313 22635 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:54.312589 23418 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:54.193295 24655 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.316054 23421 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 5. Ops: [3.5-3.5]
I20260709 13:09:54.265663 24539 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:54.318771 22750 leader_election.cc:400] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3.
I20260709 13:09:54.319451 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.324800 22750 leader_election.cc:403] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:09:54.320333 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.331430 22707 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:09:54.331748 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53817.9us
I20260709 13:09:54.275470 24450 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:54.232719 24629 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.322186 22710 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 5. Ops: [3.5-3.5]
I20260709 13:09:54.347805 22630 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:54.326257 22631 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:54.321255 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.248919 24580 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.300606 24479 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:54.293651 24676 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:54.336216 23415 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:54.342487 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.286541 24590 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.345405 23421 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:09:54.310827 24668 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.356114 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.356941 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.285521 24666 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:09:54.328120 24268 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:54.312114 24585 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:54.350256 24586 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.934s	user 0.014s	sys 0.002s
I20260709 13:09:54.367214 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.285293 24558 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.315207 22776 rpcz_store.cc:276] 0709 13:09:52.967216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.010962 (+ 43746us) service_pool.cc:224] Handling call
0709 13:09:54.248681 (+1237719us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:54.418386 23416 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:54.335000 24664 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.367811 24450 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.379174 22778 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5.
I20260709 13:09:54.353283 24677 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.359575 24479 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:54.417780 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 80479.2us
I20260709 13:09:54.350836 24679 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:54.353325 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.358201 23411 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:54.449611 24479 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:54.426179 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 125398us
I20260709 13:09:54.427537 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.456353 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.429647 22779 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:54.367183 24673 raft_consensus.cc:2764] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:09:54.431867 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.310434 24559 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.360790 24682 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.440215 22631 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:09:54.363022 24678 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.443611 24670 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.444824 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.445614 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.450178 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:54.451124 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:54.507632 23417 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:54.508001 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51760.5us
I20260709 13:09:54.508961 24670 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.509694 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.511195 24666 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.511847 24670 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.512393 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.512835 22781 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:09:54.513017 24639 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.284s	user 0.008s	sys 0.009s
I20260709 13:09:54.513923 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.430526 24651 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:09:54.515738 22630 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:54.516312 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.516959 22783 raft_consensus.cc:2376] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4. Re-sending same reply.
I20260709 13:09:54.517974 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.519160 22781 raft_consensus.cc:2410] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:54.520001 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.520102 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.520736 22781 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:54.467070 24672 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:09:54.538100 23420 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:09:54.538821 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.550882 22781 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:54.546294 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.556713 24590 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.557271 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.516852 24678 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.485715 24572 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.558801 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.567648 22782 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:54.569010 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.569708 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:09:54.571103 22780 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:09:54.560081 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:54.571995 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 121005us
I20260709 13:09:54.543220 22775 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:09:54.543557 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79049.2us
I20260709 13:09:54.544086 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.574095 22632 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:54.572160 22630 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:09:54.574316 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.577582 22707 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:54.580742 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.581738 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:54.583076 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.584406 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.594048 22781 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.585309 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.590042 24539 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.447649 24664 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.596923 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.597534 24539 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.598747 24664 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.604290 24638 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.605719 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.511878 24666 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.611004 24580 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.616222 22775 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:09:54.594525 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.620020 24585 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.580839 24675 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:09:54.559785 24678 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.621282 24627 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.596307 24604 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.622223 24604 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:54.623353 22701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1704) took 2051 ms (client timeout 2112 ms). Trace:
W20260709 13:09:54.623808 22701 rpcz_store.cc:269] 0709 13:09:52.571692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.637846 (+ 66154us) service_pool.cc:224] Handling call
0709 13:09:54.623091 (+1985245us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540010cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:54.623260 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3183634688}
I20260709 13:09:54.624488 24604 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.625108 22708 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:54.464216 24688 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.162s
I20260709 13:09:54.617190 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.580020 24676 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:54.628172 24582 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:54.627986 24676 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.515316 24490 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:54.631050 24490 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.586172 24670 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.640326 22630 raft_consensus.cc:2410] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:54.475649 24558 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.591511 22710 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:09:54.559531 24673 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.644814 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:54.621613 24587 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.645226 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:54.646201 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.646165 23415 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:54.650961 24658 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.651230 24365 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.475344 24682 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.657639 24682 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.559314 24450 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.660935 24629 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.662724 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.663271 22710 raft_consensus.cc:606] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:54.569512 24559 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.663825 22710 raft_consensus.cc:614] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:54.665678 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:54.587618 24671 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:54.667847 24671 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.668347 24671 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:54.676260 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1628) took 1585 ms. Trace:
I20260709 13:09:54.677084 22708 rpcz_store.cc:276] 0709 13:09:53.090679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.148821 (+ 58142us) service_pool.cc:224] Handling call
0709 13:09:54.624862 (+1476041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:54.625649 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.675926 (+ 50277us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540010cf50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:54.676162 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3393870080}
W20260709 13:09:54.691773 22704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.001s	sys 0.003s
I20260709 13:09:54.541723 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.591285 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.634241 24490 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:54.692401 22704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.002s	sys 0.003s
I20260709 13:09:54.707028 22778 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:54.708464 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:54.714808 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.718097 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.722806 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.726676 22775 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:09:54.709252 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113386us
I20260709 13:09:54.725281 22709 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:54.733160 22626 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:09:54.720418 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:54.738126 22628 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:09:54.740600 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 257452us
W20260709 13:09:54.711169 24557 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.285s	user 0.011s	sys 0.004s
I20260709 13:09:54.754863 22783 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:54.763865 22675 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.772s (SENT)
I20260709 13:09:54.769419 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:54.771001 22780 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:54.772589 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.753983 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.764775 23417 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:54.785059 22780 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:54.685745 24655 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:54.787344 22776 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:54.796093 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:54.801513 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58274.2us
W20260709 13:09:54.802438 22675 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.772s (SENT)
W20260709 13:09:54.803933 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:54.807950 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 339529us
I20260709 13:09:54.767352 24590 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.810511 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.810740 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.811038 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:54.818804 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.819886 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.805760 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:54.822783 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.825003 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:54.826598 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.829624 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.830298 22780 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.831439 22784 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:09:54.833050 22784 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:09:54.834851 22675 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.796s (SENT)
I20260709 13:09:54.835533 22775 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:54.835500 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.837162 22775 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:54.838928 22784 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:54.840700 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.840818 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.841722 22777 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:54.806459 24678 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.843530 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:54.849434 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.793982 24557 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.816669 24627 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:54.845909 24479 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.823951 24649 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.849323 24405 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:09:54.858918 24449 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:54.861836 24683 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.443s	user 0.015s	sys 0.002s
W20260709 13:09:54.862150 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 270992us
W20260709 13:09:54.863778 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58117.6us
I20260709 13:09:54.843003 24587 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:54.826048 24676 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:54.857090 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.849020 24629 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.841056 24658 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:54.825484 24656 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.946595 24449 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.946998 24691 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:54.948196 24691 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:54.948040 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.948648 24691 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:54.949170 24449 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.949693 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.949550 24691 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:54.951191 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:54.926735 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44762.2us
I20260709 13:09:54.929113 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:54.952229 22782 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:54.928543 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1495) took 1072 ms. Trace:
I20260709 13:09:54.846912 24687 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.857306 22780 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:54.951176 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.864900 24572 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:54.958370 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.962728 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.963724 22632 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3.
W20260709 13:09:54.934279 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98422.7us
I20260709 13:09:54.942755 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.965315 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:54.965688 22675 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.831s (SENT)
I20260709 13:09:54.966400 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:54.967330 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.844115 24555 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.082s
I20260709 13:09:54.976029 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.952842 22628 rpcz_store.cc:276] 0709 13:09:53.856097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.857123 (+  1026us) service_pool.cc:224] Handling call
0709 13:09:54.877795 (+1020672us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b540003acd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:54.928472 (+ 50677us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2005887360}
I20260709 13:09:54.978111 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.954730 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:54.824173 24590 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.955047 22777 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:54.934093 22775 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:09:54.876792 24681 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:54.876600 24670 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:54.987237 24665 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:54.988720 24587 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.988998 24590 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.989253 24587 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.990065 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.990895 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.991297 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:54.847162 24699 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
I20260709 13:09:54.992760 24463 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:54.987293 24670 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.994859 24670 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:54.999500 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:54.932337 24706 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.000480 22779 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:55.001977 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.002593 22781 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:55.002882 22626 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:55.003808 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.004976 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:55.006649 24703 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:09:55.006897 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.007634 22628 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:55.008379 24691 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:54.955367 24629 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.013558 24557 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.015470 24479 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.020560 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1646) took 1291 ms. Trace:
I20260709 13:09:55.021051 22704 rpcz_store.cc:276] 0709 13:09:53.729560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.733450 (+  3890us) service_pool.cc:224] Handling call
0709 13:09:53.774037 (+ 40587us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:53.781729 (+  7692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:53.781764 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:53.860380 (+ 78616us) log.cc:844] Serialized 52 byte log entry
0709 13:09:54.721819 (+861439us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:54.764951 (+ 43132us) log.cc:844] Serialized 14 byte log entry
0709 13:09:54.794514 (+ 29563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.794988 (+   474us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:55.020287 (+225299us) raft_consensus.cc:1685] finished
0709 13:09:55.020310 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:55.020487 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45200,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3145,"raft.queue_time_us":44750,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":62126,"thread_start_us":779143,"threads_started":1,"wal-append.queue_time_us":953814}
I20260709 13:09:54.876262 24649 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.024158 24670 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.025321 22627 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:55.025610 24649 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:54.865149 24627 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.028031 24627 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.028824 24670 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.031337 22633 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:55.032433 24657 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.032897 24657 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.035583 24676 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.035991 24657 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.036098 24676 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.037966 24676 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:54.975886 24678 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.040956 22778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:55.041666 22778 raft_consensus.cc:606] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:55.042048 22778 raft_consensus.cc:614] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:55.044323 24555 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:55.044931 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:54.968870 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:54.932103 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 83982.8us
I20260709 13:09:54.983448 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.984280 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:54.927610 24664 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:54.985524 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:54.994293 24463 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:09:55.051266 24664 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.051860 22706 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 5. Ops: [3.5-3.5]
I20260709 13:09:54.977197 24688 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:55.052896 24688 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:55.053370 24688 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:55.053575 24664 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.054500 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:55.055301 22707 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:55.051055 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:55.059278 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73888.2us
I20260709 13:09:55.061888 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:55.065807 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 100167us
W20260709 13:09:55.066847 22675 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.854s (SENT)
W20260709 13:09:55.069484 22675 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.854s (SENT)
I20260709 13:09:55.070250 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.071367 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.075582 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.093861 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:55.094384 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 151756us
I20260709 13:09:55.095803 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.098479 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.094843 22709 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:55.104794 22675 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.875s (SENT)
I20260709 13:09:55.044970 24555 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.155527 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.156736 22703 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:55.157138 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.164270 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.108778 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:55.110376 22749 leader_election.cc:400] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3.
W20260709 13:09:55.110177 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 48420.7us
I20260709 13:09:55.114925 24474 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.115224 22629 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:09:55.095520 24691 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:09:55.106729 24657 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:09:55.122793 24558 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:55.126857 24684 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:09:55.129667 24582 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.501s	user 0.014s	sys 0.000s
I20260709 13:09:54.987834 24665 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.107200 24699 raft_consensus.cc:2764] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:09:55.105014 24638 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:09:55.137269 24671 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:55.139336 24677 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.684s	user 0.004s	sys 0.007s
I20260709 13:09:55.146605 24268 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:55.149397 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.151034 24678 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.151423 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.151805 24627 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.213922 24627 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.167528 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1498) took 1224 ms. Trace:
I20260709 13:09:55.216977 22633 rpcz_store.cc:276] 0709 13:09:53.942947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.943193 (+   246us) service_pool.cc:224] Handling call
0709 13:09:55.167247 (+1224054us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540022e554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:55.167429 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2409232256}
W20260709 13:09:55.172850 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62656.8us
I20260709 13:09:55.172014 23415 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:55.181030 22707 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:55.220419 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.181330 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.076709 24557 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:55.078838 24649 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:55.196758 22632 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:55.197604 24690 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.202401 22709 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:55.218958 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.223148 24555 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.226672 24268 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.228574 24627 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.230198 24405 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:55.230247 24634 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:55.231168 22630 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:55.232671 24665 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.241182 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.198372 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.266990 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:55.267321 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 69083.5us
I20260709 13:09:55.268069 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.248374 24479 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:55.273437 24604 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.262590 24582 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.239984 24450 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:55.276738 24450 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.241973 24677 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3.
I20260709 13:09:55.251184 24692 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:55.319375 24692 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.320386 24692 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.252556 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:09:55.323510 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 71035.2us
I20260709 13:09:55.325423 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:55.326143 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.256340 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.207787 24641 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:55.217586 24559 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.003s
I20260709 13:09:55.261153 24555 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.271401 24405 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:55.272831 24268 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:55.250365 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 154719us
W20260709 13:09:55.279078 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.277942 24557 raft_consensus.cc:2764] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:09:55.281630 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.245261 24649 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:55.282857 24707 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:55.283608 23416 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:09:55.284289 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.285210 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.280763 24170 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:55.286427 24706 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.287925 24627 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:55.360491 24641 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.206996 24365 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:09:55.362013 24649 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:55.295043 24629 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.299678 23415 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:09:55.274284 24604 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:09:55.366921 24629 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:55.367074 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.367554 24629 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:09:55.368319 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112054us
I20260709 13:09:55.368729 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.369165 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.370085 24590 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.370324 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:55.370821 24590 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.370934 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.371598 24604 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:55.372296 24604 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:55.373440 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.374279 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.308813 24687 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.376305 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.378904 23417 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:55.380235 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.381141 22627 raft_consensus.cc:2376] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3. Re-sending same reply.
I20260709 13:09:55.382954 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.263477 24690 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.360491 24707 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.379189 24268 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.407932 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:55.387872 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.408646 23415 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.409113 22635 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:55.389156 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.390848 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.409966 22627 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:09:55.373538 24604 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.412909 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.395522 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.317589 24450 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:55.402954 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.379012 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:55.417351 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44015.7us
I20260709 13:09:55.411234 22629 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:55.326975 24699 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:55.413635 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.414474 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.332058 22784 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:55.289024 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.327023 24463 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:09:55.420446 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 131525us
I20260709 13:09:55.330680 24671 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.387930 24405 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:55.429425 22628 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:55.409914 24551 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:55.414366 24559 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.446976 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.458029 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.458796 22784 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:55.460722 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.461293 22784 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:55.461970 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.461308 22778 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.419571 24557 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.424288 24590 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.464896 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.465966 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.429034 24449 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:55.426993 24706 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:55.412779 24268 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.434568 24463 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.430542 24699 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.368793 24629 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.423192 24170 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:09:55.464089 24562 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.479966 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.504263 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.518455 24690 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.433243 24558 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:09:55.519164 24706 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:09:55.568503 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88690.2us
I20260709 13:09:55.568065 22779 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.570087 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.630776 22708 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:55.513841 24716 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:09:55.581207 24629 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:55.582077 24699 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.575088 24627 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:09:55.464568 24675 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:09:55.630589 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.577612 22634 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:55.636045 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:55.519873 23412 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:55.643778 22778 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:55.658826 24665 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:55.659446 24665 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.659852 24665 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.662122 24539 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:55.662683 24539 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.662840 24677 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.663775 24539 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.664484 24268 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.481762 24670 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:55.665766 24539 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:55.638383 24714 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.153s	user 0.000s	sys 0.005s
I20260709 13:09:55.585346 24590 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:09:55.638073 24688 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.584s	user 0.017s	sys 0.000s
I20260709 13:09:55.684255 24688 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.576764 24562 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.689307 24562 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.693377 24629 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:55.527608 24405 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.669600 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.669255 24463 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.670908 24706 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:55.676450 24641 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:55.701011 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.714243 22708 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:55.701848 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.715197 22702 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:55.514915 23415 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:55.724654 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.704860 24603 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:09:55.740165 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.050s	sys 0.091s
I20260709 13:09:55.734381 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.703429 24405 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.704836 24706 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:09:55.736703 22706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.006s	sys 0.009s
W20260709 13:09:55.737834 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 312880us
I20260709 13:09:55.702845 24629 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:55.744452 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.748224 22628 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.749203 22779 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:55.750412 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:55.761885 22629 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
W20260709 13:09:55.742952 22706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.006s	sys 0.009s
I20260709 13:09:55.756394 22630 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:55.768705 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.771227 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:55.740630 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.050s	sys 0.091s
I20260709 13:09:55.742898 24603 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.777815 24603 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:55.778891 24479 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:55.779407 24479 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:55.779788 24479 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.742414 24582 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:55.736379 24590 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:09:55.782689 24629 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.763350 24657 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:09:55.784092 24629 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.786084 24688 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.722867 24665 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.731210 24644 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:09:55.761649 24676 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:55.774118 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 545264us
I20260709 13:09:55.767952 24558 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:55.795920 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.798130 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.799315 22783 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:55.802743 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.803233 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.804118 22779 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
W20260709 13:09:55.806838 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 359965us
I20260709 13:09:55.825505 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.827544 23417 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:55.811524 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:55.789822 24665 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:55.790061 24676 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:55.836797 22781 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:09:55.837517 22781 raft_consensus.cc:606] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:55.837911 22781 raft_consensus.cc:614] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:55.838263 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:55.805771 24707 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.837606 24676 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:55.840706 24675 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:55.832202 24558 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.854717 24675 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.857122 24558 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.858502 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.844933 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:55.861061 24450 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:55.870647 24717 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:09:55.872517 24707 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:55.874229 24675 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:55.878443 24644 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:55.879488 24725 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:55.786490 24706 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.845147 24665 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:55.884613 24665 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:55.885596 24665 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:55.887933 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.888468 24679 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:55.888877 24679 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.878470 22779 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:55.876658 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:55.842468 24555 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:09:55.889258 24679 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:55.834499 24432 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:09:55.876256 24678 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:09:55.891906 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.894649 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.903491 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:55.905357 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.905807 23416 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:55.906333 22781 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:55.907035 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.904330 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:55.908422 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:55.920337 22707 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:55.910459 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:55.913573 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:55.928740 22709 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:55.902866 24677 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:09:55.937197 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.944661 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.907529 24671 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.927621 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:55.950387 22628 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:55.908260 24675 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.912473 24727 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:55.912919 24678 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:09:55.966344 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.961524 24449 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:55.927412 24558 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:55.968951 23417 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
W20260709 13:09:55.956984 24587 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.967s	user 0.019s	sys 0.003s
I20260709 13:09:55.988190 22600 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:55.962066 24432 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:55.957682 24170 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:55.962682 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:55.943171 24551 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:56.000361 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:56.009415 24559 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:56.009882 24726 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.986039 24449 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.990232 24558 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.961661 24539 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:09:55.989480 24690 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:55.964022 24671 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.996058 24432 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:56.004997 24170 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.002199 24557 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:56.034024 24678 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:55.970964 24675 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.022590 24449 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.031309 24558 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:55.987273 24727 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.001843 24587 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.052997 22783 raft_consensus.cc:2368] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3.
I20260709 13:09:56.053413 24587 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.039193 22776 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:56.057760 22707 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:56.011021 24726 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:09:56.068881 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:56.069682 23421 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:09:56.072160 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 301025us
I20260709 13:09:56.039748 24170 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.073269 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:56.062949 24649 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.700s	user 0.013s	sys 0.002s
W20260709 13:09:56.079736 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.079341 24604 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:56.096939 24676 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.099694 24671 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.085425 24688 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:09:56.085505 24603 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.103533 24603 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.104331 22778 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:56.104300 24676 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.107700 24675 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.107908 24679 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.109880 24727 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.082425 24649 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.112150 23420 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:56.113155 24649 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.114782 24432 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.114599 24558 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.115346 22530 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:09:56.116132 24432 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.117776 24555 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:56.118225 24555 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:56.125342 24558 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.130636 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.132011 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.084118 24582 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:56.134339 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:56.135157 22776 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:56.136291 24562 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:56.133579 24582 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.136758 24681 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:09:56.136909 24562 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:56.137652 24681 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:56.138352 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:56.139221 22777 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:56.143646 24728 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.103s	user 0.003s	sys 0.000s
I20260709 13:09:56.087690 24678 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.149153 24706 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.151955 22709 pending_rounds.cc:85] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:09:56.152377 22709 pending_rounds.cc:107] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:09:56.152634 22709 raft_consensus.cc:2889] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:56.154467 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:56.155299 23417 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:56.156096 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:56.156442 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:56.156791 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:56.157034 24603 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:56.156822 22706 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:56.157938 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:56.158483 24587 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:56.158924 24587 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:56.159838 24587 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.160873 24587 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.116499 24679 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:56.085608 24551 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.159860 22706 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:09:56.165187 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:56.100297 24479 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.167106 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.168033 22530 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 4, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:56.179082 22530 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:56.183035 22530 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.169714 22533 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 246) took 4560 ms. Trace:
I20260709 13:09:56.174204 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:56.238466 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64351.7us
I20260709 13:09:56.168895 23421 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:56.194239 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 358259us
W20260709 13:09:56.195278 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 361278us
I20260709 13:09:56.138046 24562 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.187875 24365 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:56.170190 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:56.232913 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.176417 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:56.269518 22707 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:09:56.270056 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 93765.9us
I20260709 13:09:56.270897 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:56.272706 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.279353 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:56.288024 23420 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:09:56.291909 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44959.8us
I20260709 13:09:56.076247 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:56.200075 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:56.296155 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:56.235849 22533 rpcz_store.cc:276] 0709 13:09:51.609112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.609459 (+   347us) service_pool.cc:224] Handling call
0709 13:09:56.169643 (+4560184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:54.035821 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:54.036412 (+   591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:54.036452 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:09:54.036473 (+    21us) tablet.cc:662] Decoding operations
0709 13:09:54.314753 (+278280us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:54.328991 (+ 14238us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:54.336676 (+  7685us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:09:54.356296 (+ 19620us) tablet.cc:701] Row locks acquired
0709 13:09:54.356320 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:09:54.356514 (+   194us) write_op.cc:270] Start()
0709 13:09:54.356642 (+   128us) write_op.cc:276] Timestamp: P: 1783602594356466 usec, L: 0
0709 13:09:54.356666 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:09:54.357457 (+   791us) log.cc:844] Serialized 5220 byte log entry
0709 13:09:54.851983 (+494526us) op_driver.cc:464] REPLICATION: finished
0709 13:09:55.213586 (+361603us) write_op.cc:301] APPLY: starting
0709 13:09:55.213722 (+   136us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:55.214201 (+   479us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:55.214401 (+   200us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:55.374603 (+160202us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:55.374980 (+   377us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:09:55.375029 (+    49us) write_op.cc:312] APPLY: finished
0709 13:09:55.375416 (+   387us) log.cc:844] Serialized 95 byte log entry
0709 13:09:55.376458 (+  1042us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:55.376639 (+   181us) write_op.cc:454] Released schema lock
0709 13:09:55.376701 (+    62us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:55.385540 (+  8839us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:55.385577 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1117398,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":349438,"apply.run_cpu_time_us":4992,"apply.run_wall_time_us":171951,"num_ops":13,"prepare.queue_time_us":303592,"prepare.run_cpu_time_us":17233,"prepare.run_wall_time_us":416868,"raft.queue_time_us":293033,"raft.run_cpu_time_us":16969,"raft.run_wall_time_us":385968,"replication_time_us":485897,"thread_start_us":677792,"threads_started":4,"wal-append.queue_time_us":151775}]]}
I20260709 13:09:56.156075 24555 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:56.308293 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 232114us
I20260709 13:09:56.309842 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.238590 24450 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:09:56.312812 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 43666.7us
I20260709 13:09:56.313161 22783 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:56.313635 22783 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:56.268769 24679 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.314656 22783 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.302397 24688 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:09:56.338251 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.268882 24365 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:56.303390 24557 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:56.303161 24727 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:56.342093 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:56.346402 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:56.318928 24725 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:56.311805 24707 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:09:56.350135 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:56.358047 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:56.358685 22633 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.314112 24450 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.347492 22783 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.352631 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.299546 22530 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 3, leader changed from <none> to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:56.310699 24736 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:56.318383 24627 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:56.317909 24735 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:09:56.363889 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51241.8us
W20260709 13:09:56.318512 24692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.020s	sys 0.026s
I20260709 13:09:56.367414 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.373703 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:56.336810 24688 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.363234 22530 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:56.371093 22635 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:09:56.348676 24365 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.311167 24734 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:09:56.407287 24734 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:56.407691 24734 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:56.408794 24734 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.410131 24734 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.411051 24555 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:56.378201 24742 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:56.379411 24604 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:56.379805 24479 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.379891 24590 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:56.439365 22530 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 5, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } health_report { overall_health: HEALTHY } } }
I20260709 13:09:56.463815 22530 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:09:56.389272 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 212945us
I20260709 13:09:56.391145 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:56.393270 22784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
W20260709 13:09:56.393776 24733 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:09:56.394529 22775 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:56.397953 24677 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.465775 22530 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:09:56.369841 24450 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.340950 24679 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.402788 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:56.405411 24688 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:56.386876 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1518) took 1008 ms. Trace:
I20260709 13:09:56.380966 24582 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:56.384198 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:56.516765 22778 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5.
I20260709 13:09:56.384964 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:56.518224 22781 raft_consensus.cc:2376] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6. Re-sending same reply.
I20260709 13:09:56.378784 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.444818 22633 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:56.447911 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 80645.1us
I20260709 13:09:56.388273 24736 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.467459 22784 raft_consensus.cc:606] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:09:56.387667 24732 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:09:56.377028 24692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.020s	sys 0.026s
I20260709 13:09:56.383165 24676 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.509320 22632 rpcz_store.cc:276] 0709 13:09:55.378556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.388872 (+ 10316us) service_pool.cc:224] Handling call
0709 13:09:56.386672 (+997800us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54003a6e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:56.386814 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150332288}
W20260709 13:09:56.510039 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 107347us
I20260709 13:09:56.637483 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:56.377924 24725 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:56.486267 22530 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 2 to 4, term changed from 2 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to <none>, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 3 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:56.482442 24681 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.386931 24735 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:09:56.561277 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 182623us
I20260709 13:09:56.653290 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:56.560468 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 188652us
I20260709 13:09:56.446209 24604 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:56.564908 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:56.566039 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 215876us
I20260709 13:09:56.485566 24590 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.486454 24687 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:56.508823 24657 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:09:56.381359 24449 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.508574 24671 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.386516 24170 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:56.635742 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.445341 24742 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:56.480760 24603 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.565691 24678 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:09:56.562414 24644 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:09:56.623190 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59922us
I20260709 13:09:56.717638 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:56.560712 24670 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:09:56.509810 24677 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:56.667101 24551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.011s	sys 0.016s
I20260709 13:09:56.564738 24582 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.654345 24739 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:09:56.711069 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57847.4us
I20260709 13:09:56.664819 24604 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:56.719477 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:56.670091 24720 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:56.669106 24627 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:56.732064 22630 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:56.732282 22634 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:09:56.678139 24707 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.680909 24737 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.677280 24724 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:56.690511 24629 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:56.748845 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:56.657160 24681 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.694021 24742 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.717993 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:56.653350 24676 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:56.739627 24551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.011s	sys 0.017s
I20260709 13:09:56.732178 24405 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.762737 22708 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:56.693475 24671 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:56.771076 24587 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.610s	user 0.011s	sys 0.002s
I20260709 13:09:56.765110 24699 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.765331 24691 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.806702 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 167773us
I20260709 13:09:56.789978 24479 raft_consensus.cc:2764] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:09:56.781687 24720 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:56.803293 22707 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 2 to 4, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:09:56.792621 24432 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:56.794399 24726 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.851341 24750 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:09:56.851066 24726 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.852941 24562 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.797619 24558 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.853618 24726 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.853552 24562 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.854018 24558 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.855818 24562 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.856294 24558 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.837415 22780 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:09:56.793613 24725 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.761500 24627 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:56.869236 24743 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:09:56.871032 24627 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:56.847048 24679 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:56.847908 24720 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:56.865197 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45702.4us
W20260709 13:09:56.832850 24748 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 13:09:56.864636 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:56.874883 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 155482us
W20260709 13:09:56.829900 24706 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.680s	user 0.015s	sys 0.000s
I20260709 13:09:56.733000 24603 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.933118 23417 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:09:56.896454 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.770737 24707 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.784742 24629 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:56.915082 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50678.5us
I20260709 13:09:56.770112 24737 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.922346 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:56.951912 22704 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:56.808650 24691 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.973060 24737 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:56.976466 24720 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.977571 24726 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:56.980137 24582 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:56.937119 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:56.937446 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.072s (SENT) [suppressed 54 similar messages]
W20260709 13:09:56.983474 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46117.2us
I20260709 13:09:56.818809 24727 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.102s
I20260709 13:09:56.781071 24724 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:56.988768 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1817) took 1040 ms. Trace:
I20260709 13:09:56.989328 22634 rpcz_store.cc:276] 0709 13:09:55.948399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.948619 (+   220us) service_pool.cc:224] Handling call
0709 13:09:55.968640 (+ 20021us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.750793 (+782153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.988437 (+237644us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b540003bbd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:56.988686 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752424192}
I20260709 13:09:56.990056 24724 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.808861 24699 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:56.992627 24699 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:56.993589 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 244816us
I20260709 13:09:56.994479 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:56.768178 24735 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:56.947683 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.926589 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:56.952690 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:57.002269 22702 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:57.002413 22703 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:56.911873 24557 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.013161 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 292764us
W20260709 13:09:56.962265 24745 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:09:56.962929 24772 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:09:56.964660 24749 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
I20260709 13:09:56.965373 24629 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:56.967084 24449 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:56.969425 24555 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.002s
I20260709 13:09:56.971233 24590 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.056784 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.000597 24739 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.940435 24751 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
W20260709 13:09:56.960848 24753 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:09:57.019163 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 82156.5us
I20260709 13:09:57.040985 24582 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:57.060324 24677 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:57.074059 24677 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:09:57.038398 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44000.5us
I20260709 13:09:57.076695 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:09:57.016956 24744 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:09:57.043360 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:57.087934 22702 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:57.051343 24752 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:09:57.053880 24707 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.108294 22709 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:09:57.054301 24736 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.109594 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:57.111950 24759 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:09:57.055873 22704 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:57.115551 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 477) took 1306 ms. Trace:
I20260709 13:09:57.116051 23415 rpcz_store.cc:276] 0709 13:09:55.808929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.809137 (+   208us) service_pool.cc:224] Handling call
0709 13:09:55.835305 (+ 26168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.198844 (+363539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:56.198902 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:56.252880 (+ 53978us) log.cc:844] Serialized 26 byte log entry
0709 13:09:56.662497 (+409617us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:56.662566 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.662686 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.115292 (+452606us) raft_consensus.cc:1685] finished
0709 13:09:57.115315 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.115485 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109807,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":27278,"thread_start_us":685262,"threads_started":2,"wal-append.queue_time_us":529017}
I20260709 13:09:57.115588 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.119596 24781 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.056062 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.120287 24781 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.120760 22710 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:56.997682 24766 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:57.064726 24678 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:09:57.066190 24365 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
I20260709 13:09:57.066931 24671 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:57.067499 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 65674.2us
W20260709 13:09:57.153831 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 221974us
W20260709 13:09:57.068002 24756 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.191s	user 0.004s	sys 0.000s
I20260709 13:09:57.069078 24603 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:57.069568 24676 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.070029 24675 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.042562 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.078815 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:57.175287 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.080178 22703 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:57.095113 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.095402 22627 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:57.096426 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.098138 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.177795 22633 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:57.177903 22632 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:57.098927 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.024957 24557 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.097191 24582 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:57.060014 24627 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:57.099193 24629 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:09:57.103178 24688 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.002s
I20260709 13:09:57.105945 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.106242 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.042959 24720 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.971895 24170 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.122505 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.202457 22634 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:57.161370 24604 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:57.070067 24450 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.002s
I20260709 13:09:57.132412 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:09:57.135134 22750 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.112s (SENT)
I20260709 13:09:57.208504 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:09:57.208603 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76262.7us
I20260709 13:09:57.143767 24766 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.152431 24691 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.160071 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:57.160833 24734 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.744s	user 0.007s	sys 0.009s
I20260709 13:09:57.161290 24777 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:57.168745 22780 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
W20260709 13:09:57.174286 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.178824 22628 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:57.176321 22626 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:57.185420 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 106684us
I20260709 13:09:57.185781 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.177983 22704 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:57.072706 24739 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.176055 23420 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:57.197700 22707 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:57.201776 22705 raft_consensus.cc:2410] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:57.202512 22775 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:09:57.131771 22702 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:57.203817 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.171705 24479 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:57.206633 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.244983 22780 raft_consensus.cc:606] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:57.209383 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.245843 22635 raft_consensus.cc:2410] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:57.174978 24405 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:57.211627 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:57.212276 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77189.8us
I20260709 13:09:57.249825 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.250370 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.221153 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.250519 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1818) took 1301 ms. Trace:
I20260709 13:09:57.251748 22779 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:57.224407 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:09:57.225033 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.253896 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:57.196729 24707 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:57.226531 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.255194 22783 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:57.169348 24365 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.205452 24627 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:09:57.168129 24742 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.248761 23419 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:09:57.227023 24777 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:57.215705 24170 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:57.261279 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:57.262774 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.252092 22630 rpcz_store.cc:276] 0709 13:09:55.949038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.949222 (+   184us) service_pool.cc:224] Handling call
0709 13:09:55.949667 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:56.750714 (+801047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.250228 (+499514us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b54000b77d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:57.250454 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1798857600}
I20260709 13:09:57.219326 24587 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:57.288538 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.185452 24675 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.269878 22779 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:09:57.273391 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:57.207140 24688 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.208040 24720 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.289697 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.349098 22629 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:57.350378 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60736us
I20260709 13:09:57.295650 23417 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:57.314803 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:57.353160 22627 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:57.214035 24450 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:57.356343 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 102572us
W20260709 13:09:57.361815 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.259240 24555 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:57.280656 24691 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:57.404919 22775 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:57.406737 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:57.420042 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132827us
I20260709 13:09:57.421602 23384 leader_election.cc:400] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3.
I20260709 13:09:57.447633 23384 leader_election.cc:403] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:09:57.448962 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.416294 24769 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:57.421701 22673 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.452521 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.432529 24739 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:57.434810 22776 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:57.348165 24558 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:09:57.436237 24746 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.689s	user 0.002s	sys 0.000s
I20260709 13:09:57.265749 24479 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.421435 24555 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.438993 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 240) took 1158 ms. Trace:
I20260709 13:09:57.347849 24551 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:09:57.281911 24170 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.292906 24678 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:09:57.449733 22778 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:57.422379 24707 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.450218 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 176971us
I20260709 13:09:57.271155 24736 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:57.355084 24665 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.359757 24539 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:57.447026 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:57.436873 24432 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:57.292086 24770 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:57.486364 24170 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:57.527848 24681 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.529876 24675 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.531059 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:57.531441 24780 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:09:57.531512 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:57.532310 22776 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:57.533936 24365 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.534346 24479 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.492610 24736 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:57.453473 22780 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:57.592315 23416 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:57.592907 24736 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:57.594040 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 482) took 1581 ms. Trace:
I20260709 13:09:57.594782 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:09:57.453469 22709 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:57.424213 24781 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:57.473977 22781 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:09:57.598335 22781 pending_rounds.cc:85] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 1
I20260709 13:09:57.598676 22781 pending_rounds.cc:107] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:09:57.476954 24779 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:09:57.477530 24760 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:09:57.477357 24604 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.475508 22629 rpcz_store.cc:276] 0709 13:09:56.281145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.281292 (+   147us) service_pool.cc:224] Handling call
0709 13:09:56.281682 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:57.349834 (+1068152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.438731 (+ 88897us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54001ca054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:57.438927 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2454217984}
I20260709 13:09:57.445562 24691 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:57.613312 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86448.4us
W20260709 13:09:57.484745 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65179.4us
I20260709 13:09:57.451936 24706 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:09:57.293133 24463 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:57.488646 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:57.280903 24582 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:57.491673 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.493075 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:57.452600 24769 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:57.454032 24725 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:09:57.498549 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.271253 24699 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.211s
W20260709 13:09:57.501690 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54767.7us
I20260709 13:09:57.479841 24555 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:57.471874 24558 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.497596 24707 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:57.585704 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.645165 24691 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:57.452682 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:57.739621 24758 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.956s	user 0.002s	sys 0.002s
I20260709 13:09:57.503837 24432 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:57.508656 24587 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.607331 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.704387 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.500821 24665 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.731680 22778 raft_consensus.cc:2410] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:57.743412 24769 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:57.725709 23384 leader_election.cc:400] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5.
W20260709 13:09:57.838719 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113214us
W20260709 13:09:57.705139 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 212348us
I20260709 13:09:57.492156 24627 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.603178 24727 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:57.708164 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:57.598719 24766 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:57.739625 24784 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
W20260709 13:09:57.740324 24747 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.774s	user 0.002s	sys 0.001s
I20260709 13:09:57.740705 24738 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3.
I20260709 13:09:57.817189 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:57.704535 24774 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.485s	user 0.001s	sys 0.000s
I20260709 13:09:57.818954 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:57.826998 24463 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:57.531142 24770 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.475040 24737 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:09:57.826125 22702 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:57.598927 22781 raft_consensus.cc:2889] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.614171 24779 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:57.732439 24739 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.702705 24539 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:09:57.830956 23418 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:57.516510 24734 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:57.725504 24670 raft_consensus.cc:476] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Not starting pre-election -- already a leader
I20260709 13:09:57.737498 24604 raft_consensus.cc:2764] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 13:09:57.731509 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 242987us
I20260709 13:09:57.964613 24688 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:57.852487 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:57.839603 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:57.856027 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:57.856462 24767 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.967345 22702 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:57.967804 24767 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.857824 24551 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:57.859246 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 217) took 1467 ms. Trace:
I20260709 13:09:57.969686 24795 raft_consensus.cc:2764] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 13:09:57.826152 24562 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:57.831533 24365 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:57.821779 24449 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:09:57.912388 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93515.6us
I20260709 13:09:57.820858 24676 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.918793 24590 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:57.911556 22705 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:57.920847 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1699) took 1103 ms. Trace:
I20260709 13:09:57.947695 24781 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.921365 24740 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.084s	user 0.003s	sys 0.001s
W20260709 13:09:57.921773 24789 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 13:09:57.743151 24405 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:09:57.953274 24720 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.594532 23412 rpcz_store.cc:276] 0709 13:09:56.012632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.016306 (+  3674us) service_pool.cc:224] Handling call
0709 13:09:56.044029 (+ 27723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.082451 (+ 38422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:56.085311 (+  2860us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:56.140732 (+ 55421us) log.cc:844] Serialized 26 byte log entry
0709 13:09:56.445723 (+304991us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:56.461940 (+ 16217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.484014 (+ 22074us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.435591 (+951577us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:57.593661 (+158070us) raft_consensus.cc:1685] finished
0709 13:09:57.593702 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.593956 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1988,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":3181,"spinlock_wait_cycles":12808576,"thread_start_us":284500,"threads_started":1,"wal-append.queue_time_us":536945}
I20260709 13:09:57.841437 24742 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.826220 24603 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:09:57.956749 24802 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:57.957250 24675 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:57.957753 24770 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:57.852361 23411 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:09:57.948757 24692 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:09:57.958699 24796 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:09:58.052395 24692 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.826723 24558 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.015914 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:58.056875 22702 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:57.983727 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:58.058334 22703 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:57.967443 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127927us
I20260709 13:09:58.066913 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:58.017966 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.068337 22706 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.021644 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:58.023949 24699 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:58.027007 24735 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.972281 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:58.041203 24671 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:57.972775 24590 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.042665 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.977873 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.828994 24725 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.977632 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:57.978579 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 126176us
W20260709 13:09:58.046316 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.951359 24779 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:57.954159 24734 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.976667 24463 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.189724 24767 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:58.158529 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 103024us
I20260709 13:09:57.969781 24450 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:58.089753 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68165us
I20260709 13:09:58.193709 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:58.092455 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:57.975932 24405 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:09:57.973426 22708 rpcz_store.cc:276] 0709 13:09:56.817158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.817410 (+   252us) service_pool.cc:224] Handling call
0709 13:09:56.837080 (+ 19670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.837505 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:56.837535 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:56.853123 (+ 15588us) log.cc:844] Serialized 26 byte log entry
0709 13:09:57.019948 (+166825us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:57.020078 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.020421 (+   343us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.920629 (+900208us) raft_consensus.cc:1685] finished
0709 13:09:57.920653 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.920783 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42549,"raft.queue_time_us":93310,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":20882,"thread_start_us":166150,"threads_started":1,"wal-append.queue_time_us":317998}
I20260709 13:09:58.058666 24727 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:58.104841 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132663us
I20260709 13:09:58.104836 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:58.205919 23415 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:57.851826 24587 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:09:58.020743 24551 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.970094 22782 rpcz_store.cc:276] 0709 13:09:56.391398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.396056 (+  4658us) service_pool.cc:224] Handling call
0709 13:09:56.396371 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.396744 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:56.396778 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:56.397371 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:09:56.631617 (+234246us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:56.922477 (+290860us) log.cc:844] Serialized 14 byte log entry
0709 13:09:56.997550 (+ 75073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.997647 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.703179 (+705532us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:57.858913 (+155734us) raft_consensus.cc:1685] finished
0709 13:09:57.859019 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.859174 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74750,"prepare.queue_time_us":187784,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":79028,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":5993,"spinlock_wait_cycles":5550208,"thread_start_us":498883,"threads_started":2,"wal-append.queue_time_us":340014}
I20260709 13:09:57.971228 24365 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.972594 24676 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.110220 24735 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.111037 24803 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:58.111167 24603 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:58.111652 24692 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:58.110266 24793 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.504s	user 0.005s	sys 0.001s
I20260709 13:09:58.104462 24699 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.104168 24737 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:09:58.113499 24707 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.117473 22702 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:09:58.050297 24802 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.139703 24557 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:58.054208 24539 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.016901 24720 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:57.973608 24781 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:57.971498 24449 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.970960 24562 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:57.977988 24739 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.189996 24670 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:09:58.194026 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:58.142381 24736 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.002s
I20260709 13:09:58.092535 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:58.212248 24801 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:09:58.203873 22709 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:09:58.192481 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.207150 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:58.307180 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.209991 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.307894 22706 raft_consensus.cc:2410] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:09:58.208753 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:58.206066 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:58.307902 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:09:58.308869 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 102905us
I20260709 13:09:58.214501 24450 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:58.144137 23411 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:58.148208 24665 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.053836 24675 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.192436 24590 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.195379 24734 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:58.252043 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.201140 22703 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:58.276677 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.203306 24405 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:58.273800 24603 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:58.291996 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98373.6us
I20260709 13:09:58.286474 24725 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.273983 24803 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.306360 22710 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:58.198443 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.309027 22707 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.106596 24671 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.342082 22710 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:58.316080 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 124289us
I20260709 13:09:58.310233 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:09:58.327901 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.328229 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:58.344002 22706 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.344643 24582 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:58.345109 24582 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.345449 24582 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.281258 24737 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:09:58.331399 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.349354 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.295229 23415 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:09:58.350210 22702 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.215219 24551 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:58.337801 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 129155us
I20260709 13:09:58.328909 24767 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:58.324106 24450 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.287993 24365 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.354048 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.365964 22702 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:58.356460 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:58.360052 22710 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:58.368180 22710 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:58.288201 24720 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.156318 24777 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.333375 24603 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:58.339603 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1708) took 1292 ms. Trace:
I20260709 13:09:58.369251 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.340997 24557 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:58.327208 24804 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:58.332005 24641 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:09:58.324106 24736 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.372740 22707 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:58.379041 22702 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.380404 22709 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:58.351025 24671 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.381410 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:58.341917 24803 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:58.386606 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.365163 24450 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:58.453150 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.396775 22707 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:09:58.363476 24767 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.376479 22701 rpcz_store.cc:276] 0709 13:09:57.046965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:57.055376 (+  8411us) service_pool.cc:224] Handling call
0709 13:09:57.164053 (+108677us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:09:57.164248 (+   195us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:57.171105 (+  6857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:57.171138 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:57.185054 (+ 13916us) log.cc:844] Serialized 26 byte log entry
0709 13:09:57.417856 (+232802us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:09:57.417903 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.417983 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:58.088487 (+670504us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b24000fc638. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:09:58.339391 (+250904us) raft_consensus.cc:1685] finished
0709 13:09:58.339415 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:58.339542 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14410,"raft.queue_time_us":11229,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":19543,"spinlock_wait_cycles":54536448,"thread_start_us":220951,"threads_started":1,"wal-append.queue_time_us":313054}
I20260709 13:09:58.455889 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.456573 22704 raft_consensus.cc:2410] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:09:58.460212 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71922us
I20260709 13:09:58.464404 22702 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:58.490401 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:58.471561 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.501461 22709 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:09:58.447885 24691 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 13:09:58.508616 22706 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:09:58.447527 24785 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:09:58.336803 24727 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:09:58.515370 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50609.9us
I20260709 13:09:58.518977 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:58.519647 22781 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:09:58.522809 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:58.522406 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:58.524657 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:58.523504 22781 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:58.525324 22775 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.466912 24777 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.532646 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.535929 22706 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:58.392268 24737 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.472898 24767 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.458940 24539 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:58.548470 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:58.534868 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 190706us
I20260709 13:09:58.543161 22706 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:58.540172 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:58.554594 22777 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:58.558951 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.560067 22707 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.449412 24804 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.505959 24801 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 13:09:58.566080 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 199981us
I20260709 13:09:58.577191 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:58.570972 22632 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:09:58.578908 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.534252 24587 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:58.575040 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:58.452283 24432 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.586551 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:58.591049 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.586231 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:58.557719 24676 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:58.521368 24691 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.554595 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:58.605804 22706 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:09:58.607448 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.615340 22707 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3. Re-sending same reply.
I20260709 13:09:58.615624 22775 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:58.562667 24665 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:58.574252 24365 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:09:58.585664 24815 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.585769 24670 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:58.645193 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.646016 22707 raft_consensus.cc:2376] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3. Re-sending same reply.
I20260709 13:09:58.642251 22706 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:09:58.617717 24432 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:58.617209 24557 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:09:58.682731 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.617470 24735 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:58.652837 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.618494 24699 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:58.635735 24365 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:09:58.683470 24785 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:09:58.684033 24807 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
I20260709 13:09:58.634760 24587 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:58.646111 24802 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:58.641714 24665 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:58.717559 24555 consensus_meta.cc:287] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.887s	user 0.019s	sys 0.002s
W20260709 13:09:58.729878 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.731196 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:58.638034 24449 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:58.723399 24806 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.357s	user 0.001s	sys 0.000s
I20260709 13:09:58.737774 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.763417 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:09:58.688026 24681 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:58.683619 24739 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:58.684522 24769 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.773s	user 0.010s	sys 0.003s
I20260709 13:09:58.684275 24670 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.721886 24675 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.746842 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:58.751237 22706 raft_consensus.cc:2376] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3. Re-sending same reply.
I20260709 13:09:58.756702 24804 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:58.731662 24365 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:58.759744 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:58.764727 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 208016us
I20260709 13:09:58.728159 24557 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:58.730710 24810 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
I20260709 13:09:58.647141 24815 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.758297 24736 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:58.778000 22709 raft_consensus.cc:2410] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:09:58.791401 22783 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:09:58.787096 22702 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:58.772185 24769 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.744038 24587 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.759127 24770 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.796517 24739 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.796751 24449 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:58.762652 24555 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.862342 24449 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.866989 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:58.796780 24816 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:09:58.868603 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:58.885769 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.798599 23417 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:58.782925 24779 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.596s	user 0.007s	sys 0.013s
W20260709 13:09:58.794123 24170 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.198s	user 0.016s	sys 0.011s
W20260709 13:09:58.782196 24819 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:09:58.799088 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:58.921197 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.862118 24739 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.936141 24557 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:58.937875 24562 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:58.938133 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.901415 24720 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:58.924949 24807 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:09:58.901547 24770 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:58.930418 24779 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.025413 24765 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:59.026407 24765 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.027782 24779 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:59.028431 24829 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.104s	user 0.002s	sys 0.000s
I20260709 13:09:59.028895 24665 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.934504 24170 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.029412 24765 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:59.030225 24809 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.559s	user 0.001s	sys 0.000s
I20260709 13:09:59.031558 24170 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.033178 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.006630 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.036530 22776 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:59.037415 24551 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.034513 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.039357 22783 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:09:59.040848 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.044708 24587 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.052724 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.057746 22708 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:59.060158 22630 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:09:58.975096 24449 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:58.956386 24691 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.038048 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.005281 24676 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.776191 24681 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:09:59.066140 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.003s
I20260709 13:09:58.987782 24720 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:59.077010 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1727) took 1061 ms. Trace:
I20260709 13:09:59.082233 22703 rpcz_store.cc:276] 0709 13:09:58.015399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.344238 (+328839us) service_pool.cc:224] Handling call
0709 13:09:58.344650 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:58.345025 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:58.345058 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:58.345609 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:09:58.401002 (+ 55393us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:09:58.401077 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:58.401198 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:59.076703 (+675505us) raft_consensus.cc:1685] finished
0709 13:09:59.076728 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.076922 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6702,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":12471,"thread_start_us":49141,"threads_started":1,"wal-append.queue_time_us":157551}
I20260709 13:09:59.078168 22710 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:09:59.068382 22675 connection.cc:256] RPC call timeout handler was delayed by -3.891084783999986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 13:09:59.092993 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.009s	sys 0.003s
W20260709 13:09:59.093710 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 546653us
I20260709 13:09:58.781116 24825 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:09:59.068966 24739 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:59.116466 24803 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.654s	user 0.013s	sys 0.006s
W20260709 13:09:59.116744 24603 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.667s	user 0.009s	sys 0.009s
I20260709 13:09:59.117126 24803 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.117450 24603 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.119112 24670 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.119472 24769 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.120375 24681 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.123695 24555 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:59.124542 24558 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.018s	user 0.013s	sys 0.004s
I20260709 13:09:59.125026 24558 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.096505 24676 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.127439 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.128844 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:59.129210 24590 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.795s	user 0.006s	sys 0.004s
I20260709 13:09:59.129663 24590 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.130097 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.100099 24735 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.083755 22776 raft_consensus.cc:2376] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3. Re-sending same reply.
I20260709 13:09:59.131039 24735 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.132177 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.133493 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.134902 24735 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.135921 24770 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.119125 24815 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.113238 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.139434 22783 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:59.142768 24676 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.144213 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:59.146009 24582 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.001s
I20260709 13:09:59.149963 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.145898 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.074558 24449 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.146710 22779 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:59.154052 24449 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.099475 24765 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.156941 24765 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.064054 24170 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:59.162009 22779 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
W20260709 13:09:59.150128 24734 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.001s
I20260709 13:09:59.141980 24365 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.148991 24825 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.152392 24603 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.096232 24720 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:59.171211 24450 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.692s	user 0.011s	sys 0.012s
I20260709 13:09:59.151697 24582 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:59.264312 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 677602us
I20260709 13:09:59.266788 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.267482 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.268400 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.268815 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:09:59.269722 22627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.062s	sys 0.114s
I20260709 13:09:59.269776 22633 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:09:59.271147 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.373246 22630 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:59.272497 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.374866 24590 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:59.375038 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:59.376356 22630 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:59.280537 22626 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:09:59.260902 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.005s	sys 0.006s
W20260709 13:09:59.294683 22704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1740) took 1186 ms (client timeout 1505 ms). Trace:
W20260709 13:09:59.516321 22704 rpcz_store.cc:269] 0709 13:09:58.107929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.192093 (+ 84164us) service_pool.cc:224] Handling call
0709 13:09:59.293975 (+1101882us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002bf2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:59.294340 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340552576}
I20260709 13:09:59.514588 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.302083 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.303798 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:09:59.305794 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.519115 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.520442 24449 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.309219 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1738) took 1249 ms. Trace:
I20260709 13:09:59.523223 22708 rpcz_store.cc:276] 0709 13:09:58.059589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.251633 (+192044us) service_pool.cc:224] Handling call
0709 13:09:59.308620 (+1056987us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54000e7754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:59.308935 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680133504}
I20260709 13:09:59.312098 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:09:59.525322 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 382641us
I20260709 13:09:59.525394 22779 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:59.312717 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.318569 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.527455 22782 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:59.325229 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.528810 22783 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:59.331075 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.529421 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.529878 22675 connection.cc:256] RPC call timeout handler was delayed by -3.5733145829999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260709 13:09:59.530300 22675 connection.cc:256] RPC call timeout handler was delayed by -3.1135447310000472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16820000000000002s for a response
W20260709 13:09:59.530583 22675 connection.cc:256] RPC call timeout handler was delayed by -3.0865447310000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 13:09:59.530941 22675 connection.cc:256] RPC call timeout handler was delayed by -2.9352145829999472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2808s for a response
W20260709 13:09:59.531430 22675 connection.cc:256] RPC call timeout handler was delayed by -2.7616447310000467s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 13:09:59.531847 22675 connection.cc:256] RPC call timeout handler was delayed by -2.6293963430000531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
W20260709 13:09:59.532207 22675 connection.cc:256] RPC call timeout handler was delayed by -2.2513963430000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 13:09:59.348838 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 791358us
I20260709 13:09:59.533071 24779 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:09:59.527487 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.350935 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.534106 24779 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:59.535362 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.165300 24170 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.545570 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.546593 22782 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:59.547349 22630 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:09:59.547628 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.547909 24170 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.351542 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 491) took 1824 ms. Trace:
I20260709 13:09:59.548646 23417 rpcz_store.cc:276] 0709 13:09:57.527279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:57.530706 (+  3427us) service_pool.cc:224] Handling call
0709 13:09:57.531017 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.894787 (+1363770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.350556 (+455769us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400503554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:59.351216 (+   660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914765568}
I20260709 13:09:59.549315 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.350687 24450 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.550184 22782 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:09:59.179380 24671 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.009s
I20260709 13:09:59.330194 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.551308 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.552135 22708 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:59.553148 24450 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.193113 24765 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:09:59.225034 24777 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.010s
I20260709 13:09:59.554951 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.209438 24781 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:59.204311 24821 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:59.373453 22628 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:59.556259 22779 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:09:59.556563 24821 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:09:59.292099 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1743) took 1097 ms. Trace:
W20260709 13:09:59.384320 22627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.063s	sys 0.114s
I20260709 13:09:59.557485 22778 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:09:59.558043 22778 pending_rounds.cc:85] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 2
I20260709 13:09:59.391532 24365 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.558358 22778 pending_rounds.cc:107] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:09:59.558939 22778 raft_consensus.cc:2889] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:59.399320 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.557206 22710 rpcz_store.cc:276] 0709 13:09:58.194203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.373248 (+179045us) service_pool.cc:224] Handling call
0709 13:09:59.077879 (+704631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:59.084984 (+  7105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.291433 (+206449us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54002c47d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:09:59.291762 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619345152}
I20260709 13:09:59.560530 24558 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.558257 24821 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:59.213546 24767 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.016s
I20260709 13:09:59.375178 24676 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.403961 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.434468 24803 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.562453 24676 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.561761 24821 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:09:59.564343 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.565579 24676 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.418780 24770 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:59.566679 24770 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.568300 22784 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:09:59.414686 24432 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.569677 24432 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.572626 24432 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.443364 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.574544 24826 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.575313 24826 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.462020 24582 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:09:59.439952 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.505s (SENT)
W20260709 13:09:59.464718 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57998.5us
I20260709 13:09:59.487491 24405 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.495059 24825 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.581339 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.511723 24720 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.516343 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 424) took 1502 ms. Trace:
I20260709 13:09:59.521063 22780 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:09:59.582381 22704 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:59.307083 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.520032 22776 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:09:59.347723 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.614553 22777 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:09:59.538199 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0515963430000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
I20260709 13:09:59.539583 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.539728 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.541051 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.542414 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.543754 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.544683 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.551357 24671 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.554194 22632 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:59.167495 24734 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.640856 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.642062 22710 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:59.580253 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:09:59.561872 24767 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.583530 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.585901 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:09:59.586911 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.520855 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.010s	sys 0.007s
I20260709 13:09:59.582515 23421 rpcz_store.cc:276] 0709 13:09:58.013910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.041562 (+ 27652us) service_pool.cc:224] Handling call
0709 13:09:58.041948 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.160842 (+118894us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:58.340196 (+179354us) log.cc:844] Serialized 14 byte log entry
0709 13:09:58.340533 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 13:09:58.349491 (+  8958us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.174010 (+824519us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b180041cfe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:09:59.176422 (+  2412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:59.176507 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:59.176573 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.199332 (+ 22759us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b180041cfe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1
0709 13:09:59.200725 (+  1393us) spinlock_profiling.cc:243] Waited 7.99 ms on lock 0x7b180041cfe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:59.202454 (+  1729us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b180041cfe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:59.515610 (+313156us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54000db454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:09:59.515726 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.516085 (+   359us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7310,"prepare.queue_time_us":1894902,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":107,"raft.queue_time_us":829839,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":3500,"spinlock_wait_cycles":309875840,"thread_start_us":152789,"threads_started":1}
I20260709 13:09:59.581352 24405 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:09:59.591248 24802 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:59.593019 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.660792 22784 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:59.592921 22532 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 6, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.595860 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:59.596304 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.596407 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.598433 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.600718 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.602661 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.603727 24779 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.606910 24826 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:59.608397 24770 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.611584 24720 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.613333 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:09:59.620713 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.636435 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.620339 24676 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:59.655150 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.643754 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:59.677654 22780 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:09:59.669152 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 1.08369e+06us
I20260709 13:09:59.683796 22705 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:09:59.678982 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93294.2us
I20260709 13:09:59.681425 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.645051 24734 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:59.681260 22702 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:59.694588 22779 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.688592 22708 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:59.667547 24821 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:59.702251 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.703250 22709 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:59.689184 24826 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:59.670336 24449 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:09:59.712798 22710 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:59.714944 22633 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:09:59.723929 24720 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:09:59.709329 24826 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:59.726950 23417 raft_consensus.cc:2410] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:09:59.723917 24737 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.727871 24772 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:09:59.728797 24772 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.729701 24850 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:59.730432 24850 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.733389 24850 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.728742 24737 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.736418 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.705417 22532 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.737331 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.748752 22775 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:09:59.749756 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.750658 22704 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:59.752385 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 233) took 6641 ms. Trace:
W20260709 13:09:59.755364 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 315747us
I20260709 13:09:59.743567 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.766896 24734 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:09:59.767621 22705 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:59.772302 24720 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.773393 24767 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:09:59.735292 24826 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:59.776906 24671 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.777824 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.778606 22709 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.786216 23415 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:09:59.788699 22532 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.790125 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.790992 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.791973 23421 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:59.792572 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.793004 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.793395 24450 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.793556 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:09:59.794132 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.794306 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.779534 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 499) took 1328 ms. Trace:
I20260709 13:09:59.794452 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.794382 22706 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:09:59.823935 23419 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:09:59.795989 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:59.831346 22702 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.832901 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:09:59.833285 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 94454.5us
I20260709 13:09:59.819379 23416 rpcz_store.cc:276] 0709 13:09:58.394490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.396055 (+  1565us) service_pool.cc:224] Handling call
0709 13:09:58.506079 (+110024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.537815 (+ 31736us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:58.609548 (+ 71733us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.027796 (+418248us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:09:59.653860 (+626064us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:59.678508 (+ 24648us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:59.688307 (+  9799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.706573 (+ 18266us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.722304 (+ 15731us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":521465,"prepare.queue_time_us":38709,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":9015,"raft.queue_time_us":541158,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":53827,"spinlock_wait_cycles":15313792,"thread_start_us":273363,"threads_started":2,"wal-append.queue_time_us":182090}
I20260709 13:09:59.835393 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:09:59.839202 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.840564 23412 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:09:59.841677 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.795269 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.794135 24450 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.846902 24450 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:09:59.847662 24450 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.848268 24450 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:09:59.850781 23385 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.766s (SENT)
I20260709 13:09:59.854449 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.855401 22707 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:09:59.858579 22782 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:09:59.860927 22710 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:09:59.803678 22532 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.875187 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.846482 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:59.885068 22673 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:09:59.826172 23411 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:09:59.909075 22784 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:09:59.910374 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.914990 23417 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:59.870018 24432 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:09:59.858228 24450 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.917663 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.913017 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:09:59.888176 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:09:59.922916 23415 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:59.917927 22710 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:09:59.927030 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.924551 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.928864 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:59.909933 24779 raft_consensus.cc:2764] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:09:59.752997 22530 rpcz_store.cc:276] 0709 13:09:53.111017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.111502 (+   485us) service_pool.cc:224] Handling call
0709 13:09:59.752023 (+6640521us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:57.294309 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:57.613132 (+318823us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:57.644345 (+ 31213us) write_op.cc:435] Acquired schema lock
0709 13:09:57.702970 (+ 58625us) tablet.cc:662] Decoding operations
0709 13:09:57.971136 (+268166us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:57.971211 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:57.971244 (+    33us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:09:57.972143 (+   899us) tablet.cc:701] Row locks acquired
0709 13:09:57.972167 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:09:57.972359 (+   192us) write_op.cc:270] Start()
0709 13:09:57.972505 (+   146us) write_op.cc:276] Timestamp: P: 1783602597972293 usec, L: 0
0709 13:09:57.972530 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:09:57.973331 (+   801us) log.cc:844] Serialized 4121 byte log entry
0709 13:09:58.602948 (+629617us) op_driver.cc:464] REPLICATION: finished
0709 13:09:59.215760 (+612812us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b1800179860. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1
0709 13:09:59.216638 (+   878us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b1800179860. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:09:59.216814 (+   176us) write_op.cc:301] APPLY: starting
0709 13:09:59.216944 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:59.217286 (+   342us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:59.217326 (+    40us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:59.220665 (+  3339us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:59.221018 (+   353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:09:59.221099 (+    81us) write_op.cc:312] APPLY: finished
0709 13:09:59.221588 (+   489us) log.cc:844] Serialized 77 byte log entry
0709 13:09:59.222849 (+  1261us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:59.223070 (+   221us) write_op.cc:454] Released schema lock
0709 13:09:59.223150 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:59.223714 (+   564us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:59.223760 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":256314,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":395675,"apply.run_cpu_time_us":4199,"apply.run_wall_time_us":6855,"num_ops":10,"prepare.queue_time_us":494487,"prepare.run_cpu_time_us":19517,"prepare.run_wall_time_us":1029287,"raft.queue_time_us":188031,"raft.run_cpu_time_us":12196,"raft.run_wall_time_us":531976,"replication_time_us":594369,"spinlock_wait_cycles":27569792,"thread_start_us":1005032,"threads_started":4,"wal-append.queue_time_us":369546}]]}
W20260709 13:09:59.931478 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 135765us
I20260709 13:09:59.929977 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:59.934521 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:09:59.935297 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 64558.9us
I20260709 13:09:59.935541 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:09:59.936297 22630 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:59.938061 22747 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:09:59.940478 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.943068 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107824us
W20260709 13:09:59.951630 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.954591 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:09:59.955610 24824 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:59.955865 22628 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.956257 22747 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:59.958393 24814 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.926800 24825 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:59.959159 24835 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:59.958956 24603 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:59.959949 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53942.3us
I20260709 13:09:59.963089 24737 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:09:59.963238 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116945us
I20260709 13:09:59.964216 22633 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:09:59.920099 24432 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:09:59.968357 23418 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:09:59.969796 24432 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:09:59.964372 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:59.950515 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:09:59.976825 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.959774 24835 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:09:59.984942 22779 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:59.987006 22705 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:09:59.991899 22632 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:59.993425 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:09:59.989507 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:09:59.996276 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:09:59.997081 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:00.000288 22626 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:00.000384 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:09:59.992470 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.002943 22778 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.003630 23421 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:10:00.011313 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.013046 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.014143 22779 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:00.017407 22710 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:09:59.984213 24770 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:00.006203 22532 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:59.992492 24835 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:00.060113 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.067188 22532 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:00.072865 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108668us
I20260709 13:10:00.073805 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:09:59.959199 24814 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.001927 24779 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:00.023883 24772 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:00.083359 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 110308us
I20260709 13:10:00.088081 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.090902 22782 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:00.088514 22784 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:00.097446 24795 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:10:00.099339 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 117698us
I20260709 13:10:00.100392 24463 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:00.101164 24463 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.102857 24699 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:00.080574 24770 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:00.107774 24770 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.108377 24802 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:00.109157 24802 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.109947 24463 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.111398 24802 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:00.112524 24826 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.114141 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.115456 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.116137 24814 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:00.100821 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:00.089336 24821 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:00.131283 22704 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:00.133421 23415 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:00.120851 24727 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:00.118459 24603 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:00.161145 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.170403 24735 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.170785 24837 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.171207 24735 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:00.171725 24735 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.174999 24558 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.182799 24727 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:00.183132 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 109439us
I20260709 13:10:00.184393 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.184173 24727 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.185353 24814 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:00.185822 22628 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:00.185904 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:00.187115 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.188438 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.188527 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.189221 22628 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:10:00.189596 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.190030 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.190259 22629 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:10:00.191088 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 90420.7us
I20260709 13:10:00.191203 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.191320 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.192060 24824 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:00.192555 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.192806 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:00.192970 24824 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:00.193675 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.193797 24641 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:00.195048 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.196441 22783 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:00.196899 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.198580 22705 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:10:00.199419 24603 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.200266 24603 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:00.194015 24676 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:00.204116 24676 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.204703 24676 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:00.207463 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86881.2us
I20260709 13:10:00.208389 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:00.210899 24582 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.211495 24582 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.211956 24582 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.162253 22776 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:00.214969 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51336.9us
I20260709 13:10:00.119004 24779 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.219600 24779 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.225448 24720 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:10:00.226089 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.227218 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.228248 22629 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:00.229193 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.230580 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.235841 24826 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:00.236434 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.241091 22707 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:00.245177 23421 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:00.246666 22629 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:00.247167 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61367.9us
I20260709 13:10:00.249693 22709 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:00.242149 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.251775 22778 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:00.254714 23417 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:00.256577 24463 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:00.257081 24463 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.257507 24821 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.258029 24821 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.261344 22632 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:00.263406 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.248834 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.260242 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:00.268157 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 271409us
I20260709 13:10:00.270013 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:00.272634 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.281852 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.287954 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.291052 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.257503 24463 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.294934 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.308028 22634 raft_consensus.cc:2368] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
W20260709 13:10:00.310168 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 430628us
I20260709 13:10:00.310168 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.312304 22628 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:00.317392 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 50074.5us
I20260709 13:10:00.286209 24734 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.304335 24814 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.344486 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.344761 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.306658 24837 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.344797 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.351577 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.371851 24405 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:00.372951 24769 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:00.373343 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.373574 24557 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:00.374770 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:00.419328 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83409.9us
I20260709 13:10:00.420372 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.421303 24170 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:00.421809 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.422204 24170 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.422796 24170 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:00.423274 24170 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.423543 24737 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:00.424088 24737 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:00.425642 24865 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.426349 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.428334 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.429824 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.430433 22627 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:00.343420 24734 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:00.435871 23412 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:00.436975 24734 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.440400 24767 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:00.440997 24767 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:00.342010 24826 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.443650 24826 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:00.444818 24826 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.442253 24767 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.451891 24558 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:00.386435 24803 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:00.388034 24557 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.455348 24557 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:00.461532 24767 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:00.463094 24803 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:00.434424 24170 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.466928 24170 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:00.470857 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 195521us
I20260709 13:10:00.471834 22627 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:00.450670 24405 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.483963 24405 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.387123 24450 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:10:00.340644 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:00.449025 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 327546us
I20260709 13:10:00.500905 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:00.502051 22628 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:00.426523 24865 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:00.495453 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73728.8us
I20260709 13:10:00.505548 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.511626 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:00.513103 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 222202us
W20260709 13:10:00.513917 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 162464us
I20260709 13:10:00.517261 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.516155 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.516374 22629 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:00.385741 24769 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:00.524729 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 237608us
I20260709 13:10:00.482722 22627 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:00.526103 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:00.553025 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.517987 24603 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:00.382395 24814 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.520572 24769 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:00.560384 24699 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:00.575459 24170 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:00.576011 24170 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.577028 24170 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.580322 24737 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.581794 24737 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.516709 22776 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:10:00.595393 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53749.4us
I20260709 13:10:00.578399 24170 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:00.597234 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 80090.1us
I20260709 13:10:00.556075 24803 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:00.598603 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.536896 24825 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:00.555858 24802 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:00.602079 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.528337 24835 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.621254 24449 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.621867 24449 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:00.622278 24449 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.423550 24837 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.622985 24835 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.625590 24603 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:00.626217 24603 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:00.629136 24735 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.513697 24865 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.611948 22626 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:00.611948 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:00.633178 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 338348us
I20260709 13:10:00.637159 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:00.639889 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101892us
I20260709 13:10:00.643435 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.608227 24825 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:00.632349 24802 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.667709 22635 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:00.668202 24735 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:00.673136 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.675879 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.677909 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:00.678972 24558 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.680024 24558 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.638984 24582 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.683344 24582 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:00.684136 22777 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:00.692891 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:00.662595 24555 raft_consensus.cc:2764] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:10:00.703225 24825 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.704216 24814 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.705089 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.709026 24802 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.658676 24365 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:00.666968 24699 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.717969 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:00.735543 24767 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:00.740370 24865 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.737571 24699 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.743284 24405 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:00.743988 24405 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.745694 24557 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.746366 24405 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.747706 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:00.749568 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.750734 24779 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:00.751271 24779 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.751384 24555 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:00.751732 24779 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:00.752100 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59349.1us
I20260709 13:10:00.753494 24803 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:00.754040 24803 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.755117 24803 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.756799 24821 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.743274 24699 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:00.758947 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73833.9us
I20260709 13:10:00.752133 24555 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.760726 24555 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.751468 24814 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:10:00.763227 24865 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:00.763691 24865 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:00.764211 24865 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:00.766414 24365 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:00.674449 24449 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.767200 24882 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:00.767619 24365 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.768694 24449 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.770252 24821 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:00.770967 24405 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:00.773552 24676 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.568s	user 0.000s	sys 0.010s
I20260709 13:10:00.705113 24825 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:00.774053 24603 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:00.714051 24720 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:10:00.784832 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.792237 24769 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.800923 22634 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:00.799475 24405 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.738127 24463 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.796509 24770 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.797106 24676 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:00.822893 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.824177 24557 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:00.827850 24769 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:00.835978 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:00.847761 22708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.006s	sys 0.028s
I20260709 13:10:00.872725 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:00.875428 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52669.1us
I20260709 13:10:00.881088 22779 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:00.881868 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.878911 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:00.924329 22635 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:00.915607 22784 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:00.924652 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.917430 22747 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:00.922766 22632 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:00.901074 22708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.006s	sys 0.028s
I20260709 13:10:00.929211 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.933318 22781 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:00.933931 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1750) took 1089 ms. Trace:
I20260709 13:10:00.935935 24665 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:00.936473 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:00.937369 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.915091 24558 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.915732 24641 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:00.934487 22708 rpcz_store.cc:276] 0709 13:09:59.844452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:59.946189 (+101737us) service_pool.cc:224] Handling call
0709 13:09:59.950704 (+  4515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:59.966321 (+ 15617us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:00.024762 (+ 58441us) log.cc:844] Serialized 14 byte log entry
0709 13:10:00.933262 (+908500us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:00.933441 (+   179us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:00.933533 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:00.933699 (+   166us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:00.933850 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1997,"prepare.queue_time_us":128210,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":113710,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7066,"thread_start_us":898492,"threads_started":3,"wal-append.queue_time_us":216527}
W20260709 13:10:00.979103 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 91092.8us
I20260709 13:10:00.924062 24582 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.935949 24876 raft_consensus.cc:2764] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:10:00.983528 22747 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:00.992643 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68825.4us
W20260709 13:10:00.994206 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 241270us
I20260709 13:10:01.017969 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:01.018268 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 258220us
I20260709 13:10:00.962742 24885 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:10:00.988180 24888 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:00.995879 24826 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.033562 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:01.034330 22778 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:01.035559 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:01.040669 24170 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:01.045288 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:00.992302 24889 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:10:00.989530 24641 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.009398 24582 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:01.046770 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 429665us
I20260709 13:10:01.110080 22702 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:01.079425 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.279331 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:01.153291 22778 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:01.280673 22704 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:01.096168 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:01.107817 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:01.110517 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:01.276201 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.008s	sys 0.033s
I20260709 13:10:01.285166 22784 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:01.223114 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:01.286966 22707 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:01.285826 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.008s	sys 0.033s
W20260709 13:10:01.287590 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 570818us
I20260709 13:10:01.070011 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:01.088161 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:01.112440 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:01.109760 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:01.241267 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142815us
I20260709 13:10:01.291383 22706 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.240295 24170 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:01.287670 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 293637us
W20260709 13:10:01.185874 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.293565 24170 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.171713 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:01.310249 22703 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:01.088689 24887 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:10:01.097729 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.111425 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:01.113528 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:01.332105 22701 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.116997 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.332891 22633 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:01.299346 22709 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:01.046921 24734 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:01.078359 24555 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:01.087898 24676 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.080077 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:01.082744 24463 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.023608 24699 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.345592 22782 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:01.336153 22705 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
W20260709 13:10:01.107929 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.313777 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:01.357105 22630 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:01.323741 22632 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:01.166915 24769 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:01.337483 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:01.081415 24405 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.088884 24884 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:01.385025 24884 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:01.377425 24865 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.001s
I20260709 13:10:01.386212 24884 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.387063 24865 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:01.310873 24803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.020s	sys 0.036s
I20260709 13:10:01.100571 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:01.388460 24803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.020s	sys 0.036s
I20260709 13:10:01.069453 24883 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:01.111976 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.329936 24893 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:01.389379 24883 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:01.389863 24893 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:01.390199 24883 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:01.391011 24893 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.392493 24893 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.405218 24896 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:01.407485 24767 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:01.406976 24883 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.408849 24603 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:01.409335 24603 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:01.410849 23418 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:01.089059 24890 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:01.364537 22629 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:01.411517 24890 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.411945 24890 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.412568 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.414215 22629 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:01.042255 24826 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.367134 22627 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:01.369647 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.371784 22778 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:10:01.372720 23414 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:10:01.366365 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.372951 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.374464 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:01.364535 24769 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:01.419055 22777 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:10:01.419629 22777 pending_rounds.cc:85] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 2
I20260709 13:10:01.419909 22777 pending_rounds.cc:107] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 3.3
W20260709 13:10:01.419759 22601 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.5.190:44025, user_credentials={real_user=slave}} blocked reactor thread for 55106.5us
I20260709 13:10:01.355636 24734 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:01.420439 24734 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.420204 24769 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.421787 24769 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.420187 22777 raft_consensus.cc:2889] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:01.377262 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.377804 23420 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:01.426009 22708 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:01.383842 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:01.427379 22709 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:01.427546 24890 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.428020 22630 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:01.429184 22709 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:01.312691 24450 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:01.430354 24450 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.432426 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.373010 22633 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:01.344596 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:01.410396 24603 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:01.179788 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 108261us
I20260709 13:10:01.427306 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:01.437561 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.382370 24170 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.388746 22703 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:01.449219 22707 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:01.449555 22704 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:01.439163 22783 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:01.108698 24779 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.461315 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:01.462594 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:01.463311 22629 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.444768 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:01.417032 24826 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:01.460338 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:01.472319 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.475368 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.477434 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.480916 22626 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:01.389475 24865 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.488547 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:01.489528 23421 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:01.490463 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 159214us
I20260709 13:10:01.485811 24170 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:01.494225 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.497110 22709 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:01.451797 24555 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:01.487138 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.487449 22630 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:01.481093 24885 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:10:01.462422 24405 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:10:01.483532 24770 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:10:01.511020 24770 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:01.511508 24770 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:01.512444 24770 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.513883 24770 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:01.524087 24824 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:01.525394 24641 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:01.527762 24699 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.529630 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.531430 22630 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:01.532086 24898 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:10:01.466454 24463 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 13:10:01.560011 22701 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:01.567240 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66260.9us
W20260709 13:10:01.567684 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73581.4us
W20260709 13:10:01.569545 22750 connection.cc:256] RPC call timeout handler was delayed by -1.653979872999912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:10:01.483796 24837 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:01.591454 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:01.571118 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84127.7us
I20260709 13:10:01.571949 24170 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:01.574787 22634 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:01.499627 24814 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:01.583823 24890 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:01.584844 24904 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 13:10:01.595317 24170 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:01.613678 24170 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:01.489488 24803 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 13:10:01.564599 24888 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.618486 22629 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:01.507205 24365 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:10:01.626446 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:01.627262 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 263251us
I20260709 13:10:01.630553 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:01.653712 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 59957.5us
I20260709 13:10:01.586654 24889 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:01.661257 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:01.657514 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:01.658213 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.663558 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:01.667789 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.670346 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:01.625140 24557 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:01.683239 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:01.687371 24692 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:01.687980 24865 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.667150 24699 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:01.688709 24699 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:01.689688 24692 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:01.689783 24699 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.680076 24883 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.695802 24883 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:01.696250 24883 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:01.696625 24883 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.697917 24865 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:01.657336 24767 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.043s
W20260709 13:10:01.686481 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87271us
I20260709 13:10:01.713742 22629 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:10:01.715134 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 253910us
I20260709 13:10:01.690348 24692 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:01.678669 24825 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:01.685241 24890 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:01.658601 24888 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.743211 22778 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:01.731001 24582 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:01.672107 24889 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:01.737176 24802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.075s	sys 0.083s
I20260709 13:10:01.826937 24170 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.747334 24890 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:01.829278 24170 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.830108 22630 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:01.830406 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:01.831873 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.833135 24876 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:01.836593 24876 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.837072 24876 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:01.809522 24450 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.736567 24824 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 13:10:01.817373 24779 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.834569 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:01.869279 23415 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.829264 24890 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:01.796308 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 79494.1us
W20260709 13:10:01.858786 22706 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.003s
I20260709 13:10:01.860936 24889 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:01.861835 24767 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.862134 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:01.793440 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 136021us
W20260709 13:10:01.792668 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.900391 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:01.891651 22706 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:01.897045 23415 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:01.906275 24918 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:01.906584 24405 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.906922 24918 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:01.907212 24405 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.908205 24918 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.909646 24918 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:01.909920 24929 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.910492 24929 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.913228 24929 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.913666 24865 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:01.915537 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:01.906257 24885 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:01.919652 24885 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:01.920584 24885 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.923738 24920 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.924369 24920 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:01.927557 24920 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.931327 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:01.936609 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.932667 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:01.904039 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.949337 24170 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:01.917680 24929 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:01.960053 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:01.957526 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:01.950686 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 271673us
I20260709 13:10:01.952658 24699 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:10:01.955740 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.985s (SENT) [suppressed 14 similar messages]
I20260709 13:10:01.972708 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.739682 24555 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.936522 24920 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:10:02.016686 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59222us
I20260709 13:10:02.017758 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:01.900343 24603 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:01.973892 24170 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:01.933566 24885 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:01.998574 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 330528us
I20260709 13:10:02.022378 24170 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:02.022796 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:01.953632 24405 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:01.964694 24865 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:01.975467 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.051329 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:02.052454 22708 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
W20260709 13:10:02.052459 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 96767.6us
W20260709 13:10:02.043210 22750 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.577s (SENT)
I20260709 13:10:01.964619 24929 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:02.059477 22709 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:02.062070 22709 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:01.974961 24720 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.938272 24767 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.053689 24170 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:01.938011 24734 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.026530 24727 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:02.048641 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:02.028332 24699 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:02.079838 22675 leader_election.cc:400] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:10:02.081174 22675 leader_election.cc:403] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:10:01.851135 24692 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:02.094079 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50907us
I20260709 13:10:02.095158 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:02.096004 22707 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:02.056172 24865 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.129146 22781 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:02.138186 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 73456.3us
I20260709 13:10:02.100809 24450 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:02.145851 23415 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:10:02.098322 22709 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.110908 24865 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:02.091382 24720 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.175761 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:02.167337 22750 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.953s (SENT)
I20260709 13:10:02.110338 24779 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:02.177757 24779 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.178488 24734 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.073348 24405 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
I20260709 13:10:02.182204 24938 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:02.183226 24914 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:10:02.182951 24939 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:10:02.168807 24835 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:10:02.160260 24450 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.198277 22709 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:02.199424 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:02.102591 24883 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:02.187657 24770 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.673s	user 0.017s	sys 0.000s
I20260709 13:10:02.212962 23415 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:02.208098 22707 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:10:02.184113 24769 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.762s	user 0.005s	sys 0.010s
I20260709 13:10:02.171454 24826 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:02.227192 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 186436us
I20260709 13:10:02.163256 24727 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:02.183820 24923 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:02.176929 24888 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:02.244580 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 63797.4us
I20260709 13:10:02.183141 24779 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:02.184195 24734 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.210515 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:02.257328 23421 raft_consensus.cc:2410] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:02.248222 24930 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:10:02.242497 24450 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:02.250916 24893 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.858s	user 0.017s	sys 0.000s
I20260709 13:10:02.272800 24921 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:02.273437 24917 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:10:02.273698 24720 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.287863 22709 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
W20260709 13:10:02.290158 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.322517 22709 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 1
W20260709 13:10:02.301697 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44968.3us
I20260709 13:10:02.323022 22709 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:10:02.323359 22709 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:02.267803 24931 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:02.327400 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 160112us
W20260709 13:10:02.330425 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:02.336472 24925 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:10:02.337888 24924 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
I20260709 13:10:02.336212 24928 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:10:02.263863 24826 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.291394 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:02.316367 24676 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:02.301967 24770 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.213481 24938 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.196223 24876 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.373023 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1771) took 1108 ms. Trace:
I20260709 13:10:02.245030 24929 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:02.371701 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.962s (SENT)
I20260709 13:10:02.374948 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:02.373461 22702 rpcz_store.cc:276] 0709 13:10:01.246523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.297096 (+ 50573us) service_pool.cc:224] Handling call
0709 13:10:01.324895 (+ 27799us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:01.372089 (+ 47194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:01.372131 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:01.373457 (+  1326us) log.cc:844] Serialized 520 byte log entry
0709 13:10:01.739836 (+366379us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:01.739884 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:01.739969 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:02.353857 (+613888us) raft_consensus.cc:1685] finished
0709 13:10:02.353882 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:02.354080 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22994,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":20982,"thread_start_us":365835,"threads_started":1,"wal-append.queue_time_us":525543}
I20260709 13:10:02.291774 24734 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.342787 24898 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:02.325567 24405 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:10:02.269408 24699 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.395639 24884 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:02.391271 24940 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
W20260709 13:10:02.415210 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 388592us
I20260709 13:10:02.421351 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:02.401531 24767 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.429517 22707 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:10:02.413466 24904 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.006s	sys 0.005s
I20260709 13:10:02.378608 24770 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.443285 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:02.447353 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.426587 24802 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.450024 24802 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.452056 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:02.452363 24802 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:02.452801 22629 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:02.453135 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:02.453833 22626 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:02.455132 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:02.459343 24876 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:02.431442 24898 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.462667 24770 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:02.441885 24904 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.007s	sys 0.006s
I20260709 13:10:02.464164 24883 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:02.400236 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:02.525064 22630 raft_consensus.cc:606] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Received request to transfer leadership to TS 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:02.525528 22630 raft_consensus.cc:614] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:02.436722 24928 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:02.491477 24449 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:02.496682 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 520) took 1017 ms. Trace:
I20260709 13:10:02.531471 23417 rpcz_store.cc:276] 0709 13:10:01.479601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.479747 (+   146us) service_pool.cc:224] Handling call
0709 13:10:01.506028 (+ 26281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:01.760714 (+254686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:01.760767 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:01.810155 (+ 49388us) log.cc:844] Serialized 26 byte log entry
0709 13:10:01.972952 (+162797us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:02.037328 (+ 64376us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.059773 (+ 22445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.059859 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:02.496417 (+436558us) raft_consensus.cc:1685] finished
0709 13:10:02.496443 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:02.496610 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22001,"prepare.queue_time_us":64120,"prepare.run_wall_time_us":51,"raft.queue_time_us":311553,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":9147,"spinlock_wait_cycles":5788032,"thread_start_us":333557,"threads_started":2,"wal-append.queue_time_us":576099}
I20260709 13:10:02.498430 24734 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:02.471925 24835 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:10:02.501607 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:02.517504 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.428426 24727 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.545765 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:02.548516 22702 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:02.525991 24936 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.557220 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:02.557307 24936 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.558367 24826 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:02.558302 23416 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:02.544947 24449 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:02.556449 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 51319.3us
I20260709 13:10:02.560132 24936 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.560945 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:02.545172 24734 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:02.531011 24928 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:02.564958 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:02.571434 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:02.566385 22634 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:02.573434 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:02.572171 22778 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:02.568078 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.575693 24883 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:02.556806 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66970.4us
I20260709 13:10:02.508385 24692 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:10:02.580713 22779 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:02.564232 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:02.592031 24938 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.592468 23416 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:02.593649 22779 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:10:02.549939 24893 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:02.595820 22707 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:02.597517 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:02.584901 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.574937 24928 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:02.581503 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:02.595984 22703 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:02.582217 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:02.585027 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61672.8us
I20260709 13:10:02.589552 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:02.677670 22702 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:02.560137 24449 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.577031 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:02.683878 22781 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:02.684258 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:02.625257 22779 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:10:02.587718 24920 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:02.634531 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:02.562554 24450 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:02.583009 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:02.639153 24821 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:10:02.577139 24933 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:10:02.618343 24885 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.005s
I20260709 13:10:02.674854 22777 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:10:02.709522 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 112101us
I20260709 13:10:02.675743 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:02.581187 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:02.576200 24883 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.616230 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:02.679243 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93259.3us
I20260709 13:10:02.681474 22778 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:02.721948 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:02.715250 22701 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5.
I20260709 13:10:02.553316 24735 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:02.574102 24826 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 13:10:02.595872 24921 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:02.564184 24734 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:02.683259 24950 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.232s	user 0.000s	sys 0.005s
I20260709 13:10:02.744967 22780 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 2.
I20260709 13:10:02.589970 24890 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:02.745975 24890 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.597045 24939 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:10:02.717466 22703 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:02.687620 24803 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:10:02.764127 24803 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.764919 24803 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.765130 24450 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.725293 24883 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:02.673213 24893 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:02.720510 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 110077us
I20260709 13:10:02.708822 24769 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:02.688970 24920 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.700222 24821 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:02.743285 22701 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:02.756076 24896 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.714789 24962 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:02.758714 24921 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:02.760888 24865 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.009s
I20260709 13:10:02.761998 24963 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.722183 24898 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:02.773720 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58491.7us
I20260709 13:10:02.769901 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:02.789515 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67381.6us
W20260709 13:10:02.971278 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.698s (SENT)
I20260709 13:10:03.002050 22702 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:02.995262 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.096637 22633 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:02.887343 22626 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:02.993916 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.888317 22701 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:02.852077 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:03.127941 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 358132us
W20260709 13:10:02.986310 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.128895 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:03.097988 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:02.934890 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.131753 22704 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:02.792686 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:03.130208 22628 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:03.129118 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.767027 24734 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:02.779258 24920 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:03.020293 24603 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.985s	user 0.010s	sys 0.006s
I20260709 13:10:03.181860 22626 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:03.173466 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.784832 24735 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:10:03.145655 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.173854 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.152457 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.183287 22629 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:03.172415 24931 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:03.145185 24939 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:03.054996 24893 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:03.188444 24939 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.188935 24939 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.189097 24921 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.174419 23420 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:03.176225 24946 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:03.176503 23417 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:03.190093 23416 raft_consensus.cc:2410] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:10:03.036896 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:03.116256 24957 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.348s	user 0.000s	sys 0.009s
I20260709 13:10:03.093165 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.191484 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.120112 24884 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.003993 24890 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:02.824651 24555 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.056247 24952 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.121881 24923 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:03.202800 24558 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:03.108876 24930 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:03.108901 24803 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:02.793974 24962 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.193461 24767 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:10:02.772723 24769 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:03.240553 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.241446 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.203325 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:03.162810 24963 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.209025 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.132926 24865 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:02.794503 24896 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.192093 24735 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:03.179607 24365 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:10:02.939396 24821 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.263509 24821 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.265064 24883 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.230466 24923 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:03.260488 24896 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:03.267545 24923 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.268704 24923 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:03.216220 24884 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.273101 24884 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.213436 24734 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:03.204523 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 233315us
I20260709 13:10:03.277354 22631 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:03.205377 24893 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:10:03.240962 24962 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:03.246307 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.244449 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.236281 24952 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:03.254379 24928 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.009s
I20260709 13:10:03.255559 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.300928 22703 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:03.255460 24769 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.257829 24865 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.229099 24555 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.248878 24170 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:03.260440 24963 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.260933 24949 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:03.261145 24735 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.229435 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.278040 22702 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:03.310850 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:03.311707 22702 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:03.229455 24930 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:03.276612 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94812.7us
I20260709 13:10:03.325136 24930 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.278772 24558 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:10:03.303279 24884 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.327723 24884 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.286743 24893 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:03.330890 24893 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:03.326350 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.336925 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.334604 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.343120 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:03.345582 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.346166 23420 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:03.347771 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.349466 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.350690 22779 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:03.352470 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.353420 22781 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:03.347618 24769 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.354954 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.355846 22709 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:03.357331 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.326438 24930 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:03.363951 22632 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:03.304229 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 82643.5us
W20260709 13:10:03.306062 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59900.2us
I20260709 13:10:03.307062 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:03.365523 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61041.4us
I20260709 13:10:03.319175 24883 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.306856 24921 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:03.303732 24928 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.367457 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.368438 22776 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:03.369517 22634 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:03.369987 24959 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.613s	user 0.000s	sys 0.010s
I20260709 13:10:03.366518 24949 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:03.371307 24949 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.320394 24735 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:03.374888 22707 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
I20260709 13:10:03.376296 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1790) took 1346 ms. Trace:
I20260709 13:10:03.376436 22707 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:10:03.377837 24555 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.297254 24961 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:10:03.322253 24952 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.376752 22705 rpcz_store.cc:276] 0709 13:10:02.029671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.031548 (+  1877us) service_pool.cc:224] Handling call
0709 13:10:02.032117 (+   569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:02.439388 (+407271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.439525 (+   137us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:02.468221 (+ 28696us) log.cc:844] Serialized 52 byte log entry
0709 13:10:02.535885 (+ 67664us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:02.663931 (+128046us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.666601 (+  2670us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.668555 (+  1954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.668671 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.268623 (+599952us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18002c0140. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:03.369273 (+100650us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:03.376075 (+  6802us) raft_consensus.cc:1685] finished
0709 13:10:03.376100 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.376234 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6557,"prepare.queue_time_us":157938,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":137754,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":8610,"spinlock_wait_cycles":169536000,"thread_start_us":511883,"threads_started":3,"wal-append.queue_time_us":242809}
I20260709 13:10:03.283378 24734 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.381271 22703 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:03.382539 22705 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:10:03.383193 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.386024 22633 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:03.387627 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.388500 22628 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:03.392062 22703 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:10:03.392541 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 185226us
I20260709 13:10:03.394910 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.395373 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:03.332545 24893 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.396837 22777 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:10:03.396025 22705 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:03.398276 22703 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:10:03.398277 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.403581 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.405784 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.406440 22777 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 1.
I20260709 13:10:03.408447 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:03.410924 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.401947 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.413609 22776 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:03.418298 22702 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:03.419492 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.420457 22776 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:03.402573 24961 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:03.424376 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.425274 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.426255 22776 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:03.426368 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.414824 24940 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:03.430065 22675 connection.cc:256] RPC call timeout handler was delayed by -2.9555839430000788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 13:10:03.430557 22675 connection.cc:256] RPC call timeout handler was delayed by -2.94388394300006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 13:10:03.423429 24884 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:03.430925 22675 connection.cc:256] RPC call timeout handler was delayed by -2.9258839430000307s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:10:03.431304 22675 connection.cc:256] RPC call timeout handler was delayed by -2.9096839430000045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 13:10:03.431624 22675 connection.cc:256] RPC call timeout handler was delayed by -2.8988839430001008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
I20260709 13:10:03.432487 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:03.431936 22675 connection.cc:256] RPC call timeout handler was delayed by -2.8943839430000935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 13:10:03.433028 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:03.434803 22675 connection.cc:256] RPC call timeout handler was delayed by -2.8016839430000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 13:10:03.421566 24961 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:03.435586 24170 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45654 (request call id 84) took 14893 ms. Trace:
W20260709 13:10:03.435269 22675 connection.cc:256] RPC call timeout handler was delayed by -2.7980839430000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 13:10:03.438743 24170 rpcz_store.cc:276] 0709 13:09:48.539080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.793717 (+254637us) service_pool.cc:224] Handling call
0709 13:09:49.710774 (+917057us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4c0010d2c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18051a9a20 00007f18051baa29 00007f1805180644 00007f18051a7f2d 00007f18051a7b93 00007f1806e361af 00007f1806e35f3e 00007f1806ebb888 00007f1806ed875c 00007f1806ed976a 00007f1806ed891b
0709 13:09:49.712780 (+  2006us) log.cc:844] Serialized 494 byte log entry
0709 13:09:49.807556 (+ 94776us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b540006d6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee3735 00007f1806edb9fa 00007f180bea34c9 00007f180541fc54 00007f180541fbe2 00007f180541fb11 00007f180541fa8c 00007f180541ed42 00007f180521848c 00007f18052178c6
0709 13:09:51.733815 (+1926259us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b540042a7a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e56142 00007f1806e36eaa 00007f1806e3dd53 00007f1806e3dcd9 00007f1806e3dc69 00007f1806e3dc31 00007f1806e3cdfd 000000000038f354 000000000038f199 00007f1803df0475
0709 13:10:03.428291 (+11694476us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":856034,"raft.queue_time_us":12392,"raft.run_cpu_time_us":6719,"raft.run_wall_time_us":1931214,"spinlock_wait_cycles":1519313024}
I20260709 13:10:03.440472 24961 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.432530 24963 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:10:03.427642 24983 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:10:03.432188 24981 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:10:03.449093 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f33fee9d64384b499e55f8bc58339f69 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 13:10:03.450372 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.450395 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.461699 22779 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:10:03.462396 24923 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.463199 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.465327 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:03.466333 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:03.466970 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:03.439565 22675 connection.cc:256] RPC call timeout handler was delayed by -2.7890839430000369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 13:10:03.468076 22675 connection.cc:256] RPC call timeout handler was delayed by -2.7701839430000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
W20260709 13:10:03.468464 22675 connection.cc:256] RPC call timeout handler was delayed by -2.663983943000062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:10:03.468786 22675 connection.cc:256] RPC call timeout handler was delayed by -2.5946839430000637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:10:03.469089 22675 connection.cc:256] RPC call timeout handler was delayed by -2.5442839430000959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19990000000000002s for a response
I20260709 13:10:03.461807 22776 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:10:03.469389 22675 connection.cc:256] RPC call timeout handler was delayed by -2.5334839430000784s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
W20260709 13:10:03.469760 22675 connection.cc:256] RPC call timeout handler was delayed by -2.4983839430000216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
I20260709 13:10:03.467268 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:03.462981 24923 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.472803 24980 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:03.473214 24980 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:03.473563 24980 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:03.474718 24923 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.483553 23421 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:10:03.483008 22750 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.928s (SENT)
I20260709 13:10:03.487949 22626 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:03.489428 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.470335 24921 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.491012 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:03.470043 22675 connection.cc:256] RPC call timeout handler was delayed by -2.4785839430001033s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:10:03.493144 22675 connection.cc:256] RPC call timeout handler was delayed by -2.3867839430000686s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2174s for a response
I20260709 13:10:03.492951 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.493870 22777 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:03.489809 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:03.493544 22675 connection.cc:256] RPC call timeout handler was delayed by -2.38138394300006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21800000000000003s for a response
I20260709 13:10:03.495615 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:03.495869 22675 connection.cc:256] RPC call timeout handler was delayed by -2.3687839430000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2194s for a response
I20260709 13:10:03.495992 22627 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:03.496205 22675 connection.cc:256] RPC call timeout handler was delayed by -2.3552839430000176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 13:10:03.496786 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8332839430000831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27890000000000004s for a response
I20260709 13:10:03.497448 22675 leader_election.cc:400] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:10:03.436623 24920 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.501346 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.483786 24365 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:03.507002 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.507297 24365 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.478874 24981 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:03.502481 24170 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:03.504027 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:03.507906 22628 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:03.490474 24921 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.513011 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.514504 22777 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.490852 24983 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:10:03.516563 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.517151 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.517365 24921 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.511994 24981 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.518412 24981 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:03.515010 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.507905 24365 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.500662 24979 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:03.468307 24940 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.533502 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.534684 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:03.535007 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.528822 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.503337 24727 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:10:03.537447 22776 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:03.518819 24734 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:03.527846 24938 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:03.547385 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.530745 24837 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:03.551929 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.552398 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.544961 24982 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:10:03.555248 24939 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.533390 24979 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.559980 22627 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:03.567566 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.567566 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:03.568399 22628 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.571050 24988 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:03.550351 24929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.015s	sys 0.026s
I20260709 13:10:03.561609 24926 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:03.563313 24837 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.577586 24920 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.574851 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.580874 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:03.582319 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.585516 22784 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:03.586508 22627 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:03.587399 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:03.587740 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.588517 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:03.588789 22627 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.589252 23420 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:03.590735 24915 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.591377 24915 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.593263 24940 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.593631 24915 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.595327 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.596807 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.597851 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1830) took 1020 ms. Trace:
I20260709 13:10:03.599208 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.599998 22784 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:03.601162 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.601671 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.601799 22784 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:03.602361 24365 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.603122 24961 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.604871 24365 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.607056 22778 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:03.605829 22631 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:03.607227 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.598266 22782 rpcz_store.cc:276] 0709 13:10:02.577353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.592388 (+ 15035us) service_pool.cc:224] Handling call
0709 13:10:02.592760 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.593165 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.593195 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:03.436870 (+843675us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:03.437196 (+   326us) log.cc:844] Serialized 26 byte log entry
0709 13:10:03.498858 (+ 61662us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:03.498917 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.499012 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.597496 (+ 98484us) raft_consensus.cc:1685] finished
0709 13:10:03.597519 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.597779 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2162,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":19975,"spinlock_wait_cycles":5568768,"thread_start_us":14360,"threads_started":1,"wal-append.queue_time_us":66538}
I20260709 13:10:03.608470 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:03.575685 24929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.016s	sys 0.026s
W20260709 13:10:03.624632 24956 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.950s	user 0.002s	sys 0.011s
I20260709 13:10:03.570210 24979 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.632365 22628 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:03.617739 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.627074 24938 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:03.612267 24940 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:03.615083 24889 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.612936 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:03.616184 24944 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.162s	user 0.006s	sys 0.008s
I20260709 13:10:03.614116 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:03.617516 24824 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:03.610144 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:03.589902 24837 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.619200 24893 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:03.647583 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.674926 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 78184us
I20260709 13:10:03.701879 24824 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.678210 22777 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:03.681734 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:03.683604 22631 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:03.684862 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67232.3us
I20260709 13:10:03.560101 24980 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.689370 22627 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:03.719949 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1794) took 1670 ms. Trace:
I20260709 13:10:03.693058 22630 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:03.693621 23415 raft_consensus.cc:2368] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:03.628841 24926 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:03.724275 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:03.706600 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 96520.8us
I20260709 13:10:03.677626 22784 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:03.728433 22747 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.678876 24918 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:03.723357 22628 raft_consensus.cc:606] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:03.735045 22784 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:03.700073 24692 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:03.736152 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.724073 24824 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:03.724916 22708 rpcz_store.cc:276] 0709 13:10:02.049795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.278017 (+228222us) service_pool.cc:224] Handling call
0709 13:10:02.278469 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.369230 (+ 90761us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:02.369277 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:02.403258 (+ 33981us) log.cc:844] Serialized 26 byte log entry
0709 13:10:02.465246 (+ 61988us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:02.465291 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.466227 (+   936us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.623672 (+1157445us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:03.719478 (+ 95806us) raft_consensus.cc:1685] finished
0709 13:10:03.719510 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.719864 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49003,"raft.queue_time_us":51694,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":144,"spinlock_wait_cycles":5601536,"thread_start_us":31269,"threads_started":1,"wal-append.queue_time_us":135696}
I20260709 13:10:03.708343 24365 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.728979 23415 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:03.737463 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.742691 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.744133 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1785) took 1727 ms. Trace:
W20260709 13:10:03.747236 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.747776 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.748095 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.747238 22710 rpcz_store.cc:276] 0709 13:10:02.016476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.034920 (+ 18444us) service_pool.cc:224] Handling call
0709 13:10:02.042180 (+  7260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.108825 (+ 66645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.108861 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:02.142674 (+ 33813us) log.cc:844] Serialized 26 byte log entry
0709 13:10:02.317932 (+175258us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:02.318010 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.318149 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.615155 (+1297006us) spinlock_profiling.cc:243] Waited 9.97 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:03.740585 (+125430us) raft_consensus.cc:1685] finished
0709 13:10:03.740612 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.744036 (+  3424us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8125,"raft.queue_time_us":135885,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"spinlock_wait_cycles":22941056,"thread_start_us":142125,"threads_started":2,"wal-append.queue_time_us":247560}
I20260709 13:10:03.745507 24692 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.746021 24962 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.710660 24734 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:03.756254 22635 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:03.736781 24938 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:03.757674 22747 leader_election.cc:400] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:03.761889 22747 leader_election.cc:403] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:03.762701 24962 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.763113 24582 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.764185 24975 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:03.764688 24975 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.765058 24975 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:03.759853 22675 leader_election.cc:400] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:03.734058 24929 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.729189 24918 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:03.768445 22675 leader_election.cc:403] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:03.704789 24889 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.776682 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.733880 24735 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:10:03.784174 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1832) took 1201 ms. Trace:
I20260709 13:10:03.769172 24977 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:03.760524 24952 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:03.805392 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.806363 22784 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:03.807590 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.808043 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.781316 24938 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:03.767313 24926 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.787011 22783 rpcz_store.cc:276] 0709 13:10:02.577940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.593532 (+ 15592us) service_pool.cc:224] Handling call
0709 13:10:02.593887 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.594280 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.594310 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:03.482907 (+888597us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:03.494972 (+ 12065us) log.cc:844] Serialized 26 byte log entry
0709 13:10:03.587976 (+ 93004us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:03.588021 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.588097 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.768436 (+180339us) raft_consensus.cc:1685] finished
0709 13:10:03.768465 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.779427 (+ 10962us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22942,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":148,"spinlock_wait_cycles":3569408,"thread_start_us":68578,"threads_started":1,"wal-append.queue_time_us":237349}
I20260709 13:10:03.813885 24920 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:03.761171 24365 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:03.821988 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62298.2us
I20260709 13:10:03.789920 24977 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.827793 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:03.828634 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:03.828845 22783 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:03.833473 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:03.817770 24920 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.777488 24929 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.828770 24977 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:03.879426 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 46077.6us
I20260709 13:10:03.901734 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:03.894083 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.894513 25000 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:10:03.895839 24980 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.835232 24734 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:10:03.849167 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.932747 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.934005 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.836374 24939 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.923825 24981 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:03.925485 24692 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:03.899397 24938 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:03.827493 24665 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:03.927980 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:03.967654 22783 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:03.928246 24365 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:03.836961 24918 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:03.843500 24920 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:03.983350 25001 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:10:03.836912 24908 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
W20260709 13:10:03.969735 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 128655us
I20260709 13:10:03.992944 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:03.970247 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68664.5us
I20260709 13:10:03.994674 22627 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:10:03.973680 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:03.997242 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.002022 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.942584 24734 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:04.023895 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.969177 24952 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:04.031809 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:04.043848 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.047602 24889 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:04.024955 24734 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:04.034402 24918 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:04.060631 24929 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:04.061807 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.069085 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:04.100168 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:04.017946 24940 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:04.025163 24939 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:10:04.123034 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54096.2us
I20260709 13:10:04.124933 22775 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:04.143045 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:04.143699 24981 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.143965 22775 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:04.026079 25008 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:10:04.144842 25008 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:04.145208 25008 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:04.119513 24939 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.146409 24939 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:04.146148 25008 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.131115 24952 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:04.147395 25008 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.095453 24938 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:04.126973 24940 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.152834 24940 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:04.154558 22779 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:10:04.155947 24979 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.097095 24929 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.153823 24837 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.160404 22627 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:04.165669 24961 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.005s
I20260709 13:10:04.158527 24979 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.166397 22783 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:04.167537 24837 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:04.170033 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:04.095574 24966 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.177258 22627 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:10:04.127935 24582 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:04.188647 24966 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.125514 24977 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.192881 24977 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:04.160218 22675 leader_election.cc:400] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:04.193452 24966 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:04.193583 22675 leader_election.cc:403] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:04.194974 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:04.195130 24975 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.196286 24938 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:04.164129 24824 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.196401 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:04.187242 24929 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.197458 24938 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:04.198084 24938 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:04.198573 24824 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:04.199774 24929 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:04.199491 24938 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.162516 24734 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.202723 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:04.204784 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:04.211292 22627 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:04.167637 24940 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.220614 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:04.222126 22630 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:04.222848 22783 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:04.220642 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:04.225728 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 280475us
I20260709 13:10:04.197482 24975 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:04.211814 24929 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:04.236891 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:04.253091 22627 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:04.185755 24920 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.246769 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 1910) took 1252 ms. Trace:
I20260709 13:10:04.219558 24966 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:10:04.224100 24952 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:04.253723 22635 rpcz_store.cc:276] 0709 13:10:02.994379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.994600 (+   221us) service_pool.cc:224] Handling call
0709 13:10:04.246441 (+1251841us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b540022e054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:04.246652 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446487808}
I20260709 13:10:04.220918 24824 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:10:04.238265 24582 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:04.275772 22775 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:04.222126 24940 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.307612 24940 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.310082 24939 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.311682 22778 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:04.312539 24939 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.307662 24920 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.314726 24734 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:04.317544 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:04.318312 22635 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:04.271806 24555 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:10:04.263615 24558 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:10:04.294166 24582 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.321100 24582 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.322165 24928 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.323114 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 528) took 1611 ms. Trace:
I20260709 13:10:04.323529 23421 rpcz_store.cc:276] 0709 13:10:02.711418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.716648 (+  5230us) service_pool.cc:224] Handling call
0709 13:10:02.733272 (+ 16624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:03.484008 (+750736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.322810 (+838802us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54005e2254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:04.323043 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713795328}
I20260709 13:10:04.324493 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1797) took 1137 ms. Trace:
I20260709 13:10:04.324889 22709 rpcz_store.cc:276] 0709 13:10:03.186967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.358540 (+171573us) service_pool.cc:224] Handling call
0709 13:10:04.323706 (+965166us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:10:04.323940 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.324138 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.324165 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:04.324235 (+    70us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:04.324269 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.324328 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:04.324436 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13753728}
I20260709 13:10:04.255452 24979 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:10:04.328333 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 275) took 1174 ms. Trace:
I20260709 13:10:04.328848 22632 rpcz_store.cc:276] 0709 13:10:03.154047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.154175 (+   128us) service_pool.cc:224] Handling call
0709 13:10:03.179726 (+ 25551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:03.364614 (+184888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.328009 (+963395us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400070654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:04.328226 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423054080}
W20260709 13:10:04.328557 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.255911 24929 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.335405 24929 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:04.335628 22780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 245) took 1764 ms (client timeout 2045 ms). Trace:
W20260709 13:10:04.336201 22780 rpcz_store.cc:269] 0709 13:10:02.571285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.581949 (+ 10664us) service_pool.cc:224] Handling call
0709 13:10:04.335403 (+1753454us) spinlock_profiling.cc:243] Waited 67.1 ms on lock 0x7b540033c554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:04.335532 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154293632}
W20260709 13:10:04.344671 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 174776us
W20260709 13:10:04.357091 25002 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.500s	user 0.009s	sys 0.006s
I20260709 13:10:04.375059 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:04.375685 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.375851 22629 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:04.376233 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.376287 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:04.376667 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.376945 22782 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.377247 25013 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.377905 25013 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.379086 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.379760 22780 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:04.381345 25013 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.346283 22708 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:04.383484 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.346253 22703 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:04.360864 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.355093 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.386062 22701 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.387645 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.388127 22626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1917) took 1006 ms. Trace:
I20260709 13:10:04.388571 22784 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:04.388541 22626 rpcz_store.cc:276] 0709 13:10:03.381808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.382371 (+   563us) service_pool.cc:224] Handling call
0709 13:10:03.391219 (+  8848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:03.488499 (+ 97280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.387868 (+899369us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b540006b3d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:04.388063 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":221953408}
I20260709 13:10:04.389154 22629 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:04.345077 22710 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:04.391476 24975 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:04.391942 24975 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:04.392980 24975 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.394280 24975 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:04.396482 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:04.346359 22705 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:10:04.318481 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.404505 24966 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:04.405057 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.405915 22709 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:04.406529 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1911) took 1311 ms. Trace:
I20260709 13:10:04.405054 24966 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:04.371603 24449 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:04.407759 24449 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:04.408253 24966 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.409125 22701 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:04.409960 24966 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:04.419329 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.406978 22634 rpcz_store.cc:276] 0709 13:10:03.095078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.095298 (+   220us) service_pool.cc:224] Handling call
0709 13:10:03.110105 (+ 14807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:03.369954 (+259849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.406261 (+1036307us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b540022bd54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:04.406461 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":596940672}
I20260709 13:10:04.424135 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:04.425747 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.408741 24449 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:04.430994 25007 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.276s	user 0.004s	sys 0.000s
I20260709 13:10:04.435057 22635 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:04.439126 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.440168 22776 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:04.441929 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.442930 22784 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:10:04.459574 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 202931us
I20260709 13:10:04.485852 22782 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:04.432608 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:04.489883 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57350.7us
I20260709 13:10:04.490604 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:04.491357 22635 raft_consensus.cc:606] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:04.491503 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:04.492228 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.503157 22780 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:04.507038 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:04.508088 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:04.515553 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 113212us
I20260709 13:10:04.518273 22709 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
W20260709 13:10:04.518924 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.519114 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.520897 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.522379 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.523936 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:04.524834 23420 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:04.529731 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:04.532243 24170 raft_consensus.cc:993] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:10:04.532886 24170 raft_consensus.cc:1081] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:10:04.542182 22701 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:04.544044 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:04.550213 25022 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.550774 24449 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.550972 25022 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.553555 25022 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:04.505805 24735 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.526378 24930 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:10:04.555063 24735 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.558593 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1924) took 1147 ms. Trace:
I20260709 13:10:04.559033 22631 rpcz_store.cc:276] 0709 13:10:03.411173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.726857 (+315684us) service_pool.cc:224] Handling call
0709 13:10:03.741572 (+ 14715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:03.836186 (+ 94614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:03.836219 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:03.836770 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:10:04.093498 (+256728us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:04.093558 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.364552 (+270994us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b540022e554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:04.364622 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:04.424927 (+ 60305us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:04.558371 (+133444us) raft_consensus.cc:1685] finished
0709 13:10:04.558396 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:04.558533 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2115,"raft.queue_time_us":1786,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":22761,"spinlock_wait_cycles":225044096,"thread_start_us":251586,"threads_started":1,"wal-append.queue_time_us":280254}
I20260709 13:10:04.557659 24735 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:04.561600 24926 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:04.522554 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:04.566829 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44391us
I20260709 13:10:04.569869 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:04.571990 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.605195 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:04.605746 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 322863us
I20260709 13:10:04.608809 22779 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:10:04.581707 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.575867 22776 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:04.614270 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103240us
I20260709 13:10:04.615367 24963 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:04.615680 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.616324 22705 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:04.616750 24900 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:10:04.618261 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:04.618597 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112932us
I20260709 13:10:04.616098 24928 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:04.620267 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:04.621798 24928 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:04.622366 24928 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:04.545771 24920 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:04.545905 25013 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:04.545475 24603 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:04.623677 24928 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.625064 24603 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:04.625876 24918 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:04.626107 24603 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.489348 24952 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:04.627548 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.632315 22701 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:04.622207 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:04.648600 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:04.591826 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.584529 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.530977 24943 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:04.653331 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.526539 25010 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.133s
W20260709 13:10:04.661485 22750 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.561s (SENT)
I20260709 13:10:04.668193 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.669961 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.604136 24930 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:04.673898 22701 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:10:04.676795 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63734.5us
I20260709 13:10:04.707669 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.708751 22701 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:04.709897 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.517357 24982 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.138s
W20260709 13:10:04.712230 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50797us
W20260709 13:10:04.713106 22750 leader_election.cc:336] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.571s (SENT)
I20260709 13:10:04.713605 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:04.705569 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.718519 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.660171 25022 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:04.737625 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 82588.8us
I20260709 13:10:04.722772 22701 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:04.671270 25008 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.698242 24999 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:10:04.665519 24824 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.757805 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:04.745610 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.735235 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
W20260709 13:10:04.766074 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53004.4us
I20260709 13:10:04.582847 24582 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.773656 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.779587 22709 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:04.785861 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.788713 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.790468 22710 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 1.
I20260709 13:10:04.742682 24735 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:04.792147 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:04.796504 22750 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.701s (SENT)
I20260709 13:10:04.802445 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:04.756871 24939 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:04.807876 22701 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:04.809340 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.812975 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.813889 22701 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:04.762945 25022 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:04.816399 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.738785 24463 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:10:04.824412 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 254605us
I20260709 13:10:04.836254 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:04.819514 22750 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.784s (SENT)
I20260709 13:10:04.846832 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:04.780277 24734 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:04.836998 22710 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3. Re-sending same reply.
I20260709 13:10:04.892516 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.790848 24938 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:04.898032 22635 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:10:04.893472 22709 raft_consensus.cc:2368] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:04.765636 25008 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:04.899983 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:04.823634 24944 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:10:04.865087 25011 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:10:04.924875 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.925791 22710 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:04.806726 25021 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:04.939296 22705 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:04.944303 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.954018 22701 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:04.956336 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.957667 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:04.948527 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.959532 22709 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:04.779387 24824 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.965098 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:04.899269 24938 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:04.918598 25020 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.954981 24954 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.970983 25026 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 13:10:04.918879 24734 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:04.974891 25022 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:04.966920 25001 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:04.915330 24930 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:04.979293 25025 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:10:04.904150 25017 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:04.947834 24929 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.004s
I20260709 13:10:04.987166 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:05.028975 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 209263us
W20260709 13:10:05.030229 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.879s (SENT)
I20260709 13:10:05.033560 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:05.024485 25028 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
W20260709 13:10:04.987054 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.828s (SENT)
I20260709 13:10:04.801023 24952 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:04.997427 22708 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.614s	user 0.016s	sys 0.000s
I20260709 13:10:05.001120 22710 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:05.056039 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:05.056960 22710 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:04.996500 24954 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:04.991439 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.984822 24938 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:04.988011 25020 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.058966 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:05.000130 24943 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:05.052099 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 97784.3us
W20260709 13:10:05.091909 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 104892us
W20260709 13:10:05.092278 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:05.093132 23385 leader_election.cc:336] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.985s (SENT)
I20260709 13:10:05.044977 25017 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.098580 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.044585 24929 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.100062 22702 raft_consensus.cc:2410] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:10:05.100765 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 447686us
I20260709 13:10:05.059957 25040 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.069278 25029 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.107152 22709 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:05.116917 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:05.117928 23415 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:05.090430 22710 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:10:05.091604 25019 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
W20260709 13:10:05.132534 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.139698 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:05.140717 23415 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:05.105281 24582 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:05.219769 25002 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 13:10:05.221242 25001 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:10:05.247011 24734 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:05.248806 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:05.250468 23412 raft_consensus.cc:2955] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 3 to 5, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:05.253031 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:05.253787 23417 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:05.256357 24943 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.260828 25039 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:05.265923 23417 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:05.251953 24824 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:05.269434 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.262349 25039 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.287405 23418 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:10:05.287997 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:05.288704 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:05.290671 22702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:10:05.291087 24977 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:05.296356 23419 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:05.296685 22702 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:10:05.318761 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.232s (SENT)
W20260709 13:10:05.338362 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49769.1us
I20260709 13:10:05.341580 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:05.345870 22702 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:05.338032 25027 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:10:05.293392 24939 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:05.347483 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:05.350077 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:05.352469 24966 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.942s	user 0.013s	sys 0.003s
I20260709 13:10:05.380280 23415 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
W20260709 13:10:05.305076 25035 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:10:05.110791 24930 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:05.369010 25019 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 13:10:05.453428 24975 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.036s	user 0.010s	sys 0.005s
W20260709 13:10:05.456926 25034 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:10:05.423126 24449 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:10:05.295679 25047 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.338699 24977 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:05.459646 24977 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.461056 24977 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:05.461877 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1822) took 1114 ms. Trace:
I20260709 13:10:05.369544 24982 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:05.462566 24982 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.463141 24982 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:05.464639 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:05.464941 22781 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:05.465435 22709 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:05.126400 25029 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.469291 25029 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.471200 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1870) took 1017 ms. Trace:
I20260709 13:10:05.471637 24939 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.359364 25008 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:05.122488 25040 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.471781 22783 rpcz_store.cc:276] 0709 13:10:04.454267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.457134 (+  2867us) service_pool.cc:224] Handling call
0709 13:10:04.541594 (+ 84460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.583254 (+ 41660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.583288 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:04.657464 (+ 74176us) log.cc:844] Serialized 26 byte log entry
0709 13:10:05.103221 (+445757us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:05.103260 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.103328 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:05.470877 (+367549us) raft_consensus.cc:1685] finished
0709 13:10:05.470910 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.471107 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5163,"raft.queue_time_us":41536,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":20327,"thread_start_us":352457,"threads_started":1,"wal-append.queue_time_us":592004}
I20260709 13:10:05.474627 25040 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.477626 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1821) took 1132 ms. Trace:
I20260709 13:10:05.478075 22706 rpcz_store.cc:276] 0709 13:10:04.345614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.346554 (+   940us) service_pool.cc:224] Handling call
0709 13:10:04.346894 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.651709 (+304815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.651749 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:04.664552 (+ 12803us) log.cc:844] Serialized 26 byte log entry
0709 13:10:05.102273 (+437721us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:05.102317 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.102393 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:05.477396 (+375003us) raft_consensus.cc:1685] finished
0709 13:10:05.477419 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.477563 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12073,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":638045,"threads_started":2,"wal-append.queue_time_us":706143}
I20260709 13:10:05.480036 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.462306 22707 rpcz_store.cc:276] 0709 13:10:04.347900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.348017 (+   117us) service_pool.cc:224] Handling call
0709 13:10:04.348329 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.515110 (+166781us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.515158 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:04.515712 (+   554us) log.cc:844] Serialized 26 byte log entry
0709 13:10:04.753601 (+237889us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:04.753669 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.753755 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:05.461631 (+707876us) raft_consensus.cc:1685] finished
0709 13:10:05.461657 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.461814 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10959,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":49,"thread_start_us":403765,"threads_started":2,"wal-append.queue_time_us":441724}
I20260709 13:10:05.482952 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:05.344846 25039 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.376137 24692 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:05.472350 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.418915 22710 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:05.481212 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.131108 25038 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.289s
I20260709 13:10:05.483057 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:05.481408 22706 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:05.485692 22635 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:05.487795 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.489012 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:05.466337 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.492092 22709 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:05.492365 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.493424 22706 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:05.488534 22707 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:05.495694 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.498777 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.499789 22706 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:05.500823 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.502240 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.502972 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:05.505889 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.507401 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.505584 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.510129 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.511010 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.512967 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.516321 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.490020 23415 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:05.565022 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:05.491392 24962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.016s	sys 0.009s
I20260709 13:10:05.573328 22702 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.486965 25017 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.575541 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.498066 24692 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:05.583897 24920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.013s	sys 0.013s
W20260709 13:10:05.596593 24920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.973s	user 0.013s	sys 0.013s
I20260709 13:10:05.596925 24920 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:05.596880 24692 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.598170 24920 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.511301 24449 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:05.581255 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.583777 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.638350 22709 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:05.561770 25040 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:05.487581 25047 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.582024 25038 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:05.561400 24665 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:10:05.576601 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:05.644862 22707 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:05.645365 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0102701079999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:10:05.646021 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.624082 22706 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:05.648167 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.649498 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.624323 22702 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.638350 22710 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:05.647228 22707 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:05.651019 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:05.652208 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:05.575961 24962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.016s	sys 0.009s
I20260709 13:10:05.659740 22706 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:05.606521 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:05.586724 24918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.004s	sys 0.026s
I20260709 13:10:05.652807 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:05.654738 23414 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:05.651580 24449 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:05.670279 22707 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:05.675443 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 163159us
I20260709 13:10:05.678165 22709 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:05.640450 25040 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.679798 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:05.668761 24918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.004s	sys 0.027s
I20260709 13:10:05.667917 25015 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:05.694495 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:05.697081 22706 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:10:05.697983 22709 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:05.699641 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:05.685197 25040 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:05.704777 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:05.707819 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:05.711860 22706 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:05.712380 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:05.713690 22706 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:05.714190 22706 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:05.714359 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:05.660101 25038 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.710398 24665 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:05.696353 25042 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:10:05.704115 25061 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:05.709991 25005 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:05.702977 24937 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:10:05.717162 24983 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:10:05.729101 25054 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:10:05.717610 24928 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:05.729212 22706 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.737398 24665 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:05.754725 22675 leader_election.cc:400] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
I20260709 13:10:05.767093 22675 leader_election.cc:403] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:10:05.754246 22703 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:05.761045 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.751173 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1817) took 1406 ms. Trace:
I20260709 13:10:05.702374 25015 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.769069 22706 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:05.769068 22708 rpcz_store.cc:276] 0709 13:10:04.344403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.345541 (+  1138us) service_pool.cc:224] Handling call
0709 13:10:04.345914 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.346134 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:05.054557 (+708423us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.718660 (+664103us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.750581 (+ 31921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:05.750702 (+   121us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:05.750771 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.750868 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.751082 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29997,"prepare.queue_time_us":2562,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":144548,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"spinlock_wait_cycles":3366784,"thread_start_us":660758,"threads_started":2,"wal-append.queue_time_us":535278}
I20260709 13:10:05.774760 22703 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:05.776050 24665 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:05.786482 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72236.4us
I20260709 13:10:05.768095 24983 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:05.790683 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:05.778378 25072 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6.
W20260709 13:10:05.789049 24938 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.694s	user 0.014s	sys 0.001s
I20260709 13:10:05.793048 25069 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:10:05.793497 25059 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:10:05.786028 24665 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.771847 24928 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:05.810014 24983 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.812564 25031 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:05.813071 25031 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:05.815229 25062 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:10:05.791155 24982 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.801375 24938 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:05.818216 25056 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 13:10:05.739390 24449 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:05.799369 25061 raft_consensus.cc:2764] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:10:05.799398 22706 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:10:05.863464 22706 pending_rounds.cc:85] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:10:05.863623 22629 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:05.863792 22706 pending_rounds.cc:107] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:10:05.864051 22706 raft_consensus.cc:2889] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:05.802631 25069 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.795135 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:05.864784 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 43840.3us
W20260709 13:10:05.842949 25063 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
W20260709 13:10:05.866889 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59455.5us
I20260709 13:10:05.858800 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.850759 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1869) took 1399 ms. Trace:
I20260709 13:10:05.868757 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:05.853602 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.855278 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 1826) took 1506 ms. Trace:
W20260709 13:10:05.856096 25065 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.085s	user 0.000s	sys 0.002s
I20260709 13:10:05.857964 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.859928 22707 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:05.861143 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:05.850344 22781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 1861) took 1455 ms (client timeout 1916 ms). Trace:
W20260709 13:10:05.865916 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70861.4us
I20260709 13:10:05.843663 24450 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:05.870725 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.872076 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:05.872236 22704 rpcz_store.cc:276] 0709 13:10:04.348987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.383168 (+ 34181us) service_pool.cc:224] Handling call
0709 13:10:05.818758 (+1435590us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400087d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:05.855202 (+ 36444us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3103333120}
I20260709 13:10:05.868637 22777 rpcz_store.cc:276] 0709 13:10:04.452292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.452767 (+   475us) service_pool.cc:224] Handling call
0709 13:10:04.453265 (+   498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.453602 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:04.714326 (+260724us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.842594 (+1128268us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b1800261dc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:10:05.850258 (+  7664us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:05.850365 (+   107us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:05.850409 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.850539 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.850699 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90928,"prepare.queue_time_us":357508,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":26555,"spinlock_wait_cycles":33442560,"thread_start_us":329086,"threads_started":2,"wal-append.queue_time_us":355943}
I20260709 13:10:05.861225 24734 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:05.874815 25045 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:05.814030 25031 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.875348 25038 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.918676 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:05.968449 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.062731 25064 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.363s	user 0.002s	sys 0.000s
I20260709 13:10:06.054805 25046 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:06.063241 23412 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:10:06.062485 22634 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:10:06.065248 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.920270 22703 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:05.918982 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:06.085744 23415 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:06.076877 24450 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:06.089721 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:05.918527 25015 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.069802 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:06.060796 22781 rpcz_store.cc:269] 0709 13:10:04.395050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.425458 (+ 30408us) service_pool.cc:224] Handling call
0709 13:10:05.835229 (+1409771us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400326ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:05.850256 (+ 15027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364041472}
I20260709 13:10:06.103317 25031 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:05.878072 24982 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:05.870914 25069 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:06.089934 24954 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.099786 24450 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.983850 24938 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.077522 25072 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.078855 24952 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:05.918069 25083 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.253s
I20260709 13:10:06.110392 24952 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.111820 24929 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:05.981786 25040 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:05.931756 25068 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:06.065925 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:05.980630 25073 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.112891 24952 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.097437 25038 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:06.128252 22777 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:10:06.138535 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72705.8us
I20260709 13:10:06.112890 25046 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.105711 24734 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.168779 23415 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:06.143903 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:06.189531 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45762.6us
I20260709 13:10:06.190675 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:06.202771 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.142714 24940 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:06.143395 25038 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:06.126490 25068 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.211213 25068 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:06.198293 25046 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.211820 24954 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:06.216378 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:06.155354 25073 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:06.219774 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:06.224751 22705 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:06.200079 24952 raft_consensus.cc:2764] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:10:05.932279 24928 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:06.226212 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:06.244455 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.247193 24954 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:06.248847 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:06.252568 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:06.255506 24929 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:06.255362 24938 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:06.245201 24940 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:06.213045 25038 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:06.239809 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.187249 24734 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.267153 22782 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:10:06.284112 22782 pending_rounds.cc:85] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 2
I20260709 13:10:06.318116 22782 pending_rounds.cc:107] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:10:06.126037 25072 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:06.256527 25015 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:06.272074 25061 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:06.309346 22779 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:10:06.315385 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 534) took 1199 ms. Trace:
I20260709 13:10:06.265904 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:06.398847 22779 pending_rounds.cc:85] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:10:06.399363 22779 pending_rounds.cc:107] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:10:06.399822 22779 raft_consensus.cc:2889] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:06.327003 22782 raft_consensus.cc:2889] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:06.327937 24170 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.402499 24170 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.329788 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:06.275669 24929 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.319779 24939 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:06.359241 25047 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:06.363852 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1796) took 1352 ms. Trace:
I20260709 13:10:06.411718 22707 rpcz_store.cc:276] 0709 13:10:05.012101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.673771 (+661670us) service_pool.cc:224] Handling call
0709 13:10:05.859739 (+185968us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:06.072552 (+212813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.363635 (+291083us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002bedd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:06.363783 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426959616}
W20260709 13:10:06.379753 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.398904 23420 rpcz_store.cc:276] 0709 13:10:05.115560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.116527 (+   967us) service_pool.cc:224] Handling call
0709 13:10:05.116864 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.475296 (+358432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:05.475329 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:05.475373 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:05.475402 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.315084 (+839682us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54004f4f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:06.315148 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.315302 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823793024}
W20260709 13:10:06.407027 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141243us
I20260709 13:10:06.413808 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:06.410944 24938 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:06.415678 25060 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.777s	user 0.003s	sys 0.003s
I20260709 13:10:06.416065 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:06.419420 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:06.346429 25072 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:06.421098 25049 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:06.421526 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:06.427995 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 544) took 1141 ms. Trace:
I20260709 13:10:06.421545 25049 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:06.421875 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 440) took 1129 ms. Trace:
I20260709 13:10:06.430743 25049 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.431075 23414 rpcz_store.cc:276] 0709 13:10:05.287217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.288980 (+  1763us) service_pool.cc:224] Handling call
0709 13:10:05.289329 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.680578 (+391249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.366844 (+686266us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54000da7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:06.416806 (+ 49962us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":809522048}
I20260709 13:10:06.432072 25049 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:06.438479 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.439106 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.438482 24170 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.439982 22710 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:06.439342 22706 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:06.441776 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:06.446758 22628 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:10:06.435952 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.449869 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:06.450754 22707 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:06.450942 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.408073 25061 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.455873 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:06.461895 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.466934 22710 raft_consensus.cc:2376] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8. Re-sending same reply.
I20260709 13:10:06.468328 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:06.473240 25044 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.092s	user 0.000s	sys 0.004s
I20260709 13:10:06.473469 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:06.473783 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.477553 23414 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:06.480749 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.482092 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.478058 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1876) took 1904 ms. Trace:
I20260709 13:10:06.483282 25073 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:06.478819 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:06.482767 22780 rpcz_store.cc:276] 0709 13:10:04.573715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.580446 (+  6731us) service_pool.cc:224] Handling call
0709 13:10:04.580970 (+   524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.737345 (+156375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.737382 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:04.767560 (+ 30178us) log.cc:844] Serialized 26 byte log entry
0709 13:10:04.987643 (+220083us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:04.987702 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:04.987789 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.468511 (+1480722us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:06.477700 (+  9189us) raft_consensus.cc:1685] finished
0709 13:10:06.477724 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.477978 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155811,"raft.queue_time_us":157447,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":14168,"spinlock_wait_cycles":76456320,"thread_start_us":163328,"threads_started":1,"wal-append.queue_time_us":332740}
I20260709 13:10:06.490425 22710 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:06.474637 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:06.423947 25022 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:06.492163 22710 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:06.492614 22703 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:10:06.494741 22710 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:06.493975 25022 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.497711 25022 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:06.501245 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.505877 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:06.451861 25072 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:06.465042 24929 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:06.471843 25061 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:06.483596 22531 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 2, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.422784 24939 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.428719 23411 rpcz_store.cc:276] 0709 13:10:05.286791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.286949 (+   158us) service_pool.cc:224] Handling call
0709 13:10:05.287315 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:05.287726 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:05.287758 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:05.288342 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:10:05.440917 (+152575us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:05.448481 (+  7564us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.484130 (+ 35649us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.368494 (+884364us) spinlock_profiling.cc:243] Waited 877 us on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:06.421604 (+ 53110us) raft_consensus.cc:1685] finished
0709 13:10:06.421628 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.425373 (+  3745us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152997,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":35797,"spinlock_wait_cycles":2017536,"thread_start_us":138195,"threads_started":1,"wal-append.queue_time_us":291908}
I20260709 13:10:06.476377 25040 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:06.468302 25020 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:06.541954 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:06.548049 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88546.2us
I20260709 13:10:06.552476 22705 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:06.552829 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:06.499091 24734 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:06.562815 25047 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.549685 25042 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:10:06.569850 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.588363 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.571082 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:06.595228 25020 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.595832 25020 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:06.598094 25020 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:06.595858 25005 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:06.599138 25005 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.599493 25005 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:06.579543 25039 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:06.553961 22705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1795) took 1544 ms (client timeout 2003 ms). Trace:
I20260709 13:10:06.589778 24960 raft_consensus.cc:993] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
W20260709 13:10:06.615978 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.616649 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.617144 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.564251 25083 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 13:10:06.608736 22705 rpcz_store.cc:269] 0709 13:10:05.010745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.219437 (+1208692us) service_pool.cc:224] Handling call
0709 13:10:06.553879 (+334442us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:06.607086 25072 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.624898 22707 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:06.563710 25047 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.612632 24900 raft_consensus.cc:1081] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
W20260709 13:10:06.616931 24920 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.010s	sys 0.055s
W20260709 13:10:06.638861 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.014s	sys 0.034s
W20260709 13:10:06.638932 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.639508 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.014s	sys 0.034s
W20260709 13:10:06.630857 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.006s	sys 0.051s
W20260709 13:10:06.635883 22776 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.014s	sys 0.035s
W20260709 13:10:06.609270 23416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.058s	sys 0.131s
I20260709 13:10:06.628201 25046 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:06.644554 22776 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.014s	sys 0.035s
I20260709 13:10:06.624596 22635 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:06.609238 24463 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:06.660077 22708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.029s	sys 0.032s
W20260709 13:10:06.817844 22708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.029s	sys 0.033s
I20260709 13:10:06.704468 25070 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 13:10:06.627048 24692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.021s	sys 0.035s
W20260709 13:10:06.643124 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.006s	sys 0.051s
W20260709 13:10:06.686208 23416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.058s	sys 0.131s
I20260709 13:10:06.702001 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:06.702389 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1877) took 2123 ms. Trace:
I20260709 13:10:06.834096 23419 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:06.694304 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.701124 25044 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:10:06.695696 25073 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.723448 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:06.729399 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1803) took 1382 ms. Trace:
I20260709 13:10:06.736665 24939 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:06.644418 25072 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:06.698087 24927 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.220s
W20260709 13:10:06.695194 24983 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.030s	sys 0.019s
I20260709 13:10:06.797737 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:06.799569 25017 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.756242 25058 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:06.808873 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.013s	sys 0.039s
W20260709 13:10:06.902572 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.034s	user 0.013s	sys 0.040s
W20260709 13:10:06.903164 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.03501e+06us
W20260709 13:10:06.809881 22775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.009s	sys 0.014s
W20260709 13:10:06.903986 22775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.009s	sys 0.014s
I20260709 13:10:06.904950 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1878) took 2330 ms. Trace:
W20260709 13:10:06.664561 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.030s	sys 0.015s
W20260709 13:10:06.906826 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.050s	user 0.030s	sys 0.016s
W20260709 13:10:06.907168 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 1.8055e+06us
W20260709 13:10:06.644179 24920 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.011s	sys 0.055s
I20260709 13:10:06.762648 25020 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:06.824496 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.915300 22747 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:06.810426 25051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.011s	sys 0.011s
I20260709 13:10:06.835249 23419 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
W20260709 13:10:06.903373 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.905460 22775 rpcz_store.cc:276] 0709 13:10:04.574415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.657793 (+ 83378us) service_pool.cc:224] Handling call
0709 13:10:04.700004 (+ 42211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.700343 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:10:05.102927 (+402584us) log.cc:844] Serialized 14 byte log entry
0709 13:10:06.904399 (+1801472us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:06.904480 (+    81us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:06.904555 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.904667 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.904863 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":754092,"prepare.queue_time_us":283443,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":831189,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":135,"thread_start_us":1390863,"threads_started":2}
I20260709 13:10:06.842880 22779 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
W20260709 13:10:06.855531 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.854367 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:06.936156 22779 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:06.936554 22779 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:06.828372 24692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.021s	sys 0.035s
I20260709 13:10:06.855214 22709 rpcz_store.cc:276] 0709 13:10:05.347129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.698821 (+351692us) service_pool.cc:224] Handling call
0709 13:10:05.711465 (+ 12644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:05.728778 (+ 17313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:05.728815 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:05.729381 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:10:06.087846 (+358465us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:06.109234 (+ 21388us) log.cc:844] Serialized 14 byte log entry
0709 13:10:06.121673 (+ 12439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.121794 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.729116 (+607322us) raft_consensus.cc:1685] finished
0709 13:10:06.729139 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.729313 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16797,"prepare.queue_time_us":20980,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":36802,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":3072,"thread_start_us":323125,"threads_started":1,"wal-append.queue_time_us":887045}
W20260709 13:10:06.861481 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.83115e+06us
I20260709 13:10:06.862922 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:06.937882 23421 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
W20260709 13:10:06.937975 22750 leader_election.cc:336] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.890s (SENT)
I20260709 13:10:06.828565 25046 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:06.827895 24463 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:06.897315 24983 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.031s	sys 0.019s
I20260709 13:10:06.834012 22776 rpcz_store.cc:276] 0709 13:10:04.574051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.574909 (+   858us) service_pool.cc:224] Handling call
0709 13:10:04.575386 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.575684 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:04.844648 (+268964us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.098560 (+253912us) log.cc:844] Serialized 14 byte log entry
0709 13:10:06.056471 (+957911us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b480012c258. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:10:06.694837 (+638366us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:06.694906 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:06.694951 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.695020 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.697060 (+  2040us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":758268,"prepare.queue_time_us":187851,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":1740,"raft.queue_time_us":1847049,"raft.run_cpu_time_us":2901,"raft.run_wall_time_us":198114,"spinlock_wait_cycles":17078656,"thread_start_us":999891,"threads_started":2}
W20260709 13:10:06.938796 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.938687 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:06.941365 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:06.941790 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:06.943749 22781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.033s	sys 0.056s
I20260709 13:10:06.944872 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:06.916841 25051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.012s	sys 0.011s
W20260709 13:10:07.031756 25013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.408s	user 0.012s	sys 0.029s
W20260709 13:10:07.036324 25013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.412s	user 0.012s	sys 0.029s
W20260709 13:10:07.036460 22781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.034s	sys 0.057s
I20260709 13:10:06.916872 23411 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:07.030165 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 92232.6us
I20260709 13:10:06.942960 24735 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:06.943555 24898 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.006s	sys 0.034s
I20260709 13:10:06.898502 23414 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:10:07.031167 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89291.6us
I20260709 13:10:07.038064 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:06.867612 25073 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:07.043358 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.987s (SENT) [suppressed 26 similar messages]
I20260709 13:10:07.049854 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:06.946714 25089 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.935235 25047 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.139649 22531 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 5, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:06.902558 25017 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.043284 25038 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.000s
I20260709 13:10:06.912652 25058 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.141199 22779 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.144114 22634 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:07.036731 25013 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:07.195871 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51228.8us
I20260709 13:10:07.032794 24938 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.062829 24898 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.006s	sys 0.035s
I20260709 13:10:06.943807 25020 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.173255 25068 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.961s	user 0.016s	sys 0.001s
I20260709 13:10:07.050279 25045 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:07.201699 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:07.030684 25046 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:07.210036 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.014s	sys 0.027s
I20260709 13:10:07.198894 25088 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:07.246934 25085 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:10:07.310505 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.014s	sys 0.027s
I20260709 13:10:07.250794 24900 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:07.137502 24735 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.271821 22710 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:10:07.272743 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:07.335422 23419 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:07.276453 22779 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:07.246945 25013 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.221690 24734 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.270339 25031 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.163s	user 0.009s	sys 0.008s
I20260709 13:10:07.346428 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:07.347048 22702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.005s	sys 0.041s
W20260709 13:10:07.344551 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 142978us
W20260709 13:10:07.300405 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.301764 25069 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.164s	user 0.012s	sys 0.003s
I20260709 13:10:07.243556 25038 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.353040 23415 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:10:07.353452 23415 pending_rounds.cc:85] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 5
W20260709 13:10:07.347386 22702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.005s	sys 0.041s
I20260709 13:10:07.353678 23415 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:10:07.353945 23415 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:07.354247 23415 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:10:07.354450 23415 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:07.351089 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:07.358258 22703 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 1.
W20260709 13:10:07.312515 24665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.006s	sys 0.032s
W20260709 13:10:07.355175 22598 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.002s	sys 0.024s
W20260709 13:10:07.362339 22598 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.002s	sys 0.024s
W20260709 13:10:07.362803 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 887282us
I20260709 13:10:07.364005 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:07.365279 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:07.366354 25086 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:10:07.367817 23419 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:07.368803 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:07.369410 23420 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
W20260709 13:10:07.370342 25103 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.423s	user 0.001s	sys 0.001s
I20260709 13:10:07.349756 25094 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:07.359908 24463 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.299584 24938 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.351425 25072 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.376874 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:07.362154 24665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.006s	sys 0.033s
I20260709 13:10:07.352420 24734 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.393373 22778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.007s	sys 0.048s
I20260709 13:10:07.378110 25094 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:07.396217 22778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.007s	sys 0.048s
I20260709 13:10:07.389032 25072 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:07.404083 22782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.004s	sys 0.041s
W20260709 13:10:07.405611 22782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.004s	sys 0.042s
I20260709 13:10:07.386416 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:07.397094 24449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.018s	sys 0.024s
W20260709 13:10:07.395370 24962 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.018s	sys 0.026s
I20260709 13:10:07.389894 25047 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:07.375766 25031 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:10:07.439858 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:07.330348 25068 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.356046 25069 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.491581 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 105231us
I20260709 13:10:07.492790 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:07.446959 24962 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.018s	sys 0.026s
I20260709 13:10:07.501344 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1879) took 1385 ms. Trace:
I20260709 13:10:07.501821 22704 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:07.445744 25058 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.504309 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:07.204949 25089 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.402753 25094 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:07.442862 24449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.018s	sys 0.024s
I20260709 13:10:07.555878 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:07.556984 22628 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:07.557461 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:07.558344 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:07.559157 22628 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:07.559584 22628 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:07.445007 24170 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.881s	user 0.008s	sys 0.028s
I20260709 13:10:07.432263 25072 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.558422 22777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.004s	user 0.009s	sys 0.040s
I20260709 13:10:07.447239 25047 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.561075 24170 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.997s	user 0.008s	sys 0.028s
W20260709 13:10:07.562374 22777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.009s	sys 0.040s
W20260709 13:10:07.562649 22780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.010s	sys 0.051s
W20260709 13:10:07.505834 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.567070 22780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.010s	sys 0.051s
I20260709 13:10:07.569442 25038 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.569657 25047 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:07.563445 25020 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.572149 25020 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:07.572888 25020 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:07.504586 22783 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.219s	user 0.015s	sys 0.040s
W20260709 13:10:07.575354 22783 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.289s	user 0.016s	sys 0.040s
I20260709 13:10:07.501726 25017 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.496243 25073 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.506551 25099 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:07.616144 25097 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 13:10:07.617363 25098 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:10:07.620661 25020 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.501976 22781 rpcz_store.cc:276] 0709 13:10:06.115765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.115924 (+   159us) service_pool.cc:224] Handling call
0709 13:10:06.116293 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:06.116538 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:10:06.143451 (+ 26913us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.500890 (+1357439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:07.500972 (+    82us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:07.501044 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.501130 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.501274 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30118,"prepare.queue_time_us":24936,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":208687,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":13028,"thread_start_us":1356962,"threads_started":2,"wal-append.queue_time_us":1301156}
I20260709 13:10:07.613308 25068 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:07.553191 25061 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.004s	user 0.014s	sys 0.002s
I20260709 13:10:07.611744 25101 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:07.627552 24735 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.565315 22628 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.647996 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.576385 22784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.021s	sys 0.008s
W20260709 13:10:07.648645 22784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.022s	sys 0.008s
I20260709 13:10:07.663208 22631 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:07.663733 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:07.665170 22629 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:07.665751 22632 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:07.665925 22628 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:07.643564 24938 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.705124 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:07.746343 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:07.747452 23414 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:07.690325 25046 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.763135 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:07.764372 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.011s	sys 0.012s
I20260709 13:10:07.768108 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:07.767612 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.006s	sys 0.017s
W20260709 13:10:07.769616 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.007s	sys 0.017s
W20260709 13:10:07.763406 24996 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.038s	sys 0.091s
W20260709 13:10:07.770735 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 727311us
I20260709 13:10:07.775269 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:07.764827 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.011s	sys 0.012s
I20260709 13:10:07.780735 22628 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:10:07.788686 22708 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.067s	sys 0.072s
W20260709 13:10:07.789289 22708 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.067s	sys 0.072s
I20260709 13:10:07.793823 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:07.648754 25061 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:07.798266 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1854) took 2026 ms. Trace:
I20260709 13:10:07.812809 22708 rpcz_store.cc:276] 0709 13:10:05.772035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.775631 (+  3596us) service_pool.cc:224] Handling call
0709 13:10:05.776022 (+   391us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:05.785275 (+  9253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:05.785310 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:05.795789 (+ 10479us) log.cc:844] Serialized 52 byte log entry
0709 13:10:06.833414 (+1037625us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:07.789807 (+956393us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.797872 (+  8065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.797968 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:07.798018 (+    50us) raft_consensus.cc:1685] finished
0709 13:10:07.798029 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.798198 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4772,"raft.queue_time_us":9991,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":3835,"thread_start_us":1888457,"threads_started":2,"wal-append.queue_time_us":1317428}
I20260709 13:10:07.815915 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1886) took 1632 ms. Trace:
I20260709 13:10:07.816464 22783 rpcz_store.cc:276] 0709 13:10:06.183939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.184574 (+   635us) service_pool.cc:224] Handling call
0709 13:10:06.205707 (+ 21133us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.213953 (+  8246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.214009 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.248805 (+ 34796us) log.cc:844] Serialized 26 byte log entry
0709 13:10:07.576098 (+1327293us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:07.606080 (+ 29982us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.606413 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.606500 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:07.815669 (+209169us) raft_consensus.cc:1685] finished
0709 13:10:07.815692 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.815846 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":229,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":14411,"thread_start_us":1290066,"threads_started":1,"wal-append.queue_time_us":1540354}
I20260709 13:10:07.817834 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1898) took 1313 ms. Trace:
I20260709 13:10:07.757767 24734 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:07.760668 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:07.770262 24996 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.038s	sys 0.092s
I20260709 13:10:07.771876 25046 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:07.816758 22708 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:07.818246 22778 rpcz_store.cc:276] 0709 13:10:06.504358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.507511 (+  3153us) service_pool.cc:224] Handling call
0709 13:10:06.507944 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.508455 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.508496 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.509493 (+   997us) log.cc:844] Serialized 26 byte log entry
0709 13:10:07.400833 (+891340us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:07.400887 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.400984 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:07.817610 (+416626us) raft_consensus.cc:1685] finished
0709 13:10:07.817633 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.817773 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19953,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5806,"thread_start_us":890860,"threads_started":1,"wal-append.queue_time_us":1306413}
W20260709 13:10:07.838148 22599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.012s	sys 0.030s
W20260709 13:10:07.842235 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.798506 25015 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:07.844085 22706 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:07.824177 25020 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.845503 22599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.013s	sys 0.030s
I20260709 13:10:07.820305 25069 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:07.847752 25046 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.880245 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.824469 25051 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.017s	sys 0.013s
I20260709 13:10:07.823971 25104 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.827162 25022 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.329s	user 0.013s	sys 0.005s
I20260709 13:10:07.838969 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:07.868099 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1904) took 1233 ms. Trace:
W20260709 13:10:07.888079 22709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.017s	sys 0.009s
I20260709 13:10:07.821599 25068 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:07.825176 25089 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:07.880425 25005 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.280s	user 0.018s	sys 0.000s
W20260709 13:10:07.883826 22705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.004s	sys 0.020s
I20260709 13:10:07.845268 24734 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:07.888510 22709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.017s	sys 0.009s
W20260709 13:10:07.888798 23384 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.000s	sys 0.030s
I20260709 13:10:07.888309 22780 rpcz_store.cc:276] 0709 13:10:06.634590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.634809 (+   219us) service_pool.cc:224] Handling call
0709 13:10:06.635275 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.635596 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.635634 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.745971 (+110337us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:06.755269 (+  9298us) log.cc:844] Serialized 26 byte log entry
0709 13:10:07.570101 (+814832us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:07.570156 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.570247 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:07.867703 (+297456us) raft_consensus.cc:1685] finished
0709 13:10:07.867729 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.868009 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36789120,"thread_start_us":769344,"threads_started":1,"wal-append.queue_time_us":1107501}
W20260709 13:10:07.899118 23384 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.000s	sys 0.030s
I20260709 13:10:07.841970 25072 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:07.895968 22705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.004s	sys 0.020s
W20260709 13:10:07.900729 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 845891us
W20260709 13:10:07.902206 22702 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.018s	sys 0.010s
W20260709 13:10:07.890784 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 47619.8us
W20260709 13:10:07.902559 22702 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.018s	sys 0.010s
W20260709 13:10:07.888743 22676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.008s	sys 0.018s
W20260709 13:10:07.903254 22707 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.011s	sys 0.016s
W20260709 13:10:07.909157 22707 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.013s	sys 0.017s
I20260709 13:10:07.907356 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:07.903276 22676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.009s	sys 0.018s
I20260709 13:10:07.930428 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1852) took 2945 ms. Trace:
W20260709 13:10:07.931193 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 426969us
I20260709 13:10:07.936671 22701 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:07.935662 22702 rpcz_store.cc:276] 0709 13:10:04.962026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.657229 (+695203us) service_pool.cc:224] Handling call
0709 13:10:05.657604 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:05.657958 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:05.658276 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.839179 (+180903us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b1800078b80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1
0709 13:10:05.840546 (+  1367us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b1800078b80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:06.271481 (+430935us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b1800078b80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:10:06.435606 (+164125us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b1800078b80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:07.353882 (+918276us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:07.354433 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 13:10:07.354697 (+   264us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:07.903002 (+548305us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.905852 (+  2850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.905939 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:07.905995 (+    56us) raft_consensus.cc:1685] finished
0709 13:10:07.906007 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.906166 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6193,"prepare.queue_time_us":9618,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":37650,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":35939,"spinlock_wait_cycles":49974912,"thread_start_us":1692897,"threads_started":3,"wal-append.queue_time_us":63239}
W20260709 13:10:07.905427 24898 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.014s	sys 0.015s
I20260709 13:10:07.940132 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 271) took 1498 ms. Trace:
I20260709 13:10:07.898922 25069 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:07.938853 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:07.879190 25020 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.944414 22706 rpcz_store.cc:276] 0709 13:10:06.447163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.450601 (+  3438us) service_pool.cc:224] Handling call
0709 13:10:07.936342 (+1485741us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b540010d454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:07.940033 (+  3691us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3192006528}
I20260709 13:10:07.886402 24463 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:07.886817 24802 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 13:10:07.909288 23416 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.018s	sys 0.011s
I20260709 13:10:07.895934 25022 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.953573 22710 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.000s	sys 0.034s
I20260709 13:10:07.954727 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1897) took 1450 ms. Trace:
W20260709 13:10:07.887485 25051 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.017s	sys 0.013s
W20260709 13:10:07.960436 22710 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.000s	sys 0.034s
I20260709 13:10:07.897657 25068 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:07.944296 24898 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.014s	sys 0.015s
I20260709 13:10:07.893453 25104 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.942691 24727 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:07.900053 25005 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:07.958323 22750 connection.cc:256] RPC call timeout handler was delayed by -3.1678379809999342s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 13:10:07.945612 24938 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.902017 24734 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:07.960821 22777 rpcz_store.cc:276] 0709 13:10:06.504003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.506573 (+  2570us) service_pool.cc:224] Handling call
0709 13:10:06.507015 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:06.507295 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:07.569522 (+1062227us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.954162 (+384640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:07.954247 (+    85us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:07.954328 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.954426 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:07.954633 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30797,"prepare.queue_time_us":78319,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":1314,"thread_start_us":1399817,"threads_started":2}
W20260709 13:10:07.959167 23416 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.019s	sys 0.011s
W20260709 13:10:07.975128 22750 connection.cc:256] RPC call timeout handler was delayed by -3.1614997609999591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25110000000000005s for a response
W20260709 13:10:07.987075 22750 connection.cc:256] RPC call timeout handler was delayed by -3.1319637819998434s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 13:10:07.987825 22750 connection.cc:256] RPC call timeout handler was delayed by -3.0828615919998583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:10:07.988982 22750 connection.cc:256] RPC call timeout handler was delayed by -3.0427964539999266s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
W20260709 13:10:07.989564 22750 connection.cc:256] RPC call timeout handler was delayed by -2.9662964539999166s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 13:10:07.945885 25038 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:10:07.990309 22750 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 3.216s (SENT)
I20260709 13:10:07.995833 24727 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:08.007694 24727 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.010828 25130 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:08.011449 25130 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:08.008270 25022 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.013682 25130 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.007243 23414 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:08.006691 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:08.023407 25031 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:08.025000 25073 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:08.025789 25110 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 13:10:08.025610 25073 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.026762 25061 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:07.980438 24938 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:08.027622 25043 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:08.028421 25073 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:08.029490 24938 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.029664 24920 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:08.030167 24920 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:08.031175 24920 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.032682 24920 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:08.016222 25051 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:10:08.034550 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:08.038096 22750 connection.cc:256] RPC call timeout handler was delayed by -2.769994790999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 13:10:08.039216 25105 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:10:08.039413 24450 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 13:10:08.041666 25112 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:10:08.019106 25038 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.043092 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1906) took 1301 ms. Trace:
I20260709 13:10:08.043529 25038 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:08.044400 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.043486 22784 rpcz_store.cc:276] 0709 13:10:06.741736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.742157 (+   421us) service_pool.cc:224] Handling call
0709 13:10:06.742658 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.743155 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.743197 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.748338 (+  5141us) log.cc:844] Serialized 26 byte log entry
0709 13:10:07.649218 (+900880us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:07.649267 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.652838 (+  3571us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:08.042804 (+389966us) raft_consensus.cc:1685] finished
0709 13:10:08.042834 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:08.043018 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":8604,"thread_start_us":900141,"threads_started":1,"wal-append.queue_time_us":1147034}
W20260709 13:10:08.038443 22750 connection.cc:256] RPC call timeout handler was delayed by -2.6637477669999043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:10:08.046213 25038 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:08.046954 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:08.047051 25102 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:10:08.047497 25043 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.048924 22635 raft_consensus.cc:2368] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:10:08.048837 22750 connection.cc:256] RPC call timeout handler was delayed by -2.4666990480000095s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29350000000000004s for a response
W20260709 13:10:08.049403 22750 connection.cc:256] RPC call timeout handler was delayed by -2.3118990479999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31070000000000003s for a response
W20260709 13:10:08.049764 22750 connection.cc:256] RPC call timeout handler was delayed by -2.2254990479999606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32030000000000003s for a response
W20260709 13:10:08.050138 22750 connection.cc:256] RPC call timeout handler was delayed by -2.21922189899999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 13:10:07.954988 25069 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:08.060097 22750 connection.cc:256] RPC call timeout handler was delayed by -1.7782218989999592s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 13:10:08.025595 24734 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:08.048091 25043 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.062697 25118 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:08.064913 25118 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.066123 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:08.066927 25068 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.067059 23414 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:08.067449 25031 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:08.060563 22750 connection.cc:256] RPC call timeout handler was delayed by -1.7474070139999185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:10:08.069613 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:08.070714 22750 connection.cc:256] RPC call timeout handler was delayed by -1.6657218990000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
W20260709 13:10:08.071282 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.083464 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 448) took 1381 ms. Trace:
W20260709 13:10:08.084220 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.067996 25031 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:08.064309 25043 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:08.101227 22780 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:08.103928 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:08.104696 22706 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3. Re-sending same reply.
I20260709 13:10:08.105876 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.106596 22778 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:08.106825 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:08.107600 22706 raft_consensus.cc:2368] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:08.107722 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:08.108156 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.108290 22781 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.108800 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:08.109665 22706 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:08.109839 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.110527 22782 raft_consensus.cc:2368] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4.
I20260709 13:10:08.110602 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.111449 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.091351 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.092280 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.113502 22777 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:08.111416 22702 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:08.084120 23415 rpcz_store.cc:276] 0709 13:10:06.702253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.702390 (+   137us) service_pool.cc:224] Handling call
0709 13:10:06.702743 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.354901 (+652158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.083037 (+728136us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b5400455454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:08.083360 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494566784}
I20260709 13:10:08.105890 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.114914 22776 raft_consensus.cc:2410] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:10:08.115463 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.116366 22706 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:08.117833 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.119705 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.120343 22708 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:08.120434 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.118541 22782 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:08.120962 22708 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:10:08.121131 22780 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:08.118161 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.123020 22778 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:07.959384 24463 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.126461 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.126794 22778 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:10:08.130383 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.131168 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:08.130507 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.132702 22778 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:08.133965 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.135044 22780 raft_consensus.cc:2410] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:08.139330 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.140321 22784 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:08.141594 22750 connection.cc:256] RPC call timeout handler was delayed by -1.4989230209998823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
I20260709 13:10:08.083132 25088 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:08.149096 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.171780 22784 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:08.153018 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.172763 22780 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:08.155196 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.157608 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.174304 22776 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:08.163807 22781 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:10:08.154269 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 206990us
I20260709 13:10:08.148036 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:08.169371 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:08.162770 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.180025 22783 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:08.175030 22777 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:08.183770 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.184481 22777 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:08.184885 25043 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:08.097858 25051 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.189898 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:08.190657 23415 raft_consensus.cc:2376] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6. Re-sending same reply.
W20260709 13:10:08.178906 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 229676us
I20260709 13:10:08.191058 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:08.192384 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.195343 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:08.196161 22706 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:08.137318 25068 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:08.215842 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.151355 25091 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.200649 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:08.257248 22634 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:08.201411 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.202088 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.229102 23385 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:08.258960 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60997.2us
I20260709 13:10:08.259737 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:08.237537 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:08.216076 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.179025 25058 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.202445 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.201728 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.238385 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.198348 25043 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:08.171149 25088 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.202864 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:08.278963 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.250465 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.256328 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:08.259582 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:08.186264 25013 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.285692 22630 raft_consensus.cc:2410] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:10:08.199201 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.270258 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.271067 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.277776 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.278024 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:08.279886 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:08.289911 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.239327 25101 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.293044 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:08.293486 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:08.295608 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1873) took 1002 ms. Trace:
I20260709 13:10:08.296197 22706 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:08.288166 22631 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:08.307626 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.306885 22705 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:08.330194 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.405483 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.308967 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:08.167522 25116 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:10:08.427265 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.435825 22776 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:08.298211 24981 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.390992 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.393381 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.394434 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.395350 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:08.284075 25088 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:08.276252 25038 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:08.284780 25099 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:08.400107 22702 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:08.284075 24938 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.419140 22703 rpcz_store.cc:276] 0709 13:10:07.292816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.448538 (+155722us) service_pool.cc:224] Handling call
0709 13:10:07.494157 (+ 45619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.943310 (+449153us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:08.022004 (+ 78694us) spinlock_profiling.cc:243] Waited 2.33 ms on lock 0x7b48000ce358. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:10:08.177393 (+155389us) log.cc:844] Serialized 14 byte log entry
0709 13:10:08.295097 (+117704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:08.295180 (+    83us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:08.295259 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.295362 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:08.295526 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6355,"prepare.queue_time_us":63732,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":112965,"raft.run_cpu_time_us":3155,"raft.run_wall_time_us":426598,"spinlock_wait_cycles":5349248,"thread_start_us":771358,"threads_started":3}
I20260709 13:10:08.415781 22634 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:08.421572 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.752298 22706 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:08.752817 22706 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:08.753664 22706 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.422461 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1869) took 1146 ms. Trace:
I20260709 13:10:08.754770 22710 rpcz_store.cc:276] 0709 13:10:07.275836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.341146 (+ 65310us) service_pool.cc:224] Handling call
0709 13:10:07.341466 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.394827 (+ 53361us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:07.965568 (+570741us) log.cc:844] Serialized 14 byte log entry
0709 13:10:08.250762 (+285194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:08.250825 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:08.250886 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.422196 (+171310us) spinlock_profiling.cc:243] Waited 53 ms on lock 0x7b54000e5e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:08.422253 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:08.422396 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":619169,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":24,"raft.queue_time_us":7359,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13020,"spinlock_wait_cycles":121854720,"thread_start_us":846393,"threads_started":2}
I20260709 13:10:08.423336 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1840) took 3783 ms. Trace:
I20260709 13:10:08.426082 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.437125 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.758589 22705 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:08.387322 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:08.436421 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.424988 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:08.760658 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 335756us
W20260709 13:10:08.436172 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 198745us
I20260709 13:10:08.426554 25049 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.310779 25046 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:08.317898 25039 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.459662 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:08.776027 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:08.580470 25104 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.416213 24734 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:08.615062 25047 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.031s	user 0.023s	sys 0.011s
I20260709 13:10:08.299165 25072 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:08.439996 25101 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.420087 25150 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:08.389623 25043 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:10:08.627184 25134 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:10:08.664351 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.628379 25069 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.489899 24981 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.276973 25091 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:08.653192 25042 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:08.425988 25015 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.146s
I20260709 13:10:08.199721 25051 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.299291 25013 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.663666 25089 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:08.273416 25068 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.423457 25130 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 13:10:08.643003 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.285329 25058 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:08.428294 25020 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:08.677452 24963 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:08.660255 25139 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:08.694083 24463 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:08.662425 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:08.700537 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:08.661562 25146 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
W20260709 13:10:08.687413 25127 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.403s	user 0.000s	sys 0.000s
W20260709 13:10:08.659523 25144 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 13:10:08.696651 25140 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.429292 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:08.450968 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.422341 25061 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.144s
W20260709 13:10:08.500412 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.718271 22776 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:08.752365 22702 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:08.428071 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.763379 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:08.759328 23421 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:08.766674 22703 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:08.767174 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:08.757361 22701 rpcz_store.cc:276] 0709 13:10:04.640578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.475443 (+1834865us) service_pool.cc:224] Handling call
0709 13:10:07.936484 (+1461041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.937265 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.423134 (+485869us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540010d450. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:08.423270 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3359249664}
I20260709 13:10:08.769500 22706 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:08.733417 25038 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:08.797477 22634 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:08.748456 24938 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:08.798470 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:08.798588 22673 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.009s	sys 0.004s
W20260709 13:10:09.249241 22673 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.285s	user 0.013s	sys 0.010s
W20260709 13:10:09.249617 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 1.29357e+06us
I20260709 13:10:09.250289 22781 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:09.251118 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:09.249258 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 485962us
I20260709 13:10:09.256822 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:09.251006 22782 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:09.252164 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:09.254206 22778 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:10:09.155625 22635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 1947) took 2218 ms (client timeout 1847 ms). Trace:
W20260709 13:10:09.257171 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.273473 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.272064 22635 rpcz_store.cc:269] 0709 13:10:06.940903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.941064 (+   161us) service_pool.cc:224] Handling call
0709 13:10:09.155228 (+2214164us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400127854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:09.155464 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2527676800}
I20260709 13:10:09.248194 22632 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:09.275394 22632 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:09.275485 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:09.278559 22782 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:09.278743 22780 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:09.277370 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.279881 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:09.288277 22781 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:10:08.961360 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.007s	sys 0.007s
I20260709 13:10:09.289847 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:09.296150 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49162.2us
W20260709 13:10:09.297545 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.290556 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.007s	sys 0.017s
W20260709 13:10:09.299506 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 990566us
I20260709 13:10:09.300304 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:09.290230 22632 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:09.244428 24920 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.211s	user 0.020s	sys 0.000s
W20260709 13:10:09.243624 25094 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.679s	user 0.007s	sys 0.011s
I20260709 13:10:09.151161 25046 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.324106 22779 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:10:09.324693 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.346906 22779 raft_consensus.cc:606] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:09.309864 22632 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.302713 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:09.230679 25145 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.777s	user 0.001s	sys 0.001s
W20260709 13:10:09.309579 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63024.1us
I20260709 13:10:09.152954 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 542) took 3867 ms. Trace:
I20260709 13:10:09.351125 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:09.100800 25131 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:09.348636 23420 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:09.291031 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1574) took 1001 ms. Trace:
I20260709 13:10:09.293712 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.247804 25043 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:09.283416 25089 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:09.381282 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:09.381984 23418 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:09.374347 22780 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:09.353267 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:09.387629 22779 raft_consensus.cc:614] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:09.388238 22781 rpcz_store.cc:276] 0709 13:10:08.300779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.427801 (+127022us) service_pool.cc:224] Handling call
0709 13:10:09.290956 (+863155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:09.353385 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:09.392426 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45863.3us
I20260709 13:10:09.393038 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:09.394018 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.316038 22706 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:10:09.317799 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:09.160465 25137 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.341s	user 0.003s	sys 0.000s
I20260709 13:10:09.395612 22706 pending_rounds.cc:85] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 5
W20260709 13:10:08.876610 25136 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:10:09.321110 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.324575 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:09.395955 22706 pending_rounds.cc:107] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:10:09.396466 22776 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:09.396533 22706 raft_consensus.cc:2889] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:09.340576 22784 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:09.343612 22777 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:09.224197 25005 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.345322 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:09.166384 25118 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.345690 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.303131 22702 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:08.796249 25073 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:10:09.322067 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:08.813463 25151 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:10:09.395889 22635 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:09.400507 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.402527 22633 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:10:09.408007 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57063.3us
I20260709 13:10:09.298501 25042 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:08.774884 25049 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.412963 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.412976 22781 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:09.289724 25155 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:09.419301 22776 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:10:09.430852 22776 pending_rounds.cc:85] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:10:09.431223 22776 pending_rounds.cc:107] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:10:09.421751 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.431887 22628 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:09.427874 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:09.430867 22627 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:08.796056 25039 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.284415 25047 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:09.429209 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.482674 22632 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:09.289561 24734 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.417430 25130 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:09.456202 22634 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:10:09.496336 22778 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:09.433560 22776 raft_consensus.cc:2889] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:09.383015 25043 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:09.324784 25061 consensus_queue.cc:1243] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:09.380409 25141 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:09.480010 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64750.3us
W20260709 13:10:09.480916 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62614.5us
I20260709 13:10:09.513381 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1860) took 2263 ms. Trace:
I20260709 13:10:09.289314 25072 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:09.483932 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 227207us
I20260709 13:10:09.293980 24928 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.134s
I20260709 13:10:09.484648 22783 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:09.293232 24963 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.289896 25140 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.392521 25104 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.352025 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 555) took 1149 ms. Trace:
I20260709 13:10:09.492777 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1861) took 2224 ms. Trace:
W20260709 13:10:09.492916 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.523811 22707 rpcz_store.cc:276] 0709 13:10:07.267944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.268147 (+   203us) service_pool.cc:224] Handling call
0709 13:10:07.268549 (+   402us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:07.949951 (+681402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:07.950094 (+   143us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:07.959473 (+  9379us) log.cc:844] Serialized 52 byte log entry
0709 13:10:08.184073 (+224600us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:08.184143 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.184214 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:09.492512 (+1308298us) raft_consensus.cc:1685] finished
0709 13:10:09.492536 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.492703 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":302926,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":881196,"threads_started":2,"wal-append.queue_time_us":275943}
I20260709 13:10:09.318217 25150 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:09.493727 22532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 142) took 13961 ms (client timeout 15000 ms). Trace:
I20260709 13:10:09.499339 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1571) took 1267 ms. Trace:
W20260709 13:10:09.478221 25156 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.616s	user 0.006s	sys 0.017s
I20260709 13:10:09.501539 22784 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
W20260709 13:10:09.318714 25147 consensus_queue.cc:1175] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ebb78bf12f41baae84669c1effc423" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:10:09.423089 25152 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:10:09.513892 22709 rpcz_store.cc:276] 0709 13:10:07.250388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.251571 (+  1183us) service_pool.cc:224] Handling call
0709 13:10:07.267263 (+ 15692us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:07.901228 (+633965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:07.901280 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:07.908663 (+  7383us) log.cc:844] Serialized 52 byte log entry
0709 13:10:08.178849 (+270186us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:08.178902 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.179048 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:09.513134 (+1334086us) raft_consensus.cc:1685] finished
0709 13:10:09.513159 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.513310 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":303899,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":845100,"threads_started":2,"wal-append.queue_time_us":285077}
I20260709 13:10:09.515324 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:09.432233 25122 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:09.379666 25046 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.525620 22776 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:09.520565 25141 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:09.483034 25124 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.542016 22777 rpcz_store.cc:276] 0709 13:10:08.232184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.237970 (+  5786us) service_pool.cc:224] Handling call
0709 13:10:09.499148 (+1261178us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003250d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:09.499272 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485865344}
I20260709 13:10:09.352885 25094 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.543982 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:09.574451 22778 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:09.407676 25131 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.485303 25038 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.596411 25131 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.597069 25131 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:09.298081 25139 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.549453 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:09.629703 23417 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:09.550209 24170 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:09.495277 25130 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:09.539705 25138 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.718s	user 0.000s	sys 0.009s
I20260709 13:10:09.510392 25015 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:09.521910 24939 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:09.522671 25162 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:10:09.569633 25088 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.991s	user 0.014s	sys 0.000s
I20260709 13:10:09.583117 24981 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.522588 25089 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.583418 25167 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:09.585156 24963 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:09.587533 25031 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.467s	user 0.019s	sys 0.000s
W20260709 13:10:09.589483 25099 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.854s	user 0.005s	sys 0.004s
I20260709 13:10:09.589849 25046 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.590487 25160 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:09.522297 25061 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:10:09.554632 25152 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:09.526921 25104 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.748296 25104 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:09.570639 25141 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:09.697924 25142 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.403s	user 0.000s	sys 0.000s
I20260709 13:10:09.695525 25049 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.756428 25051 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.758234 25101 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:09.761117 25094 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.695819 24938 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:09.700402 25140 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.735149 25046 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:09.744709 25160 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:10:09.744743 25157 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:10:09.733009 25099 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:09.570078 25122 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.721102 25162 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:09.742060 25013 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.717044 25015 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.776690 25160 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:09.632118 25124 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:09.788400 25132 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.924s	user 0.003s	sys 0.007s
I20260709 13:10:09.783785 25089 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:09.762293 25152 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:09.718183 25088 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.777506 25091 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:09.772976 25130 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.734100 25150 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:09.842056 25139 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:09.831326 25160 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:09.845484 25160 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:09.831813 25162 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:09.541390 22532 rpcz_store.cc:269] 0709 13:09:55.518691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.518937 (+   246us) service_pool.cc:224] Handling call
0709 13:10:09.438350 (+13919413us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:00.630746 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:00.752699 (+121953us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:00.752762 (+    63us) write_op.cc:435] Acquired schema lock
0709 13:10:00.752782 (+    20us) tablet.cc:662] Decoding operations
0709 13:10:00.753743 (+   961us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:00.753855 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:00.753884 (+    29us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:10:00.754578 (+   694us) tablet.cc:701] Row locks acquired
0709 13:10:00.783531 (+ 28953us) write_op.cc:260] PREPARE: finished
0709 13:10:00.794249 (+ 10718us) write_op.cc:270] Start()
0709 13:10:00.794449 (+   200us) write_op.cc:276] Timestamp: P: 1783602600794159 usec, L: 0
0709 13:10:00.794500 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:10:00.795472 (+   972us) log.cc:844] Serialized 2434 byte log entry
0709 13:10:02.544485 (+1749013us) op_driver.cc:464] REPLICATION: finished
0709 13:10:02.823262 (+278777us) write_op.cc:301] APPLY: starting
0709 13:10:02.823427 (+   165us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:02.840041 (+ 16614us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:02.840236 (+   195us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:02.887127 (+ 46891us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:03.022230 (+135103us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:10:03.046779 (+ 24549us) write_op.cc:312] APPLY: finished
0709 13:10:03.050729 (+  3950us) log.cc:844] Serialized 53 byte log entry
0709 13:10:03.051089 (+   360us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:03.051291 (+   202us) write_op.cc:454] Released schema lock
0709 13:10:03.051396 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:03.052024 (+   628us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:03.052078 (+    54us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":333382,"reported_tablets":17,"child_traces":[["op",{"apply.queue_time_us":212359,"apply.run_cpu_time_us":5713,"apply.run_wall_time_us":259249,"num_ops":6,"prepare.queue_time_us":361983,"prepare.run_cpu_time_us":19692,"prepare.run_wall_time_us":440602,"raft.queue_time_us":672848,"raft.run_cpu_time_us":56432,"raft.run_wall_time_us":189302,"replication_time_us":1731343,"thread_start_us":1112427,"threads_started":4,"wal-append.queue_time_us":551990,"wal-append.run_cpu_time_us":20486,"wal-append.run_wall_time_us":2689116}]]}
W20260709 13:10:09.914435 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 340185us
W20260709 13:10:09.573495 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51207.2us
I20260709 13:10:09.924973 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:09.945361 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1878) took 1513 ms. Trace:
I20260709 13:10:09.946164 22704 rpcz_store.cc:276] 0709 13:10:08.431146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.766493 (+335347us) service_pool.cc:224] Handling call
0709 13:10:08.771378 (+  4885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:08.771692 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:08.771731 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:08.795434 (+ 23703us) log.cc:844] Serialized 26 byte log entry
0709 13:10:09.565530 (+770096us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:09.565593 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:09.565694 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:09.944423 (+378729us) raft_consensus.cc:1685] finished
0709 13:10:09.944508 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.945003 (+   495us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":320073,"threads_started":1,"wal-append.queue_time_us":991664}
W20260709 13:10:09.525105 22705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.017s	sys 0.027s
W20260709 13:10:09.951249 22705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.123s	user 0.017s	sys 0.027s
I20260709 13:10:09.953563 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 288) took 1552 ms. Trace:
I20260709 13:10:09.954046 22705 rpcz_store.cc:276] 0709 13:10:08.401272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.759191 (+357919us) service_pool.cc:224] Handling call
0709 13:10:08.759594 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:08.772760 (+ 13166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:08.772806 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:08.801924 (+ 29118us) log.cc:844] Serialized 26 byte log entry
0709 13:10:09.952313 (+1150389us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:09.952532 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:09.952806 (+   274us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:09.952913 (+   107us) raft_consensus.cc:1685] finished
0709 13:10:09.952927 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.953402 (+   475us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1176,"raft.queue_time_us":4081,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":7713,"thread_start_us":1123411,"threads_started":1,"wal-append.queue_time_us":915558}
I20260709 13:10:09.957074 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:09.958151 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.561038 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 305982us
W20260709 13:10:09.963371 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.964937 22747 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:09.630832 22778 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:09.968477 22775 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:09.967144 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.971251 22778 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:10:09.971501 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.972132 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.972837 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.970968 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:09.976897 22778 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:09.978822 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.984325 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:09.978332 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:09.994978 22781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.010s	sys 0.007s
W20260709 13:10:09.592412 22784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1572) took 1342 ms (client timeout 1503 ms). Trace:
W20260709 13:10:09.997159 23384 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.008s
W20260709 13:10:09.997814 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.024s	sys 0.074s
I20260709 13:10:09.630885 23418 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:09.999686 23419 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:09.707973 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 554) took 1512 ms. Trace:
I20260709 13:10:09.378778 23416 rpcz_store.cc:276] 0709 13:10:05.286070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.350754 (+ 64684us) service_pool.cc:224] Handling call
0709 13:10:05.351170 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.574881 (+223711us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:05.686241 (+111360us) log.cc:844] Serialized 14 byte log entry
0709 13:10:07.369285 (+1683044us) spinlock_profiling.cc:243] Waited 7.44 ms on lock 0x7b180041aca0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:10:08.022695 (+653410us) log.cc:844] Serialized 14 byte log entry
0709 13:10:08.022967 (+   272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:08.023061 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:08.023142 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:09.152564 (+1129422us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b540049b6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:09.152628 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.152834 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1517689,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":110,"raft.queue_time_us":18420,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":7327,"spinlock_wait_cycles":395430656,"thread_start_us":1840392,"threads_started":3,"wal-append.queue_time_us":1252107,"wal-append.run_cpu_time_us":14227,"wal-append.run_wall_time_us":2142795}
I20260709 13:10:09.697220 23417 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:10.102803 22782 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6.
W20260709 13:10:10.042915 23421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.000s	sys 0.004s
W20260709 13:10:10.104092 23421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.001s	sys 0.004s
I20260709 13:10:09.997814 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:10.069653 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 138042us
W20260709 13:10:10.067159 22781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.011s	sys 0.007s
W20260709 13:10:10.080263 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.024s	sys 0.074s
I20260709 13:10:09.727521 25031 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:10.072038 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.003s	sys 0.009s
W20260709 13:10:10.084388 23384 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.008s	sys 0.009s
W20260709 13:10:10.099401 22784 rpcz_store.cc:269] 0709 13:10:08.249966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.250170 (+   204us) service_pool.cc:224] Handling call
0709 13:10:09.592163 (+1341993us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540015a9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:09.592327 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2439384320}
I20260709 13:10:09.801064 25047 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:10.111199 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.004s	sys 0.010s
I20260709 13:10:09.659998 23411 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:10.117744 24920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.014s	sys 0.014s
W20260709 13:10:10.118548 24920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.014s	sys 0.014s
I20260709 13:10:09.646301 25042 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.124545 25122 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.125555 25088 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.127022 25058 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.130244 24981 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:10.133500 25124 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.135177 25139 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:10.135605 24734 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:10.135792 25139 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.136041 22779 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:10.136255 25139 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:10.140646 25038 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.003s
I20260709 13:10:10.140511 25159 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:10:10.141795 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.005s	sys 0.010s
W20260709 13:10:10.142228 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.005s	sys 0.010s
W20260709 13:10:10.142546 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 710459us
W20260709 13:10:10.141402 25038 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.014s	sys 0.003s
I20260709 13:10:10.138362 22781 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:09.796219 25073 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:09.807052 25068 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:10.149947 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 602160us
I20260709 13:10:10.125600 25122 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:10.159400 25170 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:10:10.161315 25188 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:10.161971 25188 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:10.163658 25188 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.165165 24981 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:10:10.177115 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 784116us
I20260709 13:10:10.177505 24981 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:10.170686 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 775600us
I20260709 13:10:10.181103 25031 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:10.181221 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 665782us
I20260709 13:10:10.182713 23415 rpcz_store.cc:276] 0709 13:10:08.195841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.196119 (+   278us) service_pool.cc:224] Handling call
0709 13:10:08.196491 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:08.239973 (+ 43482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:08.240028 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:08.240795 (+   767us) log.cc:844] Serialized 26 byte log entry
0709 13:10:08.523093 (+282298us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:08.523162 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.523276 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:09.707595 (+1184319us) raft_consensus.cc:1685] finished
0709 13:10:09.707634 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.707877 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56401,"raft.queue_time_us":44069,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":14410,"spinlock_wait_cycles":4352,"thread_start_us":208939,"threads_started":1,"wal-append.queue_time_us":554711}
W20260709 13:10:10.185271 23411 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46262 (request call id 461) took 1893 ms (client timeout 2227 ms). Trace:
I20260709 13:10:10.179517 24981 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:10.186815 23411 rpcz_store.cc:269] 0709 13:10:08.292128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.298654 (+  6526us) service_pool.cc:224] Handling call
0709 13:10:10.184778 (+1886124us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b54006329d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:10.185030 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":721186176}
I20260709 13:10:10.187664 25015 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.175056 25039 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.189445 25177 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:10.190177 25177 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:10.190766 25177 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:10.166325 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:10.176230 25152 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.185631 24963 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:10.173038 25073 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.158159 25091 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:10:10.243238 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 259155us
I20260709 13:10:10.215677 25091 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:10.226082 25175 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:10.247916 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.251189 25091 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:10.256546 25175 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:10.278745 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.280651 25188 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:10.281105 25186 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:10:09.522502 23414 rpcz_store.cc:276] 0709 13:10:08.197901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.198079 (+   178us) service_pool.cc:224] Handling call
0709 13:10:08.198385 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:08.237714 (+ 39329us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:08.456047 (+218333us) log.cc:844] Serialized 14 byte log entry
0709 13:10:08.992355 (+536308us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:09.283582 (+291227us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:09.286563 (+  2981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:09.306163 (+ 19600us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.347377 (+ 41214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":982,"prepare.queue_time_us":46197,"prepare.run_cpu_time_us":5918,"prepare.run_wall_time_us":68100,"raft.queue_time_us":66204,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":19958,"thread_start_us":343180,"threads_started":2,"wal-append.queue_time_us":1017618}
I20260709 13:10:10.273244 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.260561 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.293570 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.303268 25046 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:10.304495 25152 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.305550 24734 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:10.307471 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108609us
I20260709 13:10:10.307979 25069 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:10.308495 25069 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:10.312800 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.313580 25047 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:10.314566 25047 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:10:10.315924 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93083.4us
I20260709 13:10:10.284235 25175 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:10.336601 22749 leader_election.cc:400] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:10.337100 22749 leader_election.cc:403] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:10:10.338490 25160 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.310129 25069 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.339324 25160 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:10.345341 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:10.345305 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83873.1us
I20260709 13:10:10.347268 24170 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.348224 24170 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.350915 24960 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:10.351788 24960 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.355139 25164 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:10.346722 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.356130 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1879) took 1885 ms. Trace:
I20260709 13:10:10.357129 25046 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.358069 25046 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.359222 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.355657 25164 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.362263 25195 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:10.362581 25164 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:10.362926 25195 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:10.363489 25195 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:10.364755 25072 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.342523 25160 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.370867 24170 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.306164 24734 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:10.380457 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.371799 25047 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:10.361559 25139 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.369148 25046 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.392724 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.368959 24960 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.391911 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.379108 24734 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:10.356547 22708 rpcz_store.cc:276] 0709 13:10:08.431402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.776731 (+345329us) service_pool.cc:224] Handling call
0709 13:10:09.246860 (+470129us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:09.247136 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:09.546897 (+299761us) log.cc:844] Serialized 14 byte log entry
0709 13:10:10.249056 (+702159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:10.256826 (+  7770us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:10.273632 (+ 16806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.286942 (+ 13310us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:10.317085 (+ 30143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81629,"prepare.queue_time_us":488050,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":95720,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":267,"thread_start_us":889220,"threads_started":3,"wal-append.queue_time_us":476112}
I20260709 13:10:10.409703 25072 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:10.410392 25072 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:10.410952 25072 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:10.409729 25165 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:10.375681 25038 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 13:10:10.421651 25070 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.326s	user 0.030s	sys 0.027s
I20260709 13:10:10.422966 22673 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.425983 25204 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:10.427027 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 80490.5us
I20260709 13:10:10.434520 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.435173 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:10.436009 23411 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:10.436704 25198 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:10.416810 25046 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:10.415836 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:10.432797 25070 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.031s	sys 0.027s
W20260709 13:10:10.494463 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:10.507233 25089 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.011s
W20260709 13:10:10.512934 25141 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.016s
I20260709 13:10:10.459852 25073 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:10.466797 25162 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:10:10.528100 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:10.539235 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:10.539666 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.539752 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:10.532265 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 161423us
I20260709 13:10:10.542290 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.545667 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.540047 23414 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:10.546765 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:10.547464 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.548655 25088 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:10.549042 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.550508 25013 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:10.555207 25088 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:10.555935 25088 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:10.555682 25013 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.554746 22706 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:10.554389 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 119482us
I20260709 13:10:10.560707 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.561060 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:10.561224 25091 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.562182 25165 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:10.562390 24938 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:10.563251 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:10.564195 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:10.564397 24938 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:10.564932 24938 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:10.565430 25204 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:10.565465 24938 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:10.566025 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:10.537678 25043 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.991s	user 0.011s	sys 0.005s
W20260709 13:10:10.554896 25196 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.261s	user 0.002s	sys 0.000s
I20260709 13:10:10.567603 25165 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:10.567121 25043 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.569139 22780 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 2. Current term is 3. Ops: []
W20260709 13:10:10.570739 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.572618 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.573410 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:10.574687 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:10.576436 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:10.580564 25089 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:10.583978 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:10.577666 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:10.586081 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:10.586122 22626 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:10.567031 25141 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.589973 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:10.592170 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:10.593178 25141 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:10.588759 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:10.594058 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:10.594892 25089 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:10.546906 25195 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.586462 25139 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.597723 22705 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:10.599115 25195 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:10.528034 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 191719us
I20260709 13:10:10.596680 25088 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.558526 25013 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.603273 25088 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:10.605093 22673 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:10.605698 25165 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:10.606727 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.607795 22710 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:10.611752 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.611557 25165 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:10.613046 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:10.614018 22777 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:10.602259 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.623813 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.637984 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:10.550242 25177 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.645768 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.618875 25043 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.644099 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.650236 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.650761 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.651773 23411 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:10.652705 25165 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:10.653213 25165 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.653628 25165 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:10.659431 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.650110 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.659442 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.630103 25046 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:10.641328 25195 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:10:10.672771 25148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.793s	user 0.073s	sys 0.076s
I20260709 13:10:10.687343 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.673305 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:10.673593 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.675005 25047 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:10.657524 24938 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.678092 25177 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:10.681751 24734 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.679752 22634 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:10.682682 22673 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:10.689160 25148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.817s	user 0.073s	sys 0.076s
I20260709 13:10:10.689565 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:10.693496 22777 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:10.691365 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:10.695067 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:10.625828 25164 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.707784 22701 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:10.708164 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.709167 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.709673 22778 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:10.710023 24734 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:10.682322 25175 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:10.729027 25175 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:10.729519 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 93087.4us
I20260709 13:10:10.729838 25175 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:10.721304 22706 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:10.721735 22632 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:10:10.710994 25195 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.725679 25091 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.726352 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:10.751142 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101758us
I20260709 13:10:10.729027 24938 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:10.761036 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.761583 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.766852 23415 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:10.768827 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.731679 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.754142 25195 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:10.792400 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.800195 22708 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
W20260709 13:10:10.804831 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.819947 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.823742 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.805004 25088 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:10.806479 25188 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:10:10.835196 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65993.5us
I20260709 13:10:10.833287 23418 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:10.851581 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:10.859632 23419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 557) took 1505 ms. Trace:
I20260709 13:10:10.860316 23419 rpcz_store.cc:276] 0709 13:10:09.354010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.354223 (+   213us) service_pool.cc:224] Handling call
0709 13:10:09.354733 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:10.106071 (+751338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.859178 (+753107us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b54006ff4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.859518 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480512896}
I20260709 13:10:10.849808 24170 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:10.862869 24170 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:10.863832 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:10.865406 22705 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:10.869585 24928 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:10.870270 24928 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:10.873257 24928 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:10.883589 25211 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
W20260709 13:10:10.901422 25212 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:10:10.908914 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 285253us
I20260709 13:10:10.910117 22750 leader_election.cc:400] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:10.913877 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 1956) took 1668 ms. Trace:
I20260709 13:10:10.914469 22629 rpcz_store.cc:276] 0709 13:10:09.245610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.245783 (+   173us) service_pool.cc:224] Handling call
0709 13:10:09.246161 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:09.936221 (+690060us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:10.068813 (+132592us) log.cc:844] Serialized 14 byte log entry
0709 13:10:10.204290 (+135477us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:10.204356 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:10.204418 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.913323 (+708905us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b540022e554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.913397 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:10.913617 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106993,"prepare.queue_time_us":213101,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":1475,"raft.queue_time_us":114497,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":199,"spinlock_wait_cycles":1584935168,"thread_start_us":154848,"threads_started":2,"wal-append.queue_time_us":76163}
I20260709 13:10:10.862353 25042 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:10.916949 25042 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:10.917431 25042 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:10.921031 23411 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:10.910459 22750 leader_election.cc:403] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:10:10.825904 25118 raft_consensus.cc:2764] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 13:10:10.925127 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77343.4us
I20260709 13:10:10.818945 25047 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:10:10.927383 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.826256 24938 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:10.847429 25188 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:10.929071 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 165251us
I20260709 13:10:10.831247 25088 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.940122 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:10.947576 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 457) took 1661 ms. Trace:
I20260709 13:10:10.948127 23412 rpcz_store.cc:276] 0709 13:10:09.285660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.289500 (+  3840us) service_pool.cc:224] Handling call
0709 13:10:09.439836 (+150336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:10.074245 (+634409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:10.074332 (+    87us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:10.156433 (+ 82101us) log.cc:844] Serialized 26 byte log entry
0709 13:10:10.200484 (+ 44051us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:10.204548 (+  4064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.946920 (+742372us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5400455454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.946975 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:10.947024 (+    49us) raft_consensus.cc:1685] finished
0709 13:10:10.947044 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:10.947352 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":214727,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":135,"spinlock_wait_cycles":49203968,"thread_start_us":530734,"threads_started":2,"wal-append.queue_time_us":67797}
I20260709 13:10:10.950815 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:10.843005 25070 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 13:10:10.954996 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:10.942111 25088 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:10.924403 25072 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:10.979228 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.980106 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 264) took 2315 ms. Trace:
I20260709 13:10:10.979277 25188 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:10.981734 25182 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.560s	user 0.000s	sys 0.001s
I20260709 13:10:10.984989 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 265) took 2320 ms. Trace:
I20260709 13:10:10.986810 25217 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
W20260709 13:10:10.987454 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77516us
I20260709 13:10:10.976138 24170 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.023149 24170 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:10.999554 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.022529 22780 rpcz_store.cc:276] 0709 13:10:08.664609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.412978 (+748369us) service_pool.cc:224] Handling call
0709 13:10:09.413282 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:10.569670 (+1156388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.984604 (+414934us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003408d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.984890 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2657207552}
I20260709 13:10:11.005419 22775 rpcz_store.cc:276] 0709 13:10:08.664253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.664418 (+   165us) service_pool.cc:224] Handling call
0709 13:10:08.664870 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:09.969366 (+1304496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.979626 (+1010260us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400299854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.979973 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2996384128}
I20260709 13:10:11.030982 22701 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:10:11.023571 22675 connection.cc:256] RPC call timeout handler was delayed by -1.259732281999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:10:11.037760 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59866.7us
I20260709 13:10:10.989960 25164 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.041152 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.033140 24938 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:11.061641 25197 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.049089 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:11.077540 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 138420us
I20260709 13:10:11.055784 24981 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:11.067842 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0590322819999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1726s for a response
W20260709 13:10:11.103011 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 254444us
W20260709 13:10:11.109256 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73747.5us
I20260709 13:10:11.086086 25164 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:11.127995 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:11.128844 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 478742us
I20260709 13:10:11.130079 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.079587 25204 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.124002 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.125351 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.138010 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:11.151736 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:11.153014 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.154886 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.157390 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 556) took 1810 ms. Trace:
I20260709 13:10:11.160251 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:11.170362 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:11.141649 23411 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:11.160512 23421 rpcz_store.cc:276] 0709 13:10:09.346990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.347256 (+   266us) service_pool.cc:224] Handling call
0709 13:10:09.409431 (+ 62175us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:09.497914 (+ 88483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:09.497964 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:09.500555 (+  2591us) log.cc:844] Serialized 26 byte log entry
0709 13:10:10.188871 (+688316us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:10.189028 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.708704 (+519676us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b5400515754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:10.708764 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:10.857696 (+148932us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:11.154492 (+296796us) raft_consensus.cc:1685] finished
0709 13:10:11.154523 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:11.157296 (+  2773us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352474240,"thread_start_us":588186,"threads_started":1,"wal-append.queue_time_us":689860}
W20260709 13:10:11.186121 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 150717us
I20260709 13:10:11.192677 25046 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:11.193437 25046 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:11.194746 25155 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:11.195215 25072 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:11.195601 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.195834 25061 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:11.196610 25061 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:11.197109 25061 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.198859 25020 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:11.199812 25219 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:10:11.200063 25020 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.203228 25070 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:11.203770 25070 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:11.204191 25070 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.208556 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:11.209209 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.200696 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.139020 25197 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.183655 25139 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:10:11.155972 25204 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.177999 25175 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:11.145526 25165 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.174700 25147 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.233515 25147 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.237772 25195 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.177686 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.222223 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:11.218215 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.248222 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:11.251250 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:11.252014 22710 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:11.252319 25197 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:11.252913 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64108.1us
I20260709 13:10:11.253342 24802 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:11.254756 24996 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:11.256191 24802 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:11.256644 24802 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:11.257849 24802 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.259768 24802 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:11.265439 25204 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:11.266232 22701 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:10:11.279027 22782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 273) took 2542 ms (client timeout 2175 ms). Trace:
W20260709 13:10:11.279587 22782 rpcz_store.cc:269] 0709 13:10:08.737004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.279516 (+542512us) service_pool.cc:224] Handling call
0709 13:10:11.278700 (+1999184us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54002d5854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:11.278915 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805456640}
I20260709 13:10:11.247394 25155 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:11.282198 25155 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.283912 25069 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.286422 25061 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.289686 25061 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.296177 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:11.297598 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:11.299801 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:11.306705 23418 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:11.308161 22701 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
W20260709 13:10:11.316053 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:11.316774 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.320022 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.230305 25175 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.249989 25165 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.299573 24665 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:11.352684 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 134607us
W20260709 13:10:11.354144 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 54435.6us
I20260709 13:10:11.355144 24170 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:11.356313 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 141515us
I20260709 13:10:11.357115 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.357266 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.358124 23415 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.319370 25141 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:11.229948 25042 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.363529 22747 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:11.350795 25175 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:11.377820 22702 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:11.378257 25042 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:11.382647 25020 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.369765 25223 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:11.319154 24802 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:10:11.365581 25217 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:10:11.388444 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.375795 25147 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:11.368099 24920 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:11.403183 24920 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.405328 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:11.406253 25047 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:11.408212 22708 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:11.409865 22708 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:11.410959 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 202585us
I20260709 13:10:11.406438 22705 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:11.413714 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:11.417024 24170 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:11.417474 24170 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:11.420384 25141 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:11.429387 22708 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:11.418145 25165 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:11.416755 25217 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:11.478941 24170 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.484714 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.487293 25197 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.487993 25197 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.490877 25195 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.492919 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:11.494055 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.498771 25130 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:11.503028 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60739.5us
I20260709 13:10:11.503614 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.505204 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.508270 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.519464 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.509455 22708 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:11.454485 25088 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.522552 25088 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.511312 25216 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:11.523437 25216 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.481506 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:11.525085 23412 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:10:11.525401 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44048.8us
I20260709 13:10:11.525863 25216 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:11.526451 25152 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:11.512576 24920 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:11.503159 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.526903 25152 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.527398 25152 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:11.529681 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:11.520306 23416 raft_consensus.cc:2376] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6. Re-sending same reply.
I20260709 13:10:11.532601 25069 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.533270 25069 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.535020 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:11.535759 22633 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:11.526737 24981 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:10:11.537343 25147 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:11.537818 25147 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:11.538976 25147 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.541060 25070 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.541913 25197 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.465334 24802 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.543758 24802 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.543814 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:11.546065 24802 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.429584 24928 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:10:11.546383 24981 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:11.547020 24981 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.547561 24981 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.444973 25047 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:10:11.549297 25047 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:11.549672 25047 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:11.572177 24170 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:11.556875 22704 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260709 13:10:11.583989 22704 pending_rounds.cc:85] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:10:11.589289 23415 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:11.553186 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.584293 22704 pending_rounds.cc:107] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:10:11.617434 22704 raft_consensus.cc:2889] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:11.571376 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:11.473068 25165 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:11.557503 23411 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:11.620266 24928 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.636092 24928 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.636610 25165 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.624753 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:11.628326 23384 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.217s (SENT)
I20260709 13:10:11.559895 25217 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.550567 25047 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.556999 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:11.616151 25223 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:11.562328 25195 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:11.687841 24170 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:11.532644 25216 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.117s
W20260709 13:10:11.691676 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 186568us
W20260709 13:10:11.697400 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 340325us
I20260709 13:10:11.698833 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:11.689911 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 373620us
I20260709 13:10:11.638521 24928 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:11.655714 24996 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:11.726054 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:11.731276 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:11.708209 24170 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:11.732614 24170 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:11.735140 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 441814us
I20260709 13:10:11.747462 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.747663 24170 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.758844 25236 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:11.731644 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1888) took 1180 ms. Trace:
I20260709 13:10:11.762781 22704 rpcz_store.cc:276] 0709 13:10:10.551764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.551897 (+   133us) service_pool.cc:224] Handling call
0709 13:10:10.552255 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:11.625340 (+1073085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:11.729563 (+104223us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400314554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:11.731557 (+  1994us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2309005056}
W20260709 13:10:11.768904 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51220.5us
I20260709 13:10:11.770756 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.689070 23416 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:11.772225 24170 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:11.772573 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.729856 24920 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:11.725123 24996 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.793864 22626 raft_consensus.cc:1217] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Deduplicated request from leader. Original: 3.6->[4.7-4.7]   Dedup: 4.7->[]
I20260709 13:10:11.799407 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:11.800168 22626 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260709 13:10:11.795070 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 100193us
I20260709 13:10:11.700678 25195 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:11.762308 25047 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:11.802390 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78437.8us
I20260709 13:10:11.772056 25147 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:11.803588 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:11.806578 25089 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:11.806900 25069 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.876475 25070 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.877552 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.924642 22630 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 2: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:11.824927 25089 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:11.793761 25141 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.925802 25089 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:11.801482 24996 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:11.925976 25141 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:11.928663 25141 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:11.929548 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:11.790776 24920 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:11.930863 24920 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:11.890161 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 261888us
W20260709 13:10:11.935322 25233 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 13:10:11.936239 23384 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.515s (SENT)
W20260709 13:10:11.937455 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.768086 25175 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:11.940727 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57517.7us
I20260709 13:10:11.941105 25175 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:11.941874 25175 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:11.894943 25159 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:11.891803 22704 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:10:11.895136 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 95864.1us
I20260709 13:10:11.894937 22633 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:11.944518 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.784197 25042 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:11.783252 24960 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:11.788699 25197 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:11.941821 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:11.919935 25246 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:10:11.960227 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 152771us
I20260709 13:10:11.888434 24665 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:10:11.973296 22633 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:11.991094 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:11.992259 22784 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:11.995000 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:11.989312 25088 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:11.943009 25159 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:12.002245 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:11.959369 24960 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:11.959654 25197 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:11.959147 25042 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.023008 22777 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:11.987103 25141 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:12.028230 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83816us
I20260709 13:10:12.015301 24960 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.032689 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:11.988016 25246 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:12.046742 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 243506us
I20260709 13:10:12.047217 22782 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:12.008591 25223 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:12.040803 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:12.052783 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.008949 25159 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.036545 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.026340 25042 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:12.078426 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:12.054910 24981 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.003s
I20260709 13:10:12.081918 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:12.056600 25046 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.863s	user 0.013s	sys 0.006s
I20260709 13:10:12.049266 25223 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:12.013913 25249 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:10:12.067524 22783 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:12.083166 22635 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:12.087001 22784 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:12.079062 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.129233 22777 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:12.068022 22779 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:12.110854 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:12.137545 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52756us
I20260709 13:10:12.137152 25131 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:12.156874 24938 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:12.093094 24981 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.095649 25046 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:12.180670 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.522s (SENT)
I20260709 13:10:12.182951 22784 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:12.183169 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:12.159708 25240 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.217s	user 0.001s	sys 0.001s
W20260709 13:10:12.163374 25225 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:12.127632 25165 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:12.195848 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 254127us
I20260709 13:10:12.198904 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.199802 22784 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:12.205602 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 196198us
I20260709 13:10:12.207036 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:12.177858 25130 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.678s	user 0.015s	sys 0.006s
I20260709 13:10:12.188151 25069 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:12.160068 25047 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:10:12.208031 22783 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:12.209297 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.104727 25197 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.313642 25197 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:12.321382 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:12.322361 22634 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:12.198263 25088 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:12.276360 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260709 13:10:12.328231 22709 pending_rounds.cc:85] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 8
I20260709 13:10:12.328758 22709 pending_rounds.cc:107] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260709 13:10:12.328301 25088 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.276214 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.329763 22782 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:12.277534 25195 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:12.269637 22675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.976s (SENT) [suppressed 39 similar messages]
W20260709 13:10:12.331296 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61731.5us
I20260709 13:10:12.281260 25271 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:12.219795 24938 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.331308 25088 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:12.337668 24938 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:12.338119 25256 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:10:12.284006 25046 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:12.161971 25246 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:12.208870 25152 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.681s	user 0.014s	sys 0.001s
I20260709 13:10:12.202593 25131 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:10:12.288920 25260 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:10:12.289683 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1595) took 1185 ms. Trace:
I20260709 13:10:12.295194 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:12.296245 25089 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.296329 25047 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:10:12.303360 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 87954.1us
I20260709 13:10:12.304256 24981 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.354535 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.310519 22635 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:12.329095 22709 raft_consensus.cc:2889] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:12.208070 25165 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.284377 25223 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.343096 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.177846 25261 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.178s
I20260709 13:10:12.188468 24996 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.364408 22781 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:12.346946 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.350337 22629 rpcz_store.cc:276] 0709 13:10:11.104437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.111544 (+  7107us) service_pool.cc:224] Handling call
0709 13:10:11.128506 (+ 16962us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:11.325045 (+196539us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:11.325081 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:11.339638 (+ 14557us) log.cc:844] Serialized 52 byte log entry
0709 13:10:11.652236 (+312598us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:11.719142 (+ 66906us) log.cc:844] Serialized 14 byte log entry
0709 13:10:11.727953 (+  8811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:11.777441 (+ 49488us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540022d154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:11.777517 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.289289 (+511772us) raft_consensus.cc:1685] finished
0709 13:10:12.289412 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.289614 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97453,"prepare.queue_time_us":65504,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":129606,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":14841,"spinlock_wait_cycles":245745408,"thread_start_us":290448,"threads_started":1,"wal-append.queue_time_us":594037}
I20260709 13:10:12.305974 25141 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:12.338364 24938 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:12.357703 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:12.359084 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 318405us
I20260709 13:10:12.350335 25152 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.361153 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.365553 22778 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:12.367609 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:12.348004 25046 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:12.397719 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.368757 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:12.404713 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:12.397529 22631 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:12.334496 25271 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:10:12.339280 25042 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:12.407936 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127255us
I20260709 13:10:12.401024 22775 raft_consensus.cc:2376] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4. Re-sending same reply.
I20260709 13:10:12.408840 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:12.371961 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.412822 22783 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6.
I20260709 13:10:12.364483 25231 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:12.417336 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59745.5us
I20260709 13:10:12.360388 25270 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:10:12.419831 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:12.420706 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:12.422523 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.395573 25223 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:12.397059 25165 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:12.429531 25263 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:10:12.430236 25131 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:12.439366 25131 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.400306 25089 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:12.431140 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:12.431329 25270 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:12.431983 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:12.432848 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:12.361912 24920 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.423322 25231 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.434132 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:12.434811 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.426345 25069 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:12.413775 25046 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.409410 24938 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.434741 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:12.409747 24960 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.397089 25152 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.415529 25130 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.477s
I20260709 13:10:12.835320 25020 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.894459 24920 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.894246 25020 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:12.659492 25159 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 13:10:12.442567 25131 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.652870 22634 raft_consensus.cc:2410] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:12.895696 25159 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.412220 25164 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:12.896981 22633 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:12.831035 25247 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.715s	user 0.003s	sys 0.001s
I20260709 13:10:12.449398 25270 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:12.664268 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 318) took 1022 ms. Trace:
I20260709 13:10:12.408687 25195 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:12.899426 22783 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6.
W20260709 13:10:12.900279 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.689884 22626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1606) took 1213 ms. Trace:
W20260709 13:10:12.652781 22675 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.836s (SENT)
I20260709 13:10:12.557147 22782 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:12.687947 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:12.729835 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:12.903378 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.831673 22784 raft_consensus.cc:2410] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:10:12.911243 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 181558us
I20260709 13:10:12.913522 22630 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:12.893620 25046 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:12.905716 22705 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
W20260709 13:10:12.907745 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.909158 22626 rpcz_store.cc:276] 0709 13:10:11.476704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.506774 (+ 30070us) service_pool.cc:224] Handling call
0709 13:10:11.507245 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.803776 (+296531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.688598 (+884822us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b5400059ea0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0a02 00007f1806edcdd6 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:12.689559 (+   961us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400070654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:12.689789 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693818368}
I20260709 13:10:12.918187 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:12.671537 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 514233us
I20260709 13:10:12.893502 25130 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:12.919588 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:12.920892 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:12.419469 25141 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.684016 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1889) took 2131 ms. Trace:
I20260709 13:10:12.532680 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:12.893487 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.752s (SENT)
I20260709 13:10:12.927521 22709 rpcz_store.cc:276] 0709 13:10:10.552142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.552351 (+   209us) service_pool.cc:224] Handling call
0709 13:10:10.552814 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:12.364305 (+1811491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.683648 (+319343us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400087d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:12.683907 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3931320192}
I20260709 13:10:12.923295 22775 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:12.847384 25277 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:10:12.933573 23415 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:12.898439 22627 rpcz_store.cc:276] 0709 13:10:11.642211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.643702 (+  1491us) service_pool.cc:224] Handling call
0709 13:10:11.686874 (+ 43172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:11.687464 (+   590us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:12.089247 (+401783us) log.cc:844] Serialized 14 byte log entry
0709 13:10:12.160037 (+ 70790us) log.cc:844] Serialized 14 byte log entry
0709 13:10:12.663401 (+503364us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b18003cb500. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:10:12.663662 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:12.663746 (+    84us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:12.663828 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.663953 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.664178 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":601749,"prepare.queue_time_us":105009,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":4466,"raft.queue_time_us":581281,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":2670,"spinlock_wait_cycles":4983680,"thread_start_us":328745,"threads_started":1}
I20260709 13:10:12.898764 25159 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.623026 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.914067 25020 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.955964 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.898887 25270 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.957228 22781 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:12.958242 25270 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:12.415392 25272 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.527s
I20260709 13:10:12.940779 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:12.967674 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.671236 25249 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:12.971329 25249 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:12.927155 25195 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:12.973058 25195 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.972447 25249 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.952059 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.976948 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:12.916097 25069 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.984395 22598 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:12.985335 25236 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:12.985731 25236 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.985773 23411 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:12.986128 25236 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:12.986860 25069 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.929059 25152 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:12.449626 22701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:56860
I20260709 13:10:12.989296 25269 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:12.989714 25269 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:12.989778 22701 raft_consensus.cc:606] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:12.990348 22701 raft_consensus.cc:614] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:12.990638 25269 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:12.991575 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.992079 25269 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:12.992604 22779 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 3: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:12.994082 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:12.994899 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:12.912285 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 224445us
I20260709 13:10:12.930780 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:12.936398 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:12.943888 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:12.942271 23420 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:12.998689 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.946388 25150 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:12.946369 23416 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:12.587327 25072 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.947094 22628 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:12.936136 25131 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:12.947777 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:12.948568 24938 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:12.948663 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:13.027360 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 82301.9us
I20260709 13:10:13.029635 22776 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:12.949468 25231 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:12.689168 25261 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:12.916018 25147 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:10:12.950496 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57080.9us
W20260709 13:10:13.038465 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.018s (SENT)
I20260709 13:10:12.940718 25141 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:12.935861 24981 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:10:13.001900 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.610442 24996 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:12.627703 25042 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:12.686951 25262 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.924296 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:13.009696 22777 raft_consensus.cc:2410] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:13.014978 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.016006 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.053108 22781 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:10:13.016170 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85539.2us
W20260709 13:10:13.054020 22710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.034s	sys 0.053s
W20260709 13:10:13.054389 22710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.034s	sys 0.053s
I20260709 13:10:13.018908 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:12.924834 25130 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:13.032536 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.028177 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.054872 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:13.039286 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.020071 25249 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:13.055827 22784 raft_consensus.cc:2410] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:13.033388 24963 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:13.037719 23414 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:13.052594 25197 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.736s	user 0.008s	sys 0.005s
I20260709 13:10:13.037550 25261 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:13.085981 23384 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.036s (SENT)
I20260709 13:10:13.086740 23421 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:13.057734 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.087801 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.088349 22779 raft_consensus.cc:2410] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:10:13.074276 22783 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:10:13.088686 22781 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:13.065487 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 301) took 1257 ms. Trace:
I20260709 13:10:13.048385 23412 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:13.050768 25242 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.000s	sys 0.029s
W20260709 13:10:13.107055 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 487402us
I20260709 13:10:13.060225 24981 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:13.043570 25164 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:10:13.039533 25072 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.066563 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.116492 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.128618 22777 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:13.119575 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.071130 24963 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:13.039826 24802 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:10:13.099403 25242 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.000s	sys 0.030s
I20260709 13:10:13.091938 22710 rpcz_store.cc:276] 0709 13:10:11.808360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.808509 (+   149us) service_pool.cc:224] Handling call
0709 13:10:11.885066 (+ 76557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:11.917222 (+ 32156us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:10:11.987645 (+ 70423us) log.cc:844] Serialized 14 byte log entry
0709 13:10:12.498335 (+510690us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800337540. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:10:13.064922 (+566587us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:13.065037 (+   115us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:13.065111 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.065224 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.065426 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70368,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":20228,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":12482,"spinlock_wait_cycles":32436608,"thread_start_us":556335,"threads_started":1}
I20260709 13:10:13.070793 25046 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.090368 24938 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:13.145444 22782 raft_consensus.cc:2368] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4.
I20260709 13:10:13.062462 25262 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.143707 24963 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:13.154340 25141 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:13.155827 22809 heartbeater.cc:646] Failed to heartbeat to 127.21.5.190:44025 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.5.190:44025 timed out after 15.000s (SENT)
W20260709 13:10:13.170101 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98601us
I20260709 13:10:13.157059 24960 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:13.145917 25197 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:13.159152 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:13.091005 24996 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 13:10:13.170974 23384 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.050s (SENT)
I20260709 13:10:13.192339 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:13.171633 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.164819 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.116700 22675 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.872s (SENT)
I20260709 13:10:13.135582 25130 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.187150 24981 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.190773 23411 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:13.197098 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:13.191327 25046 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.203908 24996 consensus_queue.cc:1243] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer 6a31ac6451ff4f73bee634487d84d001 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:10:13.205375 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.216395 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.191977 25197 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:13.227934 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.236728 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 190812us
I20260709 13:10:13.240304 22782 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:13.240729 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.239630 25280 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 13:10:13.229420 25197 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:13.226982 25236 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.269802 25236 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:13.272673 25262 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:13.274798 25231 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.275388 25231 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.275859 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.277163 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.277670 25231 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:13.278362 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.259938 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 152024us
W20260709 13:10:13.279603 22675 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.937s (SENT)
I20260709 13:10:13.279700 23411 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:13.280041 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:13.228283 25130 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:13.226439 24938 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:13.263037 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.264612 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1609) took 1442 ms. Trace:
I20260709 13:10:13.266530 25072 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:13.258667 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.295678 22634 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:10:13.295841 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.287810 22628 rpcz_store.cc:276] 0709 13:10:11.821808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.821987 (+   179us) service_pool.cc:224] Handling call
0709 13:10:11.885808 (+ 63821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.010981 (+1125173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.264290 (+253309us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003a7354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:13.264530 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434324352}
I20260709 13:10:13.284317 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:13.285514 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 256630us
W20260709 13:10:13.258427 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 203631us
W20260709 13:10:13.289070 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.293860 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.292922 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.300772 22782 raft_consensus.cc:2410] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:13.301578 22627 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:13.310475 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.325794 22708 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:13.310911 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.304652 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.328348 22775 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:13.315477 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.311411 22710 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:13.330397 22706 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:13.317467 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.332793 22702 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:13.311834 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.333979 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:13.273942 25261 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.308318 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.376464 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.377457 22782 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:13.340130 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.378605 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:13.351217 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45614.5us
I20260709 13:10:13.394083 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.395043 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113811us
I20260709 13:10:13.393697 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.301874 25069 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.402225 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:13.403026 23415 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:13.401204 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.404490 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.404799 22783 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:13.292327 25231 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:13.397413 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.415992 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.416456 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.396116 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.417155 23418 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:13.417606 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.405421 22782 raft_consensus.cc:2376] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5. Re-sending same reply.
I20260709 13:10:13.418850 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.419981 22783 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:13.408023 22779 raft_consensus.cc:2376] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6. Re-sending same reply.
I20260709 13:10:13.421377 22783 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:13.305173 25285 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:13.424674 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:13.423226 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 210200us
I20260709 13:10:13.398118 24960 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:10:13.394476 24802 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:10:13.428584 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.452760 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.479197 22708 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:13.474728 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.482203 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.482805 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:13.453909 23384 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.175s (SENT)
I20260709 13:10:13.447985 23415 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:13.459620 22631 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:10:13.501578 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 75587.7us
I20260709 13:10:13.503835 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.505347 22709 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:13.422063 25261 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:13.510315 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.511148 22782 raft_consensus.cc:2376] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8. Re-sending same reply.
I20260709 13:10:13.424966 25139 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:13.576537 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.577824 22782 raft_consensus.cc:2410] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:13.588826 25291 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:10:13.593137 25295 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:10:13.599874 25278 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.349s	user 0.001s	sys 0.001s
I20260709 13:10:13.561394 25155 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.509284 25223 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.622889 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 296) took 1090 ms. Trace:
I20260709 13:10:13.420249 25069 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.563025 25284 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.577791 25269 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.002s
I20260709 13:10:13.613569 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:13.617733 22634 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:13.618973 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.620206 22630 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:13.635308 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 198813us
I20260709 13:10:13.664050 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:13.667029 23384 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.423s (SENT)
I20260709 13:10:13.652112 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:13.456552 25285 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.646750 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:13.664047 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.606712 25046 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:13.607954 25139 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.679589 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:13.671826 22627 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:13.683241 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.683981 22707 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:13.686255 22632 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:13.625068 22778 rpcz_store.cc:276] 0709 13:10:12.532758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.038999 (+506241us) service_pool.cc:224] Handling call
0709 13:10:13.051127 (+ 12128us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:10:13.161504 (+110377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.161542 (+    38us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:10:13.195918 (+ 34376us) log.cc:844] Serialized 104 byte log entry
0709 13:10:13.316704 (+120786us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:13.451829 (+135125us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.456065 (+  4236us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.459140 (+  3075us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.480145 (+ 21005us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.480366 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.480495 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.622530 (+142035us) raft_consensus.cc:1685] finished
0709 13:10:13.622562 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.622803 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18766,"prepare.queue_time_us":627279,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":95,"raft.queue_time_us":31049,"raft.run_cpu_time_us":580,"raft.run_wall_time_us":19228,"spinlock_wait_cycles":22093568,"thread_start_us":210170,"threads_started":2,"wal-append.queue_time_us":200497}
I20260709 13:10:13.670653 25302 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:13.696749 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.646327 25223 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.683753 25046 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.703209 25141 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.653251 25155 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.708057 23419 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:13.704689 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.693523 25302 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:13.733222 22705 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:13.737349 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:13.704397 25297 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:10:13.671072 25284 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:13.738663 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.741477 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:13.744261 22635 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:13.745359 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 237454us
I20260709 13:10:13.706986 25164 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.746170 25289 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.751398 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.751449 25164 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.752882 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.754045 25164 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:13.754112 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.755023 22705 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:13.756130 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.756909 22705 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:13.757689 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.758518 22705 raft_consensus.cc:2410] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:13.759310 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.759900 22705 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:13.760718 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.760970 25197 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.770540 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.771684 22703 raft_consensus.cc:2368] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:13.772554 25197 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:13.772783 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.775100 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.764307 25155 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:13.708897 25046 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:13.765013 23414 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.001s
I20260709 13:10:13.765442 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.781780 22628 raft_consensus.cc:2368] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:13.767777 25159 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:13.768778 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:13.782712 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:13.783365 25164 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:13.775470 25295 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:10:13.739879 25270 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.781s	user 0.012s	sys 0.003s
I20260709 13:10:13.738723 25269 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.786298 24963 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.607s	user 0.003s	sys 0.007s
W20260709 13:10:13.795529 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.793254 22708 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:13.798930 25164 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.774327 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.717947 25311 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:13.816176 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68697.9us
I20260709 13:10:13.814565 24963 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.816709 25164 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:13.815553 22703 raft_consensus.cc:2368] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:13.815934 25311 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:13.809098 22707 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:13.731348 25141 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:13.815080 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.835335 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:13.836333 22710 raft_consensus.cc:2368] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4.
I20260709 13:10:13.839291 22778 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:13.840559 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.845947 22705 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:13.789589 23414 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:13.792114 25155 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.790426 25159 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:13.818364 24963 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:13.869293 22706 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.003s
I20260709 13:10:13.874910 22706 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:13.878948 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.879969 22779 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:13.858713 25164 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.896168 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 151141us
I20260709 13:10:13.897344 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:13.894760 22749 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:13.898178 22782 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:13.961410 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 64009.2us
I20260709 13:10:13.967641 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:13.751713 25289 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:14.036465 22775 consensus_meta.cc:287] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.682s	user 0.011s	sys 0.003s
I20260709 13:10:14.037012 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:14.037269 22775 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:14.037984 22780 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:10:14.039952 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:14.041224 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:14.042265 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74797.8us
W20260709 13:10:14.036633 22702 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.703s	user 0.016s	sys 0.002s
I20260709 13:10:14.044697 22702 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:13.865700 25242 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:13.865178 25195 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:13.825902 25269 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:13.897162 25295 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:10:14.109561 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 214810us
I20260709 13:10:14.113294 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.125229 25147 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.125530 25288 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:14.126334 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83284.1us
I20260709 13:10:14.126420 25150 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:10:14.127413 25315 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
W20260709 13:10:14.128226 25279 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.069s	user 0.005s	sys 0.000s
I20260709 13:10:14.130409 25069 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.134967 25139 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:14.147787 25308 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
W20260709 13:10:14.171923 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:14.201102 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.015s	sys 0.091s
W20260709 13:10:14.241039 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.015s	sys 0.091s
I20260709 13:10:13.819617 25270 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.222s
I20260709 13:10:14.211213 22706 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:14.211063 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.212517 25318 raft_consensus.cc:2764] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:10:13.800367 25313 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.418s
I20260709 13:10:14.215785 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:14.119575 25304 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
W20260709 13:10:14.279449 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63816.4us
I20260709 13:10:14.217268 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:13.892313 25155 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:14.125351 25236 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:14.126085 25164 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.130332 25303 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 13:10:14.207403 25285 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:14.296686 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 168152us
I20260709 13:10:14.280711 25223 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:14.218362 25159 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:14.216965 25311 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:14.251760 25282 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:14.243395 25069 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.253286 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 578) took 1325 ms. Trace:
I20260709 13:10:14.247207 25284 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.242038 25267 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:14.219717 25195 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:14.264463 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:14.278194 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67246.3us
I20260709 13:10:14.282126 22710 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:14.282689 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:13.866195 25302 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.217419 25147 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:14.291951 22709 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.727s	user 0.008s	sys 0.010s
I20260709 13:10:14.348685 22709 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:14.349228 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 1939) took 1030 ms. Trace:
I20260709 13:10:14.349547 22709 rpcz_store.cc:276] 0709 13:10:13.318406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.503547 (+185141us) service_pool.cc:224] Handling call
0709 13:10:14.349156 (+845609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:14.244269 22780 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:14.281580 24963 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:14.321060 23421 rpcz_store.cc:276] 0709 13:10:12.928161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.928303 (+   142us) service_pool.cc:224] Handling call
0709 13:10:12.928682 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.087205 (+158523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.252947 (+1165742us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54005f7654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:14.253207 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":318864128}
I20260709 13:10:14.351364 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:14.299543 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.012s
I20260709 13:10:14.276135 25288 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.302759 25313 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.329123 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:10:14.341395 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58810.4us
I20260709 13:10:14.358280 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.359015 22702 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.344525 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.339644 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:14.352252 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.008s	sys 0.012s
I20260709 13:10:14.331871 25223 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:14.360138 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 656136us
I20260709 13:10:14.297948 25046 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:14.335237 25264 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.365784 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.373622 22780 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:14.346768 25303 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:14.362861 25313 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:14.380831 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.371626 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:14.334728 25267 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:14.375694 22710 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:10:14.383935 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.384804 22710 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:14.385890 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.369745 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1966) took 1450 ms. Trace:
I20260709 13:10:14.360291 25312 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:14.380905 22709 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:10:14.343318 25282 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.387230 22776 rpcz_store.cc:276] 0709 13:10:12.919014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.072938 (+153924us) service_pool.cc:224] Handling call
0709 13:10:13.120623 (+ 47685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.233844 (+113221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.233878 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:13.233926 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:13.233960 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.369438 (+1135478us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54003c37d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:14.369491 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.369680 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259765632}
W20260709 13:10:14.401194 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56812.5us
I20260709 13:10:14.391351 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:14.403169 22780 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:14.386531 22710 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:14.399938 22779 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.004s
I20260709 13:10:14.368335 25141 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.407874 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.409392 22779 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:14.336136 25069 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.377367 25303 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:14.438903 25303 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:14.441815 25292 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:14.410159 25326 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:14.443150 25326 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:14.443625 25326 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:14.452615 25147 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.454742 25195 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.455083 25164 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.456866 25288 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.457167 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:14.458274 25319 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:10:14.378412 25264 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:14.460232 25069 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.460815 25264 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.460960 25289 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:14.461730 25267 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.462661 25155 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:14.463209 25267 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:14.464469 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.465183 22775 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:14.426509 22702 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:14.419073 25311 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:14.428292 25269 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.429374 25313 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.467337 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.469888 22702 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:14.430140 25046 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 13:10:14.431663 25282 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:14.413055 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.467799 25141 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:14.468371 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 471) took 1558 ms. Trace:
I20260709 13:10:14.472414 22782 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:14.472736 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.475832 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.477443 22706 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:14.477669 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.463663 25292 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:14.423547 25312 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:14.482717 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69725.6us
I20260709 13:10:14.480924 22705 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:14.484133 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.498925 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.498674 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.500624 22706 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:14.493225 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.500748 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.501598 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.482980 25292 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:14.505779 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:14.475876 23412 rpcz_store.cc:276] 0709 13:10:12.909594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.915656 (+  6062us) service_pool.cc:224] Handling call
0709 13:10:13.027010 (+111354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.096353 (+ 69343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.468115 (+1371762us) spinlock_profiling.cc:243] Waited 72.6 ms on lock 0x7b540020ab50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:14.468283 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167030144}
W20260709 13:10:14.507929 22531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 235) took 21804 ms (client timeout 15000 ms). Trace:
I20260709 13:10:14.509766 25159 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:14.510196 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.512041 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.512301 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.514575 25292 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:14.516300 22706 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:14.494069 25282 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:14.516729 22703 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:14.518093 22706 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:14.562115 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57924.7us
I20260709 13:10:14.564453 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.521616 25164 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:10:14.565450 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.567060 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.568889 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.569635 23411 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:14.570187 23411 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:14.544780 22780 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:14.573972 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.576359 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:14.577521 22702 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:14.566286 23412 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:10:14.580214 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 181958us
I20260709 13:10:14.572547 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:14.582360 22749 leader_election.cc:400] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:14.582856 22749 leader_election.cc:403] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:14.585225 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:14.585415 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.595803 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:14.596441 22703 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:14.596948 22706 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:10:14.503424 25155 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:14.603256 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.603402 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.604315 22627 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:14.603519 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.605984 22709 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:14.603256 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.608235 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.608901 22635 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:14.608973 22675 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.973s (SENT)
I20260709 13:10:14.609500 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:14.610275 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:14.611426 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 146182us
I20260709 13:10:14.609687 22635 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 13:10:14.503919 25313 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
W20260709 13:10:14.616966 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 259619us
I20260709 13:10:14.621189 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.622057 22703 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:14.623296 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.593680 25223 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:14.625087 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:14.595474 25282 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:14.631866 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:14.624351 23421 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:14.633960 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:14.634564 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8446397220000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
I20260709 13:10:14.634517 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:14.637085 22675 connection.cc:256] RPC call timeout handler was delayed by -1.831139721999989s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22440000000000004s for a response
W20260709 13:10:14.637531 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8032397219999439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2275s for a response
W20260709 13:10:14.637876 22675 connection.cc:256] RPC call timeout handler was delayed by -1.7707087700000557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
I20260709 13:10:14.531569 25320 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:10:14.580611 25046 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 13:10:14.643265 22706 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:14.611943 25270 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:14.644785 22675 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.986s (SENT)
W20260709 13:10:14.667174 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.672s (SENT)
I20260709 13:10:14.671814 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:14.669080 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:14.674481 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:14.675499 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.681443 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.676347 22706 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:14.679608 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.683573 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.683925 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.684561 22776 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:14.684659 22783 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:14.680148 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:14.685969 22778 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:14.635774 25282 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:14.687486 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:14.663865 25272 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:14.700685 22749 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:14.710731 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71788.1us
W20260709 13:10:14.713815 22675 leader_election.cc:336] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.004s (SENT)
I20260709 13:10:14.714334 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:14.717887 22777 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:14.718425 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:14.720333 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77087.7us
I20260709 13:10:14.723181 25195 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:14.718374 22777 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:14.687676 25261 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.124s	user 0.014s	sys 0.001s
I20260709 13:10:14.676031 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:14.508536 22531 rpcz_store.cc:269] 0709 13:09:52.632925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.635757 (+  2832us) service_pool.cc:224] Handling call
0709 13:09:52.925523 (+289766us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:10:14.437016 (+21511493us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:08.628123 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:08.774755 (+146632us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:08.774797 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:10:08.793375 (+ 18578us) tablet.cc:662] Decoding operations
0709 13:10:08.813555 (+ 20180us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:08.813639 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:08.813705 (+    66us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:10:08.844336 (+ 30631us) tablet.cc:701] Row locks acquired
0709 13:10:08.844375 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:10:08.862605 (+ 18230us) write_op.cc:270] Start()
0709 13:10:08.862775 (+   170us) write_op.cc:276] Timestamp: P: 1783602608862530 usec, L: 0
0709 13:10:08.862798 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:10:08.869015 (+  6217us) log.cc:844] Serialized 923 byte log entry
0709 13:10:10.445254 (+1576239us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b
0709 13:10:11.196780 (+751526us) op_driver.cc:464] REPLICATION: finished
0709 13:10:11.507909 (+311129us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806eda32a 00007f1806e5ff14 00007f1806e5fec3 00007f1806e5fe38 00007f1806e5fdf3 00007f1806e5f0dc 00007f1806e88e16 00007f1806e70cbb 00007f1806e5b39d 00007f1806e5ed75
0709 13:10:11.779829 (+271920us) write_op.cc:301] APPLY: starting
0709 13:10:11.811756 (+ 31927us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:11.878875 (+ 67119us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:11.878923 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:11.922217 (+ 43294us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:11.922562 (+   345us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:10:11.922643 (+    81us) write_op.cc:312] APPLY: finished
0709 13:10:11.923040 (+   397us) log.cc:844] Serialized 29 byte log entry
0709 13:10:11.923247 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:11.923383 (+   136us) write_op.cc:454] Released schema lock
0709 13:10:11.923448 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:11.923996 (+   548us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:11.924027 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3471072,"reported_tablets":19,"spinlock_wait_cycles":64377088,"child_traces":[["op",{"apply.queue_time_us":492746,"apply.run_cpu_time_us":3324,"apply.run_wall_time_us":200381,"num_ops":2,"prepare.queue_time_us":538339,"prepare.run_cpu_time_us":155244,"prepare.run_wall_time_us":2166919,"raft.queue_time_us":286958,"raft.run_cpu_time_us":97965,"raft.run_wall_time_us":328841,"replication_time_us":2333716,"spinlock_wait_cycles":17312000,"thread_start_us":2380004,"threads_started":4,"wal-append.queue_time_us":1937151,"wal-append.run_cpu_time_us":29857,"wal-append.run_wall_time_us":2113386}]]}
I20260709 13:10:14.726848 22777 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.745966 22777 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:14.694121 25282 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:14.636052 25330 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:10:14.690573 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.717414 25272 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:14.718919 25285 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.680346 25270 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:14.730942 25069 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:14.725003 25336 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:10:14.759778 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59203.2us
I20260709 13:10:14.761710 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:14.749809 24960 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:14.751996 24996 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.772238 24996 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:14.772825 25328 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:10:14.780357 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1969) took 1860 ms. Trace:
I20260709 13:10:14.780821 22781 rpcz_store.cc:276] 0709 13:10:12.919814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.089507 (+169693us) service_pool.cc:224] Handling call
0709 13:10:13.119748 (+ 30241us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:13.127450 (+  7702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.127489 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:14.106151 (+978662us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:14.231178 (+125027us) log.cc:844] Serialized 26 byte log entry
0709 13:10:14.477804 (+246626us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:14.477863 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.477933 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:14.779976 (+302043us) raft_consensus.cc:1685] finished
0709 13:10:14.779998 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.780279 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3405,"raft.queue_time_us":15087,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":13906,"spinlock_wait_cycles":46049152,"thread_start_us":242808,"threads_started":1,"wal-append.queue_time_us":366485}
I20260709 13:10:14.757797 25272 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:14.753607 23416 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:14.785665 25242 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:14.793226 22710 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:14.800905 25337 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:14.801599 25337 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:14.802029 25337 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:14.770238 25336 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:14.789661 25195 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:14.902818 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.912892 25342 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:14.913941 25329 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:14.760195 25164 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:10:14.811906 24960 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:14.925850 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 1974) took 1088 ms. Trace:
I20260709 13:10:14.607777 25331 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:10:14.964112 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 295095us
W20260709 13:10:14.969813 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 335234us
I20260709 13:10:14.929345 25313 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:10:14.976775 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 252992us
I20260709 13:10:14.943886 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:14.634819 25312 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:14.964874 22784 rpcz_store.cc:276] 0709 13:10:13.836527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.838737 (+  2210us) service_pool.cc:224] Handling call
0709 13:10:14.919346 (+1080609us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b5800043598. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:10:14.919679 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:14.920013 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:14.922742 (+  2729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:14.922826 (+    84us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:14.922884 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.922971 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.925254 (+  2283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7262976}
W20260709 13:10:15.021772 22675 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.104s (SENT)
W20260709 13:10:15.008306 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131747us
I20260709 13:10:14.940704 25195 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.002049 22779 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:14.985224 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 343358us
W20260709 13:10:15.022250 22750 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.987s (SENT)
I20260709 13:10:15.027540 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.033952 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.029090 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:14.984050 25313 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:14.964602 25350 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:14.935536 25329 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:14.977950 25164 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:15.046820 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.047166 23419 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:10:15.048710 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64554.5us
W20260709 13:10:15.049827 22675 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.107s (SENT)
I20260709 13:10:15.029475 25348 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:10:15.024613 25267 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.552s	user 0.009s	sys 0.002s
I20260709 13:10:15.001921 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.040655 25197 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:15.053483 25348 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:15.055879 25197 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:15.012419 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.056728 25348 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.035496 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:15.058241 25348 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.058444 25336 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:15.064947 25303 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.002s
I20260709 13:10:15.065598 25303 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.067322 25270 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:15.067713 25326 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.004s
I20260709 13:10:15.068012 25356 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.068131 25270 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:15.068951 25356 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:15.069298 25270 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:15.069434 25356 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.069867 25356 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.027449 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.071368 25270 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.072515 25355 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.072752 25270 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.073053 25355 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.050407 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:15.078755 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:15.035202 25292 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.002s
W20260709 13:10:15.057489 25346 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:10:15.068385 25326 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.080499 22775 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:15.038929 25313 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:15.034549 23421 raft_consensus.cc:2376] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:10:15.087837 22630 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:15.092051 25147 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.092141 25292 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:15.093080 25292 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.092975 25147 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:15.108603 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73190.6us
I20260709 13:10:15.109967 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:15.109982 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.094309 25264 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.117977 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.119087 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.120055 22628 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:15.120847 22626 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:15.121726 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.123311 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.124990 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:15.118340 25343 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:10:15.118433 25147 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:15.135710 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85945.6us
W20260709 13:10:15.136824 22675 connection.cc:256] RPC call timeout handler was delayed by -1.6060087700000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 13:10:15.137094 22675 connection.cc:256] RPC call timeout handler was delayed by -1.4036397219999799s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2719s for a response
W20260709 13:10:15.137264 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2811087700000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260709 13:10:15.137419 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2245397219999177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2918s for a response
W20260709 13:10:15.137578 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1489397219999091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3002s for a response
W20260709 13:10:15.137737 22675 connection.cc:256] RPC call timeout handler was delayed by -1.14525908600001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 13:10:15.138527 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.140610 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.140964 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:15.140637 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 156286us
W20260709 13:10:15.141956 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.007s (SENT)
I20260709 13:10:15.141815 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:15.147863 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.150701 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.151583 22626 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:15.151867 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.155841 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.156495 22626 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:15.157562 25360 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.157747 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.051728 24960 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.158551 22750 leader_election.cc:400] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:15.158747 22628 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:15.167595 22750 leader_election.cc:403] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:10:15.164625 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:15.168098 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 446757us
I20260709 13:10:15.162250 22634 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:15.120118 25264 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.176395 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.180331 23421 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:15.172058 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.146896 25236 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:15.166097 25340 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.377s	user 0.001s	sys 0.000s
W20260709 13:10:15.187116 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 77209us
I20260709 13:10:15.162983 25284 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:15.166807 25352 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.187950 22633 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:15.160657 25326 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:15.174034 25069 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.158493 25360 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:15.196565 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:15.161294 25345 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.374s	user 0.002s	sys 0.000s
I20260709 13:10:15.193934 25284 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:15.199565 25365 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:15.207521 25320 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
W20260709 13:10:15.210565 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52196.4us
I20260709 13:10:15.143415 25261 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:10:15.201226 22628 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:10:15.224562 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1613) took 2303 ms (client timeout 2092 ms). Trace:
I20260709 13:10:15.215099 22633 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:15.215616 22627 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:15.226745 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:15.225009 22631 rpcz_store.cc:269] 0709 13:10:12.913072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.914703 (+  1631us) service_pool.cc:224] Handling call
0709 13:10:15.216020 (+2301317us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540022b854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:15.216139 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061185280}
I20260709 13:10:15.231304 25302 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:15.235292 25303 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.154968 25329 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.227119 25355 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:15.223842 25365 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:15.239188 25223 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:15.239681 25223 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:15.240029 25365 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.242635 25344 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:15.243115 25344 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:15.243781 25292 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.244141 25344 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.244988 22750 leader_election.cc:400] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:15.245726 25344 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.251075 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.251986 22781 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:15.238219 25284 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.252875 23411 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:15.253324 22777 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:10:15.253731 22777 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:15.205945 25366 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:10:15.255628 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.255968 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.254572 22777 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.256971 25292 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.257808 22777 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:10:15.257529 25292 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.259819 25292 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.246201 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.263077 25264 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:15.263885 25267 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.217479 25360 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.263181 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.264945 25360 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.265311 22781 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:15.270253 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.283660 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.285284 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:15.287928 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.292908 22777 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:10:15.294090 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.222596 25262 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:15.221253 25046 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:10:15.309695 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:15.313831 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.294935 25267 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.290259 25342 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.317102 22782 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:15.295656 25331 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.319802 22783 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:15.302448 25262 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.325947 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.334234 22777 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
W20260709 13:10:15.338181 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.316298 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:15.308390 25354 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:15.340274 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.307888 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:15.318171 25282 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.001s
I20260709 13:10:15.356123 24996 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.356792 25325 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:15.357273 25325 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.357388 25350 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:15.357712 25325 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.359462 25147 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.360477 23416 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 5: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:15.363389 22784 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:15.318092 25336 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:10:15.373232 25311 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
W20260709 13:10:15.374181 25159 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.864s	user 0.004s	sys 0.009s
I20260709 13:10:15.340054 25262 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.333453 25046 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.334041 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.409346 23412 raft_consensus.cc:2410] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:15.411043 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:15.411885 25336 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:15.412981 25336 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:15.384205 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.304508 25337 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.322744 25352 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:10:15.347746 25354 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.365574 25334 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:15.368192 25282 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:15.430658 25321 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:10:15.427106 25272 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.000s
I20260709 13:10:15.437270 25272 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.426898 25354 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:15.445976 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 79788.5us
I20260709 13:10:15.426244 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.368808 25356 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.464112 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:15.465050 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 171075us
I20260709 13:10:15.425388 25360 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.473260 25360 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.465951 25334 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:10:15.448078 25272 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:15.455458 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 316966us
W20260709 13:10:15.485185 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 301840us
I20260709 13:10:15.491230 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:15.491348 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.497124 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:15.508677 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.503425 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:15.523569 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.503114 24981 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:15.507184 25288 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:15.515353 25303 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:15.508379 25311 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:10:15.538858 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.505704 25046 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.531539 24981 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.527812 25360 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.547827 25337 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.551509 25282 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:15.527400 25373 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:15.562038 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70991.6us
I20260709 13:10:15.564491 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.532157 25288 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.572463 25360 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:15.577695 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:15.561022 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69750.2us
I20260709 13:10:15.581288 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:15.583401 23421 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:15.564450 24981 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.583657 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.584856 25236 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.589524 25356 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:15.590219 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.593291 25152 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.588280 25313 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.542037 25303 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.602555 22776 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:15.561978 25141 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:15.623773 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.624787 25356 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.622085 25313 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.626565 25303 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:15.627754 25338 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.825s	user 0.004s	sys 0.000s
I20260709 13:10:15.612305 25366 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.617632 25155 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:15.630690 25155 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:15.631626 25155 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.632472 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.633147 23414 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:15.636854 25284 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:15.639218 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.641628 22776 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:15.643182 25303 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.644202 25354 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.645350 22782 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:15.567294 25046 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.620071 25147 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:15.647993 25046 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.599133 25288 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:15.565353 25311 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.651664 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1946) took 1914 ms. Trace:
I20260709 13:10:15.652081 22708 rpcz_store.cc:276] 0709 13:10:13.737439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.859902 (+122463us) service_pool.cc:224] Handling call
0709 13:10:13.860462 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:13.870517 (+ 10055us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:14.447670 (+577153us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b18003be360. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:10:14.448119 (+   449us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.605227 (+157108us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:14.607805 (+  2578us) log.cc:844] Serialized 26 byte log entry
0709 13:10:14.607984 (+   179us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:14.788463 (+180479us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b18003be360. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:10:14.788756 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.788988 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.789070 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:15.627152 (+838082us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:15.651434 (+ 24282us) raft_consensus.cc:1685] finished
0709 13:10:15.651458 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:15.651600 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44779,"prepare.queue_time_us":752661,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":102,"raft.queue_time_us":17038,"raft.run_cpu_time_us":495,"raft.run_wall_time_us":37213,"spinlock_wait_cycles":320714112,"thread_start_us":156491,"threads_started":1,"wal-append.queue_time_us":308143}
I20260709 13:10:15.617103 25152 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.622076 25360 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:15.606694 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.648417 25195 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:15.655577 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.632898 25155 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:15.661464 23421 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:15.649262 25303 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:15.662864 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.690002 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.690428 23411 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.690659 25311 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:15.690784 25313 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:15.691371 25311 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:15.691411 25313 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.695509 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.697044 25366 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:15.697853 25366 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.672721 25262 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.651062 25341 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:15.699749 25341 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:15.700083 25262 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:15.675050 25374 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:15.700760 25341 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.702190 25341 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:15.668233 24981 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.676376 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:15.677436 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:15.703609 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 97075.1us
I20260709 13:10:15.703979 22631 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:15.681834 25325 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.683284 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.668109 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 1991) took 1078 ms. Trace:
I20260709 13:10:15.687575 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:15.696918 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.675486 22626 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:15.707154 22630 rpcz_store.cc:276] 0709 13:10:14.589149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.589782 (+   633us) service_pool.cc:224] Handling call
0709 13:10:15.667773 (+1077991us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540022bd54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:15.667978 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1109544704}
I20260709 13:10:15.718220 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.719295 22782 raft_consensus.cc:2410] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:15.721639 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 303) took 1137 ms. Trace:
I20260709 13:10:15.724078 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:15.725260 25242 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:15.711320 25262 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:15.729288 22778 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:15.729784 25262 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:15.730773 25262 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:15.734117 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:15.722114 22775 rpcz_store.cc:276] 0709 13:10:14.584599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.595487 (+ 10888us) service_pool.cc:224] Handling call
0709 13:10:15.721443 (+1125956us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54000cb054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:15.721571 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065990016}
I20260709 13:10:15.737752 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.737752 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:15.740336 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.741501 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.741596 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:15.742318 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.750697 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.751832 23421 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:15.742339 22775 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4. Re-sending same reply.
I20260709 13:10:15.781916 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:15.785250 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.743747 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:15.805929 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62296.3us
I20260709 13:10:15.748087 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.812350 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:15.812767 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.816422 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.816430 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.817641 22630 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:15.820245 22776 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:10:15.822098 22628 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:15.823539 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:15.824239 22626 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:15.827867 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:15.828922 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.829885 22775 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:15.709915 25365 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:15.840627 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:15.841490 22632 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:15.831187 22780 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 13:10:15.831715 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72946.6us
I20260709 13:10:15.832048 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:15.832348 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:15.650235 25197 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.846208 25197 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:15.850979 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:15.851786 22781 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:15.718113 23412 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:15.831066 22635 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:15.805058 25360 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:15.804369 25366 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:15.717200 25195 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:15.739056 25313 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:15.856678 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43986us
I20260709 13:10:15.821691 25320 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:15.821854 25354 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:15.859179 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:15.865396 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.852950 25356 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:10:15.915691 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50388.8us
I20260709 13:10:15.917183 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:15.873909 25320 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.905639 25365 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:10:15.936564 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 122506us
I20260709 13:10:15.890043 23411 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:15.918857 25195 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:15.919142 25373 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:15.934635 25320 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:15.936655 25356 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:15.956737 22626 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:10:15.958779 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.941707 25365 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:15.963730 25325 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:15.973546 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56509.7us
I20260709 13:10:15.955344 25373 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.977998 25365 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:15.979835 25373 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:15.981966 25380 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:10:15.984472 25223 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.985839 25223 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:15.989902 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.975013 25288 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:15.997294 25288 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:16.003562 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.006434 25141 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:10:15.991638 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.007443 25141 consensus_queue.cc:1243] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:16.022547 25141 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:16.023140 25141 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:16.023267 22673 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:16.024325 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:16.024129 25141 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.025255 25373 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:16.025560 25141 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:16.025665 25373 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:16.026239 25373 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.027004 25360 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:16.027917 25360 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:16.028810 22627 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:16.028960 25360 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.013217 25152 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.017686 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:16.031960 25330 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:16.032490 23414 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.032594 25330 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:16.035573 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.038360 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:16.039253 23412 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.018975 25329 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:16.046933 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.051492 22631 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:10:16.052672 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 78209.5us
I20260709 13:10:16.053262 22635 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:16.053545 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:16.044082 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:16.059746 23416 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:10:16.061412 25195 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:16.064705 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.033954 25330 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.071993 25373 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.079939 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:16.089206 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57254.5us
I20260709 13:10:16.099223 22779 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:16.100535 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.101294 22701 raft_consensus.cc:2376] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4. Re-sending same reply.
I20260709 13:10:16.102126 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:16.103461 22778 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:16.109660 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.112102 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:16.066962 25152 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:16.121002 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:16.122498 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.122355 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.123502 22710 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:16.122009 22704 raft_consensus.cc:2376] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3. Re-sending same reply.
I20260709 13:10:16.089452 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:16.089290 25195 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:16.138065 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:16.139006 22780 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.066833 25320 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.152491 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:16.153615 22775 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:16.158077 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.105242 25334 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:10:16.147193 22710 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.146961 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:16.161206 22779 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.154901 25152 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:16.136454 25257 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:16.165748 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 107722us
I20260709 13:10:16.135296 25376 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.167956 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:16.168787 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.151098 23414 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.190289 22780 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:16.192157 22781 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:16.202867 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:16.205466 22775 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:16.207157 22530 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:16.210280 22530 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:16.193411 25303 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.249965 22702 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:16.252249 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:16.253538 22710 raft_consensus.cc:2368] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6.
I20260709 13:10:16.278447 22779 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:16.278712 22780 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:10:16.278447 22778 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.178978 25257 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.214994 25356 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:10:16.285663 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.296273 22530 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:16.285573 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:16.299117 22708 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:16.182787 25376 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.303359 22703 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:10:16.284696 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:16.298440 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.305160 22704 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:16.316030 25373 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.280260 25303 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:16.317267 25386 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:10:16.319080 25303 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:16.321422 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:16.322904 23419 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.309571 24960 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:16.324422 24960 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:16.325582 24960 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.319489 25320 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:16.334589 22530 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:10:16.351075 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 271220us
I20260709 13:10:16.358793 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.359807 25376 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.360615 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.361975 24981 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.278700 25152 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.368115 22530 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 4 to 5, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:16.370419 22530 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:16.372043 22530 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 2 to 3, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:16.367981 22635 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:16.375344 25329 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.375988 25329 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:16.378988 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.383037 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:16.383746 25257 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:16.386883 23412 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:16.381524 25284 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:16.390374 25141 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:16.391162 25141 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:16.392424 25141 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.394443 25329 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:16.398732 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.399353 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.400066 22705 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.400082 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:16.408187 23419 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:16.410481 22673 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:16.402318 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.413040 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.415006 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:16.418990 22777 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:16.422659 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:16.432886 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:16.435571 22701 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:10:16.425971 25320 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:16.443783 25320 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:16.444377 25320 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.447014 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:16.447858 22703 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:16.425843 25348 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:16.449838 25348 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:16.451759 25348 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.458760 25391 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 13:10:16.426120 25223 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:10:16.469316 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.470221 23414 raft_consensus.cc:2410] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:16.470760 22703 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:10:16.475613 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 327476us
I20260709 13:10:16.412863 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:16.483532 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.504992 22705 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:16.507217 22530 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 3, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:16.510154 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 334542us
I20260709 13:10:16.432132 25366 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:16.407059 25329 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:16.527829 22701 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:16.536339 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123501us
I20260709 13:10:16.538998 22530 catalog_manager.cc:5697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:16.551177 22635 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:16.559388 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.513562 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:16.522014 25387 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:16.565300 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51867.8us
I20260709 13:10:16.522571 25223 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:16.570363 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:16.570358 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 374800us
I20260709 13:10:16.572170 25155 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.575554 25155 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.579093 25331 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:16.581540 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:16.586937 25387 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:16.588285 25387 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.590796 25393 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.591471 25393 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.592039 25242 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.592643 25242 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.594015 25393 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:16.564668 25366 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:16.595878 25366 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.598815 25373 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:16.565903 25356 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.599478 25373 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.600066 25356 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.601883 25373 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.602128 25356 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:16.603787 25223 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.612118 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.570665 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:16.542279 22530 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 reported cstate change: leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:16.571795 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:16.612841 25141 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:16.613519 25303 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:10:16.570986 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.567018 25195 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:16.623113 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:16.647599 22781 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:16.624397 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:16.628333 25284 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:10:16.629539 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.629961 22782 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:10:16.631431 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:16.657107 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.609833 25320 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.612126 25242 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:16.652369 25325 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:16.649076 22780 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:16.664892 25396 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:16.671121 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 99400.3us
W20260709 13:10:16.666292 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127979us
I20260709 13:10:16.646270 22530 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:16.674993 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:16.680286 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:16.688699 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.671461 25325 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:16.673406 25155 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.678719 25284 raft_consensus.cc:2764] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:10:16.698582 25397 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.699345 25325 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.704540 25396 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:16.700251 25155 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:10:16.737520 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63206.2us
I20260709 13:10:16.739346 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:16.715375 25344 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.000s	sys 0.031s
W20260709 13:10:16.742694 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70218.1us
I20260709 13:10:16.702395 25303 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:10:16.766079 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:16.727577 25397 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.770334 25290 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:16.726703 25195 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.809195 25290 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:16.810355 25290 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.811806 25290 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:16.784549 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:16.813114 22631 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:16.746294 25155 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:16.804630 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:16.807479 25344 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.000s	sys 0.032s
I20260709 13:10:16.808465 25288 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:10:16.819291 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 80048.4us
W20260709 13:10:16.854001 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55554.4us
I20260709 13:10:16.865502 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:16.772768 25396 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.794924 25331 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:16.861290 25329 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:16.862411 25407 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:16.899091 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:16.900225 22627 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
W20260709 13:10:16.899863 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.862592 25288 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:16.892086 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.895223 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:16.877267 25320 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:16.880059 25348 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:16.922806 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:16.923682 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.888377 25373 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:16.892853 25366 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:16.896785 25407 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:16.930955 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:16.927548 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:16.931943 22631 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:16.937348 25400 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:16.937142 25288 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.938227 25400 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:16.938766 25400 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:16.939345 25288 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:16.878911 25396 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.945029 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.948779 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.914100 25329 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:16.943020 25320 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:16.952340 25334 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:10:16.954741 25320 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.956246 25398 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:16.949558 25303 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:16.957333 25320 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:16.949853 25402 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:16.958061 25320 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.958392 25329 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:16.961390 25329 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:16.962988 25331 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.926858 25399 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:10:16.971659 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:16.972775 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:16.929790 25348 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:16.946077 25360 raft_consensus.cc:2764] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:10:16.947963 25410 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.946061 25407 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:16.966398 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:16.950387 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:17.000204 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.023201 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.025648 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:17.025861 22780 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:17.026691 22782 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:16.968070 25387 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:17.034354 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68081.1us
I20260709 13:10:16.970718 25398 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:16.988449 25399 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.041325 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:16.932977 25373 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.041381 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:17.065492 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.966529 25320 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:17.079600 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:17.098418 22750 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.522s (SENT)
I20260709 13:10:17.098995 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.072810 25329 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.970438 25264 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:10:17.102308 25400 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:17.089970 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 194861us
W20260709 13:10:17.086071 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.106992 25400 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.113302 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:17.120505 22750 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.557s (SENT)
I20260709 13:10:17.121594 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.121976 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:17.119776 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 192263us
W20260709 13:10:17.123056 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 191528us
W20260709 13:10:17.125348 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.913s (SENT)
W20260709 13:10:17.126642 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.915s (SENT)
I20260709 13:10:17.127183 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.064366 25399 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:17.091534 25411 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.104326 25329 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:17.136597 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 99309.9us
W20260709 13:10:17.140138 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.936s	user 0.117s	sys 0.100s
W20260709 13:10:17.140697 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.117s	sys 0.100s
W20260709 13:10:17.127343 25405 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
W20260709 13:10:17.141518 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 939108us
I20260709 13:10:17.120430 24981 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:17.062942 25387 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.130307 23412 raft_consensus.cc:2410] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:10:17.151436 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.561s (SENT)
I20260709 13:10:17.128135 25223 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:17.120505 25396 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:17.019773 25410 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.138401 25419 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.160712 25410 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.161496 25410 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.162127 25410 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.127912 25391 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:17.158075 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.583s (SENT)
I20260709 13:10:17.132460 25400 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:10:17.187808 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53252.9us
I20260709 13:10:17.195849 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.196851 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.199891 23414 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:17.143347 25366 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:17.155649 25402 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:17.213150 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55103.7us
W20260709 13:10:17.214934 22750 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.639s (SENT)
I20260709 13:10:17.156313 25303 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:17.164551 25419 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:17.219816 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.230962 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:17.207624 25396 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:10:17.206694 25366 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:17.223917 25403 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 13:10:17.148432 25411 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.164017 25398 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.348793 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:17.210455 25223 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:17.350382 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:17.253541 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:17.233229 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 80681.9us
I20260709 13:10:17.374491 22781 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
W20260709 13:10:17.344905 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.376610 22780 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:17.377599 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:17.334830 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.090s (SENT) [suppressed 35 similar messages]
W20260709 13:10:17.385129 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50351.6us
W20260709 13:10:17.305137 25141 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.000s
I20260709 13:10:17.371685 25397 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.388345 23421 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:10:17.221608 25155 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:17.375617 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:17.263685 25408 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
W20260709 13:10:17.227999 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:17.354003 25325 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.624s	user 0.014s	sys 0.000s
I20260709 13:10:17.391078 23421 pending_rounds.cc:85] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 6
W20260709 13:10:17.295439 25414 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:10:17.388391 25366 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:17.420416 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:17.263180 25412 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:10:17.437563 23421 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:10:17.478199 23421 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:17.408447 22780 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:17.268375 25422 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:17.267175 25413 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.194s	user 0.000s	sys 0.000s
I20260709 13:10:17.318215 25329 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.292485 25387 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.355048 25436 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.451215 23412 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:17.370683 25242 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:10:17.306749 25348 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.229447 25195 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.210s
W20260709 13:10:17.224735 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70160us
I20260709 13:10:17.305555 25373 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:17.311192 25320 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.232291 25421 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.293329 25434 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:17.399667 24981 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:17.478240 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:17.477047 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70791.2us
I20260709 13:10:17.321507 25423 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
W20260709 13:10:17.487321 23385 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.512s (SENT)
I20260709 13:10:17.441646 25393 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:17.502046 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 126579us
I20260709 13:10:17.536904 23385 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:17.505187 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.470162 25391 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.511391 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.311739 25402 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:17.540311 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.525804 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.526795 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:17.484854 25344 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:17.441358 25325 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.498155 25365 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:17.537930 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 310007us
W20260709 13:10:17.545455 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58205.7us
I20260709 13:10:17.536067 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.514930 25387 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:17.551995 25387 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.552924 25325 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:17.554397 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.511804 25329 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.556466 25329 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.557197 22629 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:17.559103 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.558979 25365 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.266914 25303 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.561733 25365 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.512573 25433 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.496959 25155 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:10:17.571081 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:17.532328 25436 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.547955 25430 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:10:17.548573 25423 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:17.537068 25421 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.541373 22631 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:17.542374 25320 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:17.622849 24981 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.578320 25397 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:17.626840 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89414.8us
I20260709 13:10:17.565388 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.628064 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:17.628535 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 63281.4us
I20260709 13:10:17.629935 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.566095 23419 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:17.573742 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:17.632473 22780 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:17.575341 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.634178 22626 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:17.579676 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.535144 25440 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:17.581696 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.582683 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:17.543586 25344 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:17.564038 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:17.585951 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.599238 25434 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.601460 25331 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.606807 25401 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.562479 25433 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.580904 25423 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:17.617657 22632 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:10:17.580899 25330 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:17.619613 25438 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:10:17.552963 25411 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:17.583817 25365 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:17.620339 25436 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:17.622604 25421 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.627290 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.646494 23412 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:17.650462 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.564195 25387 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:17.716082 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65760.6us
I20260709 13:10:17.675701 25360 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:17.651551 22631 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:17.649573 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.664798 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.513881 25223 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.545555 25393 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.669328 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.632639 25195 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.644385 25446 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:10:17.730916 25329 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:17.679586 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 143624us
W20260709 13:10:17.676702 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48769.2us
W20260709 13:10:17.676496 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48140.1us
I20260709 13:10:17.681892 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:17.734956 22750 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.710s (SENT)
I20260709 13:10:17.736053 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.650686 25320 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.707948 25365 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:17.704412 22627 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:17.712875 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.646476 22629 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:17.761078 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.762070 25398 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:17.762779 25398 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:17.720604 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 71184.4us
W20260709 13:10:17.720881 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56195.2us
I20260709 13:10:17.764655 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.655175 25290 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:10:17.765941 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.675459 25443 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:10:17.767580 25409 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.768834 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:17.769940 25290 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:17.770673 25290 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:17.650041 25434 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.651212 25401 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.771946 25290 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.773231 25434 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.727138 25410 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.704095 25334 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:17.774422 25334 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.658928 25344 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:17.774856 25334 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:17.775170 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:17.776101 22783 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:17.773291 25401 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.779263 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.651160 25331 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:17.786936 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.646204 25440 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.675489 25330 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.747598 22777 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:17.724622 25360 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:17.752689 22750 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.735s (SENT)
I20260709 13:10:17.704864 25411 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.824553 25411 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.827876 25411 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:17.675325 25373 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:10:17.836697 25398 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.792109 25423 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.786194 25329 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.794061 25446 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:17.845000 25446 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.789283 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:17.846134 22601 leader_election.cc:400] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6.
W20260709 13:10:17.846395 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57172.2us
I20260709 13:10:17.846727 22601 leader_election.cc:403] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:17.790700 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.770150 25443 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:17.796113 25410 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:17.798199 25409 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.801074 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:17.732352 25242 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:17.807829 25366 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.733036 25387 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:17.750257 25407 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.717s	user 0.006s	sys 0.009s
I20260709 13:10:17.735493 25421 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:10:17.834954 25455 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 13:10:17.835279 25303 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:10:17.835603 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 82962.1us
I20260709 13:10:17.536123 25399 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.849224 22629 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:10:17.849598 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59014.3us
I20260709 13:10:17.850423 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:17.845613 25446 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:17.754870 25356 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.855742 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.767109 25444 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:17.781769 25450 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:17.822029 25437 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:10:17.946254 25450 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.948913 25450 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.848246 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:17.950162 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102040us
I20260709 13:10:17.951339 25432 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:17.952057 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.953130 25261 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:17.865301 22783 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:17.865969 22630 raft_consensus.cc:606] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.956296 22630 raft_consensus.cc:614] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:17.782310 24981 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 13:10:17.831852 25360 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:17.958123 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:17.950165 25303 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:17.960291 25303 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:17.961172 25303 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.814097 25440 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:17.964663 25435 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:17.881153 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:17.966383 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.770s (SENT)
I20260709 13:10:17.784590 25320 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.870700 25152 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:10:17.865808 25329 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:17.852767 25443 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:17.927140 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.877331 25242 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.872488 25373 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:17.937438 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:17.900833 25421 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.865334 25334 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:17.966950 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:17.971475 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:17.971913 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:18.017164 25303 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:18.037249 22705 raft_consensus.cc:2376] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6. Re-sending same reply.
I20260709 13:10:17.950325 24960 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:10:18.040253 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 102911us
I20260709 13:10:18.047104 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:18.041039 25195 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:18.055730 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:18.040511 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61335.9us
I20260709 13:10:17.877184 25409 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:18.058868 22704 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:18.060501 25409 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:17.913618 25407 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.035766 25242 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:18.039345 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.029985 25329 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:18.051139 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84791.5us
I20260709 13:10:18.064543 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.083323 22703 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:18.034060 25373 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.052042 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:18.083323 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.791s (SENT)
W20260709 13:10:18.088922 22750 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.840s (SENT)
I20260709 13:10:18.089618 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:18.041265 25421 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:17.931640 25365 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:17.967336 25356 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:18.116535 22750 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.907s (SENT)
I20260709 13:10:18.121208 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:18.063627 25411 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:18.119884 23414 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:18.125324 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.130528 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.924s (SENT)
I20260709 13:10:18.131170 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:18.132174 22750 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.930s (SENT)
I20260709 13:10:18.132556 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:18.127202 25223 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:18.099125 25373 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:18.048995 24960 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:18.062855 25433 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:18.091030 25329 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:18.100473 25195 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:18.128734 25443 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:18.265656 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.165859 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.137606 25155 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:18.133674 25434 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:18.356369 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.258526 25462 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 13:10:18.257653 25461 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:10:18.167858 24960 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:18.325958 22702 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:18.310786 25195 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:18.225092 25320 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:18.143445 25445 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:18.248350 25469 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:18.444124 25469 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.163694 25436 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:18.413857 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.250375 25366 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:18.144381 25444 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:18.224251 25397 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:18.396984 25443 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:18.187166 25330 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:18.428393 25334 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:18.429370 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.430960 25407 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:18.422689 25195 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:18.414036 25440 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:18.435364 25401 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:18.485812 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:18.437294 25393 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:18.439113 25356 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:18.441748 25399 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:18.388852 25410 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 13:10:18.472990 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 426007us
I20260709 13:10:18.394946 25434 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:18.471691 25414 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 13:10:18.471979 25360 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.006s
I20260709 13:10:18.473472 25152 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6.
I20260709 13:10:18.492702 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:18.486161 25331 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:18.518003 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 380050us
I20260709 13:10:18.481914 25397 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:18.546921 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 229967us
I20260709 13:10:18.489346 25401 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.534721 25477 raft_consensus.cc:2764] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:10:18.558955 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.565035 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:18.562386 25439 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.496891 25444 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.572153 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.518723 25360 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.532718 25407 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:10:18.575340 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.573472 23421 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:18.577538 22780 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:18.579994 22778 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:18.579983 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.583161 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.580924 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:18.603611 23417 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:18.595453 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.604843 22777 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:18.605751 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:18.585278 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.606700 22779 raft_consensus.cc:2410] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:10:18.587890 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 49077.1us
I20260709 13:10:18.586416 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.611651 22778 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:18.608565 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:18.613742 23417 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:18.613857 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.610001 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.627300 25469 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:18.576105 25446 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 13:10:18.636559 22704 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:18.600168 25477 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:18.564306 25331 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.672044 25397 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:18.643087 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.670493 25446 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.670830 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.671574 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.672278 25331 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:18.638726 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57876.4us
W20260709 13:10:18.673360 25487 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:10:18.698825 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:18.572350 25439 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:18.682540 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.690205 22707 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
W20260709 13:10:18.667186 25344 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.892s	user 0.007s	sys 0.010s
I20260709 13:10:18.695163 22778 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:18.707131 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.695992 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.696561 22777 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:18.696969 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:18.710285 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:18.712180 22777 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:18.680419 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.701820 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.713161 22705 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:18.713680 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.714365 22775 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:18.702107 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.679111 25261 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:18.708326 22704 raft_consensus.cc:2410] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:18.704104 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.714298 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.708952 22629 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:18.719511 22784 pending_rounds.cc:85] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:10:18.717844 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:18.719799 22784 pending_rounds.cc:107] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:10:18.720026 22704 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:18.718742 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.721556 22708 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:18.718848 22634 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:18.717904 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.717053 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.720077 22784 raft_consensus.cc:2889] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:18.708014 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:18.718318 22705 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:18.766953 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.797308 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.798388 22705 raft_consensus.cc:2410] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:18.741950 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:18.754194 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.799484 22782 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:18.729126 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:18.802946 22778 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:18.741772 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.749332 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.805330 22707 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:18.755963 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.745674 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:18.725397 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:18.755438 22631 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:18.753657 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.819489 23417 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:18.819720 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:18.843079 22708 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:18.823572 22634 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:18.844484 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:18.810375 22710 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:18.847903 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:18.849198 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.846724 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:18.689523 25477 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.717103 25261 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:19.139901 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.879715 22778 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:18.852804 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:18.945659 25242 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.003s
I20260709 13:10:19.143007 25242 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.893127 25401 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:18.888432 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:19.146401 22705 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:18.752122 25470 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:10:19.143402 22634 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:19.145679 25242 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:19.237541 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.256596 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.264622 22705 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:19.146199 23385 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.849s (SENT)
I20260709 13:10:19.268507 22634 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:18.859694 25456 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:10:18.957609 25360 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:18.803192 25434 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:19.280609 25261 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:18.798753 25446 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:18.774799 25334 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:19.016007 25421 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.866s	user 0.011s	sys 0.002s
W20260709 13:10:18.966430 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 177098us
I20260709 13:10:19.308995 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:19.310070 22705 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:19.311504 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.716753 25329 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:19.154636 22702 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:18.959427 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:19.316741 23418 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:19.268626 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 122478us
W20260709 13:10:19.216291 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.884055 25284 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:18.858775 25483 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 13:10:18.769611 25393 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:18.858480 25373 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.687s	user 0.005s	sys 0.005s
I20260709 13:10:19.283965 23419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 620) took 1607 ms. Trace:
I20260709 13:10:19.328743 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.269616 22708 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:18.903644 25444 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:18.963413 25303 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.003s
I20260709 13:10:18.992316 25331 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:19.138553 25488 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.414s	user 0.000s	sys 0.001s
I20260709 13:10:18.776563 24960 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:19.323508 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:19.094717 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.066730 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:19.327807 23416 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.013s	sys 0.020s
W20260709 13:10:19.384251 23416 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.013s	sys 0.020s
I20260709 13:10:19.328670 23419 rpcz_store.cc:276] 0709 13:10:17.676905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.677122 (+   217us) service_pool.cc:224] Handling call
0709 13:10:17.868942 (+191820us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:10:17.869156 (+   214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:17.909322 (+ 40166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:17.909378 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:17.910043 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 13:10:17.931856 (+ 21813us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:17.931902 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:17.931981 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:19.283515 (+1351534us) raft_consensus.cc:1685] finished
0709 13:10:19.283546 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:19.283872 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22702,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":19678,"spinlock_wait_cycles":10805632,"thread_start_us":60622,"threads_started":2,"wal-append.queue_time_us":564645}
I20260709 13:10:19.330158 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.333292 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:19.386502 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.387184 22780 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:19.387367 22708 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:19.386963 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.331483 25284 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:18.724874 25344 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:19.283215 25334 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:19.407949 22635 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:19.409732 25334 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:19.368180 22704 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
I20260709 13:10:19.366127 23418 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:19.371935 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.254551 25397 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:19.412144 25397 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:19.380942 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:19.416138 22704 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:19.417595 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.417620 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.418932 22708 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:19.420117 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:19.420260 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.395792 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:19.422886 22704 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:19.321105 25434 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:19.403105 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:19.405323 25284 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:19.406296 23411 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:19.407341 25344 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:19.434394 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.435384 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:19.426093 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.424600 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:19.438038 22710 raft_consensus.cc:2368] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:19.444213 25397 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:19.444813 25423 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:19.384965 25439 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:19.451148 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.452112 22705 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:19.453263 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:19.465548 22629 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.002s
I20260709 13:10:19.468204 25446 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:19.347550 25393 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:19.479014 25473 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.016s	user 0.000s	sys 0.011s
I20260709 13:10:19.484627 25329 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:19.315737 25421 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:19.488051 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.489560 22629 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:19.491268 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:19.470393 22750 connection.cc:256] RPC call timeout handler was delayed by -1.4262182199998961s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
I20260709 13:10:19.490259 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 1993) took 1442 ms. Trace:
W20260709 13:10:19.477758 25457 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.444s	user 0.000s	sys 0.011s
W20260709 13:10:19.499338 25468 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.454s	user 0.003s	sys 0.003s
I20260709 13:10:19.499415 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:19.349057 25373 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:19.798378 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:19.862141 22709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.049s	sys 0.116s
W20260709 13:10:20.104417 22709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.103s	sys 0.174s
I20260709 13:10:20.105726 22708 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:19.428368 25331 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:19.531193 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.106591 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1916) took 1315 ms. Trace:
W20260709 13:10:20.107539 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.107942 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:19.564654 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.108807 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 324) took 1456 ms. Trace:
I20260709 13:10:20.109411 22705 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:20.110435 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 327) took 1456 ms. Trace:
I20260709 13:10:19.571259 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2024) took 1163 ms. Trace:
I20260709 13:10:19.500224 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.107214 22708 rpcz_store.cc:276] 0709 13:10:18.791600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.487580 (+695980us) service_pool.cc:224] Handling call
0709 13:10:20.106485 (+618905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.109318 22704 rpcz_store.cc:276] 0709 13:10:18.652943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.437086 (+784143us) service_pool.cc:224] Handling call
0709 13:10:20.108720 (+671634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.111044 22705 rpcz_store.cc:276] 0709 13:10:18.653917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.564273 (+910356us) service_pool.cc:224] Handling call
0709 13:10:20.110348 (+546075us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.118929 22630 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:19.875919 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:19.687893 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:20.122756 22631 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:20.110965 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1630) took 1385 ms (client timeout 1742 ms). Trace:
I20260709 13:10:20.130272 22703 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:20.130584 22629 rpcz_store.cc:269] 0709 13:10:18.725164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.728791 (+  3627us) service_pool.cc:224] Handling call
0709 13:10:20.110868 (+1382077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.101290 22626 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:10:20.103137 22706 rpcz_store.cc:276] 0709 13:10:18.048123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.048321 (+   198us) service_pool.cc:224] Handling call
0709 13:10:18.048782 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:18.142975 (+ 94193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:18.148945 (+  5970us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:18.309800 (+160855us) log.cc:844] Serialized 26 byte log entry
0709 13:10:18.496192 (+186392us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:18.572803 (+ 76611us) log.cc:844] Serialized 14 byte log entry
0709 13:10:18.573047 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.573123 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:19.489943 (+916820us) raft_consensus.cc:1685] finished
0709 13:10:19.489975 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:19.490175 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33056,"prepare.queue_time_us":270558,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":5122,"raft.queue_time_us":71842,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":39299,"thread_start_us":216068,"threads_started":3,"wal-append.queue_time_us":140273}
I20260709 13:10:20.111893 22628 rpcz_store.cc:276] 0709 13:10:18.407721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.409217 (+  1496us) service_pool.cc:224] Handling call
0709 13:10:18.409779 (+   562us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:18.416619 (+  6840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:18.416657 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:18.442926 (+ 26269us) log.cc:844] Serialized 52 byte log entry
0709 13:10:18.540446 (+ 97520us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:18.649437 (+108991us) log.cc:844] Serialized 14 byte log entry
0709 13:10:18.892509 (+243072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.892626 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:19.570107 (+677481us) raft_consensus.cc:1685] finished
0709 13:10:19.570138 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:19.571161 (+  1023us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2169,"prepare.queue_time_us":332086,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":55,"raft.queue_time_us":32308,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":9580,"thread_start_us":420917,"threads_started":3,"wal-append.queue_time_us":171611}
I20260709 13:10:20.143034 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.143805 22706 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:20.145440 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.147006 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.147609 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.148396 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1917) took 1348 ms. Trace:
I20260709 13:10:20.145499 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.151963 22706 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:20.153354 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.154318 22706 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 2.
I20260709 13:10:20.155658 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.150715 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 325) took 1497 ms. Trace:
I20260709 13:10:20.157539 22703 rpcz_store.cc:276] 0709 13:10:18.653278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.499057 (+845779us) service_pool.cc:224] Handling call
0709 13:10:20.150591 (+651534us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.148684 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1907) took 1448 ms. Trace:
W20260709 13:10:20.145620 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.161679 22704 rpcz_store.cc:276] 0709 13:10:18.700433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.147223 (+1446790us) service_pool.cc:224] Handling call
0709 13:10:20.148593 (+  1370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.156354 22705 rpcz_store.cc:276] 0709 13:10:18.799996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.146571 (+1346575us) service_pool.cc:224] Handling call
0709 13:10:20.148294 (+  1723us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:20.072631 25488 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
W20260709 13:10:20.015725 25504 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.593s	user 0.000s	sys 0.001s
I20260709 13:10:20.165946 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:19.593102 25471 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.354s	user 0.000s	sys 0.011s
I20260709 13:10:20.168114 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:20.225657 22778 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:20.162941 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1908) took 1462 ms. Trace:
I20260709 13:10:20.173651 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1634) took 1327 ms. Trace:
I20260709 13:10:20.227380 22630 rpcz_store.cc:276] 0709 13:10:18.846522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.846691 (+   169us) service_pool.cc:224] Handling call
0709 13:10:20.110348 (+1263657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.119625 (+  9277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.173431 (+ 53806us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003d3950. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.173580 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2897020288}
I20260709 13:10:20.229041 22708 rpcz_store.cc:276] 0709 13:10:18.700753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.145081 (+1444328us) service_pool.cc:224] Handling call
0709 13:10:20.162857 (+ 17776us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.230803 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.231782 22708 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:20.174154 25431 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:20.174688 22635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1627) took 1476 ms (client timeout 1520 ms). Trace:
I20260709 13:10:20.176533 22777 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:10:20.265462 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:20.178941 25500 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.912s	user 0.000s	sys 0.011s
I20260709 13:10:20.179633 22775 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:10:20.182593 22702 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:20.186957 23385 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.009s	sys 0.005s
I20260709 13:10:20.064541 25477 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:20.188705 22779 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:20.064018 24960 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:19.447082 25434 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.262243 25373 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:20.233156 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.241353 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:20.189028 25480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.015s	sys 0.010s
W20260709 13:10:20.164646 25261 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.014s	sys 0.015s
I20260709 13:10:20.247201 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:20.169831 25329 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:20.251701 22780 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1636) took 1672 ms (client timeout 1760 ms). Trace:
I20260709 13:10:20.256103 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 625) took 1691 ms. Trace:
I20260709 13:10:20.266281 22777 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:20.250403 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:20.263433 22635 rpcz_store.cc:269] 0709 13:10:18.698432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.698565 (+   133us) service_pool.cc:224] Handling call
0709 13:10:20.174468 (+1475903us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b540006b8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.174589 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1605735808}
I20260709 13:10:20.277639 22634 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:10:20.278307 23385 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.009s	sys 0.005s
I20260709 13:10:20.356899 22708 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:20.357648 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.163471 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:20.297339 23416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.015s	sys 0.011s
W20260709 13:10:20.209775 25423 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.000s
W20260709 13:10:20.309356 23414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46262 (request call id 506) took 3098 ms (client timeout 1636 ms). Trace:
I20260709 13:10:20.356899 22778 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
W20260709 13:10:20.359845 23414 rpcz_store.cc:269] 0709 13:10:17.208374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.209765 (+  1391us) service_pool.cc:224] Handling call
0709 13:10:20.305900 (+3096135us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b54006b0154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.306163 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2041903488}
W20260709 13:10:20.361366 23416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.976s	user 0.015s	sys 0.011s
W20260709 13:10:20.357692 22780 rpcz_store.cc:269] 0709 13:10:18.579348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.579687 (+   339us) service_pool.cc:224] Handling call
0709 13:10:20.251378 (+1671691us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b540033cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.251606 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1831125888}
I20260709 13:10:20.311002 22710 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:20.356971 23411 rpcz_store.cc:276] 0709 13:10:18.564544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.564729 (+   185us) service_pool.cc:224] Handling call
0709 13:10:20.255778 (+1691049us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b54004f4f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.256003 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931079040}
W20260709 13:10:20.362528 22710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1906) took 1662 ms (client timeout 2046 ms). Trace:
I20260709 13:10:20.358225 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.355630 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:20.364672 22777 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:20.365212 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.365554 22781 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:20.366017 22703 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:20.358505 22707 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:19.525005 25446 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.366817 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:20.359561 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 874031us
I20260709 13:10:20.369933 22703 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:20.370385 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.090s (SENT)
I20260709 13:10:20.370388 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:20.370896 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.371277 22783 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:20.371524 22703 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:20.372474 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.372991 22703 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:20.367034 22708 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:20.367766 22705 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:10:20.363164 22710 rpcz_store.cc:269] 0709 13:10:18.700143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.438888 (+738745us) service_pool.cc:224] Handling call
0709 13:10:20.362445 (+923557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.377918 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.378789 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:20.370103 25477 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:20.366495 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:20.359014 25261 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.014s	sys 0.020s
I20260709 13:10:20.391677 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2028) took 1678 ms. Trace:
I20260709 13:10:20.391454 25477 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.392379 22704 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:20.392078 22632 rpcz_store.cc:276] 0709 13:10:18.713652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.713793 (+   141us) service_pool.cc:224] Handling call
0709 13:10:18.745812 (+ 32019us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:18.746321 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:18.746363 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:18.746997 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 13:10:18.891159 (+144162us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:18.891231 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.891351 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:20.178213 (+1286862us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:20.391267 (+213054us) raft_consensus.cc:1685] finished
0709 13:10:20.391299 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.391602 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22010,"raft.queue_time_us":7235,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":12853,"spinlock_wait_cycles":13629568,"thread_start_us":143684,"threads_started":1,"wal-append.queue_time_us":271941}
I20260709 13:10:20.393576 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.394254 22704 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:20.364451 25373 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:20.323235 25443 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.841s	user 0.000s	sys 0.016s
I20260709 13:10:20.381955 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1922) took 1369 ms. Trace:
W20260709 13:10:20.358946 25480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.218s	user 0.018s	sys 0.011s
I20260709 13:10:20.386852 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 512) took 2543 ms. Trace:
W20260709 13:10:20.322999 25195 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.834s	user 0.016s	sys 0.001s
I20260709 13:10:20.418314 23412 rpcz_store.cc:276] 0709 13:10:17.843809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.844033 (+   224us) service_pool.cc:224] Handling call
0709 13:10:17.844453 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:17.844901 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:17.844935 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:17.845515 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 13:10:18.038969 (+193454us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:18.039025 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.039109 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:19.592171 (+1553062us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:20.248114 (+655943us) raft_consensus.cc:1685] finished
0709 13:10:20.248150 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.386739 (+138589us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7799,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":13582,"spinlock_wait_cycles":16432896,"thread_start_us":193014,"threads_started":1,"wal-append.queue_time_us":297448}
W20260709 13:10:20.380570 22784 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.008s	sys 0.019s
I20260709 13:10:20.395076 22701 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:20.395439 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.396415 22710 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
W20260709 13:10:20.421197 22784 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.009s	sys 0.019s
I20260709 13:10:20.422041 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.381866 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:20.417472 22705 rpcz_store.cc:276] 0709 13:10:19.012260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.240897 (+1228637us) service_pool.cc:224] Handling call
0709 13:10:20.381872 (+140975us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.390811 25391 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:20.424108 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.424033 25391 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.424921 22705 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:20.425474 22705 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:20.425890 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1888) took 1770 ms. Trace:
I20260709 13:10:20.426194 22705 rpcz_store.cc:276] 0709 13:10:18.655154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.423788 (+1768634us) service_pool.cc:224] Handling call
0709 13:10:20.425826 (+  2038us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.427120 25391 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:20.385459 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:20.427815 22705 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:20.428001 22707 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:20.428525 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 321) took 1786 ms. Trace:
I20260709 13:10:20.428910 22705 rpcz_store.cc:276] 0709 13:10:18.646501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.427074 (+1780573us) service_pool.cc:224] Handling call
0709 13:10:20.427445 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.428250 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.428468 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.429411 22707 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:20.430202 22705 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:20.430792 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.431005 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 319) took 1789 ms. Trace:
I20260709 13:10:20.431367 22710 rpcz_store.cc:276] 0709 13:10:18.641230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.421672 (+1780442us) service_pool.cc:224] Handling call
0709 13:10:20.430938 (+  9266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.432637 22710 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:20.433169 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1919) took 1621 ms. Trace:
I20260709 13:10:20.433853 25423 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.434599 25423 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.436908 25423 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:20.437459 25423 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.437918 25423 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:20.299476 25450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.014s	sys 0.040s
I20260709 13:10:20.433925 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1890) took 1778 ms. Trace:
I20260709 13:10:20.438931 22710 rpcz_store.cc:276] 0709 13:10:18.812107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.431995 (+1619888us) service_pool.cc:224] Handling call
0709 13:10:20.432323 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.432960 (+   637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.433118 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.441634 22706 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:10:20.321493 25434 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:20.443739 22708 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:20.373656 25440 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:20.412391 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:20.439285 25360 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.447039 22705 rpcz_store.cc:276] 0709 13:10:18.655722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.429727 (+1774005us) service_pool.cc:224] Handling call
0709 13:10:20.430012 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.433666 (+  3654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.433869 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.459920 25360 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.460677 22705 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:20.461160 22705 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:20.447070 25450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.014s	sys 0.041s
I20260709 13:10:20.394136 25477 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:20.457461 22708 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:10:20.463611 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:20.464527 22710 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:20.306308 25431 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.462080 22705 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.467278 22705 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:20.444927 22707 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:20.468710 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 322) took 1820 ms. Trace:
I20260709 13:10:20.469178 22707 rpcz_store.cc:276] 0709 13:10:18.649065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.428887 (+1779822us) service_pool.cc:224] Handling call
0709 13:10:20.429262 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.468349 (+ 39087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.468619 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:20.473228 22708 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:20.490955 22708 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:10:20.495455 22750 connection.cc:256] RPC call timeout handler was delayed by -1.3757643059999509s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
I20260709 13:10:20.367362 25329 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.501335 22710 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:20.504928 22710 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:20.456228 25409 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:20.520226 23421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 493) took 2035 ms (client timeout 1503 ms). Trace:
W20260709 13:10:20.520961 23421 rpcz_store.cc:269] 0709 13:10:18.485259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.485431 (+   172us) service_pool.cc:224] Handling call
0709 13:10:20.519921 (+2034490us) spinlock_profiling.cc:243] Waited 86.6 ms on lock 0x7b54005a3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.520111 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199145344}
I20260709 13:10:20.440932 25373 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.487596 25434 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.444108 25195 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.552012 25393 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:20.554060 25421 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:20.527606 25409 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:20.571213 25508 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:10:20.563592 25446 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:20.549678 25434 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.642263 25517 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:20.643249 25517 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.630367 25514 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.647017 25514 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:20.651391 25510 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:10:20.656379 25474 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 2.241s	user 0.013s	sys 0.009s
W20260709 13:10:20.667106 25502 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.329s	user 0.005s	sys 0.010s
W20260709 13:10:20.788703 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.007s	sys 0.006s
W20260709 13:10:20.789821 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.008s	sys 0.007s
I20260709 13:10:20.793109 22632 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
W20260709 13:10:20.794988 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.002s	sys 0.011s
I20260709 13:10:20.795552 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:20.795554 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.011s
I20260709 13:10:20.797003 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:20.795830 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:20.803591 22628 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:20.812783 22601 connection.cc:256] RPC call timeout handler was delayed by -1.4778061029999208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
W20260709 13:10:20.814153 22601 connection.cc:256] RPC call timeout handler was delayed by -1.3176061030000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:10:20.816448 22601 connection.cc:256] RPC call timeout handler was delayed by -1.1277061029999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:10:20.818552 22633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1631) took 2076 ms (client timeout 1760 ms). Trace:
W20260709 13:10:20.819375 22633 rpcz_store.cc:269] 0709 13:10:18.742120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.742438 (+   318us) service_pool.cc:224] Handling call
0709 13:10:20.817862 (+2075424us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002f5154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.818216 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3667108608}
I20260709 13:10:20.822934 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:20.828339 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:20.829321 22701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 1985) took 2858 ms (client timeout 3090 ms). Trace:
W20260709 13:10:20.829896 22701 rpcz_store.cc:269] 0709 13:10:17.970482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.038891 (+ 68409us) service_pool.cc:224] Handling call
0709 13:10:20.828728 (+2789837us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54001ea854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.829024 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5314342656}
W20260709 13:10:20.832054 22702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1901) took 2133 ms (client timeout 1907 ms). Trace:
W20260709 13:10:20.832573 22702 rpcz_store.cc:269] 0709 13:10:18.698553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:19.371521 (+672968us) service_pool.cc:224] Handling call
0709 13:10:20.831549 (+1460028us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400315954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.831786 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1772621696}
I20260709 13:10:20.834152 22704 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:20.834837 22706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1912) took 2108 ms (client timeout 2624 ms). Trace:
W20260709 13:10:20.835335 22706 rpcz_store.cc:269] 0709 13:10:18.726076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.155295 (+1429219us) service_pool.cc:224] Handling call
0709 13:10:20.834301 (+679006us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400316354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.834587 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653125888}
I20260709 13:10:20.836473 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:20.838451 22628 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:20.840279 22708 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:20.844183 22705 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:20.844552 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 317) took 2214 ms. Trace:
I20260709 13:10:20.846709 22632 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:10:20.847673 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.005s	sys 0.012s
W20260709 13:10:20.848258 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.005s	sys 0.012s
I20260709 13:10:20.848114 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2051) took 3057 ms. Trace:
I20260709 13:10:20.850174 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:20.848878 22776 rpcz_store.cc:276] 0709 13:10:17.790980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.804144 (+ 13164us) service_pool.cc:224] Handling call
0709 13:10:17.804636 (+   492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:17.805207 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:17.805254 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:17.806401 (+  1147us) log.cc:844] Serialized 52 byte log entry
0709 13:10:17.909216 (+102815us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:17.909664 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 13:10:17.910181 (+   517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:17.910286 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:19.384553 (+1474267us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:20.674055 (+1289502us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b24003b9318. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2efb0 00007f1806aa5488 00007f1806aa5be8 00007f1806a7d398 00007f1806a7876a 00007f1806a77bb6
0709 13:10:20.847465 (+173410us) raft_consensus.cc:1685] finished
0709 13:10:20.847508 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.847776 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":219,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":2239,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":162,"spinlock_wait_cycles":149762688,"thread_start_us":55947,"threads_started":1,"wal-append.queue_time_us":137657}
W20260709 13:10:20.851457 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.000s	sys 0.017s
W20260709 13:10:20.852049 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.000s	sys 0.017s
W20260709 13:10:20.853312 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:20.854537 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.856014 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1644) took 2158 ms. Trace:
W20260709 13:10:20.858985 22782 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.013s	sys 0.005s
W20260709 13:10:20.859756 22782 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.014s	sys 0.005s
W20260709 13:10:20.860860 22779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1641) took 2247 ms (client timeout 1912 ms). Trace:
W20260709 13:10:20.861501 22779 rpcz_store.cc:269] 0709 13:10:18.622568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.631076 (+  8508us) service_pool.cc:224] Handling call
0709 13:10:20.860137 (+2229061us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400131354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.860495 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411702016}
I20260709 13:10:20.864785 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:20.861517 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:20.859011 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.000s	sys 0.016s
W20260709 13:10:20.867871 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.000s	sys 0.016s
I20260709 13:10:20.838853 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:20.868410 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 749667us
I20260709 13:10:20.856705 22784 rpcz_store.cc:276] 0709 13:10:18.697416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.697741 (+   325us) service_pool.cc:224] Handling call
0709 13:10:18.719300 (+ 21559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:18.801425 (+ 82125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:18.801478 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:18.802108 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:10:19.065969 (+263861us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:19.616632 (+550663us) log.cc:844] Serialized 14 byte log entry
0709 13:10:20.421673 (+805041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.421765 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:20.666307 (+244542us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:20.855045 (+188738us) raft_consensus.cc:1685] finished
0709 13:10:20.855275 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.855715 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1253619,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":53,"raft.queue_time_us":862327,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":31807,"spinlock_wait_cycles":13559168,"thread_start_us":1563385,"threads_started":3,"wal-append.queue_time_us":535575}
I20260709 13:10:20.873661 22775 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:20.874979 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 623) took 2320 ms. Trace:
I20260709 13:10:20.875183 22784 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:10:20.868314 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.875532 23415 rpcz_store.cc:276] 0709 13:10:18.554328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.554778 (+   450us) service_pool.cc:224] Handling call
0709 13:10:20.522910 (+1968132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:20.523247 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:20.523287 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:20.523353 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:20.523408 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.523488 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.874408 (+350920us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54005a3cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:20.874686 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4525100800}
I20260709 13:10:20.888043 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:20.888954 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:20.867659 22776 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
W20260709 13:10:20.891785 25290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.825s	user 0.007s	sys 0.074s
I20260709 13:10:20.893105 25152 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.845042 22709 rpcz_store.cc:276] 0709 13:10:18.630128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.631628 (+  1500us) service_pool.cc:224] Handling call
0709 13:10:18.722815 (+ 91187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:19.283533 (+560718us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:19.327663 (+ 44130us) log.cc:844] Serialized 14 byte log entry
0709 13:10:20.105025 (+777362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:20.105091 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:20.105145 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.843806 (+738661us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b54000e7754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:20.843895 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:20.844260 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43899,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":13871,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":24648,"spinlock_wait_cycles":1288888192,"thread_start_us":776839,"threads_started":1,"wal-append.queue_time_us":1028399}
W20260709 13:10:20.919210 25331 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.008s	sys 0.017s
W20260709 13:10:20.955343 25290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.894s	user 0.007s	sys 0.075s
W20260709 13:10:20.955698 25331 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.009s	sys 0.018s
W20260709 13:10:20.940364 23382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.005s	sys 0.003s
I20260709 13:10:20.941097 25516 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.942891 25520 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.582221 25421 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:20.951340 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81584.3us
W20260709 13:10:20.936020 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 117922us
I20260709 13:10:20.938630 25446 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.960395 22704 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:20.831971 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:20.963112 23382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.005s	sys 0.003s
I20260709 13:10:20.957651 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:20.968534 25526 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:20.969576 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.006s	sys 0.006s
W20260709 13:10:20.970111 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.007s	sys 0.006s
I20260709 13:10:20.969918 25526 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:20.972643 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 602258us
I20260709 13:10:20.975028 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:20.974931 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:20.976624 23415 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:20.978708 25433 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:20.978979 25446 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:20.979911 25433 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:20.981679 25152 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:20.983433 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 166587us
I20260709 13:10:20.983927 25525 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:20.985430 25152 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:20.974085 25331 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:20.986400 25391 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:20.987152 25391 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:20.986826 25516 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.988338 25331 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:20.989596 25485 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:20.990162 25485 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.990526 25485 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:20.987783 25391 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:20.991235 25446 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:20.995885 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:20.996258 25506 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:20.996392 25521 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:20.997809 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.003861 25524 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:21.003563 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:21.004165 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:20.986231 25152 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:21.004920 25440 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:21.008642 25520 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.009244 25497 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:21.012770 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:21.010040 25440 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.014765 25398 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.015360 25443 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.026536 25443 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.006907 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:21.021211 25520 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.022027 22709 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:21.025409 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.025550 25393 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:21.028339 22784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37296
I20260709 13:10:20.981864 25433 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.034796 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.035764 22779 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.035974 22777 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:21.037595 22784 raft_consensus.cc:606] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:21.041095 22784 raft_consensus.cc:614] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:21.028817 25440 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:21.022549 25497 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.044486 22783 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:21.056051 25498 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:21.056957 25514 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.057202 25498 consensus_queue.cc:1243] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:10:21.053391 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.013s
I20260709 13:10:21.061506 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.064332 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:21.076732 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123438us
I20260709 13:10:21.050220 25497 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.088439 25514 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.089344 25514 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.100529 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:21.076221 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.000s	sys 0.014s
I20260709 13:10:21.004946 25524 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:21.079566 23416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.007s	sys 0.003s
I20260709 13:10:21.047477 25152 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:21.112505 23416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.008s	sys 0.004s
I20260709 13:10:21.119210 22705 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:21.105221 25155 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.008s	sys 0.004s
I20260709 13:10:21.130889 25261 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:21.131491 25261 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:21.132311 25514 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.137247 25526 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.138792 25532 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:21.139097 25434 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.139503 25288 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:21.140587 25450 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:21.140327 25288 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.142402 25464 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 13:10:21.143298 25195 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.143405 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:21.144441 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.144948 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:21.145819 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:21.132889 25261 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.151875 25261 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:21.154765 25373 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.155284 25421 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:21.175740 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 187131us
I20260709 13:10:21.111176 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:21.140134 25531 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.127s	user 0.000s	sys 0.002s
I20260709 13:10:21.167716 22706 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:10:21.120294 25520 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.187641 23421 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:21.188277 25520 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.189028 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.190997 25517 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:21.161209 25155 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.008s	sys 0.004s
I20260709 13:10:21.186450 22706 pending_rounds.cc:85] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 1
I20260709 13:10:21.190514 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:21.182313 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 685729us
I20260709 13:10:21.194936 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.132961 25360 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.208009 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.198248 22706 pending_rounds.cc:107] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:10:21.209975 22706 raft_consensus.cc:2889] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:21.202543 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 164091us
I20260709 13:10:21.219002 25421 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.219446 25288 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.206872 25373 raft_consensus.cc:2764] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:10:21.226920 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:21.229684 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.225041 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:21.247745 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.218348 25391 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.255810 25431 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.262198 25152 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:21.262823 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 70566.5us
W20260709 13:10:21.263985 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.266686 25423 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.266961 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.269114 25360 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:21.270275 22750 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.967s (SENT)
W20260709 13:10:21.272835 22781 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.907s	user 0.010s	sys 0.014s
I20260709 13:10:21.273329 22781 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:21.274027 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2062) took 1092 ms. Trace:
I20260709 13:10:21.274391 22781 rpcz_store.cc:276] 0709 13:10:20.181831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.181952 (+   121us) service_pool.cc:224] Handling call
0709 13:10:21.273926 (+1091974us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:21.275388 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:21.277081 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.288215 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 97812us
I20260709 13:10:21.279430 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.154784 25398 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.295140 25507 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.213790 25517 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:21.299109 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70609.7us
I20260709 13:10:21.301162 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2056) took 1127 ms. Trace:
I20260709 13:10:21.305038 23421 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:21.293556 25155 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:21.309497 22632 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:21.310690 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 492) took 2861 ms. Trace:
I20260709 13:10:21.312549 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.313028 23384 connection.cc:256] RPC call timeout handler was delayed by -1.60303774099998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1942s for a response
I20260709 13:10:21.320542 25527 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.312347 23416 rpcz_store.cc:276] 0709 13:10:18.444974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.445161 (+   187us) service_pool.cc:224] Handling call
0709 13:10:18.445554 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:18.445779 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:19.384778 (+938999us) log.cc:844] Serialized 14 byte log entry
0709 13:10:21.185966 (+1801188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:21.204207 (+ 18241us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:21.229410 (+ 25203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:21.296402 (+ 66992us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:21.306294 (+  9892us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":442155,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":620755,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":357,"thread_start_us":2671670,"threads_started":3,"wal-append.queue_time_us":1179690}
I20260709 13:10:21.321406 25527 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.322736 25329 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:21.323890 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.324087 25524 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.320046 25440 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.307530 25532 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:21.314337 25391 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.310796 25155 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.303364 22782 rpcz_store.cc:276] 0709 13:10:20.168968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.169102 (+   134us) service_pool.cc:224] Handling call
0709 13:10:20.225397 (+ 56295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:20.860609 (+635212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:20.860676 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:20.861789 (+  1113us) log.cc:844] Serialized 26 byte log entry
0709 13:10:20.961499 (+ 99710us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:20.961540 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:20.975879 (+ 14339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:21.279864 (+303985us) raft_consensus.cc:1685] finished
0709 13:10:21.279884 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:21.296058 (+ 16174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":408154,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":149,"thread_start_us":733378,"threads_started":2,"wal-append.queue_time_us":112405}
I20260709 13:10:21.322804 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.360431 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:21.361061 25155 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.361732 22709 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:21.361975 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:21.363005 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:21.309883 25507 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.369372 25507 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:21.330528 23384 connection.cc:256] RPC call timeout handler was delayed by -1.0496625889999223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
I20260709 13:10:21.328925 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.371737 23384 connection.cc:256] RPC call timeout handler was delayed by -1.0454852319999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 13:10:21.372002 22631 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
W20260709 13:10:21.372294 23384 connection.cc:256] RPC call timeout handler was delayed by -1.0454852319999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 13:10:21.331748 25528 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.331485 25517 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.333823 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.328981 25407 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:21.337639 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:21.339088 25398 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.336817 25532 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.343041 22779 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:21.343218 25527 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.395113 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.395188 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.349992 22750 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.046s (SENT)
I20260709 13:10:21.312381 25421 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:10:21.391338 22706 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:21.351255 22635 raft_consensus.cc:2410] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:21.351775 25440 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.354771 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:21.355126 22710 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:21.371253 22629 raft_consensus.cc:2410] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:10:21.372627 23384 connection.cc:256] RPC call timeout handler was delayed by -1.0454852319999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 13:10:21.377158 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:21.399401 23384 connection.cc:256] RPC call timeout handler was delayed by -1.0454852319999191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 13:10:21.395823 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:21.405483 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.400573 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 102144us
W20260709 13:10:21.403095 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.405076 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.409273 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.400197 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:21.411931 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.413335 23421 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:21.390594 25507 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.416298 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.409709 25391 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:21.416631 25546 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:21.417331 25391 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.417850 25391 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:21.418696 25525 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:21.419546 25525 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.400059 25431 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.420063 25525 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:21.420841 25431 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.423184 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:10:21.423753 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:21.425299 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.426112 25528 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.403964 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:21.427188 25421 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:21.416786 25507 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:21.428164 22703 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.023s	user 0.007s	sys 0.011s
I20260709 13:10:21.432125 22702 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:21.416318 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:21.428681 22703 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:21.418022 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.437162 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:21.439204 25407 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.440176 25507 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.441939 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.439790 23414 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:10:21.442153 25407 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.443733 23414 pending_rounds.cc:85] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 5
I20260709 13:10:21.455360 23414 pending_rounds.cc:107] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:10:21.455719 23414 raft_consensus.cc:2889] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:21.462978 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 1997) took 1090 ms. Trace:
I20260709 13:10:21.406375 25446 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.463743 22703 rpcz_store.cc:276] 0709 13:10:20.372053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.372219 (+   166us) service_pool.cc:224] Handling call
0709 13:10:21.447377 (+1075158us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:21.478334 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 62139.7us
I20260709 13:10:21.479938 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:21.483474 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.488039 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.491760 23411 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
W20260709 13:10:21.495359 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 145434us
W20260709 13:10:21.497169 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.219s (SENT)
I20260709 13:10:21.496809 25525 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:21.489391 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55467.7us
W20260709 13:10:21.498586 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1141567589999113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1552s for a response
I20260709 13:10:21.529847 25446 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:21.531529 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 51770.3us
I20260709 13:10:21.568701 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:10:21.548280 25538 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.292s	user 0.004s	sys 0.000s
I20260709 13:10:21.520432 25525 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.561486 25195 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:21.561915 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 1998) took 1189 ms. Trace:
W20260709 13:10:21.563160 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.484982 25440 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.520344 25421 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:21.568439 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.530375 25544 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:21.515921 25527 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:21.570255 22627 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:21.587955 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:21.590209 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 161680us
I20260709 13:10:21.591982 25431 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:21.626278 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:21.594187 22775 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:21.594672 22783 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:21.627439 22784 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:10:21.512741 25497 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.595701 25443 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.616088 22704 rpcz_store.cc:276] 0709 13:10:20.372390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.395136 (+ 22746us) service_pool.cc:224] Handling call
0709 13:10:21.561571 (+1166435us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54002bedd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:21.561821 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948928384}
I20260709 13:10:21.596110 25360 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:21.581341 25546 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.593652 25391 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:21.627509 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:21.630447 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 148036us
I20260709 13:10:21.617935 25195 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.638231 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45654
I20260709 13:10:21.619787 25450 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:21.621299 25421 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.639277 22632 raft_consensus.cc:606] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Received request to transfer leadership
I20260709 13:10:21.650007 22777 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:10:21.623899 25544 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.626906 25527 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.623106 25517 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.657326 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:21.631455 25360 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.669143 22781 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:21.644088 25195 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:21.619827 25155 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.668880 25497 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:21.678860 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.678959 25443 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:10:21.680531 25446 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.681433 25443 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:21.682021 25443 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:21.687704 25443 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:21.693586 25552 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:10:21.696662 25485 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.706s	user 0.009s	sys 0.008s
I20260709 13:10:21.696107 25484 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:21.697248 25402 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:21.698329 25261 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:21.697335 25399 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:21.699905 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.700858 25423 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:21.701643 25423 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:21.701997 25423 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.702342 25423 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.704322 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.705062 25288 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:21.650161 25421 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:21.669334 25546 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.720438 25261 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.721877 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.666731 25527 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:21.724778 25435 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:10:21.728210 25484 raft_consensus.cc:993] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:10:21.728994 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:21.672380 25360 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:21.662283 25544 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.634764 25440 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.769389 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:21.733297 22776 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:21.730554 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:21.690246 25443 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:21.749217 22784 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:21.759363 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55181.5us
I20260709 13:10:21.732995 25155 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:21.775794 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54062.1us
I20260709 13:10:21.776027 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:21.772768 25528 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:21.764143 25288 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:21.782058 25497 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:21.734819 25485 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.767864 25450 raft_consensus.cc:2764] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:10:21.758275 25261 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.790795 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:21.792414 25391 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:21.793502 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 64648.1us
I20260709 13:10:21.799047 25521 raft_consensus.cc:1081] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Signalling peer 6a31ac6451ff4f73bee634487d84d001 to start an election
I20260709 13:10:21.792464 25525 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:21.780184 25484 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:21.792752 25407 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:10:21.831791 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51766us
I20260709 13:10:21.836416 22634 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:21.777499 25303 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
W20260709 13:10:21.841675 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 111300us
I20260709 13:10:21.834471 25434 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:21.844985 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58137.2us
I20260709 13:10:21.878711 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:21.883140 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.887660 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:21.888759 25533 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:10:21.892869 22775 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:21.893337 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.894273 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:21.892967 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.854007 25484 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:21.897884 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.897686 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.894912 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.904537 23414 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:21.910590 22784 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:21.911078 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.916231 25288 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.912714 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:21.925817 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.926456 23421 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:21.927603 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:21.901086 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:21.931560 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:21.933068 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 161446us
I20260709 13:10:21.944025 22784 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:21.931193 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:21.910516 25506 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.934074 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:21.922200 25431 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:10:21.883276 25152 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:10:21.924774 25331 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:21.966292 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:21.968624 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:21.955487 25528 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.956338 25155 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:21.956811 25521 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:10:21.985141 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:21.989074 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98480.5us
W20260709 13:10:21.999801 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.004282 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:21.988008 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:21.972510 25433 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:21.974720 25402 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:22.017082 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:21.990916 25440 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.015981 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:22.023756 25568 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:10:22.024385 22780 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:22.015892 25402 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:22.027180 25402 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:21.994879 25373 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:22.040409 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:22.041124 25517 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:22.041630 25517 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:22.017999 22631 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:22.045045 25409 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:22.045325 23420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
 from {username='slave'} at 127.0.0.1:46252
I20260709 13:10:22.046211 23420 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:22.046690 23420 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:22.048563 25528 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:22.049703 25560 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 13:10:22.050896 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 82375.1us
I20260709 13:10:22.051546 25528 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:22.054379 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.054533 25325 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:10:22.055071 25558 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:22.055537 22704 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 3.
I20260709 13:10:22.055868 25558 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.059015 22710 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:10:22.062706 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.013185 25484 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.065883 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.018038 22784 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:22.067042 25546 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:22.068308 25391 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:22.069200 25391 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:21.984565 25155 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:21.963258 25506 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.031018 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:22.044766 25507 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:22.078076 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101971us
W20260709 13:10:22.082029 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52266.6us
I20260709 13:10:22.084542 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.085333 22704 raft_consensus.cc:2376] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6. Re-sending same reply.
I20260709 13:10:22.086169 25440 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:22.086800 25440 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:22.087904 25440 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.012387 25303 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:22.090202 25303 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:22.090678 25303 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:22.090919 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.081377 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:22.012643 25433 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:22.013195 22779 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:21.991478 25497 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:10:22.118885 25507 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:22.113539 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.119442 25507 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:22.119971 25507 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:22.124624 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.126426 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.127298 22708 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:22.114202 25517 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.080799 25155 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:22.143654 22710 raft_consensus.cc:2376] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5. Re-sending same reply.
I20260709 13:10:22.058511 25558 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:22.169062 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73609.2us
I20260709 13:10:22.117149 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:22.080282 25485 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.132781 25517 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:22.133136 25433 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:22.212096 25261 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:22.215411 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 184484us
I20260709 13:10:22.208498 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:22.237344 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 139514us
W20260709 13:10:22.213766 25565 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:10:22.243593 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:22.253911 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.259372 22704 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:22.260897 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:22.263252 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.264447 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
W20260709 13:10:22.266036 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 193944us
I20260709 13:10:22.269886 25573 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:22.271000 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.271854 25360 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.233366 25521 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:22.270836 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:22.283784 22778 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:22.284987 22705 raft_consensus.cc:2368] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 5. Current term is 6.
I20260709 13:10:22.285383 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:22.286135 22704 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:22.290843 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70059.3us
I20260709 13:10:22.292801 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:22.294356 25533 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:22.295012 25533 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:22.296445 25533 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.298031 25533 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:22.298920 25290 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:22.314958 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:22.326768 23417 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:10:22.328222 22702 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:22.270746 25523 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.336079 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:22.337311 25528 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:22.337756 25528 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:22.338169 25528 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:22.338349 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:22.339128 22778 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:22.342345 22709 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:22.342890 22709 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:22.344529 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.281018 25360 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.351150 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:22.302450 25290 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:22.340430 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:22.285398 25521 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:22.356091 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.364841 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:22.357897 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:22.359342 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:22.373612 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:22.375595 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:22.378199 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.372095 22784 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:22.377529 23420 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.315198 25431 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:10:22.399454 22704 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:22.405258 25572 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:10:22.407636 25290 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:22.408810 25290 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.410511 25431 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:22.411042 25431 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:22.412046 25431 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.413771 25431 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:22.376418 25576 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:10:22.371984 25521 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:22.419828 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62052.4us
I20260709 13:10:22.374969 25391 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.425503 23416 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
W20260709 13:10:22.431783 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 139119us
I20260709 13:10:22.446556 23417 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:22.383356 25532 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:22.445175 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:22.453053 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.453917 22704 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:22.401597 25546 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:10:22.458843 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.465107 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.441771 22709 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.467509 22703 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:22.343727 25523 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.438961 25329 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:22.426748 25514 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:22.466378 25391 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:22.484813 22708 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:22.486063 23420 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:22.467276 25546 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.499182 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:22.502159 22709 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:22.500648 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:22.501348 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 174000us
I20260709 13:10:22.500663 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:22.497598 25421 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.734s	user 0.006s	sys 0.024s
W20260709 13:10:22.497999 25443 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.717s	user 0.019s	sys 0.000s
I20260709 13:10:22.513761 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:22.490111 25329 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.493526 25506 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.478884 25402 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.497207 25485 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:10:22.787596 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:22.894456 25517 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.006s
I20260709 13:10:22.623752 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.791324 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:22.549175 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:22.545584 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.643559 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2077) took 1050 ms. Trace:
I20260709 13:10:22.548789 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:23.078007 22776 rpcz_store.cc:276] 0709 13:10:21.592812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.618908 (+ 26096us) service_pool.cc:224] Handling call
0709 13:10:21.628628 (+  9720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:21.771135 (+142507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.643119 (+871984us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400130e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:22.643456 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240009088}
I20260709 13:10:23.078804 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.888868 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:22.941975 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.747128 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:22.587764 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 350) took 1570 ms. Trace:
I20260709 13:10:22.658704 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:22.680917 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45654
I20260709 13:10:23.091643 22635 raft_consensus.cc:606] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Received request to transfer leadership
I20260709 13:10:22.579005 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:22.551508 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:22.798219 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 357086us
W20260709 13:10:22.636854 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:23.053476 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.578552 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:22.580737 25195 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.819s	user 0.005s	sys 0.010s
I20260709 13:10:22.584786 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.610834 22709 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:10:22.779855 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:22.550083 22703 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:22.620731 25399 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:22.501451 25391 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:22.507177 25423 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.802s	user 0.017s	sys 0.012s
I20260709 13:10:22.901207 22701 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:22.507385 25546 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:23.237321 23416 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:22.515976 22780 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:22.968094 22630 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:23.078955 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:22.682436 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 134831us
I20260709 13:10:23.107412 23419 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:22.935731 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.519s (SENT) [suppressed 25 similar messages]
I20260709 13:10:23.107656 22704 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:10:23.244908 22704 pending_rounds.cc:85] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 3
I20260709 13:10:23.245199 22704 pending_rounds.cc:107] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:10:23.245491 22704 raft_consensus.cc:2889] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:23.108377 22783 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:22.647512 25527 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.003s
W20260709 13:10:22.729645 25584 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 13:10:22.910636 25507 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.790s	user 0.024s	sys 0.002s
W20260709 13:10:22.870082 25528 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.004s
I20260709 13:10:22.537868 25421 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:23.269757 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:23.202745 25360 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:23.216626 25399 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:22.868106 25433 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.001s
I20260709 13:10:22.823885 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:22.584978 25578 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:10:22.798219 25288 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.000s	sys 0.006s
I20260709 13:10:23.179204 22706 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:22.948350 25524 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:22.618443 25581 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:10:23.210944 22628 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:22.806021 25264 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.247687 25423 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:22.513142 25516 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:22.747586 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:22.728324 25523 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:23.216213 23414 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:23.211709 22709 pending_rounds.cc:85] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 3
I20260709 13:10:23.070309 25329 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:23.223641 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:22.649636 25520 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:22.554085 25582 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:22.550549 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:22.509258 25435 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.706s
I20260709 13:10:23.255467 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:23.258527 22775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.002s	sys 0.019s
I20260709 13:10:23.258642 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:22.822427 25533 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.001s
I20260709 13:10:23.269596 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:23.090816 22634 rpcz_store.cc:276] 0709 13:10:21.031603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.035797 (+  4194us) service_pool.cc:224] Handling call
0709 13:10:22.587532 (+1551735us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54001ca054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:22.587693 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684547840}
I20260709 13:10:23.229568 25391 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:22.551368 25303 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:22.652338 25544 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.872s	user 0.006s	sys 0.007s
I20260709 13:10:23.279421 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:23.187000 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:23.276132 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:23.286389 22708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.022s	sys 0.023s
I20260709 13:10:23.254763 25398 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:23.302395 23417 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:23.273582 25528 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:23.270282 25507 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:23.264708 25573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.023s	sys 0.026s
I20260709 13:10:23.318799 25360 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:23.274987 25521 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.819s	user 0.014s	sys 0.001s
W20260709 13:10:23.337198 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 557451us
I20260709 13:10:23.337901 22782 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:23.213687 25195 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:22.656957 25155 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:23.262691 25527 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:23.999353 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.025s	sys 0.038s
I20260709 13:10:24.081132 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 642) took 1341 ms. Trace:
W20260709 13:10:23.614638 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:23.905210 22748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.005s	sys 0.006s
W20260709 13:10:24.083034 22748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.872s	user 0.006s	sys 0.007s
I20260709 13:10:23.515471 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:24.080811 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.025s	sys 0.043s
W20260709 13:10:24.042012 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.032s	sys 0.045s
W20260709 13:10:23.622553 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:23.690706 22779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.013s	sys 0.067s
I20260709 13:10:24.081799 22778 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:23.905822 23385 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.003s	sys 0.008s
I20260709 13:10:23.808717 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:24.082258 22709 pending_rounds.cc:107] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 6.4
W20260709 13:10:24.084376 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.034s	sys 0.050s
W20260709 13:10:24.084728 22775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.007s	sys 0.029s
W20260709 13:10:24.085146 22779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.304s	user 0.013s	sys 0.092s
W20260709 13:10:24.084802 22708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.026s	sys 0.032s
W20260709 13:10:24.086364 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.087769 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 500) took 1504 ms. Trace:
I20260709 13:10:24.083950 22628 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:24.081820 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 643) took 1296 ms. Trace:
I20260709 13:10:24.087464 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1655) took 1197 ms. Trace:
I20260709 13:10:24.097731 22781 rpcz_store.cc:276] 0709 13:10:22.890122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.220491 (+330369us) service_pool.cc:224] Handling call
0709 13:10:24.087411 (+866920us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:24.086941 23385 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.003s	sys 0.009s
W20260709 13:10:24.099550 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.16381e+06us
I20260709 13:10:24.089906 22709 raft_consensus.cc:2889] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:24.096623 23414 rpcz_store.cc:276] 0709 13:10:22.583053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.584426 (+  1373us) service_pool.cc:224] Handling call
0709 13:10:24.087718 (+1503292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.081701 23417 rpcz_store.cc:276] 0709 13:10:22.740278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.228625 (+488347us) service_pool.cc:224] Handling call
0709 13:10:24.081056 (+852431us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:24.107151 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 920283us
W20260709 13:10:24.109541 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 839584us
I20260709 13:10:24.113339 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 504) took 1187 ms. Trace:
W20260709 13:10:24.115960 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.113910 23416 rpcz_store.cc:276] 0709 13:10:22.926541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.238446 (+311905us) service_pool.cc:224] Handling call
0709 13:10:24.113231 (+874785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.097213 23420 rpcz_store.cc:276] 0709 13:10:22.785250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.246134 (+460884us) service_pool.cc:224] Handling call
0709 13:10:24.081745 (+835611us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:23.329260 25288 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.006s	sys 0.017s
W20260709 13:10:23.547091 25290 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.995s	user 0.036s	sys 0.046s
W20260709 13:10:24.075269 25546 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.813s	user 0.015s	sys 0.000s
W20260709 13:10:23.985055 25595 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
I20260709 13:10:24.132858 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:24.026326 25498 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.000s	sys 0.012s
W20260709 13:10:24.137118 23418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 498) took 1623 ms (client timeout 1801 ms). Trace:
W20260709 13:10:24.083186 25573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.530s	user 0.035s	sys 0.038s
I20260709 13:10:24.137882 23412 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:24.138171 22626 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:24.139199 23415 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:24.139789 22784 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:24.144372 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2033) took 1878 ms. Trace:
I20260709 13:10:24.194778 22709 rpcz_store.cc:276] 0709 13:10:22.248833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.265338 (+ 16505us) service_pool.cc:224] Handling call
0709 13:10:22.316207 (+ 50869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.127021 (+1810814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.127387 (+   366us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.184603 22710 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:24.146767 22701 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:23.508564 25434 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:24.205794 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 119007us
I20260709 13:10:24.207026 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:24.201196 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2057) took 1652 ms. Trace:
W20260709 13:10:24.140952 23418 rpcz_store.cc:269] 0709 13:10:22.513963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.514606 (+   643us) service_pool.cc:224] Handling call
0709 13:10:24.137011 (+1622405us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.209992 22704 rpcz_store.cc:276] 0709 13:10:22.549030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.313336 (+764306us) service_pool.cc:224] Handling call
0709 13:10:24.120947 (+807611us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.137815 (+ 16868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:24.137863 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.137919 (+    56us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:24.200825 (+ 62906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.200916 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.201112 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:23.538381 25583 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:23.361178 25519 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.079820 25402 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:23.809695 25407 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:23.365418 25421 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.220367 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:24.130378 25290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.578s	user 0.045s	sys 0.076s
W20260709 13:10:24.231719 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 113766us
I20260709 13:10:24.238981 25597 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:10:24.242594 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 109882us
I20260709 13:10:24.243996 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:24.244565 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.224845 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.135547 25516 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:24.228554 22519 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.031s	sys 0.112s
W20260709 13:10:24.143198 25498 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.000s	sys 0.013s
I20260709 13:10:23.361685 22531 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 8, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:24.245651 22747 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:24.104203 25521 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.250051 22627 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:24.250578 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.251801 22633 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:24.252815 22519 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.031s	sys 0.112s
I20260709 13:10:24.204080 25582 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:23.455571 25588 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 1.278s
W20260709 13:10:24.224884 25595 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:10:24.192467 25443 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.228755 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:24.296844 22634 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:23.558699 25524 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:24.272014 22519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 748613us
I20260709 13:10:24.148136 25546 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.254511 25520 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:10:24.226130 25288 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.112183 25544 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:23.329871 25433 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.297372 22634 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:23.643102 25399 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.254086 25523 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:24.303180 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 644) took 1502 ms. Trace:
I20260709 13:10:24.326140 25521 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.326555 23417 rpcz_store.cc:276] 0709 13:10:22.791803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.198403 (+1406600us) service_pool.cc:224] Handling call
0709 13:10:24.252718 (+ 54315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.259744 (+  7026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:24.259798 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.272875 (+ 13077us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:24.293575 (+ 20700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.293685 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.293908 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.327056 25288 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.256141 25573 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.259819 25435 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.259336 25440 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.124603 25264 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.338636 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:24.309029 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.270829 25421 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:24.314265 25582 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.316458 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.315851 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.318560 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.319568 25528 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.359365 25528 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:24.359889 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.360204 25528 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:24.360837 25528 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.361737 22630 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:24.362157 25592 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.578s	user 0.000s	sys 0.011s
I20260709 13:10:24.319872 25303 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.320822 25360 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.362977 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.363346 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.321023 25423 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.252605 25407 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.256371 25519 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.305104 25443 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.322966 25433 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.323287 23411 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:24.323928 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97094.7us
I20260709 13:10:24.324345 25507 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.324815 25544 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.325942 25399 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:24.288529 25290 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:24.322847 22631 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:24.335839 25601 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
W20260709 13:10:24.389988 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 1.64242e+06us
I20260709 13:10:24.391192 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2053) took 1843 ms. Trace:
I20260709 13:10:24.391474 22626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 361) took 2103 ms. Trace:
I20260709 13:10:24.391623 22701 rpcz_store.cc:276] 0709 13:10:22.547922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.248171 (+700249us) service_pool.cc:224] Handling call
0709 13:10:23.274522 (+ 26351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.196787 (+922265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.390958 (+194171us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b54002c38d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:24.391127 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986985344}
I20260709 13:10:24.335189 25435 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.256245 25583 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.336459 25604 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:24.337002 25440 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.337631 25524 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.329071 22634 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.406980 22634 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:24.349314 23421 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:24.251305 25434 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.351545 22627 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 4: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:24.372328 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:24.417249 25434 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.363476 25360 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.344010 25520 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:24.373656 23419 raft_consensus.cc:2410] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:24.381436 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2108) took 1865 ms. Trace:
I20260709 13:10:24.382140 22703 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:24.382659 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2103) took 1871 ms. Trace:
I20260709 13:10:24.446682 22784 rpcz_store.cc:276] 0709 13:10:22.511491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.512227 (+   736us) service_pool.cc:224] Handling call
0709 13:10:22.588311 (+ 76084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.142742 (+1554431us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.382406 (+239664us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400132254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:24.382561 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3567490432}
W20260709 13:10:24.386886 23415 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 634) took 2506 ms (client timeout 2219 ms). Trace:
W20260709 13:10:24.381572 22710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 335) took 2202 ms (client timeout 2248 ms). Trace:
W20260709 13:10:24.448184 23415 rpcz_store.cc:269] 0709 13:10:21.880182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.880664 (+   482us) service_pool.cc:224] Handling call
0709 13:10:24.386563 (+2505899us) spinlock_profiling.cc:243] Waited 2.24 s on lock 0x7b5400503554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:24.386779 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5142940416}
W20260709 13:10:24.389367 22782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2089) took 2620 ms (client timeout 2219 ms). Trace:
W20260709 13:10:24.396966 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 588368us
I20260709 13:10:24.398654 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 533) took 1962 ms. Trace:
W20260709 13:10:24.399600 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.127s	user 0.031s	sys 0.078s
I20260709 13:10:24.344386 25573 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.384262 25423 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:24.391884 22626 rpcz_store.cc:276] 0709 13:10:22.278962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.279157 (+   195us) service_pool.cc:224] Handling call
0709 13:10:22.321643 (+ 42486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.200415 (+1878772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.381821 (+181406us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b540006d6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:24.381997 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4177265536}
I20260709 13:10:24.399211 22783 raft_consensus.cc:2376] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:24.403386 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:24.403348 22675 connection.cc:256] RPC call timeout handler was delayed by -2.412558912999998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
W20260709 13:10:24.455461 22675 connection.cc:256] RPC call timeout handler was delayed by -2.3522589129999005s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1912s for a response
I20260709 13:10:24.380475 25407 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.418756 22635 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:10:24.414387 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.192s (SENT)
W20260709 13:10:24.450380 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.178s	user 0.031s	sys 0.078s
W20260709 13:10:24.450045 22782 rpcz_store.cc:269] 0709 13:10:21.770246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.778676 (+  8430us) service_pool.cc:224] Handling call
0709 13:10:24.389145 (+2610469us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540015a4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:24.389286 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3555723136}
W20260709 13:10:24.448299 22710 rpcz_store.cc:269] 0709 13:10:22.179705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.179938 (+   233us) service_pool.cc:224] Handling call
0709 13:10:24.381351 (+2201413us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400314554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:24.381509 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4523275648}
W20260709 13:10:24.456907 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 1.18469e+06us
I20260709 13:10:24.450266 23412 rpcz_store.cc:276] 0709 13:10:22.428023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.430700 (+  2677us) service_pool.cc:224] Handling call
0709 13:10:22.453148 (+ 22448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.141831 (+1688683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.390271 (+248440us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400354b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:24.390457 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3873795968}
I20260709 13:10:24.403602 25155 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.456881 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:24.458549 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:24.459697 22632 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:24.445302 22775 rpcz_store.cc:276] 0709 13:10:22.516163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.552795 (+ 36632us) service_pool.cc:224] Handling call
0709 13:10:22.687116 (+134321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.745539 (+ 58423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.745593 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.746283 (+   690us) log.cc:844] Serialized 26 byte log entry
0709 13:10:24.093762 (+1347479us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:24.093807 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.093904 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.381176 (+287272us) raft_consensus.cc:1685] finished
0709 13:10:24.381210 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.381366 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58829,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":1347029,"threads_started":1,"wal-append.queue_time_us":1558345}
I20260709 13:10:24.461485 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 648) took 1569 ms. Trace:
I20260709 13:10:24.459686 25528 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.461926 23416 rpcz_store.cc:276] 0709 13:10:22.885729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.442664 (+1556935us) service_pool.cc:224] Handling call
0709 13:10:24.454379 (+ 11715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.454588 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:24.454661 (+    73us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.454710 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:24.454745 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.454825 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.454962 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.420457 25527 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.501612 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 649) took 1615 ms. Trace:
W20260709 13:10:24.501814 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43849.4us
I20260709 13:10:24.442777 25195 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.420336 25520 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:24.561848 23416 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:24.444700 25360 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.506461 23415 rpcz_store.cc:276] 0709 13:10:22.886141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.449786 (+1563645us) service_pool.cc:224] Handling call
0709 13:10:24.454496 (+  4710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.500612 (+ 46116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:24.501135 (+   523us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.501192 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:24.501238 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.501324 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.501533 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1460,"spinlock_wait_cycles":87808}
W20260709 13:10:24.565145 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 161864us
W20260709 13:10:24.573088 25599 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:10:24.566542 25288 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.567401 25443 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.567807 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:24.585912 22784 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:24.587294 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:24.588075 22784 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:24.559041 25614 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:24.589934 22626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45654
I20260709 13:10:24.590139 25614 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:24.567461 25596 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 13:10:24.503573 25155 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.418787 25604 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.613168 22628 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:24.565547 25524 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:10:24.574736 25590 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:10:24.615283 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2056) took 1243 ms. Trace:
I20260709 13:10:24.602501 25360 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.619061 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2109) took 2095 ms. Trace:
I20260709 13:10:24.620599 23412 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:24.574491 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:24.611240 25520 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:24.648238 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 233870us
I20260709 13:10:24.620393 22777 rpcz_store.cc:276] 0709 13:10:22.523927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.553866 (+ 29939us) service_pool.cc:224] Handling call
0709 13:10:22.627446 (+ 73580us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.686084 (+ 58638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.686122 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.751601 (+ 65479us) log.cc:844] Serialized 26 byte log entry
0709 13:10:23.113324 (+361723us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:23.113432 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:23.113556 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.618733 (+1505177us) raft_consensus.cc:1685] finished
0709 13:10:24.618766 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.618969 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11686,"raft.queue_time_us":14872,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":148,"thread_start_us":240242,"threads_started":1,"wal-append.queue_time_us":1038138}
W20260709 13:10:24.651014 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45586.9us
W20260709 13:10:24.677767 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 216656us
I20260709 13:10:24.602563 25583 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.604841 25443 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.627099 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 505) took 1667 ms. Trace:
I20260709 13:10:24.692835 23412 rpcz_store.cc:276] 0709 13:10:22.960484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.458299 (+1497815us) service_pool.cc:224] Handling call
0709 13:10:24.618360 (+160061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.621109 (+  2749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.626980 (+  5871us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.678758 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:24.612465 25615 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.677198 23417 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:10:24.621410 22628 rpcz_store.cc:276] 0709 13:10:23.372250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.372469 (+   219us) service_pool.cc:224] Handling call
0709 13:10:24.615200 (+1242731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.616125 25611 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.647959 25434 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:24.722143 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:24.723070 22777 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:24.622076 25524 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.725147 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:24.728518 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:24.701136 22784 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:24.694536 22626 raft_consensus.cc:606] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Received request to transfer leadership
I20260709 13:10:24.704675 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2049) took 2203 ms. Trace:
I20260709 13:10:24.622787 25407 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 13:10:24.729844 22626 raft_consensus.cc:614] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:24.706743 25443 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.680279 25544 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:24.730048 22707 rpcz_store.cc:276] 0709 13:10:22.501614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.896764 (+395150us) service_pool.cc:224] Handling call
0709 13:10:22.897150 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.897672 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.897715 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.898342 (+   627us) log.cc:844] Serialized 26 byte log entry
0709 13:10:24.084113 (+1185771us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:24.084160 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.084241 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.704395 (+620154us) raft_consensus.cc:1685] finished
0709 13:10:24.704423 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.704607 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":1172246,"threads_started":1,"wal-append.queue_time_us":1543934}
I20260709 13:10:24.733749 25524 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:24.733560 25544 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.735198 25612 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:24.736389 25544 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.647699 25360 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.739114 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:24.738999 25360 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.739990 25612 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:24.740427 25612 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.740835 25612 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:24.741225 25360 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.699056 25288 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:10:24.711339 25615 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.711618 25611 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.742427 25288 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:24.742841 25288 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:24.716008 25604 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.648562 25613 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:24.743788 25288 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.618338 25546 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.730293 25621 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:24.745829 25621 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.746249 25621 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:24.746891 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 172578us
I20260709 13:10:24.705698 25520 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:24.749761 25615 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.751143 25611 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.751459 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:24.752708 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:24.753455 22784 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:24.752895 25612 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.721205 25434 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:24.731360 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2045) took 2351 ms. Trace:
I20260709 13:10:24.704874 25264 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.647439 25614 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.703677 25423 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:24.726989 25600 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:10:24.733866 25443 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.726491 25618 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:10:24.603726 25195 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:24.718590 22676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.023s	sys 0.026s
I20260709 13:10:24.770928 25288 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:24.771278 22676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.023s	sys 0.026s
W20260709 13:10:24.774421 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:24.775681 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 530112us
W20260709 13:10:24.776264 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.002s	sys 0.013s
W20260709 13:10:24.778760 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.002s	sys 0.013s
I20260709 13:10:24.779711 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:24.776593 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.001s	sys 0.014s
W20260709 13:10:24.780835 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.001s	sys 0.015s
W20260709 13:10:24.781724 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 574802us
W20260709 13:10:24.783986 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 222597us
W20260709 13:10:24.784313 22778 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.005s
I20260709 13:10:24.784916 22778 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:24.785214 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:24.785669 22778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 357) took 1562 ms (client timeout 1902 ms). Trace:
I20260709 13:10:24.616628 25155 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.786513 22780 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:24.761161 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2106) took 2246 ms. Trace:
I20260709 13:10:24.792688 22779 rpcz_store.cc:276] 0709 13:10:22.514850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.515005 (+   155us) service_pool.cc:224] Handling call
0709 13:10:22.579756 (+ 64751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.580288 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.580324 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.740682 (+160358us) log.cc:844] Serialized 26 byte log entry
0709 13:10:24.092989 (+1352307us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:24.093037 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.093111 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.760819 (+667708us) raft_consensus.cc:1685] finished
0709 13:10:24.760852 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.761073 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2622,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":117,"thread_start_us":1311481,"threads_started":1,"wal-append.queue_time_us":1631221}
W20260709 13:10:24.766139 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.754320 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:24.761269 22776 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:24.756675 22708 rpcz_store.cc:276] 0709 13:10:22.380177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.683556 (+303379us) service_pool.cc:224] Handling call
0709 13:10:22.684005 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.684603 (+   598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.684644 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.685355 (+   711us) log.cc:844] Serialized 26 byte log entry
0709 13:10:24.093446 (+1408091us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:24.093487 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.093549 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.731068 (+637519us) raft_consensus.cc:1685] finished
0709 13:10:24.731096 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.731284 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1553,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":165,"thread_start_us":1407648,"threads_started":1,"wal-append.queue_time_us":1691174}
I20260709 13:10:24.766607 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:24.796924 22784 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
W20260709 13:10:24.795954 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.784137 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:24.790023 22778 rpcz_store.cc:269] 0709 13:10:23.223119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.223297 (+   178us) service_pool.cc:224] Handling call
0709 13:10:24.785588 (+1562291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:24.801187 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:24.802208 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:24.802915 22778 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:24.803826 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:24.803318 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:24.767079 25391 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.005s
I20260709 13:10:24.808281 23416 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:24.755921 25434 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:24.813810 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:24.817461 22784 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:24.764788 25421 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:24.764070 25402 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:24.816816 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:24.815649 22781 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.003s	sys 0.029s
I20260709 13:10:24.828766 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:24.755067 25573 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:24.755415 25440 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.794872 25264 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:24.791981 25618 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:24.786975 25507 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:24.765515 25435 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:24.802066 25443 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:10:24.812052 25516 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:10:24.758322 25423 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.807201 25544 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:25.072502 22777 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:10:25.123533 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 296675us
W20260709 13:10:24.849399 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 139275us
W20260709 13:10:25.282333 22675 connection.cc:256] RPC call timeout handler was delayed by -2.2037589130000015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 13:10:25.075528 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 256325us
W20260709 13:10:25.293005 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.293643 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.293926 22781 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.009s	sys 0.030s
I20260709 13:10:24.873390 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 645) took 2041 ms. Trace:
I20260709 13:10:24.860654 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:25.185457 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.156044 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 340787us
W20260709 13:10:25.295495 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.809154 25523 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:24.804337 25605 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:25.045892 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.942s (SENT)
I20260709 13:10:25.296753 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2054) took 2748 ms. Trace:
I20260709 13:10:25.297116 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.303483 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.303210 22703 rpcz_store.cc:276] 0709 13:10:22.548209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.262833 (+714624us) service_pool.cc:224] Handling call
0709 13:10:24.381912 (+1119079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.446685 (+ 64773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.296550 (+849865us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400314550. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.296682 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519026432}
W20260709 13:10:25.096043 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 220341us
I20260709 13:10:25.294525 23417 rpcz_store.cc:276] 0709 13:10:22.831775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.328328 (+1496553us) service_pool.cc:224] Handling call
0709 13:10:24.328785 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:24.700926 (+372141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.872969 (+172043us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54000da7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:24.873295 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402184448}
I20260709 13:10:25.304126 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:25.305128 22675 connection.cc:256] RPC call timeout handler was delayed by -1.9526589129999365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2356s for a response
W20260709 13:10:25.305840 22675 connection.cc:256] RPC call timeout handler was delayed by -1.6025589129999389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2745s for a response
W20260709 13:10:25.306164 22675 connection.cc:256] RPC call timeout handler was delayed by -1.3217589129999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3057s for a response
W20260709 13:10:25.158352 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 383381us
I20260709 13:10:25.282703 22778 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:24.816797 25607 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:25.309588 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:25.070683 25329 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.866s	user 0.015s	sys 0.000s
I20260709 13:10:25.310128 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.313042 23418 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:24.914160 25399 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.002s
I20260709 13:10:24.817978 25615 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:25.311715 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.288614 25264 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.072116 25520 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:24.915736 25155 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:24.831948 25402 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:24.893607 25612 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:25.153106 25421 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:25.319098 23416 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:24.814163 25195 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.320053 25616 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:10:25.343598 23421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 501) took 2760 ms (client timeout 2226 ms). Trace:
I20260709 13:10:25.343647 25616 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:25.298079 25507 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:25.344244 23421 rpcz_store.cc:269] 0709 13:10:22.583476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.623332 (+ 39856us) service_pool.cc:224] Handling call
0709 13:10:25.343341 (+2720009us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54005163d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.343514 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2905269632}
I20260709 13:10:25.344923 25616 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.345600 23420 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:10:25.346536 25507 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:25.346997 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 647) took 2462 ms. Trace:
I20260709 13:10:25.347380 23420 rpcz_store.cc:276] 0709 13:10:22.884450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.422401 (+1537951us) service_pool.cc:224] Handling call
0709 13:10:25.345344 (+922943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:25.346049 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.346787 (+   738us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54005163d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.346931 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121499008}
I20260709 13:10:25.335260 25628 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.335948 25421 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:24.894816 25521 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.423s
I20260709 13:10:25.339279 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.071201 25621 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.341665 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:25.343145 25533 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:25.355904 25533 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:25.341740 25612 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:25.356971 25533 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:25.358162 25533 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.359102 22784 raft_consensus.cc:2410] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:10:25.359900 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.362689 25621 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.362843 25628 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.364867 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:24.817378 25391 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.368435 22705 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:25.349121 23419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 641) took 2716 ms. Trace:
I20260709 13:10:25.370995 23419 rpcz_store.cc:276] 0709 13:10:22.632709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.249188 (+616479us) service_pool.cc:224] Handling call
0709 13:10:25.348897 (+2099709us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b540064aad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.349055 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669210496}
I20260709 13:10:25.012100 25524 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:24.854055 25583 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.465s
I20260709 13:10:25.373088 25583 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:25.373463 25583 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:25.374325 25583 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:25.375813 25583 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:25.376959 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.318957 25264 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:25.375813 25391 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.379673 25524 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:25.307798 25544 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:25.351238 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.079896 25434 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.365825 25520 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:25.366951 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.324076 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.346406 25616 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.383229 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 632) took 3515 ms. Trace:
I20260709 13:10:25.455364 23411 rpcz_store.cc:276] 0709 13:10:21.867525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.867701 (+   176us) service_pool.cc:224] Handling call
0709 13:10:21.896120 (+ 28419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.375525 (+2479405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.382937 (+1007412us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54005f7654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:25.383161 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5581488000}
I20260709 13:10:25.351912 25195 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.447626 22777 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:25.454191 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 139700us
I20260709 13:10:25.459043 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.462118 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1656) took 1170 ms. Trace:
W20260709 13:10:25.462173 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.462571 22777 rpcz_store.cc:276] 0709 13:10:24.291479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.317661 (+1026182us) service_pool.cc:224] Handling call
0709 13:10:25.462043 (+144382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.402794 22531 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 5, term changed from 3 to 7, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to <none>, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 7 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:25.366452 25628 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.317022 25329 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:25.443352 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.467923 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.469518 23418 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:25.471292 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:25.474762 25582 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:25.476122 25619 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:25.476643 25619 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:25.477883 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 140788us
W20260709 13:10:25.479224 25587 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 2.260s	user 0.007s	sys 0.021s
W20260709 13:10:25.485136 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 185842us
I20260709 13:10:25.320262 25615 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:25.492851 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 128174us
I20260709 13:10:25.453799 25520 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.503598 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:25.507723 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 461861us
I20260709 13:10:25.511420 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.541920 22777 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:25.453475 25524 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:25.544673 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.544914 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.477727 25619 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:25.478892 25544 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.474762 25618 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:10:25.537611 22628 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:25.541128 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2114) took 1455 ms. Trace:
W20260709 13:10:25.543056 23384 leader_election.cc:336] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 2.008s (SENT)
I20260709 13:10:25.545511 22630 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:25.546118 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:25.583837 23384 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 2.141s (SENT)
I20260709 13:10:25.531436 22779 raft_consensus.cc:2368] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 7.
W20260709 13:10:25.548255 25631 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:10:25.548722 25524 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.549109 25615 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.549640 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:25.544080 25520 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:25.555735 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.556048 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.548739 25329 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:25.539983 25195 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:25.557956 25519 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.559686 25434 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.562253 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.564823 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.566218 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.467252 25402 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:25.594700 22626 raft_consensus.cc:2410] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:10:25.568346 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:25.594951 22627 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:25.591212 22633 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:25.573076 25605 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:10:25.579454 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107149us
I20260709 13:10:25.582278 22781 rpcz_store.cc:276] 0709 13:10:24.085247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.197886 (+112639us) service_pool.cc:224] Handling call
0709 13:10:24.243710 (+ 45824us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.244007 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:25.294392 (+1050385us) log.cc:844] Serialized 14 byte log entry
0709 13:10:25.540694 (+246302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.540757 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:25.540812 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.540885 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.541051 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":631933,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":69,"raft.queue_time_us":29614,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9163,"thread_start_us":1232157,"threads_started":2}
I20260709 13:10:25.585846 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.585222 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.587256 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:25.588130 23384 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 2.151s (SENT)
I20260709 13:10:25.580111 25523 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:25.614727 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:25.616320 23384 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.151s (SENT)
I20260709 13:10:25.614725 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.615698 22630 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:25.616914 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.596467 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:25.600769 22634 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:25.617163 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1659) took 1216 ms. Trace:
I20260709 13:10:25.618853 22775 rpcz_store.cc:276] 0709 13:10:24.400569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.564493 (+1163924us) service_pool.cc:224] Handling call
0709 13:10:25.617089 (+ 52596us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.615746 22632 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:25.589635 25524 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.625435 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:25.620381 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.614478 25621 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:25.586130 25264 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.621148 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.641274 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.641968 22626 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.644338 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2046) took 3226 ms. Trace:
I20260709 13:10:25.644809 22705 rpcz_store.cc:276] 0709 13:10:22.417777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.831295 (+413518us) service_pool.cc:224] Handling call
0709 13:10:22.877354 (+ 46059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.368927 (+2491573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.644060 (+275133us) spinlock_profiling.cc:243] Waited 2.49 s on lock 0x7b5400087d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:25.644263 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5728368384}
I20260709 13:10:25.646827 25288 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:25.648031 25288 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:25.648402 25288 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:25.648757 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.649369 25288 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.650465 22784 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:10:25.651199 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2116) took 1565 ms. Trace:
I20260709 13:10:25.651535 22784 rpcz_store.cc:276] 0709 13:10:24.085783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.649778 (+1563995us) service_pool.cc:224] Handling call
0709 13:10:25.650210 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:25.650921 (+   711us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.651138 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.652717 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 369) took 1503 ms. Trace:
I20260709 13:10:25.653156 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2117) took 1567 ms. Trace:
I20260709 13:10:25.653124 22635 rpcz_store.cc:276] 0709 13:10:24.149330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.346561 (+197231us) service_pool.cc:224] Handling call
0709 13:10:25.652536 (+1305975us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b540022d654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.652653 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126463872}
I20260709 13:10:25.653615 22784 rpcz_store.cc:276] 0709 13:10:24.085939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.652312 (+1566373us) service_pool.cc:224] Handling call
0709 13:10:25.652660 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.652848 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:25.652874 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.652917 (+    43us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:25.652946 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.652998 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.653104 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.638491 25621 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.615000 25615 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:10:25.650282 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:25.623991 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 358) took 2400 ms (client timeout 2151 ms). Trace:
I20260709 13:10:25.625456 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1658) took 1229 ms. Trace:
I20260709 13:10:25.626312 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.654611 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.675657 22634 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:25.654976 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.655510 23411 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:25.655558 22708 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:25.655637 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2052) took 3311 ms. Trace:
I20260709 13:10:25.655874 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.657729 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.680397 23419 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:25.659399 22709 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:25.658130 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:25.661262 23420 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:10:25.681730 22710 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.627471 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.661346 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2113) took 1576 ms. Trace:
W20260709 13:10:25.658188 22530 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 247) took 25298 ms (client timeout 15000 ms). Trace:
I20260709 13:10:25.662006 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:25.664527 22627 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:25.684255 22777 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:25.684332 22706 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:25.685237 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:25.686015 22626 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:25.628443 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 360) took 3614 ms. Trace:
I20260709 13:10:25.660465 22782 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:25.626412 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.630823 25528 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:25.679556 23416 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:25.680397 22747 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:25.665175 22531 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.631335 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:25.631496 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88482.1us
I20260709 13:10:25.627586 25523 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.637102 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1654) took 2747 ms. Trace:
I20260709 13:10:25.693285 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.670863 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:25.674983 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1662) took 1254 ms. Trace:
I20260709 13:10:25.637822 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.682987 22780 rpcz_store.cc:276] 0709 13:10:24.084909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.085069 (+   160us) service_pool.cc:224] Handling call
0709 13:10:24.785588 (+700519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.792329 (+  6741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.661131 (+868802us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54003250d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.661273 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511059712}
I20260709 13:10:25.693488 22776 rpcz_store.cc:276] 0709 13:10:22.889510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.228171 (+338661us) service_pool.cc:224] Handling call
0709 13:10:24.761053 (+1532882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.803807 (+ 42754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.636856 (+833049us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540015a9d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.637027 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3430029184}
I20260709 13:10:25.654812 22703 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:25.675397 22779 rpcz_store.cc:276] 0709 13:10:24.395989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.803602 (+407613us) service_pool.cc:224] Handling call
0709 13:10:25.625161 (+821559us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54001865d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.625370 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489190656}
I20260709 13:10:25.673736 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:25.671574 22781 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:25.689889 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.658502 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.678364 22631 rpcz_store.cc:276] 0709 13:10:22.343721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.400612 (+ 56891us) service_pool.cc:224] Handling call
0709 13:10:22.405147 (+  4535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.392843 (+1987696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.655362 (+1262519us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b540022ea54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:25.655564 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4409857408}
I20260709 13:10:25.614043 25195 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:25.675258 22783 rpcz_store.cc:269] 0709 13:10:23.223533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.246330 (+ 22797us) service_pool.cc:224] Handling call
0709 13:10:25.623781 (+2377451us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54002d5854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:25.623909 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740011264}
I20260709 13:10:25.665606 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:25.694139 22778 rpcz_store.cc:276] 0709 13:10:24.420659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.625158 (+1204499us) service_pool.cc:224] Handling call
0709 13:10:25.674912 (+ 49754us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.698009 22702 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.589684 25619 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.695277 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.699510 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:25.706780 22707 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:25.697731 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.699811 23416 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:25.694087 22701 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:25.687036 22629 rpcz_store.cc:276] 0709 13:10:22.013700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.127629 (+113929us) service_pool.cc:224] Handling call
0709 13:10:22.128115 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.197607 (+ 69492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.197640 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.210036 (+ 12396us) log.cc:844] Serialized 26 byte log entry
0709 13:10:22.415856 (+205820us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b180025d200. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:10:22.503461 (+ 87605us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:22.503521 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.503621 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:25.308821 (+2805200us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:25.628205 (+319384us) raft_consensus.cc:1685] finished
0709 13:10:25.628231 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.628376 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52752,"raft.queue_time_us":80057,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3905,"spinlock_wait_cycles":28381952,"thread_start_us":87359,"threads_started":1,"wal-append.queue_time_us":725694}
I20260709 13:10:25.700026 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.707969 22531 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.700951 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2119) took 1614 ms. Trace:
W20260709 13:10:25.701653 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85358.1us
I20260709 13:10:25.709971 22782 rpcz_store.cc:276] 0709 13:10:24.086242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.659917 (+1573675us) service_pool.cc:224] Handling call
0709 13:10:25.660279 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.700666 (+ 40387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.700891 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.711166 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.703708 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2120) took 1617 ms. Trace:
I20260709 13:10:25.713044 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:25.713246 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:25.713840 23416 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:25.713891 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2122) took 1593 ms. Trace:
I20260709 13:10:25.714381 22779 rpcz_store.cc:276] 0709 13:10:24.120609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.695701 (+1575092us) service_pool.cc:224] Handling call
0709 13:10:25.713386 (+ 17685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.713577 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:25.713602 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.713645 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:25.713675 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.713730 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.713832 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11648}
I20260709 13:10:25.655117 25621 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:25.718116 25591 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.143s	user 0.017s	sys 0.010s
I20260709 13:10:25.693131 25288 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.664669 25402 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.720023 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:25.720592 25195 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.722730 25434 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:25.722823 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:25.724525 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:25.724949 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93710.5us
I20260709 13:10:25.725811 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:25.718272 22704 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:25.726823 25519 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:25.727002 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:25.727274 25519 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.727675 25519 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:25.727795 25628 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:25.729447 23419 raft_consensus.cc:2410] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:25.728976 22531 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.729954 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 362) took 1227 ms. Trace:
I20260709 13:10:25.711779 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.725867 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.730321 22783 rpcz_store.cc:276] 0709 13:10:24.502681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.697643 (+1194962us) service_pool.cc:224] Handling call
0709 13:10:25.729474 (+ 31831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.729646 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:25.729669 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.729706 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:25.729735 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.729788 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.729893 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.711800 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:25.731750 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a0e9f7dc2a2e4bdb9287d910e4b491f7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:25.733040 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.727744 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:25.734642 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.734658 22776 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:25.736014 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2121) took 1633 ms. Trace:
I20260709 13:10:25.736212 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:25.736488 22776 rpcz_store.cc:276] 0709 13:10:24.102447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.695266 (+1592819us) service_pool.cc:224] Handling call
0709 13:10:25.734391 (+ 39125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.735696 (+  1305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.735938 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:25.737215 22628 raft_consensus.cc:2410] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:25.737061 22710 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:25.711930 22777 rpcz_store.cc:276] 0709 13:10:24.086406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.686310 (+1599904us) service_pool.cc:224] Handling call
0709 13:10:25.703007 (+ 16697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.703264 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:25.703303 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.703366 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:25.703418 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.703494 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.703629 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:25.738890 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.737757 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:25.740083 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:25.699693 25605 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:25.745820 22747 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:25.718107 22780 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:25.755812 22531 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.751034 22633 raft_consensus.cc:2376] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4. Re-sending same reply.
I20260709 13:10:25.759966 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:25.760871 22777 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:25.763808 22780 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
W20260709 13:10:25.767216 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.769455 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.742111 25423 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:25.733332 25434 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:25.776371 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.742916 25527 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:25.777794 22777 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:25.775300 22531 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 6, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.779531 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.733995 25628 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.780575 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:25.757274 25155 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:25.756655 25391 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:25.785910 22783 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:25.787783 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.788764 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:25.790015 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.790841 22780 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:25.797909 22780 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:25.795872 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:25.802320 22706 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:25.803694 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:25.776347 25329 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:25.810591 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84865.3us
I20260709 13:10:25.776706 25605 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:25.812945 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.755486 25325 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:10:25.783782 25521 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:25.787597 25628 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.825006 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.826251 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:25.776192 25507 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:10:25.806102 25546 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:25.836827 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:25.805498 25528 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:25.804435 25544 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:25.810868 25407 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:25.798221 25527 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:25.842537 22630 raft_consensus.cc:2410] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:10:25.859169 22776 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:10:25.847615 22531 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 4, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.856488 25407 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:25.864988 25407 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.865671 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:25.866144 25605 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.869691 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:25.833662 25611 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:25.807082 25612 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:10:25.856616 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:25.834270 25613 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:25.861660 22531 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.877686 22701 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:10:25.683328 22530 rpcz_store.cc:269] 0709 13:10:00.359454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:00.359780 (+   326us) service_pool.cc:224] Handling call
0709 13:10:25.657891 (+25298111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:17.250941 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:17.307735 (+ 56794us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:17.307814 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:10:17.307844 (+    30us) tablet.cc:662] Decoding operations
0709 13:10:17.309382 (+  1538us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:17.309493 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:17.309535 (+    42us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:10:17.347139 (+ 37604us) tablet.cc:701] Row locks acquired
0709 13:10:17.347188 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:10:17.347566 (+   378us) write_op.cc:270] Start()
0709 13:10:17.347833 (+   267us) write_op.cc:276] Timestamp: P: 1783602617347473 usec, L: 0
0709 13:10:17.347863 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:10:17.510031 (+162168us) log.cc:844] Serialized 4432 byte log entry
0709 13:10:18.099678 (+589647us) op_driver.cc:464] REPLICATION: finished
0709 13:10:18.543203 (+443525us) write_op.cc:301] APPLY: starting
0709 13:10:18.547887 (+  4684us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:18.612399 (+ 64512us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:18.698341 (+ 85942us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:19.140057 (+441716us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:19.140436 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 13:10:19.140511 (+    75us) write_op.cc:312] APPLY: finished
0709 13:10:19.141029 (+   518us) log.cc:844] Serialized 83 byte log entry
0709 13:10:20.452585 (+1311556us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:20.485832 (+ 33247us) write_op.cc:454] Released schema lock
0709 13:10:20.496498 (+ 10666us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:20.550130 (+ 53632us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:20.551328 (+  1198us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":33894,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":423383,"apply.run_cpu_time_us":39069,"apply.run_wall_time_us":1992405,"num_ops":11,"prepare.queue_time_us":260536,"prepare.run_cpu_time_us":95517,"prepare.run_wall_time_us":482025,"raft.queue_time_us":170008,"raft.run_cpu_time_us":77903,"raft.run_wall_time_us":460397,"replication_time_us":693423,"thread_start_us":1930716,"threads_started":5,"wal-append.queue_time_us":1656893,"wal-append.run_cpu_time_us":22546,"wal-append.run_wall_time_us":2303623}]]}
I20260709 13:10:25.858645 25583 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:25.890764 25583 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.891360 25583 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:25.890556 22531 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.892602 22531 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.894775 25365 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:25.895501 25264 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.895758 25407 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.871937 25329 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:25.899062 25329 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:25.894239 22531 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.904536 25523 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.906417 25365 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:25.904978 22531 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.902915 25407 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:25.908932 22531 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.910574 22531 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.912212 22531 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.914027 22531 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 2, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.878576 25627 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:25.921208 25627 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:25.921698 25627 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:25.884366 25614 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:25.923768 25583 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.921778 22531 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 4, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.884727 25421 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:10:25.927349 25527 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.928668 25612 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:25.928849 25546 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:25.929131 25612 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:25.930712 22702 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:25.930966 25613 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:25.932075 25613 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:25.847481 25528 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:25.938416 25528 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:25.875308 25615 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:10:25.940076 23418 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:10:25.925750 22531 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:25.961681 23415 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:25.966910 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.961423 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 104923us
I20260709 13:10:25.966482 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 363) took 1407 ms. Trace:
I20260709 13:10:25.971359 22780 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:25.982955 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:25.855155 25598 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:10:25.990653 22531 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: config changed from index 3 to 5, term changed from 5 to 8, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to <none>, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 8 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:25.930135 25612 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.007484 22531 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 3, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:25.975075 25583 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:25.888806 25434 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.104s
I20260709 13:10:26.020488 25598 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:26.021026 25598 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:25.965238 25520 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.028122 25365 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:25.974221 25544 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:10:26.034399 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 225902us
I20260709 13:10:26.029392 25365 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:26.029100 25621 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.035445 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:25.960992 25524 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.053987 25621 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.063707 25528 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.066931 22784 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:26.068063 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.070142 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2118) took 1984 ms. Trace:
I20260709 13:10:26.075781 22784 rpcz_store.cc:276] 0709 13:10:24.086092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.654524 (+1568432us) service_pool.cc:224] Handling call
0709 13:10:25.671783 (+ 17259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:26.067565 (+395782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.069778 (+  2213us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54000cbf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:26.070052 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907911936}
I20260709 13:10:26.071306 25520 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.071573 22630 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:10:26.096755 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46505.1us
I20260709 13:10:26.031126 25544 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.033313 22531 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:26.099968 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:26.102192 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:26.107391 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:26.022018 25598 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.111313 22780 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:26.036870 25365 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:26.111858 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:26.112751 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:26.099570 22531 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 6, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:26.120466 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2115) took 2034 ms. Trace:
I20260709 13:10:26.074150 25612 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:26.121953 22775 rpcz_store.cc:276] 0709 13:10:24.085588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.621987 (+1536399us) service_pool.cc:224] Handling call
0709 13:10:25.622340 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.622583 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:10:25.786567 (+163984us) log.cc:844] Serialized 14 byte log entry
0709 13:10:26.119761 (+333194us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:26.120003 (+   242us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:26.120078 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.120184 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:26.120375 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3621,"prepare.queue_time_us":231267,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"thread_start_us":481133,"threads_started":2}
W20260709 13:10:26.122390 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 341859us
I20260709 13:10:26.147423 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:26.100962 25544 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:26.153944 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.181367 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 294281us
W20260709 13:10:26.185752 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 134785us
W20260709 13:10:26.265694 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.267220 25639 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:26.268702 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:26.269551 22628 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:26.270759 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.270855 25288 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:26.271486 25288 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.271759 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 369) took 1471 ms. Trace:
I20260709 13:10:26.272356 25288 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.272195 22778 rpcz_store.cc:276] 0709 13:10:24.800207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.709667 (+909460us) service_pool.cc:224] Handling call
0709 13:10:25.710040 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:25.731184 (+ 21144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:25.731250 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:25.768623 (+ 37373us) log.cc:844] Serialized 26 byte log entry
0709 13:10:26.025171 (+256548us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:26.025221 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.025296 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:26.271479 (+246183us) raft_consensus.cc:1685] finished
0709 13:10:26.271503 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:26.271689 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":654,"raft.queue_time_us":95,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":22173,"thread_start_us":181211,"threads_started":1,"wal-append.queue_time_us":302464}
W20260709 13:10:26.268954 25647 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:10:26.274977 25649 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:10:26.271064 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:26.277953 22630 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:26.279529 25646 log.cc:927] Time spent T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:10:26.225528 22581 auto_rebalancer.cc:893] Removing replace marker failed: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:26.213778 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 344174us
W20260709 13:10:26.283123 22581 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:10:26.288717 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:26.138520 25516 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:26.283846 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:26.286667 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 110628us
W20260709 13:10:26.283716 22581 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:10:26.286475 22628 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:26.297365 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:26.297649 22633 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:26.302234 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:26.174556 25523 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:26.150458 25528 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.312886 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:26.207150 25573 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:26.308827 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.347779 25573 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:26.351063 25433 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:26.317457 25628 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.313786 25627 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.361359 25628 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.362452 22777 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:26.186187 25657 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:26.365574 25657 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.366799 25657 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:26.368809 25651 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:10:26.305214 25325 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:26.369495 25598 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:26.369637 25325 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:26.317966 25329 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.370136 25325 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:26.370693 25329 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:26.151321 25632 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
W20260709 13:10:26.313498 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 369397us
I20260709 13:10:26.373359 25632 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:26.373739 25632 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:26.375942 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63194.5us
W20260709 13:10:26.322129 25619 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.001s
I20260709 13:10:26.322021 22630 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:26.168900 25519 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.299192 25618 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:10:26.343472 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:26.311666 25440 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:26.314725 25615 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:26.386945 23420 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:26.349452 25573 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.374630 25632 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:26.390643 25655 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:10:26.390903 23417 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:26.392544 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:26.392660 25195 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:25.968794 22781 rpcz_store.cc:276] 0709 13:10:24.559576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.697767 (+1138191us) service_pool.cc:224] Handling call
0709 13:10:25.698047 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.698254 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:25.826870 (+128616us) log.cc:844] Serialized 14 byte log entry
0709 13:10:25.966077 (+139207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:25.966136 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:25.966188 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.966273 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.966417 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220461,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":494,"thread_start_us":263123,"threads_started":2}
I20260709 13:10:26.206059 25402 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.208s
I20260709 13:10:26.206378 25583 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.169s
I20260709 13:10:26.250401 25365 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.396602 25627 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.398120 25440 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.404093 23418 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:26.387483 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:26.314098 25613 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:26.416679 22776 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:26.394994 25607 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:10:26.399958 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56617.9us
W20260709 13:10:26.435448 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48056.9us
I20260709 13:10:26.417450 25443 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:26.428884 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.417366 25546 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:26.433290 25632 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:26.508572 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 82912.5us
I20260709 13:10:26.434674 25440 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:26.556114 25519 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.445264 25619 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:26.422233 25656 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:10:26.568327 25582 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:26.568758 25582 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.569622 25582 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.573789 22781 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:26.575572 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.576447 25521 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:26.576654 25433 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:26.576960 25521 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.579088 25519 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:26.584393 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:26.556095 25423 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.777s	user 0.020s	sys 0.001s
I20260709 13:10:26.442541 25152 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
W20260709 13:10:26.557106 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118363us
I20260709 13:10:26.440383 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:26.436586 25607 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.432017 25391 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:26.577960 25521 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.566435 25619 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:26.597410 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 157062us
I20260709 13:10:26.560709 25613 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:26.598913 25288 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:26.600826 25627 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:26.598924 25615 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:26.596253 25519 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:26.638376 25519 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:26.639413 25519 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.436916 25653 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
I20260709 13:10:26.644582 25653 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:26.645022 25653 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.612676 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:26.592999 23385 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.519s (SENT)
W20260709 13:10:26.591442 25587 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.160s	user 0.001s	sys 0.002s
W20260709 13:10:26.648766 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55804us
W20260709 13:10:26.649451 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.723s (SENT)
I20260709 13:10:26.649852 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:26.429834 25661 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.175s
I20260709 13:10:26.651726 25661 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:26.652005 25365 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:26.652176 25661 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:26.612780 25443 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:10:26.653159 25521 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:26.616024 25524 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.624172 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.655548 25582 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:26.656337 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.657398 25264 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:26.657806 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.658130 25264 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.659109 25264 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.666898 25616 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:26.668138 25616 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:26.668907 25616 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:26.669376 25325 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.631346 25619 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.632920 25613 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.618036 25391 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:26.680809 25391 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.682044 25391 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:26.634928 25639 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:26.614456 25607 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:26.684473 25325 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:26.611053 25195 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:10:26.669425 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.653201 25661 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.689790 25661 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:26.697511 22632 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:26.435670 25597 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260709 13:10:26.609617 25423 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:26.675323 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.673143 25544 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.582108 25440 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.715266 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:26.716300 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.717012 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:26.645974 25653 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.726653 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:26.723551 25195 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:26.728291 25195 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:26.728737 25195 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:26.729089 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79674.5us
I20260709 13:10:26.730872 25528 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:26.722199 25616 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:26.792486 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:26.793274 22706 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:26.723830 25443 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:10:26.796020 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 209915us
W20260709 13:10:26.800485 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73932.8us
I20260709 13:10:26.801481 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:26.802520 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.807379 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.802644 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:26.808974 22708 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:26.809136 23419 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:26.809516 22710 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:26.770787 25288 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.812791 23420 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:10:26.825747 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.814116 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:26.827509 23416 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:26.820185 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:26.837320 23411 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:26.839221 25616 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:26.843154 25616 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:26.843588 25616 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:26.844744 25507 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.845329 25507 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.847694 25507 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:26.854025 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52630.3us
I20260709 13:10:26.797106 25443 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:26.859754 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:26.860504 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.862015 23416 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:26.881961 22675 leader_election.cc:400] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 7.
I20260709 13:10:26.882427 22675 leader_election.cc:403] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:10:26.884011 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:26.889837 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62879.5us
I20260709 13:10:26.893636 22706 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:26.894304 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 305172us
I20260709 13:10:26.895728 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:26.896505 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:26.878822 25329 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.002s
I20260709 13:10:26.897794 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:26.864737 25443 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:26.912850 22633 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:26.836758 25288 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:26.857776 25673 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:10:26.923521 22630 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:10:26.924970 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.902s (SENT)
I20260709 13:10:26.909117 25639 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:26.901823 25507 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:10:26.936262 25507 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:26.937815 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:26.938062 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:26.938544 22630 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:26.929558 25433 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:26.943778 25674 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:10:26.944939 23385 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.141s (SENT)
I20260709 13:10:26.945856 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:26.946350 22633 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:26.943358 25433 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:26.949770 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:26.954280 22706 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:26.900509 25519 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:10:26.956073 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:26.956863 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.936749 25507 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:26.944893 25544 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:26.947327 25639 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:26.954804 25329 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:26.981844 22708 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:26.979074 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77162.1us
I20260709 13:10:26.963992 25627 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:26.971004 25391 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:27.016325 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.242s (SENT)
W20260709 13:10:27.017872 23385 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.248s (SENT)
I20260709 13:10:27.018054 22706 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:27.021984 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:27.024031 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.028451 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 155387us
I20260709 13:10:27.029663 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.966557 25421 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:10:27.004940 25440 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.010078 25673 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:27.011689 25627 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.010411 25329 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:27.006589 25546 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:27.046741 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:26.985515 25434 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:10:27.068951 23415 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:27.069995 25684 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:10:27.071069 25670 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.401s	user 0.000s	sys 0.003s
I20260709 13:10:27.072450 25546 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.049453 25682 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:27.058394 25627 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:27.057682 25685 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.053043 25656 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:10:27.055429 25683 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:10:27.066887 25440 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:27.088069 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58531.4us
I20260709 13:10:27.109460 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:27.118666 22779 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:27.126513 22709 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:27.137923 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.139561 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.140446 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.139619 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.148604 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.154261 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.154477 22633 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:27.155100 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.154415 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.155792 22780 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 2.
I20260709 13:10:27.156517 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.156853 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.082219 25677 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:10:27.160983 22632 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:27.162657 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.162909 25325 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:27.164826 22775 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:27.166005 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.166337 22779 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:27.166790 22634 raft_consensus.cc:2368] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:27.064021 25639 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.167485 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:27.167758 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.168181 22707 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:27.168645 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.168692 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.169948 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.170233 25682 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:27.167590 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.170914 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.172263 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.166910 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.175612 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.176371 22631 raft_consensus.cc:2368] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:27.177124 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.177927 22631 raft_consensus.cc:2410] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:10:27.183771 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.184544 22776 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:27.185788 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:27.186826 25607 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.187644 25433 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.188282 22634 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:27.189711 25607 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.191025 22706 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:27.192121 25329 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.194808 25288 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.170943 22633 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:27.207062 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.130141 25391 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:27.234630 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 105656us
I20260709 13:10:27.242159 22627 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:27.243920 22709 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:27.243876 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.245179 22777 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:27.246249 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.249133 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.252404 22631 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:10:27.254472 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.241926 25661 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.003s
I20260709 13:10:27.236749 25697 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:27.243409 25155 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:27.259707 25691 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.259933 25697 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.260429 25691 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.260514 25697 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.257551 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.265437 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:27.274677 22627 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:27.279479 22631 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:27.281143 25672 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:27.281980 25672 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:27.263751 25691 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.283540 25672 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.289132 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:27.304612 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:27.305190 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.112576 25685 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.312127 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:27.328014 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.330400 25443 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.333398 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:27.333963 25433 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:27.333863 25704 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:27.274531 25524 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:10:27.334359 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:27.343753 22706 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:10:27.327456 25435 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:27.335573 25421 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:27.306375 25632 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:27.259853 25155 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.341699 22701 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:10:27.355345 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.364295 22703 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:27.367154 25627 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:27.350081 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.337212 25325 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:10:27.369733 25627 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.371061 25433 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:27.372367 25195 consensus_meta.cc:287] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
I20260709 13:10:27.372148 25433 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.372852 25195 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.374069 22704 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:27.374797 25329 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:27.375253 25329 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:27.347131 25435 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.393886 22783 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:27.397712 22707 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:27.404524 25653 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:27.410404 25521 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:27.410957 25521 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:27.415762 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 113283us
I20260709 13:10:27.419602 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:27.420301 22627 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:27.411924 25521 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.424013 25521 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:27.424247 25615 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.000s	sys 0.012s
W20260709 13:10:27.424819 25615 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.000s	sys 0.012s
I20260709 13:10:27.425145 25615 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:27.426407 25615 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.439083 22633 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:27.437579 25612 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.448119 25524 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:27.501097 22701 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:27.400267 25435 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:27.399168 25632 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:27.510149 22627 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:27.499969 22706 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:27.475826 25421 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.470139 25615 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.475636 25627 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.464206 25402 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 13:10:27.527557 25616 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.000s
I20260709 13:10:27.501040 25407 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:27.555130 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.531509 25195 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.510825 25612 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.541466 25704 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.552219 25688 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.544663 25714 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:27.521095 25524 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:27.575495 25671 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.931s	user 0.005s	sys 0.000s
I20260709 13:10:27.545674 25615 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.581020 25155 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.583118 25685 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.600934 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.539634 25627 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.535737 25421 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:27.616930 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.554414 25616 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:27.598023 25619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.004s	sys 0.018s
I20260709 13:10:27.563756 25407 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.538892 25329 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.633996 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.639215 23415 raft_consensus.cc:2410] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:10:27.637732 25619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.004s	sys 0.019s
I20260709 13:10:27.593698 25704 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.636777 25697 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.636108 25443 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.599138 25688 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.640079 25720 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.599750 25714 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:27.684661 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.603801 25632 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.692011 25421 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:27.692651 25714 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.693245 25288 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.693406 25524 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.688470 25616 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.693836 25288 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.695029 22629 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:27.696336 25288 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.697902 25612 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.707553 25720 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.672739 25708 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:27.686116 25618 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.725610 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.727077 25615 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.743955 25688 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:27.744570 25710 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:10:27.748333 25721 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
I20260709 13:10:27.748544 25704 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:27.717623 25720 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:27.750757 25708 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.751442 25627 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:27.754426 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:27.758920 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.751611 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.787369 25407 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:27.793138 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.794014 22628 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:27.807176 25716 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:10:27.807927 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:27.807024 25152 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:27.813180 25152 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:27.814159 25152 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:27.815204 25391 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.000s
I20260709 13:10:27.815652 25152 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:27.816717 25507 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.820000 25632 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.820770 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:27.820540 25632 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.821377 22628 raft_consensus.cc:606] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:27.739491 25618 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.826469 25632 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.812562 25435 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:27.834571 22701 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:27.815666 25391 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 7.
I20260709 13:10:27.836854 25708 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:27.846318 25607 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:27.852176 25407 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.837316 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:27.842134 22634 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:10:27.842527 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:27.844648 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:27.869283 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:27.869867 22635 raft_consensus.cc:606] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:27.870251 22635 raft_consensus.cc:614] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:27.856781 22781 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:27.870678 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.871178 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.868453 25688 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:27.832899 25657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.178s	user 0.016s	sys 0.022s
W20260709 13:10:27.879824 25657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.254s	user 0.016s	sys 0.022s
I20260709 13:10:27.868595 25612 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.881088 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.881624 25612 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.884109 25612 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.884256 23419 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:10:27.886778 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.888168 22779 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:27.858752 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:27.889624 22782 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:27.892253 25435 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:27.901614 22780 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:27.841228 25507 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:27.845095 25516 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.877731 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.878316 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:27.879351 22629 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:27.873977 25688 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:27.871306 22777 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:27.907074 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.917645 22775 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:27.920145 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.925128 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.926214 22781 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:27.927320 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:28.051299 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.051864 22775 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:28.053043 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2090) took 1272 ms. Trace:
I20260709 13:10:27.921097 22780 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:28.054313 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.966933 22777 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:28.053486 22702 rpcz_store.cc:276] 0709 13:10:26.780974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.781096 (+   122us) service_pool.cc:224] Handling call
0709 13:10:26.781484 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:27.299053 (+517569us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b180024e500. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ed5bfb 00007f1806ee064b
0709 13:10:27.299275 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:27.299316 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:27.300037 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 13:10:27.505367 (+205330us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:27.530730 (+ 25363us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.893010 (+362280us) spinlock_profiling.cc:243] Waited 979 us on lock 0x7b180024e500. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:10:28.052660 (+159650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.052760 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.052809 (+    49us) raft_consensus.cc:1685] finished
0709 13:10:28.052822 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.052977 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7750,"prepare.queue_time_us":29076,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":505595,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":22895,"spinlock_wait_cycles":12049536,"thread_start_us":488050,"threads_started":3,"wal-append.queue_time_us":247556}
W20260709 13:10:28.054914 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 192939us
W20260709 13:10:28.058583 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 421227us
W20260709 13:10:28.059973 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.886796 25656 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:28.061831 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.618s (SENT) [suppressed 25 similar messages]
I20260709 13:10:28.064019 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.049906 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:27.868749 25434 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:27.914893 25712 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.054968 22780 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:28.074221 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:27.874850 25618 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.076910 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.077181 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.077853 22702 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:10:28.079514 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:27.915705 25707 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:28.080444 22635 raft_consensus.cc:606] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:28.080838 22635 raft_consensus.cc:614] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:28.008075 25727 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:27.894721 25407 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.108388 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:28.079202 25434 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.073855 25685 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.083218 25288 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:10:28.079504 25712 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:28.115330 25732 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:10:27.960994 25516 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.071288 25607 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:28.141485 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.142266 22780 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:28.143518 25288 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:10:28.143929 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.147948 25288 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:28.149046 25288 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.149724 25434 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:28.150653 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.150539 25288 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:28.071416 25656 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.171787 22702 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:28.171576 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58977.7us
I20260709 13:10:28.183894 22778 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:28.087035 25707 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:28.193463 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 105619us
I20260709 13:10:28.166739 25673 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:10:28.182024 25688 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.139231 25685 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.197319 22600 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.163237 25607 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.217741 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:28.219594 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.220429 22775 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:28.123015 25727 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:28.229074 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.225522 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.232853 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:28.234979 22750 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.772s (SENT)
I20260709 13:10:28.235514 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:28.234486 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 155699us
I20260709 13:10:28.239563 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:28.214505 25735 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:10:28.212317 25656 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.261067 22704 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:28.263455 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.505s (SENT)
I20260709 13:10:28.244262 25433 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:28.255661 25632 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:10:28.276952 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.231954 25726 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:28.255501 25627 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:28.246080 25618 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.288076 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca809087a6724923ab06ace347425d31"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:28.291955 22631 raft_consensus.cc:606] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:28.292526 22631 raft_consensus.cc:614] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:28.288964 22782 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:28.255002 25653 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 13:10:28.279857 25433 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.281867 25632 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:28.352205 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 112525us
W20260709 13:10:28.353710 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 114199us
I20260709 13:10:28.292078 25434 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:28.291530 25627 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.355697 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 549) took 1566 ms. Trace:
I20260709 13:10:28.358495 23417 rpcz_store.cc:276] 0709 13:10:26.789435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.789642 (+   207us) service_pool.cc:224] Handling call
0709 13:10:27.810984 (+1021342us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:10:27.811188 (+   204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:27.811597 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:27.811626 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:27.812154 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 13:10:28.054567 (+242413us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:28.054637 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.054729 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.355392 (+300663us) raft_consensus.cc:1685] finished
0709 13:10:28.355417 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.355630 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1255,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"spinlock_wait_cycles":4348800,"thread_start_us":241995,"threads_started":1,"wal-append.queue_time_us":265409}
W20260709 13:10:28.361269 22750 leader_election.cc:336] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.013s (SENT)
I20260709 13:10:28.352849 25433 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:28.356070 25632 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:28.354846 25686 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:28.380694 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.381613 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:28.363237 25627 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:28.362785 25152 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.349077 25618 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.367193 25726 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:28.421373 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60171us
W20260709 13:10:28.426770 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.429508 22780 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:28.404358 25712 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.411615 25533 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:28.430790 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:28.411279 25407 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:28.438936 22780 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
W20260709 13:10:28.440850 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 85378.1us
I20260709 13:10:28.423619 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.404714 25750 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:10:28.407084 25152 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.411552 25686 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:28.446422 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.431707 25685 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.437703 25533 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:28.461855 25745 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:10:28.461340 25533 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:28.464056 25743 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:28.464485 25533 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:28.466835 25741 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:10:28.467710 25516 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.469054 22631 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 13:10:28.469446 25747 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:28.442832 25727 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.436337 25607 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.477138 25618 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.469323 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.501123 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.502009 22631 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:28.492466 25737 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:10:28.481801 25656 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:28.441592 25677 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:10:28.494287 25618 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:10:28.494000 25758 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:28.520031 25618 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:28.520262 25686 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.520610 25618 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:28.507575 25729 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:28.521721 25618 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.533756 25618 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:28.541455 25707 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.543838 25155 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:28.544422 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.528429 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:28.545882 23418 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:28.530547 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.548684 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.549713 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.550551 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.551275 22702 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:28.552129 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.537649 25433 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.561134 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.553356 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:28.550479 23385 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.540s (SENT)
I20260709 13:10:28.559355 23419 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:28.563024 23416 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:10:28.563572 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 3.001s (SENT)
I20260709 13:10:28.564172 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:28.567517 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.567911 25152 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:28.567978 23411 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:10:28.568799 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.569661 22705 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:28.569856 25736 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.412s	user 0.001s	sys 0.002s
I20260709 13:10:28.571967 25760 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.553114 25750 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:28.572551 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.493670 25726 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.576135 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.576776 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:28.517031 25656 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:28.605840 22703 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:28.609763 23418 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:28.602200 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.577687 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.579559 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.579816 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.580581 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.616537 22634 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:28.617161 22626 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:28.589039 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.592706 22632 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:28.592970 25399 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:28.593676 22635 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:28.626782 22628 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:28.627521 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:28.595011 22627 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:10:28.628458 22627 pending_rounds.cc:85] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 5
I20260709 13:10:28.628826 22627 pending_rounds.cc:107] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:10:28.629137 22627 raft_consensus.cc:2889] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:28.597355 25573 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:28.597499 22633 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 13:10:28.598946 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.598909 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.603976 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:28.604127 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:28.538477 25507 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.629s	user 0.014s	sys 0.008s
I20260709 13:10:28.577628 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:28.618768 22704 raft_consensus.cc:2368] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
W20260709 13:10:28.619247 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.619380 22702 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:10:28.631245 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.632140 22701 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:10:28.656812 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.660270 22703 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:28.660496 22777 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:28.662710 22779 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:10:28.663053 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51981.5us
I20260709 13:10:28.589944 25755 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:28.667281 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.668290 22629 raft_consensus.cc:606] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:28.688760 22629 raft_consensus.cc:614] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:28.632787 25573 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:28.613701 25760 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.806607 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.892968 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.892804 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:28.735380 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:28.807124 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 410123us
I20260709 13:10:28.735991 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.900943 22704 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:28.905186 22701 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:28.905449 22707 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:28.899753 22703 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:28.671913 25733 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.540s	user 0.000s	sys 0.003s
I20260709 13:10:28.909018 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.635440 25712 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:28.915262 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.915347 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.916674 22703 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:28.917047 22702 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:10:28.669133 25740 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:28.658176 25399 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.806625 25507 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.846045 25288 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:28.917780 25755 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:28.919539 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.924575 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:28.928380 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 324320us
I20260709 13:10:28.813351 25533 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.108s
I20260709 13:10:28.933002 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.932510 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.669504 25750 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:28.929312 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:28.936803 22701 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:28.938060 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:28.837584 25772 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.117s
I20260709 13:10:28.927439 25755 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.941072 22633 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:10:28.942215 25632 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.004s
I20260709 13:10:28.944169 22703 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:28.936111 25727 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.934880 25754 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:28.958459 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:28.952208 22750 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.212s (SENT)
I20260709 13:10:28.954764 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:28.959756 22703 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:10:28.946508 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.921742 25712 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.990096 22701 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:28.992717 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2114) took 1565 ms. Trace:
I20260709 13:10:28.997421 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:28.996245 22709 rpcz_store.cc:276] 0709 13:10:27.427496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.427849 (+   353us) service_pool.cc:224] Handling call
0709 13:10:27.428237 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:27.632165 (+203928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:27.632219 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:27.667796 (+ 35577us) log.cc:844] Serialized 26 byte log entry
0709 13:10:27.910155 (+242359us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:27.910197 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:27.910269 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.542593 (+632324us) spinlock_profiling.cc:243] Waited 7.06 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:28.960298 (+417705us) raft_consensus.cc:1685] finished
0709 13:10:28.960325 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.992619 (+ 32294us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194510,"raft.queue_time_us":464947,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":16242944,"thread_start_us":360932,"threads_started":2,"wal-append.queue_time_us":341178}
I20260709 13:10:28.998299 22702 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:28.952535 25288 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:28.949695 25760 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:28.928351 25740 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.956453 25751 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:28.953433 25755 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:28.931351 25507 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.012495 25751 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:29.016196 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.957842 25754 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:29.016564 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.016716 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.016963 25754 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:29.017760 22633 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:29.018235 25751 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:29.017040 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51260.6us
I20260709 13:10:29.016724 22776 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:29.019411 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.020853 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:29.025692 25765 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:29.026235 25765 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.026980 25774 raft_consensus.cc:993] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:10:29.026793 25507 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:29.027623 25774 raft_consensus.cc:1081] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:10:29.029244 25765 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:29.031598 25754 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.034585 25750 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.001881 25768 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:29.039265 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:28.959894 25433 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.039885 22702 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:10:29.040885 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.046600 22710 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:10:28.867710 25656 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:29.053192 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 64904.6us
W20260709 13:10:29.038928 25627 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.631s	user 0.013s	sys 0.001s
I20260709 13:10:28.992329 25766 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:28.959266 25632 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.957531 25727 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:28.944494 25573 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.204512 22701 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:10:29.074585 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.507547 22703 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:29.514760 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.242468 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.553748 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:29.530848 22707 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.622s	user 0.014s	sys 0.000s
I20260709 13:10:29.554296 22784 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:29.523205 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.556479 22706 raft_consensus.cc:2368] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:29.555194 22707 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:29.542030 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:29.110121 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.557078 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.557610 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:29.134464 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 464839us
I20260709 13:10:29.557670 22702 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
W20260709 13:10:29.075637 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.554338 22704 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:29.555207 22708 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:29.557086 22706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1940) took 1506 ms (client timeout 1539 ms). Trace:
W20260709 13:10:29.562754 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:29.562397 22706 rpcz_store.cc:269] 0709 13:10:28.050433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.170452 (+1120019us) service_pool.cc:224] Handling call
0709 13:10:29.557016 (+386564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40704}
I20260709 13:10:29.564193 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.553946 25768 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.565315 22701 raft_consensus.cc:2368] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
W20260709 13:10:29.186105 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 165298us
I20260709 13:10:29.566298 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.567162 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:10:29.132751 25618 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.598s	user 0.015s	sys 0.002s
I20260709 13:10:29.570444 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.582633 22702 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:29.553390 25434 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.887s	user 0.000s	sys 0.022s
I20260709 13:10:29.589710 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.575760 22706 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:29.576903 22707 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:29.583886 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:29.197329 25763 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.317s	user 0.000s	sys 0.002s
I20260709 13:10:29.571511 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:29.588022 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.433714 25677 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.499642 25712 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.169777 25740 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.153251 25750 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:29.591871 22709 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 3. Current term is 4. Ops: [2.2-3.3]
I20260709 13:10:29.606815 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.360751 25612 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:29.593087 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
W20260709 13:10:29.590346 25434 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.079s	user 0.000s	sys 0.023s
I20260709 13:10:29.401201 25772 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.593770 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:29.405494 25751 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:29.575268 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.376423 25399 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:29.514335 25754 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.598275 22703 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:29.623598 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.623234 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.624459 22703 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.624646 22706 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:29.569522 25768 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.586982 25779 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:10:29.096813 25767 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:29.168033 25673 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.444s
I20260709 13:10:29.560801 25766 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.569962 25627 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.614099 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.615919 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.704546 22702 raft_consensus.cc:2410] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:10:29.705580 25627 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:29.705803 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:29.625952 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.626138 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.708907 22784 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:29.712769 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.713617 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:29.714269 22750 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.539s (SENT)
I20260709 13:10:29.615497 25521 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:29.504676 25755 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.606662 25750 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.772688 25750 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:29.773973 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 199008us
I20260709 13:10:29.630504 22704 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
W20260709 13:10:29.631593 22778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2166) took 1754 ms (client timeout 1682 ms). Trace:
W20260709 13:10:29.775251 22778 rpcz_store.cc:269] 0709 13:10:27.877370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.877500 (+   130us) service_pool.cc:224] Handling call
0709 13:10:29.631288 (+1753788us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54003250d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:29.631479 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629081344}
I20260709 13:10:29.776712 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:29.777648 22778 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:29.780164 25433 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.781075 25433 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:29.783034 25778 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:10:29.634680 22781 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:29.786132 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2177) took 1590 ms. Trace:
I20260709 13:10:29.786677 22775 rpcz_store.cc:276] 0709 13:10:28.195813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.448069 (+252256us) service_pool.cc:224] Handling call
0709 13:10:28.448963 (+   894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:28.449335 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:28.449379 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:28.450114 (+   735us) log.cc:844] Serialized 26 byte log entry
0709 13:10:28.459862 (+  9748us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:28.459926 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.460040 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:29.785827 (+1325787us) raft_consensus.cc:1685] finished
0709 13:10:29.785851 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:29.786057 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":9247,"threads_started":1,"wal-append.queue_time_us":215162}
I20260709 13:10:29.672945 22707 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:29.610333 25740 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:29.604869 25677 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.701467 25632 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.616175 25772 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.622292 25751 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.615252 25686 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:29.187803 25288 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.714593 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:29.808218 22784 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:29.714991 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:29.811328 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97128.9us
I20260709 13:10:29.811642 25766 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:29.812733 22750 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.618s (SENT)
I20260709 13:10:29.813211 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:29.716717 22702 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:10:29.729622 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 172620us
I20260709 13:10:29.815466 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 529) took 1933 ms. Trace:
I20260709 13:10:29.756793 22703 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:29.815943 23415 rpcz_store.cc:276] 0709 13:10:27.882455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.882835 (+   380us) service_pool.cc:224] Handling call
0709 13:10:27.883217 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:27.975185 (+ 91968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:27.975236 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:27.975855 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:10:28.186156 (+210301us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:28.205273 (+ 19117us) log.cc:844] Serialized 14 byte log entry
0709 13:10:28.205703 (+   430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.570487 (+364784us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b18000b6260. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:29.171703 (+601216us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18000b6260. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:29.633521 (+461818us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18000b6260. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419
0709 13:10:29.815123 (+181602us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b54000da7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:29.815189 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:29.815237 (+    48us) raft_consensus.cc:1685] finished
0709 13:10:29.815250 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:29.815403 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9185,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":1192,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":8675,"spinlock_wait_cycles":295356160,"thread_start_us":171942,"threads_started":1,"wal-append.queue_time_us":227241,"wal-append.run_cpu_time_us":3758,"wal-append.run_wall_time_us":1234620}
I20260709 13:10:29.822167 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.823733 25727 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.826056 25772 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:29.828199 25677 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:29.830933 25673 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:29.769447 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.771901 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:29.614986 25612 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.764475 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.791392 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:29.795022 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.846194 22706 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:29.795846 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.846596 22707 raft_consensus.cc:2376] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6. Re-sending same reply.
I20260709 13:10:29.848479 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:29.850467 22750 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.671s (SENT)
I20260709 13:10:29.823863 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 390) took 1774 ms. Trace:
I20260709 13:10:29.836086 22633 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:29.782840 25619 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.707623 25767 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.839655 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:29.841251 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:29.841572 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.863076 22630 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:29.843999 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:29.632414 22708 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:29.847493 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.853008 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.864686 22706 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:29.865195 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.844974 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.865821 22708 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:29.859150 22781 rpcz_store.cc:276] 0709 13:10:28.048990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.049745 (+   755us) service_pool.cc:224] Handling call
0709 13:10:29.634033 (+1584288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:29.783834 (+149801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:29.823663 (+ 39829us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54003250d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:29.823801 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3642834688}
I20260709 13:10:29.867053 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.867899 22781 raft_consensus.cc:2410] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:10:29.859129 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:29.871476 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:29.869365 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:29.875735 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.879103 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.880496 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.882172 22784 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:29.883948 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:29.884579 22784 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:29.885797 22634 raft_consensus.cc:2368] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:29.887097 22707 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:29.894423 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.896219 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.919143 22708 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:29.923727 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.925114 22707 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:10:29.926098 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.926998 22708 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:29.928324 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:29.922758 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51366.1us
W20260709 13:10:29.934643 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.935344 22778 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:29.849066 25764 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:29.940634 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:29.861199 25779 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:10:29.852958 25521 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.853499 25708 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:29.937261 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2187) took 1358 ms. Trace:
I20260709 13:10:29.950315 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.932698 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2185) took 1354 ms. Trace:
I20260709 13:10:29.991349 22781 rpcz_store.cc:276] 0709 13:10:28.578713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.931614 (+1352901us) service_pool.cc:224] Handling call
0709 13:10:29.931972 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:29.932223 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.932262 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:29.932322 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:29.932368 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:29.932442 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:29.932611 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:29.864253 25766 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:10:29.996795 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2189) took 1414 ms. Trace:
I20260709 13:10:29.963315 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:29.843861 25686 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:29.947615 25434 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:10:29.967337 25607 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:29.969276 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.011639 22708 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:29.970786 22626 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:29.797252 25740 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:29.946419 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2777618819999361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 13:10:30.017297 22784 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
I20260709 13:10:30.023480 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:29.982909 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:29.943326 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:29.850768 25744 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:10:29.924293 25729 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:29.968741 25765 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.938s	user 0.013s	sys 0.000s
I20260709 13:10:30.028087 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.028831 22708 raft_consensus.cc:2376] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:30.029831 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.030737 22708 raft_consensus.cc:2376] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4. Re-sending same reply.
I20260709 13:10:29.831571 25673 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.947355 25754 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:29.997289 22782 rpcz_store.cc:276] 0709 13:10:28.582699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.995879 (+1413180us) service_pool.cc:224] Handling call
0709 13:10:29.996232 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:29.996463 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.996488 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:29.996534 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:29.996564 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:29.996623 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:29.996731 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.035095 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:29.999121 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.015861 25755 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:29.999109 22710 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1945) took 1450 ms (client timeout 1884 ms). Trace:
I20260709 13:10:30.000430 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:30.037446 22632 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:30.037222 22710 rpcz_store.cc:269] 0709 13:10:28.549081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.552927 (+  3846us) service_pool.cc:224] Handling call
0709 13:10:29.996536 (+1443609us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b540010ca54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:29.999008 (+  2472us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":867659136}
I20260709 13:10:29.870045 25619 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:29.990753 22778 rpcz_store.cc:276] 0709 13:10:28.579194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.936215 (+1357021us) service_pool.cc:224] Handling call
0709 13:10:29.936561 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:29.936776 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.936802 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:29.936849 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:29.936882 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:29.937075 (+   193us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:29.937200 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.039726 22778 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:30.040478 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2191) took 1457 ms. Trace:
W20260709 13:10:30.026429 25720 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.729s	user 0.006s	sys 0.029s
W20260709 13:10:30.020407 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2777618819999361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 13:10:30.045609 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2750618819999318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:10:30.046130 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2021618819999276s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
I20260709 13:10:30.046284 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.020828 22701 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:30.018646 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.025550 22707 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:30.019743 25288 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:29.992936 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:30.060025 22633 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:30.056439 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.089462 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.115911 22709 raft_consensus.cc:2376] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:30.117005 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.072431 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.118346 22709 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:30.118687 22705 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:30.109681 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.118997 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1979) took 1081 ms. Trace:
I20260709 13:10:30.119503 22709 rpcz_store.cc:276] 0709 13:10:29.037256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.116707 (+1079451us) service_pool.cc:224] Handling call
0709 13:10:30.118929 (+  2222us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:30.120157 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101628us
I20260709 13:10:30.120909 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.121021 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.121789 22709 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:30.116959 22781 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:30.101410 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.125212 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.025550 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 82331.3us
I20260709 13:10:30.006799 25607 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:30.036100 22782 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:30.127138 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:30.128767 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2190) took 1545 ms. Trace:
I20260709 13:10:30.032292 25704 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:30.040926 22778 rpcz_store.cc:276] 0709 13:10:28.583150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.038918 (+1455768us) service_pool.cc:224] Handling call
0709 13:10:30.039322 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.040159 (+   837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.040409 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:30.002241 23385 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.995s (SENT)
I20260709 13:10:30.044060 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.129380 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:30.129352 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.122035 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:30.130468 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 128228us
W20260709 13:10:30.046531 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1895618820000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:10:30.096779 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.131244 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:30.131492 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1625618819999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1645s for a response
I20260709 13:10:30.131745 22704 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:30.131891 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1301618820000385s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16810000000000003s for a response
W20260709 13:10:30.132254 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1148618820000138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
I20260709 13:10:30.132511 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.133308 22709 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:30.134297 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.135253 22709 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:30.136759 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.137753 22782 raft_consensus.cc:2368] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:30.138654 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.138839 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.139516 23416 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:30.139822 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.140940 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.119386 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1971) took 1084 ms. Trace:
I20260709 13:10:29.992594 25521 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.126339 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:30.132577 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1130618820000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:10:30.085815 25434 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.144083 22635 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:30.142825 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1130618820000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 13:10:30.144243 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:30.144877 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1058618819999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1708s for a response
I20260709 13:10:30.145141 22707 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:30.145310 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1040618819999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:10:30.145817 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1040618819999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
W20260709 13:10:30.146132 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1022618819999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 13:10:30.146427 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0851618819999658s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17310000000000003s for a response
I20260709 13:10:30.147010 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.142318 22705 rpcz_store.cc:276] 0709 13:10:29.034843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.039963 (+  5120us) service_pool.cc:224] Handling call
0709 13:10:30.072050 (+1032087us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:10:30.119320 (+ 47270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16179072}
I20260709 13:10:29.980872 22706 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:10:30.146793 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0752618819999498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
I20260709 13:10:30.148290 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1973) took 1112 ms. Trace:
W20260709 13:10:30.148527 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0590618820000373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 13:10:30.148711 22706 rpcz_store.cc:276] 0709 13:10:29.035514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.852674 (+817160us) service_pool.cc:224] Handling call
0709 13:10:30.148227 (+295553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.149271 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:30.149739 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:30.045523 25720 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.756s	user 0.007s	sys 0.029s
W20260709 13:10:30.074460 25791 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:30.088627 25750 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:30.075331 25796 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:10:30.151608 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:30.108640 25770 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.921s	user 0.000s	sys 0.012s
I20260709 13:10:30.175364 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.060943 25673 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:30.176299 22710 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:30.177444 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.178088 22710 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:30.178774 25399 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:30.179388 25399 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:30.179944 25399 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:30.151170 25546 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:30.142459 22704 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:30.032074 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:30.064214 25769 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.326s	user 0.010s	sys 0.000s
I20260709 13:10:30.154157 22600 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.154901 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.155036 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.158052 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.189286 22707 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:30.078917 25751 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:30.164377 22632 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:30.092322 25573 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.168308 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:30.219676 22776 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2188) took 1637 ms (client timeout 1518 ms). Trace:
I20260709 13:10:30.219790 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.220312 22709 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:30.106573 25329 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:30.170255 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.171216 22781 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:30.242748 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:30.171821 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.243532 22777 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.243417 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.245086 22781 raft_consensus.cc:2410] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:30.034343 25765 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:30.170840 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45827.7us
I20260709 13:10:30.173427 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.271832 22705 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:30.174178 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.174376 25754 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:30.175423 25785 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:30.183393 22708 raft_consensus.cc:2376] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:30.286175 22708 raft_consensus.cc:1217] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 6.6->[8.7-8.7]   Dedup: 8.7->[]
I20260709 13:10:30.286808 22708 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:10:30.152838 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.184223 22704 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:30.289252 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1982) took 1251 ms. Trace:
I20260709 13:10:30.289733 22704 rpcz_store.cc:276] 0709 13:10:29.038136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.183518 (+1145382us) service_pool.cc:224] Handling call
0709 13:10:30.183883 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.288856 (+104973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.289167 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.184702 22630 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:30.079864 25612 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:30.077102 25767 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:30.301602 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:30.186399 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.641s (SENT)
W20260709 13:10:30.327683 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 141345us
I20260709 13:10:30.044746 25433 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:30.189297 22706 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:30.063562 25740 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.084008 25686 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:30.158237 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:30.342825 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 184687us
W20260709 13:10:30.337015 23385 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.178s (SENT)
I20260709 13:10:30.343817 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:30.344022 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.269425 25772 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:30.196547 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.199404 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2098) took 1637 ms. Trace:
I20260709 13:10:30.153371 25797 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:30.200109 25677 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:30.151099 25708 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:30.027194 25768 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:30.210810 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56749.6us
I20260709 13:10:30.213758 22707 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:30.303130 25786 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:10:30.393124 22707 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:30.386348 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.140143 25656 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:10:30.220435 22776 rpcz_store.cc:269] 0709 13:10:28.582452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.695936 (+113484us) service_pool.cc:224] Handling call
0709 13:10:30.219414 (+1523478us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540033c054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:30.219575 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266729728}
W20260709 13:10:30.222572 22779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 389) took 2174 ms (client timeout 1643 ms). Trace:
W20260709 13:10:30.394999 22779 rpcz_store.cc:269] 0709 13:10:28.048574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.049520 (+   946us) service_pool.cc:224] Handling call
0709 13:10:30.222238 (+2172718us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400186fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:30.222465 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1365645696}
I20260709 13:10:30.397607 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 392) took 1465 ms. Trace:
I20260709 13:10:30.225229 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2186) took 1646 ms. Trace:
I20260709 13:10:30.398809 22775 rpcz_store.cc:276] 0709 13:10:28.578960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.945369 (+1366409us) service_pool.cc:224] Handling call
0709 13:10:29.945762 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:29.964172 (+ 18410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.964207 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:29.964256 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:29.964297 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.224851 (+260554us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b54000cba54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:30.224930 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.225140 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41746304}
I20260709 13:10:30.243825 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1986) took 1095 ms. Trace:
I20260709 13:10:30.401130 22701 rpcz_store.cc:276] 0709 13:10:29.148576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.613812 (+465236us) service_pool.cc:224] Handling call
0709 13:10:30.243594 (+629782us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400316354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:30.243756 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727979264}
I20260709 13:10:30.129209 22784 rpcz_store.cc:276] 0709 13:10:28.582897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.024892 (+1441995us) service_pool.cc:224] Handling call
0709 13:10:30.128201 (+103309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.128411 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.128439 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.128487 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:30.128519 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.128587 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.128704 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.243288 22702 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11.
I20260709 13:10:30.246134 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.247319 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.178369 25521 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.264148 22710 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:30.410161 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 1987) took 1261 ms. Trace:
I20260709 13:10:30.143905 25792 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:30.171262 25704 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.219439 25402 raft_consensus.cc:2764] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:10:30.167169 25407 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.386051 25612 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:10:30.279116 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107381us
I20260709 13:10:30.449232 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:30.452113 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:30.190866 25760 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:30.272647 23411 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:30.301602 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.302065 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:30.458535 22703 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:30.288132 22778 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:30.135196 25288 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.183041 25546 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:30.316075 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.325549 25435 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:30.050503 25619 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.336535 22777 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:10:30.159852 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.345878 22704 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:30.314355 22708 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:10:30.476963 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:30.156937 25434 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:30.353183 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.214s (SENT)
W20260709 13:10:30.478699 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 125561us
I20260709 13:10:30.349207 22631 rpcz_store.cc:276] 0709 13:10:28.562261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.562433 (+   172us) service_pool.cc:224] Handling call
0709 13:10:28.562897 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:28.592010 (+ 29113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:28.592044 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:28.592617 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:10:28.670811 (+ 78194us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:28.671008 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.671372 (+   364us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:30.107997 (+1436625us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:30.183882 (+ 75885us) raft_consensus.cc:1685] finished
0709 13:10:30.183903 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.199310 (+ 15407us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11956,"raft.queue_time_us":31112,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":93,"spinlock_wait_cycles":25254144,"thread_start_us":77526,"threads_started":1,"wal-append.queue_time_us":346298}
I20260709 13:10:30.479944 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:30.403703 23417 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:30.398118 22779 rpcz_store.cc:276] 0709 13:10:28.932055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.396380 (+1464325us) service_pool.cc:224] Handling call
0709 13:10:30.396860 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.397139 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.397169 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.397231 (+    62us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:30.397284 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.397362 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.397522 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.410636 22710 rpcz_store.cc:276] 0709 13:10:29.148862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.187166 (+1038304us) service_pool.cc:224] Handling call
0709 13:10:30.263658 (+ 76492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.409872 (+146214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.410095 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:30.412485 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 140836us
I20260709 13:10:30.482844 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.424559 22701 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:10:30.483942 22777 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:30.486277 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.486989 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:30.487318 22634 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:30.425259 22705 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:10:30.488335 22777 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:30.430164 22707 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:10:30.354246 25673 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:30.354604 25797 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.424794 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.500996 23414 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:30.437907 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 395) took 1495 ms. Trace:
I20260709 13:10:30.367337 25688 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:30.442229 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.369927 25772 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:30.274557 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 1991) took 1124 ms. Trace:
I20260709 13:10:30.507890 22709 rpcz_store.cc:276] 0709 13:10:29.150217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.166190 (+1015973us) service_pool.cc:224] Handling call
0709 13:10:30.274472 (+108282us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.391062 25708 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.388062 25751 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:30.460426 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:30.521046 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60712.3us
I20260709 13:10:30.522017 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:30.464202 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.457497 25760 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:30.405079 25656 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:30.407145 25786 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.407491 25729 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.428431 25792 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.492057 22708 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:30.492904 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.482892 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:30.539899 22779 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:30.499197 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.501717 22776 rpcz_store.cc:276] 0709 13:10:28.942787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.436920 (+1494133us) service_pool.cc:224] Handling call
0709 13:10:30.437297 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.437518 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.437548 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.437601 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:30.437640 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.437701 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.437829 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.505043 22704 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:30.492877 25740 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.505092 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.506116 22630 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
W20260709 13:10:30.479961 23385 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.736s (SENT)
I20260709 13:10:30.510946 22705 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:30.464749 25686 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:30.444918 25704 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.278404 25607 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:30.597185 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 117241us
I20260709 13:10:30.504513 25797 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:30.473006 25785 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:30.473551 25612 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:30.536383 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:30.537057 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:30.522933 23421 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:30.533883 25656 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.502100 25685 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:30.576275 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2199) took 1002 ms. Trace:
I20260709 13:10:30.538534 25786 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:30.630683 22779 rpcz_store.cc:276] 0709 13:10:29.574176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.540684 (+966508us) service_pool.cc:224] Handling call
0709 13:10:30.575679 (+ 34995us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.575885 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.575913 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.575962 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:30.575996 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.576053 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.576202 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.521348 25777 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.633364 22782 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:30.556813 25727 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:30.529783 25707 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:30.457762 25757 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.070s
W20260709 13:10:30.582175 25794 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:10:30.581203 22532 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.591670 25805 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.592371 25546 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:30.526235 22775 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:30.528421 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:30.599779 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:30.693483 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 165140us
I20260709 13:10:30.600672 22784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
W20260709 13:10:30.607983 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56494.2us
I20260709 13:10:30.500252 25755 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:30.694355 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:30.541569 25751 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:30.695643 25800 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:10:30.606058 22776 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:30.625546 22531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 234) took 14434 ms (client timeout 15000 ms). Trace:
I20260709 13:10:30.530404 25708 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:30.635293 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 100420us
I20260709 13:10:30.556013 25792 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.692992 25612 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.693696 22707 raft_consensus.cc:2410] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:10:30.624114 25785 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.462077 25407 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.701885 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:30.700691 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 166152us
I20260709 13:10:30.685730 25757 consensus_queue.cc:1243] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:10:30.634274 25685 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:30.700037 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2193) took 1133 ms. Trace:
I20260709 13:10:30.596388 25740 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.635180 25727 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:30.552711 25729 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:30.716636 22775 rpcz_store.cc:276] 0709 13:10:29.566576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.525751 (+959175us) service_pool.cc:224] Handling call
0709 13:10:30.526044 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.692574 (+166530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.699967 (+  7393us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:30.716298 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:30.697329 25765 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.708820 25546 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:10:30.721817 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.734910 25798 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:10:30.718154 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:30.808974 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69538us
I20260709 13:10:30.736066 25619 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:30.707176 25793 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:30.705853 25673 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:30.704592 25754 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.733629 22532 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.747656 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2106) took 1040 ms. Trace:
I20260709 13:10:30.730247 25727 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:30.754258 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:30.832588 22630 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:30.755621 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:30.719658 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 238215us
I20260709 13:10:30.716277 25785 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.654513 25777 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:30.810948 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 92942.1us
I20260709 13:10:30.724016 25685 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:30.828696 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:30.727629 25773 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:30.829193 22628 rpcz_store.cc:276] 0709 13:10:29.707337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.715335 (+  7998us) service_pool.cc:224] Handling call
0709 13:10:29.715826 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:29.716159 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.716199 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:29.768895 (+ 52696us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.116791 (+347896us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:30.116831 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.116902 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:30.747370 (+630468us) raft_consensus.cc:1685] finished
0709 13:10:30.747394 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.747572 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":306731,"threads_started":1,"wal-append.queue_time_us":520103}
W20260709 13:10:30.697062 22531 rpcz_store.cc:269] 0709 13:10:16.168620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.168827 (+   207us) service_pool.cc:224] Handling call
0709 13:10:30.603369 (+14434542us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:26.851168 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:26.851916 (+   748us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:26.851970 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:10:26.851991 (+    21us) tablet.cc:662] Decoding operations
0709 13:10:26.933794 (+ 81803us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:26.933888 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:26.933922 (+    34us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:10:26.935873 (+  1951us) tablet.cc:701] Row locks acquired
0709 13:10:26.935931 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:10:26.936137 (+   206us) write_op.cc:270] Start()
0709 13:10:26.936319 (+   182us) write_op.cc:276] Timestamp: P: 1783602626936062 usec, L: 0
0709 13:10:26.936348 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:10:27.028642 (+ 92294us) log.cc:844] Serialized 9020 byte log entry
0709 13:10:27.535212 (+506570us) op_driver.cc:464] REPLICATION: finished
0709 13:10:27.714078 (+178866us) write_op.cc:301] APPLY: starting
0709 13:10:27.714187 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:27.714698 (+   511us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:27.752041 (+ 37343us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:27.817806 (+ 65765us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:27.818240 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 13:10:27.818286 (+    46us) write_op.cc:312] APPLY: finished
0709 13:10:27.873647 (+ 55361us) log.cc:844] Serialized 158 byte log entry
0709 13:10:27.873883 (+   236us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:27.874110 (+   227us) write_op.cc:454] Released schema lock
0709 13:10:27.874204 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:27.874729 (+   525us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:27.874765 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1434845,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":168452,"apply.run_cpu_time_us":7056,"apply.run_wall_time_us":160605,"num_ops":23,"prepare.queue_time_us":228356,"prepare.run_cpu_time_us":55843,"prepare.run_wall_time_us":291530,"raft.queue_time_us":248231,"raft.run_cpu_time_us":12753,"raft.run_wall_time_us":246985,"replication_time_us":542861,"spinlock_wait_cycles":1792,"thread_start_us":359918,"threads_started":4,"wal-append.queue_time_us":93002,"wal-append.run_cpu_time_us":27924,"wal-append.run_wall_time_us":1747613}]]}
I20260709 13:10:30.833550 22631 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:30.656433 25772 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:30.835484 22532 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:30.849756 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.862213 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.808974 25546 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:30.812876 25619 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.701511 25760 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:30.828629 25152 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:30.867147 22630 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:30.895550 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:30.903652 25546 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:30.910301 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:30.900332 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 363336us
I20260709 13:10:30.861104 22532 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 6, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.911962 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:30.909341 22701 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:30.912283 22532 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 8, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.827306 25746 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.914222 22532 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:30.916097 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 315997us
I20260709 13:10:30.828325 25793 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.831610 25768 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:30.809437 25618 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:10:30.931250 22532 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.933221 22532 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 5, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:30.707784 25805 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:30.935804 25288 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.848423 25673 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:30.860570 25773 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.959479 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:30.902689 25619 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:30.960743 25773 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:30.977340 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 117219us
I20260709 13:10:30.980549 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.981894 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:30.982931 22626 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:30.986361 22784 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:30.986956 22784 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:30.988016 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2196) took 1419 ms. Trace:
I20260709 13:10:31.000787 22778 rpcz_store.cc:276] 0709 13:10:29.568377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.532715 (+964338us) service_pool.cc:224] Handling call
0709 13:10:30.533062 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.533300 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:30.903835 (+370535us) log.cc:844] Serialized 14 byte log entry
0709 13:10:30.984632 (+ 80797us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.985164 (+   532us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:30.985670 (+   506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.986447 (+   777us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:30.987516 (+  1069us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24274,"thread_start_us":356684,"threads_started":2}
I20260709 13:10:30.999392 22784 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:30.934738 25793 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:30.928712 25751 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:30.835119 25754 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:30.914928 25746 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:31.015086 25810 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 13:10:31.029009 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:31.034837 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 122969us
I20260709 13:10:31.048264 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:31.058282 22675 leader_election.cc:400] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:31.058713 22675 leader_election.cc:403] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:31.046316 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:31.075683 22778 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:31.079341 22701 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:31.010433 25434 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:31.135802 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 396) took 1572 ms. Trace:
I20260709 13:10:31.026922 25828 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:31.110711 25704 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:31.157006 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 238194us
I20260709 13:10:31.863492 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2112) took 1581 ms. Trace:
W20260709 13:10:31.865999 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.000s	sys 0.015s
W20260709 13:10:31.866645 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.000s	sys 0.015s
W20260709 13:10:31.867535 22702 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.015s	sys 0.018s
W20260709 13:10:31.871719 22702 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.016s	sys 0.019s
W20260709 13:10:31.840188 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:31.872979 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:31.873462 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2133) took 1757 ms. Trace:
W20260709 13:10:31.867419 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 809253us
W20260709 13:10:31.558197 25685 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.004s
W20260709 13:10:31.765507 25708 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.048s	user 0.015s	sys 0.003s
I20260709 13:10:31.317701 25740 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:31.886430 25708 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:31.417127 25746 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:31.888904 25727 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.029s	user 0.016s	sys 0.011s
I20260709 13:10:31.324134 25729 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:31.571717 25754 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:31.149183 25766 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.749s
I20260709 13:10:31.899864 25754 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:31.900482 25754 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:31.885986 25685 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:31.844635 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 403) took 1348 ms. Trace:
W20260709 13:10:31.843190 22635 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.008s	sys 0.019s
I20260709 13:10:31.889921 25727 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:31.864532 22633 rpcz_store.cc:276] 0709 13:10:30.281959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.283686 (+  1727us) service_pool.cc:224] Handling call
0709 13:10:30.340772 (+ 57086us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.511522 (+170750us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:30.511913 (+   391us) log.cc:844] Serialized 14 byte log entry
0709 13:10:30.716583 (+204670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.716653 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:30.716721 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.862455 (+1145734us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b540006c2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:31.862586 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.863130 (+   544us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18501,"prepare.queue_time_us":826,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":5902,"raft.queue_time_us":4835,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":24910,"spinlock_wait_cycles":391937792,"thread_start_us":200658,"threads_started":1,"wal-append.queue_time_us":319712}
W20260709 13:10:31.420351 25399 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.240s	user 0.004s	sys 0.013s
I20260709 13:10:31.932315 22776 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:31.932683 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.004s	sys 0.005s
I20260709 13:10:31.208604 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:31.933238 25399 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:31.614022 25546 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.704s	user 0.008s	sys 0.007s
I20260709 13:10:31.300530 25407 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:31.912482 25708 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:31.933259 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.004s	sys 0.005s
I20260709 13:10:31.941754 25546 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:31.138258 22777 rpcz_store.cc:276] 0709 13:10:29.563741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.509303 (+945562us) service_pool.cc:224] Handling call
0709 13:10:30.509621 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.509882 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:30.647169 (+137287us) log.cc:844] Serialized 14 byte log entry
0709 13:10:31.132905 (+485736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:31.132991 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:31.133058 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.133153 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.133306 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56685,"prepare.queue_time_us":367535,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":55,"raft.queue_time_us":200984,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":67920,"spinlock_wait_cycles":125184,"thread_start_us":593209,"threads_started":3}
W20260709 13:10:31.945238 22635 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.011s	sys 0.020s
I20260709 13:10:31.873991 22702 rpcz_store.cc:276] 0709 13:10:30.115648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.409818 (+294170us) service_pool.cc:224] Handling call
0709 13:10:30.453685 (+ 43867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.454041 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:10:30.763292 (+309251us) log.cc:844] Serialized 14 byte log entry
0709 13:10:31.108932 (+345640us) log.cc:844] Serialized 14 byte log entry
0709 13:10:31.872497 (+763565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:31.872650 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:31.872839 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.872999 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.873300 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55398,"prepare.queue_time_us":583208,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":5620,"raft.queue_time_us":328338,"spinlock_wait_cycles":45215104,"thread_start_us":1333342,"threads_started":4,"wal-append.queue_time_us":760841}
I20260709 13:10:31.215097 25673 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:31.970777 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.090s	user 0.000s	sys 0.003s
W20260709 13:10:31.971444 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.124s	user 0.000s	sys 0.003s
I20260709 13:10:31.973093 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2154) took 1437 ms. Trace:
I20260709 13:10:31.289433 25837 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:31.973811 22702 rpcz_store.cc:276] 0709 13:10:30.536008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.969850 (+1433842us) service_pool.cc:224] Handling call
0709 13:10:31.971071 (+  1221us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:31.972254 (+  1183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:31.972310 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:31.972575 (+   265us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:31.972665 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.972760 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.972966 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:31.292026 25807 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.430s	user 0.001s	sys 0.000s
I20260709 13:10:31.107715 25627 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.399442 25777 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:31.981256 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.000s	sys 0.027s
W20260709 13:10:31.988790 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.000s	sys 0.027s
W20260709 13:10:31.561934 25797 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.929s	user 0.013s	sys 0.004s
W20260709 13:10:31.215499 25809 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.501s	user 0.000s	sys 0.002s
W20260709 13:10:31.325174 25816 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 13:10:31.280458 25818 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.321s	user 0.001s	sys 0.001s
W20260709 13:10:31.559414 25785 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.701s	user 0.009s	sys 0.003s
I20260709 13:10:31.987185 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2155) took 1450 ms. Trace:
I20260709 13:10:31.987044 25627 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:31.294242 25823 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
W20260709 13:10:31.615746 25573 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.353s	user 0.010s	sys 0.007s
W20260709 13:10:31.756726 25786 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.059s	user 0.010s	sys 0.006s
W20260709 13:10:31.581771 25773 consensus_meta.cc:287] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.004s
W20260709 13:10:31.563524 25793 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
W20260709 13:10:31.563401 25792 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.848s	user 0.010s	sys 0.004s
I20260709 13:10:31.287161 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 394) took 2347 ms. Trace:
W20260709 13:10:31.580845 25619 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.002s
I20260709 13:10:31.998914 22702 rpcz_store.cc:276] 0709 13:10:30.536296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.981295 (+1444999us) service_pool.cc:224] Handling call
0709 13:10:31.986083 (+  4788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:31.986691 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:31.986720 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:31.986794 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:31.986846 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.986917 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.987085 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:31.417258 25329 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.154s	user 0.011s	sys 0.003s
W20260709 13:10:31.992350 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 1.02818e+06us
I20260709 13:10:31.233902 22778 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:31.984032 25805 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:31.970849 25152 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.990183 25797 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:31.237412 25829 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:31.321995 25839 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:31.962527 25754 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:31.919914 22634 rpcz_store.cc:276] 0709 13:10:30.496125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.518644 (+ 22519us) service_pool.cc:224] Handling call
0709 13:10:30.519049 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:30.592612 (+ 73563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.592654 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:30.593385 (+   731us) log.cc:844] Serialized 52 byte log entry
0709 13:10:30.739647 (+146262us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:30.830978 (+ 91331us) log.cc:844] Serialized 14 byte log entry
0709 13:10:30.831213 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.831300 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:31.843648 (+1012348us) raft_consensus.cc:1685] finished
0709 13:10:31.843745 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.844295 (+   550us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145485,"prepare.queue_time_us":95911,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":75582,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":22146,"spinlock_wait_cycles":31691136,"thread_start_us":145809,"threads_started":1,"wal-append.queue_time_us":731882}
W20260709 13:10:32.038600 22598 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Timed out: RequestConsensusVote RPC to 127.21.5.132:36877 timed out after 1.518s (SENT)
I20260709 13:10:32.039784 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:32.034893 25848 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:32.038789 25656 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.368s	user 0.013s	sys 0.008s
I20260709 13:10:32.041556 25853 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.045356 25546 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.026475 25152 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.086835 25152 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.087457 25627 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.028560 22781 rpcz_store.cc:276] 0709 13:10:28.939813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.408027 (+1468214us) service_pool.cc:224] Handling call
0709 13:10:30.408433 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.408817 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.408850 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.409423 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.504260 (+ 94837us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:30.504321 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.504414 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:31.286726 (+782312us) raft_consensus.cc:1685] finished
0709 13:10:31.286767 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.287020 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2999,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":9971,"thread_start_us":39805,"threads_started":1,"wal-append.queue_time_us":379221}
I20260709 13:10:32.090844 25792 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.092774 22781 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:10:32.093605 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2209) took 1648 ms. Trace:
I20260709 13:10:32.052421 22631 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:32.053138 23416 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:10:32.053788 25786 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.041456 25825 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:32.053131 22701 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:32.056353 25785 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.057955 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2218) took 1016 ms. Trace:
I20260709 13:10:32.094069 22781 rpcz_store.cc:276] 0709 13:10:30.445378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.091600 (+1646222us) service_pool.cc:224] Handling call
0709 13:10:32.092172 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.093256 (+  1084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.093508 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.062776 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.064352 25656 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.065799 25848 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.067689 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2156) took 1252 ms. Trace:
I20260709 13:10:32.111434 22702 rpcz_store.cc:276] 0709 13:10:30.815270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.020903 (+1205633us) service_pool.cc:224] Handling call
0709 13:10:32.036297 (+ 15394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.067048 (+ 30751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.067084 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.067174 (+    90us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:32.067269 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.067358 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.067575 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.112200 22784 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.041297 25854 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:32.070959 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:32.071110 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet f33fee9d64384b499e55f8bc58339f69 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50366
I20260709 13:10:32.103852 22778 rpcz_store.cc:276] 0709 13:10:31.020434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.045985 (+ 25551us) service_pool.cc:224] Handling call
0709 13:10:32.033941 (+987956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.071430 25793 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.072747 25754 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.073956 25772 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:10:32.074872 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 57413.5us
W20260709 13:10:32.076771 25857 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:10:32.080451 25773 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.046082 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2208) took 1601 ms. Trace:
I20260709 13:10:32.125730 25825 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.127771 22777 rpcz_store.cc:276] 0709 13:10:30.444139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.036829 (+1592690us) service_pool.cc:224] Handling call
0709 13:10:32.044832 (+  8003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.045389 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.045451 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.045537 (+    86us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:32.045574 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.045642 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.045782 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:32.128850 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66376.4us
W20260709 13:10:32.123927 22706 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.063s	user 0.014s	sys 0.023s
W20260709 13:10:32.129637 22706 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.193s	user 0.015s	sys 0.024s
I20260709 13:10:32.130018 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.131062 25686 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:32.132388 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2157) took 1316 ms. Trace:
I20260709 13:10:32.133311 25767 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:32.133980 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 371) took 2362 ms. Trace:
I20260709 13:10:32.047446 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 404) took 1550 ms. Trace:
I20260709 13:10:32.095705 23416 pending_rounds.cc:85] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 7
I20260709 13:10:32.090220 25727 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:32.094560 25826 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.868s	user 0.014s	sys 0.009s
I20260709 13:10:32.094188 25792 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.078226 25685 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:32.120025 22782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 401) took 1958 ms (client timeout 1643 ms). Trace:
I20260709 13:10:32.119907 25854 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.113061 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50372 (request call id 2197) took 2544 ms. Trace:
I20260709 13:10:32.134073 25853 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:32.134828 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 64025us
I20260709 13:10:32.135345 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.081873 25399 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:32.038231 22703 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.007s	sys 0.029s
W20260709 13:10:32.166960 22703 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.008s	sys 0.030s
W20260709 13:10:32.167341 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 96807.8us
I20260709 13:10:32.168526 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2134) took 2052 ms. Trace:
I20260709 13:10:32.168841 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 403) took 1815 ms. Trace:
I20260709 13:10:32.132870 22702 rpcz_store.cc:276] 0709 13:10:30.815644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.112780 (+1297136us) service_pool.cc:224] Handling call
0709 13:10:32.113352 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.131868 (+ 18516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.131899 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.131979 (+    80us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:32.132031 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.132112 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.132295 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.172760 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:32.173221 25766 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:32.173872 25766 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:32.174010 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2159) took 1357 ms. Trace:
I20260709 13:10:32.137421 23416 pending_rounds.cc:107] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:10:32.175014 23416 raft_consensus.cc:2889] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:32.175232 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.176216 25688 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.176949 25685 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:32.177134 25688 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.177745 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:32.146044 22782 rpcz_store.cc:269] 0709 13:10:30.161840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.196237 (+ 34397us) service_pool.cc:224] Handling call
0709 13:10:32.113225 (+1916988us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400186fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:32.119706 (+  6481us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653171968}
I20260709 13:10:32.149709 25727 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:32.179112 25727 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:32.179550 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.180259 25786 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.180585 22782 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:32.180563 25727 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.182435 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.182878 25656 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.149714 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:32.183344 22782 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:32.082438 25619 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.185765 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2212) took 1678 ms. Trace:
I20260709 13:10:32.185873 25854 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.122593 25785 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.186690 25619 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.150202 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.187561 25848 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.169174 22703 rpcz_store.cc:276] 0709 13:10:30.115911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.459573 (+343662us) service_pool.cc:224] Handling call
0709 13:10:30.494205 (+ 34632us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:30.497227 (+  3022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.497282 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:30.508244 (+ 10962us) log.cc:844] Serialized 52 byte log entry
0709 13:10:30.812335 (+304091us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:31.045285 (+232950us) log.cc:844] Serialized 14 byte log entry
0709 13:10:32.167773 (+1122488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.167948 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.168022 (+    74us) raft_consensus.cc:1685] finished
0709 13:10:32.168041 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.168434 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2344,"prepare.queue_time_us":320051,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":63,"raft.queue_time_us":241225,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":14269,"thread_start_us":1546379,"threads_started":3,"wal-append.queue_time_us":432831}
I20260709 13:10:32.189713 22702 rpcz_store.cc:276] 0709 13:10:30.816277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.172520 (+1356243us) service_pool.cc:224] Handling call
0709 13:10:32.173123 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.173462 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.173499 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.173557 (+    58us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:32.173644 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.173728 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.173890 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.114965 25797 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.186306 22782 rpcz_store.cc:276] 0709 13:10:30.507433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.184285 (+1676852us) service_pool.cc:224] Handling call
0709 13:10:32.184820 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.185360 (+   540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.185394 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.185463 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:32.185502 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.185567 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.185700 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:32.192853 25842 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.880s	user 0.007s	sys 0.017s
I20260709 13:10:32.193763 22782 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:10:32.195457 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.195856 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 681) took 1443 ms. Trace:
I20260709 13:10:32.196233 23421 rpcz_store.cc:276] 0709 13:10:30.752713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.753709 (+   996us) service_pool.cc:224] Handling call
0709 13:10:30.827446 (+ 73737us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.827889 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.827921 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.832137 (+  4216us) log.cc:844] Serialized 26 byte log entry
0709 13:10:31.035429 (+203292us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:31.035508 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.035693 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.191639 (+1155946us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:32.195144 (+  3505us) raft_consensus.cc:1685] finished
0709 13:10:32.195163 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.195601 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6819,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16791,"spinlock_wait_cycles":25019264,"thread_start_us":107047,"threads_started":1,"wal-append.queue_time_us":446404}
I20260709 13:10:32.169286 22779 rpcz_store.cc:276] 0709 13:10:30.353492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.632050 (+278558us) service_pool.cc:224] Handling call
0709 13:10:30.632423 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.696242 (+ 63819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.696280 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:30.696328 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:30.696357 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.168166 (+1471809us) spinlock_profiling.cc:243] Waited 63.6 ms on lock 0x7b540033d954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.168253 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.168571 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146211712}
I20260709 13:10:32.197731 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.198699 23411 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:32.198997 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 405) took 1502 ms. Trace:
I20260709 13:10:32.199548 22782 rpcz_store.cc:276] 0709 13:10:30.696451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.192713 (+1496262us) service_pool.cc:224] Handling call
0709 13:10:32.193340 (+   627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:32.194381 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.198888 (+  4507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.200287 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.200873 25607 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:32.201488 23417 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:32.201946 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2214) took 1191 ms. Trace:
I20260709 13:10:32.151232 25773 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.138424 22706 rpcz_store.cc:276] 0709 13:10:29.771908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.356345 (+584437us) service_pool.cc:224] Handling call
0709 13:10:30.356825 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.357302 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.357346 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.411239 (+ 53893us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.936578 (+525339us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:32.130493 (+1193915us) log.cc:844] Serialized 14 byte log entry
0709 13:10:32.133444 (+  2951us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.133580 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.133630 (+    50us) raft_consensus.cc:1685] finished
0709 13:10:32.133646 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.133892 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":255892,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":566,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":24856,"thread_start_us":1631414,"threads_started":2,"wal-append.queue_time_us":864532}
W20260709 13:10:32.153527 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57964.7us
I20260709 13:10:32.151674 22709 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:32.155720 25329 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.163949 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:32.134294 25767 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.176081 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.147960 22784 rpcz_store.cc:276] 0709 13:10:29.569036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.534148 (+965112us) service_pool.cc:224] Handling call
0709 13:10:32.086356 (+1552208us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b5400341cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed4a3d 00007f180bea4cd0 00007f1805429254 00007f18054291e2 00007f1805429111 00007f180542908c 00007f1805428342 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:32.112882 (+ 26526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":671559040}
I20260709 13:10:32.183040 22632 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:32.152745 25793 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.155213 25627 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:32.135710 25825 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.200886 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 406) took 1215 ms. Trace:
I20260709 13:10:32.174474 25766 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.223477 22779 rpcz_store.cc:276] 0709 13:10:30.985961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.198510 (+1212549us) service_pool.cc:224] Handling call
0709 13:10:32.199003 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.200383 (+  1380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.200424 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.200510 (+    86us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:32.200559 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.200665 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.200806 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.202503 22782 rpcz_store.cc:276] 0709 13:10:31.011000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.200570 (+1189570us) service_pool.cc:224] Handling call
0709 13:10:32.201029 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.201465 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.201505 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.201578 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:32.201638 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.201707 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.201860 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.137329 22635 rpcz_store.cc:276] 0709 13:10:30.496520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.498715 (+  2195us) service_pool.cc:224] Handling call
0709 13:10:30.499134 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:30.519507 (+ 20373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.519552 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:30.531032 (+ 11480us) log.cc:844] Serialized 52 byte log entry
0709 13:10:30.837023 (+305991us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:31.986839 (+1149816us) log.cc:844] Serialized 14 byte log entry
0709 13:10:32.015691 (+ 28852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.025434 (+  9743us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.029504 (+  4070us) raft_consensus.cc:1685] finished
0709 13:10:32.029522 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.047362 (+ 17840us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":130599,"prepare.queue_time_us":117272,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":49818,"raft.run_cpu_time_us":747,"raft.run_wall_time_us":96049,"thread_start_us":1262947,"threads_started":2,"wal-append.queue_time_us":799900}
I20260709 13:10:32.256625 22783 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:32.226847 25797 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.266543 22629 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:32.267237 22629 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:32.268857 22629 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.270336 22629 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.272396 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:32.267283 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2215) took 1256 ms. Trace:
I20260709 13:10:32.276252 22779 rpcz_store.cc:276] 0709 13:10:31.011307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.248982 (+1237675us) service_pool.cc:224] Handling call
0709 13:10:32.258795 (+  9813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.266869 (+  8074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.266907 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.266984 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:10:32.267017 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.267100 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.267222 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:32.276731 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 89210.8us
I20260709 13:10:32.292220 25758 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.293040 25758 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.293581 25853 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.257313 25848 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:10:32.274094 25825 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:32.299968 25825 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.300760 25837 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:32.264050 25329 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.302737 25837 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.309401 25750 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.280222 25627 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.314873 22778 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:32.315848 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:32.318841 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2146) took 2170 ms. Trace:
I20260709 13:10:32.319028 25848 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:32.319509 25685 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:32.306169 25329 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.357051 25744 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:32.357564 25825 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:32.358695 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.359143 25797 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.375756 25866 tablet_replica.cc:333] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:10:32.358103 25744 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:32.400859 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 44300.3us
I20260709 13:10:32.416014 25793 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:32.319492 25848 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:32.433027 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.481143 22784 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:32.319383 22707 rpcz_store.cc:276] 0709 13:10:30.148166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.708385 (+560219us) service_pool.cc:224] Handling call
0709 13:10:30.725333 (+ 16948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.725571 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:31.035540 (+309969us) log.cc:844] Serialized 14 byte log entry
0709 13:10:32.259989 (+1224449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.277132 (+ 17143us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:32.318382 (+ 41250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.318529 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.318773 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17193,"prepare.queue_time_us":382900,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":5300,"raft.queue_time_us":61095,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":197,"thread_start_us":1378410,"threads_started":3,"wal-append.queue_time_us":1228956}
I20260709 13:10:32.468469 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2211) took 1987 ms. Trace:
I20260709 13:10:32.485378 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2216) took 1464 ms. Trace:
W20260709 13:10:32.485991 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.486364 22784 rpcz_store.cc:276] 0709 13:10:31.019721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.273896 (+1254175us) service_pool.cc:224] Handling call
0709 13:10:32.479604 (+205708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.481700 (+  2096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.483694 (+  1994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:32.475180 25329 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:32.494282 25329 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:32.494892 25329 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.495014 25744 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:32.488675 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:32.496110 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.496572 25767 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.489161 22779 pending_rounds.cc:85] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 1
I20260709 13:10:32.489751 25627 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.490672 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:32.491119 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.492156 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 673) took 2172 ms. Trace:
I20260709 13:10:32.492995 25848 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.499866 22779 pending_rounds.cc:107] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 2.2
W20260709 13:10:32.504936 22775 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2217) took 1484 ms (client timeout 1856 ms). Trace:
W20260709 13:10:32.503489 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 231269us
I20260709 13:10:32.511266 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:32.511590 25825 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:32.513288 25750 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:32.513834 25750 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:32.508615 25804 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:10:32.506090 22775 rpcz_store.cc:269] 0709 13:10:31.020126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.942745 (+922619us) service_pool.cc:224] Handling call
0709 13:10:32.490788 (+548043us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5800043598. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:10:32.504859 (+ 14071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37606144}
W20260709 13:10:32.506822 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 74045.7us
I20260709 13:10:32.505963 22779 raft_consensus.cc:2889] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:32.482259 22777 rpcz_store.cc:276] 0709 13:10:30.480804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.129228 (+1648424us) service_pool.cc:224] Handling call
0709 13:10:32.129774 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.132037 (+  2263us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:10:32.265944 (+133907us) log.cc:844] Serialized 14 byte log entry
0709 13:10:32.467944 (+202000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.468022 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:32.468092 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.468172 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.468354 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":182677,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":54,"raft.queue_time_us":413,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":315,"thread_start_us":260251,"threads_started":2}
I20260709 13:10:32.515792 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:32.515268 25804 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:32.516410 22747 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:32.508283 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:32.518245 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 242582us
I20260709 13:10:32.520511 22782 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:32.503000 23420 rpcz_store.cc:276] 0709 13:10:30.319791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.320015 (+   224us) service_pool.cc:224] Handling call
0709 13:10:30.458207 (+138192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.458524 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.458553 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.459157 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.933232 (+474075us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:30.933304 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.933421 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.060209 (+1126788us) raft_consensus.cc:1685] finished
0709 13:10:32.060266 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.491533 (+431267us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54006fe850. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:32.491866 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316664192,"thread_start_us":428177,"threads_started":1,"wal-append.queue_time_us":668045}
I20260709 13:10:32.520656 25704 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:32.524979 25856 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:32.525880 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.525880 25856 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.526500 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:32.527554 25727 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:32.529011 25856 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.529289 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.530292 25727 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:32.531505 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.531960 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.532256 22782 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:32.536679 25804 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.539803 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:32.534318 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.540416 22629 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:32.545503 25848 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:32.546092 25759 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:32.546530 25759 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.546939 25759 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.547318 25627 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:32.547679 22784 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.551246 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.541287 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:32.552075 22777 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:32.499575 25766 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.553424 22778 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:32.515281 25750 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.560566 22782 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.558013 25727 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:32.555713 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.574541 25793 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:10:32.597703 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86794.3us
I20260709 13:10:32.599213 25843 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.599833 25843 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.602156 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.603032 22629 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:32.605542 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.607239 25843 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.609021 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 405) took 2112 ms. Trace:
I20260709 13:10:32.609045 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.609620 22632 rpcz_store.cc:276] 0709 13:10:30.496926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.499510 (+  2584us) service_pool.cc:224] Handling call
0709 13:10:30.499819 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.233649 (+1733830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.608736 (+375087us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400236ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.608940 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3791606528}
I20260709 13:10:32.610208 22629 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:32.611301 25766 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:32.611744 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2202) took 2170 ms. Trace:
I20260709 13:10:32.612280 22776 rpcz_store.cc:276] 0709 13:10:30.441485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.697783 (+256298us) service_pool.cc:224] Handling call
0709 13:10:30.698117 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:31.933139 (+1235022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.611336 (+678197us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54002d5d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.611662 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2799520000}
I20260709 13:10:32.613116 25708 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:32.614375 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2152) took 2338 ms. Trace:
I20260709 13:10:32.613351 23419 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 680) took 1861 ms. Trace:
I20260709 13:10:32.614024 25708 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.614820 22701 rpcz_store.cc:276] 0709 13:10:30.276506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.085008 (+808502us) service_pool.cc:224] Handling call
0709 13:10:31.145874 (+ 60866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.137728 (+991854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.614102 (+476374us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b540010e354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.614311 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852816640}
W20260709 13:10:32.616343 23385 connection.cc:256] RPC call timeout handler was delayed by -1.2001922579999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
W20260709 13:10:32.617390 23385 connection.cc:256] RPC call timeout handler was delayed by -1.1110922579999851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:10:32.617926 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.619053 22627 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:32.619405 25708 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.620052 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:32.620986 22627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2111) took 2375 ms (client timeout 1748 ms). Trace:
I20260709 13:10:32.614953 23419 rpcz_store.cc:276] 0709 13:10:30.752231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.756581 (+  4350us) service_pool.cc:224] Handling call
0709 13:10:30.757084 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.809752 (+ 52668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.809786 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.834933 (+ 25147us) log.cc:844] Serialized 26 byte log entry
0709 13:10:31.033594 (+198661us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:31.033677 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.612707 (+1579030us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b54004e9654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.612776 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.612844 (+    68us) raft_consensus.cc:1685] finished
0709 13:10:32.612855 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.613164 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":786,"raft.queue_time_us":3220,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":11699,"spinlock_wait_cycles":120207616,"thread_start_us":101771,"threads_started":1,"wal-append.queue_time_us":432329,"wal-append.run_cpu_time_us":2669,"wal-append.run_wall_time_us":1124541}
W20260709 13:10:32.621405 22627 rpcz_store.cc:269] 0709 13:10:30.245816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.245992 (+   176us) service_pool.cc:224] Handling call
0709 13:10:32.620625 (+2374633us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b5400127d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:32.620859 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4969053952}
I20260709 13:10:32.624421 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2210) took 2176 ms. Trace:
I20260709 13:10:32.626173 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.626775 25685 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:32.630499 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.632867 25533 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:32.633541 25750 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:32.634424 25840 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.321s	user 0.010s	sys 0.014s
W20260709 13:10:32.636349 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 95208.3us
I20260709 13:10:32.638563 22781 rpcz_store.cc:276] 0709 13:10:30.447979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.145105 (+1697126us) service_pool.cc:224] Handling call
0709 13:10:32.258467 (+113362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.312876 (+ 54409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.312918 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:32.312973 (+    55us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:32.313006 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.624126 (+311120us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b54003282d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.624186 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.624347 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124400512}
I20260709 13:10:32.614147 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:32.638085 22750 connection.cc:256] RPC call timeout handler was delayed by -1.3606860339999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1734s for a response
W20260709 13:10:32.639998 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 123685us
W20260709 13:10:32.642231 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59445.1us
I20260709 13:10:32.644697 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.645120 22747 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:32.646255 22782 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:32.646858 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:32.648674 22750 connection.cc:256] RPC call timeout handler was delayed by -1.0521985230000155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
I20260709 13:10:32.593191 25793 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.652184 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.641824 25685 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:32.653415 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.637545 25759 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.649857 25856 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:32.662704 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.651610 25793 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:32.666960 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 408) took 1709 ms. Trace:
W20260709 13:10:32.670230 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2155) took 4825 ms (client timeout 1639 ms). Trace:
I20260709 13:10:32.667692 22631 rpcz_store.cc:276] 0709 13:10:30.958120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.958390 (+   270us) service_pool.cc:224] Handling call
0709 13:10:30.958865 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.094953 (+1136088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.655981 (+561028us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400128254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.666899 (+ 10918us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308209024}
W20260709 13:10:32.671541 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 139774us
W20260709 13:10:32.672266 22783 rpcz_store.cc:269] 0709 13:10:27.841451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.842216 (+   765us) service_pool.cc:224] Handling call
0709 13:10:32.663227 (+4821011us) spinlock_profiling.cc:243] Waited 4.22 s on lock 0x7b5400324bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:32.666854 (+  3627us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9699108096}
I20260709 13:10:32.674002 23412 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:32.666913 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 535) took 2526 ms. Trace:
I20260709 13:10:32.659642 25812 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.677011 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.678050 22701 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:32.676450 23416 rpcz_store.cc:276] 0709 13:10:30.140679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.141184 (+   505us) service_pool.cc:224] Handling call
0709 13:10:30.141468 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.175851 (+2034383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.660658 (+484807us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54005f7654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.666841 (+  6183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4394048896}
I20260709 13:10:32.679757 23421 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:32.685149 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:32.685333 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:32.686192 22705 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:32.668167 25856 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.665637 25627 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:32.691256 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:32.700906 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:32.704689 23412 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:32.709373 25764 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:32.712417 22780 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:10:32.720383 25866 raft_consensus.cc:2243] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:32.722123 25866 pending_rounds.cc:70] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Trying to abort 1 pending ops.
I20260709 13:10:32.707319 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:32.724738 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2219) took 1704 ms. Trace:
I20260709 13:10:32.728785 22779 rpcz_store.cc:276] 0709 13:10:31.020749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.279769 (+1259020us) service_pool.cc:224] Handling call
0709 13:10:32.488712 (+208943us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:32.542479 (+ 53767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.542519 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:32.543283 (+   764us) log.cc:844] Serialized 52 byte log entry
0709 13:10:32.639343 (+ 96060us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:10:32.639377 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.639473 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:32.724459 (+ 84986us) raft_consensus.cc:1685] finished
0709 13:10:32.724485 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:32.724668 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2279,"raft.queue_time_us":26122,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":170,"thread_start_us":95635,"threads_started":1,"wal-append.queue_time_us":152832}
I20260709 13:10:32.706138 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.698558 25856 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.737181 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:32.731336 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:32.739001 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:32.698249 25533 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.743821 22778 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:32.742988 22701 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:32.737819 22705 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:32.734273 22777 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:32.745934 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57716.8us
I20260709 13:10:32.749486 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:32.754081 22631 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:32.722548 25866 pending_rounds.cc:77] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting op as it isn't in flight: id { term: 3 index: 4 } timestamp: 7305636190420013056 op_type: NO_OP noop_request { }
I20260709 13:10:32.747331 25399 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.755894 25851 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:32.756129 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:32.756795 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:32.763327 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2144) took 2615 ms. Trace:
I20260709 13:10:32.764279 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:32.731411 25627 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:32.770938 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:32.776932 22703 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:32.752861 25871 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.252s	user 0.002s	sys 0.001s
I20260709 13:10:32.768369 22709 rpcz_store.cc:276] 0709 13:10:30.147549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.508744 (+361195us) service_pool.cc:224] Handling call
0709 13:10:30.509091 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.226116 (+1717025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.762969 (+536853us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b54002c42d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:32.763231 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3764935552}
I20260709 13:10:32.783893 22706 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:32.778980 22707 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:32.793457 22704 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.753110 25766 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:32.782332 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:32.791917 25781 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.006s
W20260709 13:10:32.810137 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 115317us
I20260709 13:10:32.725603 25812 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.774363 25866 raft_consensus.cc:2889] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:10:32.824569 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:32.826958 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:32.827860 22600 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:32.839890 22776 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:32.849413 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.797670 25399 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.778645 25533 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.859467 22701 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:32.851013 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:32.860610 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.509s (SENT)
I20260709 13:10:32.864287 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:32.866209 22702 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:32.842901 25866 raft_consensus.cc:2272] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:32.828186 25766 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:32.882421 22707 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:32.883599 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48779.2us
I20260709 13:10:32.900422 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:32.859372 25803 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:32.907418 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.910523 22709 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:10:32.852439 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55351.8us
W20260709 13:10:32.916600 22750 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.786s (SENT)
I20260709 13:10:32.917258 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:32.922170 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.923255 22632 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.923807 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.924861 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.924602 22633 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:32.923843 22675 leader_election.cc:400] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:32.931953 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.933221 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:32.935411 25819 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:10:32.935431 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:32.943827 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.944010 25766 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.882522 25851 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:10:32.946604 25399 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.927644 25759 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:32.910020 25803 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.948453 25843 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:32.949699 22779 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:10:32.954941 25781 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:10:32.949860 25288 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:10:32.951434 25803 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:32.962741 22629 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:10:32.962893 25627 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:32.968000 25533 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:32.968436 22778 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:32.926110 25329 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.974208 25707 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:32.958802 25797 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:32.952983 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 112112us
I20260709 13:10:32.951436 22709 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:10:32.949414 25759 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:10:32.977507 22782 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:32.953912 25843 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:32.925974 25744 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:33.010231 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:33.014562 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.014793 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:33.018848 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.020587 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.024813 22632 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:33.029376 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:33.030495 22784 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:33.033629 22704 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:33.035255 22703 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:32.992911 25329 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:33.036481 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:33.030886 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:33.039880 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.974434 25793 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:33.042048 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 180875us
I20260709 13:10:33.051458 22631 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:33.052928 25866 ts_tablet_manager.cc:1916] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:10:33.053946 25882 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:10:33.055791 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 147761us
W20260709 13:10:33.056805 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.801s (SENT)
I20260709 13:10:33.057284 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.057495 25803 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.059055 25815 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.059834 25815 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.060954 25843 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:33.062150 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85249.5us
I20260709 13:10:33.062829 25533 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:33.063376 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.063726 22784 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:33.064136 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.016821 25704 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:33.076200 22702 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:33.042858 25797 raft_consensus.cc:2764] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:10:33.081344 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.093210 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:33.070895 25836 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.749s	user 0.016s	sys 0.023s
W20260709 13:10:33.106851 25836 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.785s	user 0.016s	sys 0.023s
I20260709 13:10:33.070149 25863 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:33.110865 25607 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:33.111855 25803 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:33.113555 25793 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:33.114955 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 99737.8us
I20260709 13:10:33.042255 25754 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:10:33.127795 25754 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:33.128261 25754 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:33.129278 25754 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.131846 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.133934 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.134891 25759 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:33.135407 25759 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.136404 25759 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.140260 25288 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:33.140816 25288 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.141767 25288 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.071110 25815 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:33.144655 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.145905 25618 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:33.146445 25618 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.147512 25618 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.070282 25329 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:33.089764 22750 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.825s (SENT)
I20260709 13:10:33.100008 25765 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:33.176469 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.187800 22631 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:33.189100 22781 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:33.192068 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.086680 25744 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.093s
I20260709 13:10:33.173801 25765 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.131431 25803 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 13:10:33.234908 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 155613us
W20260709 13:10:33.328473 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 247718us
I20260709 13:10:33.270191 22777 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:33.263789 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:33.330363 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 301310us
I20260709 13:10:33.187674 25863 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:33.195101 25754 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:33.346575 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 270419us
I20260709 13:10:33.360284 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.264899 25793 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:33.192972 25627 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:10:33.361101 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.665s (SENT) [suppressed 50 similar messages]
I20260709 13:10:33.331030 25765 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:33.382853 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 203744us
W20260709 13:10:33.390129 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.877s (SENT)
I20260709 13:10:33.395109 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.395404 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:33.391144 25793 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.410578 22701 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:33.423457 25885 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:33.424978 25688 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:33.425789 25760 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:33.431275 25888 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 13:10:33.431540 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.981s (SENT)
I20260709 13:10:33.432034 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.433178 25758 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:33.434336 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.438999 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81941.2us
W20260709 13:10:33.439903 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.884s (SENT)
I20260709 13:10:33.440393 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:33.441888 22750 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.983s (SENT)
I20260709 13:10:33.443351 25793 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.445483 25767 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:33.446926 25863 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:33.451004 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67606.4us
I20260709 13:10:33.451720 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2184) took 4873 ms. Trace:
I20260709 13:10:33.452140 22675 leader_election.cc:400] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:33.452299 22780 rpcz_store.cc:276] 0709 13:10:28.578444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.581668 (+  3224us) service_pool.cc:224] Handling call
0709 13:10:32.701827 (+4120159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.713082 (+ 11255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.451324 (+738242us) spinlock_profiling.cc:243] Waited 4.09 s on lock 0x7b5400324bd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:33.451606 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9395887872}
W20260709 13:10:33.453718 25685 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.784s	user 0.005s	sys 0.005s
I20260709 13:10:33.438665 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.442335 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.453773 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:33.468531 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.454229 25685 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.469206 25758 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:33.474686 25863 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:33.465474 25890 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:33.450459 25768 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:10:33.498566 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 306589us
W20260709 13:10:33.510646 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.488162 25766 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:33.468137 25688 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:10:33.519133 25688 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:33.519788 22708 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 2. Current term is 3. Ops: []
I20260709 13:10:33.522706 25851 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:33.523316 25851 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:33.524896 25851 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.527508 25851 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:33.498070 25768 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:33.532692 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 79049.5us
W20260709 13:10:33.534123 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102624us
W20260709 13:10:33.536767 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.091s (SENT)
I20260709 13:10:33.542868 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.491266 25759 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:10:33.549299 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:33.550026 22705 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:33.541926 25766 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:33.551576 25856 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.774s	user 0.013s	sys 0.000s
I20260709 13:10:33.551689 25766 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.552115 25856 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.552688 25812 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:33.554126 25853 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:33.554592 25853 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:33.554857 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:33.557215 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.558115 25727 raft_consensus.cc:2764] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:10:33.558955 25727 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:33.550084 25895 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
W20260709 13:10:33.559790 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44232us
W20260709 13:10:33.560710 22750 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.110s (SENT)
I20260709 13:10:33.560864 22705 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:33.561161 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:33.565537 22750 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.110s (SENT)
I20260709 13:10:33.565634 25765 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.566092 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.563380 25768 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.566565 25815 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:33.566896 25764 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:33.567497 25758 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:33.567790 25764 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:33.571206 25768 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:33.568675 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.125s (SENT)
I20260709 13:10:33.574892 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.554363 25856 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:33.591823 22750 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.139s (SENT)
I20260709 13:10:33.592880 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:33.594983 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.106s (SENT)
I20260709 13:10:33.599059 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:33.598557 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:33.601370 22708 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:10:33.600792 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 205492us
I20260709 13:10:33.538872 25533 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:10:33.603801 22702 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:33.603811 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.602550 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:33.594635 25764 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:33.602432 25759 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:33.617813 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:33.602124 25758 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:33.597880 25863 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:10:33.621241 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:33.623984 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:33.625560 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:33.629873 22710 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:33.626435 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.610140 25688 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:33.609395 25685 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:33.640260 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48492us
I20260709 13:10:33.627921 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.644730 25688 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:33.645764 22750 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.148s (SENT)
I20260709 13:10:33.646303 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.628293 25765 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:33.617877 25759 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.651618 25759 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.651968 25853 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.655119 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:33.655942 22782 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:33.656286 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:33.656942 22778 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:33.657007 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:33.645946 25688 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.658368 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:33.631126 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.661734 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.615425 25837 raft_consensus.cc:2764] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:10:33.677265 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:33.683269 22779 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:10:33.678570 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 51291us
I20260709 13:10:33.677409 22782 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:33.687637 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:33.678045 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.152s (SENT)
I20260709 13:10:33.689954 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.684739 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:33.691365 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.692749 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:33.692149 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:33.694720 22779 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:33.697129 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:33.681464 25902 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:33.701845 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:33.711210 22632 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:33.713260 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:33.715813 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.714447 22775 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:33.727629 25896 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:10:33.707329 25902 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.694561 25863 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.738564 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.738922 25863 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.739277 25902 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.741075 25902 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:33.741472 25863 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:33.743464 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2225) took 1592 ms. Trace:
I20260709 13:10:33.743916 22777 rpcz_store.cc:276] 0709 13:10:32.150553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.752393 (+601840us) service_pool.cc:224] Handling call
0709 13:10:33.269941 (+517548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.329236 (+ 59295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.743190 (+413954us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400159fd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:33.743361 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1125076224}
I20260709 13:10:33.749094 23414 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:33.749683 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 121870us
I20260709 13:10:33.694342 25758 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:33.694324 25812 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:10:33.751626 25720 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:33.709522 25607 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:33.730006 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.710146 25627 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:33.757239 25899 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 13:10:33.757612 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:33.757228 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.739790 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:33.763240 22708 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:33.702739 25707 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:33.743307 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:33.787936 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:33.713289 25750 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:33.777901 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 132171us
I20260709 13:10:33.791399 22783 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:33.792132 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.799371 25618 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:33.801636 25908 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:10:33.802860 25908 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.806118 25533 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:10:33.810365 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58058.8us
I20260709 13:10:33.767732 25848 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:10:33.817261 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 155705us
I20260709 13:10:33.814505 22779 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:33.822691 22601 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:33.779911 25607 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.826876 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:33.820636 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:33.828279 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:33.789902 25903 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:33.779615 25627 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:33.843624 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:33.851002 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:33.851922 22750 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.181s (SENT)
I20260709 13:10:33.852494 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:33.858952 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 96947.2us
W20260709 13:10:33.860167 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.825480 25760 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:33.863368 22710 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:33.864107 23420 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:10:33.866367 23415 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:10:33.872184 22778 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:33.866873 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:33.867115 25768 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:33.846266 22782 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:33.869761 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:33.818403 25890 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.867507 25435 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.896152 22707 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:33.888729 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:33.789665 25843 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:10:33.878289 25837 raft_consensus.cc:2764] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:10:33.843863 25607 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:33.903689 22709 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:33.851253 25903 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.863628 25627 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:33.868142 25815 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:33.869103 25793 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:33.830767 25895 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:10:33.917943 22779 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:33.916615 25329 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:33.919876 25768 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.920364 25433 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:10:33.921375 25908 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:33.926860 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.939419 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:33.829419 25707 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.912964 25765 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:33.936708 25793 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.947461 25804 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:33.927682 25903 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:34.008591 25804 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:33.936587 25755 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:33.964092 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 145828us
I20260709 13:10:33.924609 25890 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:33.971179 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 150765us
I20260709 13:10:33.908274 25435 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:33.980173 25768 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:33.980803 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:33.987571 25793 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:33.999328 25901 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:10:33.948005 25848 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.010257 23417 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:34.029707 23416 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:33.945220 25815 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:34.031560 22779 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:10:34.036623 23412 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:34.028213 25908 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:10:34.061497 22703 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:34.055029 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:34.042918 22750 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.190s (SENT)
I20260709 13:10:34.023804 25765 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:34.078649 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.080963 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:34.062009 25917 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 13:10:34.009754 25804 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.108368 23420 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:34.112872 22775 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:34.083992 25837 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:34.117290 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 71994.3us
W20260709 13:10:34.115808 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.121560 25707 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.121280 25765 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.087044 25848 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.124259 25848 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.124518 25804 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:34.107867 25329 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:34.127141 25914 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:10:34.128086 25914 consensus_queue.cc:1243] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Peer 6a31ac6451ff4f73bee634487d84d001 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:34.110244 25766 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:34.136668 25768 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.139515 25890 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:34.140084 25759 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:34.140748 25916 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:10:34.142455 25815 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.159407 22708 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:34.159802 22675 leader_election.cc:400] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
W20260709 13:10:34.161586 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118729us
I20260709 13:10:34.165262 25866 ts_tablet_manager.cc:1929] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 13:10:34.166013 25866 log.cc:1199] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/f33fee9d64384b499e55f8bc58339f69
I20260709 13:10:34.167702 25866 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50366 (request call id 85) took 2097 ms. Trace:
I20260709 13:10:34.168111 25866 rpcz_store.cc:276] 0709 13:10:32.070108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.070457 (+   349us) service_pool.cc:224] Handling call
0709 13:10:32.374999 (+304542us) ts_tablet_manager.cc:1151] Deleting tablet f33fee9d64384b499e55f8bc58339f69
0709 13:10:32.375134 (+   135us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:10:33.600033 (+1224899us) tablet_metadata.cc:665] Metadata flushed
0709 13:10:34.162189 (+562156us) tablet_metadata.cc:665] Metadata flushed
0709 13:10:34.167457 (+  5268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":357928,"tablet-delete.queue_time_us":254110,"thread_start_us":131376,"threads_started":1}
I20260709 13:10:34.178012 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:34.182327 25837 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:34.221084 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:34.242604 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 115512us
I20260709 13:10:34.120822 25760 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:34.251519 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 381812us
I20260709 13:10:34.252463 23420 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:34.253618 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:34.256340 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:34.257165 23420 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:34.258670 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:34.260465 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.211s (SENT)
I20260709 13:10:34.261008 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:34.223105 25914 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.248241 25720 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:34.266260 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:34.273663 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:34.274979 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54083.8us
I20260709 13:10:34.265935 25848 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:34.278688 25627 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.279701 25765 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:34.291287 25765 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:34.291853 25765 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:34.295380 25760 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.296463 25853 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:34.275640 25764 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:10:34.305421 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:34.312638 25755 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:34.315589 25627 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:34.316304 25902 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.001s
W20260709 13:10:34.324625 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 75242.5us
I20260709 13:10:34.327670 23412 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:34.332841 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:34.274019 25914 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:34.339854 22750 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.220s (SENT)
W20260709 13:10:34.341703 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.275282 25903 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.344964 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:34.346582 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:34.340444 25940 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:34.355652 25938 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:34.347539 25848 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:34.375013 25607 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.387341 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:34.367446 25729 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:34.394838 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55057.2us
I20260709 13:10:34.394069 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:34.364708 25938 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:34.404919 22750 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.227s (SENT)
I20260709 13:10:34.405668 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:34.391789 25793 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.390571 25903 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.394274 25914 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:34.408550 25764 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:34.408982 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.234s (SENT)
I20260709 13:10:34.419744 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:34.409371 25938 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.410643 25435 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:34.411172 25793 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:34.422960 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.423388 25903 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:34.423213 25902 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:34.423151 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:34.425366 23412 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:34.417878 22750 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.244s (SENT)
W20260709 13:10:34.412926 25925 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:10:34.412807 25765 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.431612 22781 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:10:34.431948 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.407835 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:34.427176 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:34.430238 22783 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:34.409396 25607 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:34.434142 25765 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.436533 25720 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:34.438730 25914 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:34.461645 25914 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.462208 25914 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:34.462500 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:34.462749 22629 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:34.463505 22633 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:34.440514 25288 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:10:34.457391 22631 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:34.458362 22627 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:34.473079 25720 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:34.474268 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.476122 25750 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 13:10:34.480561 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.474648 25720 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.494560 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:34.494720 25825 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:34.496924 22627 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:34.487296 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:34.491288 22709 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:34.483875 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:34.511979 23421 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:34.514552 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:34.516786 25533 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:34.518249 25618 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:34.518769 25618 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:34.516005 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:34.520704 22627 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:34.535811 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.509503 25915 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.512979 25766 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:34.510974 25851 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:34.536887 22707 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:10:34.547132 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.538569 25759 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:34.547694 25915 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.554755 25812 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:34.567903 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.491493 25920 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:10:34.489038 25755 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:10:34.562446 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:34.600186 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:34.601262 22781 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:34.529834 25618 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:34.605216 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:34.605955 22779 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:10:34.575431 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 122606us
I20260709 13:10:34.579154 22709 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:10:34.606954 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:34.607527 22779 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:34.607905 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:34.613216 25759 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:34.614607 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:34.614413 25759 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.615713 25760 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:34.616240 25760 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:34.620378 25932 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.306s	user 0.004s	sys 0.000s
I20260709 13:10:34.620566 25914 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.624336 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:34.586432 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:34.633695 22777 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:34.637662 22780 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:34.637415 25760 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.642299 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.642272 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.643360 22710 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:34.647953 22777 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:34.514063 25399 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:10:34.560930 25766 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:34.660681 22707 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:34.664407 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:34.668988 25952 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:10:34.673045 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.673554 22780 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:10:34.680737 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:34.690976 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.691648 22707 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:34.696259 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:34.664175 25533 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:10:34.704042 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.704810 22704 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:34.670631 25928 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:10:34.714941 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.715734 22705 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5. Re-sending same reply.
I20260709 13:10:34.700976 25435 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:34.723328 22706 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:10:34.704617 25399 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:34.738281 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:34.735529 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:34.741685 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.720114 25914 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.742417 22706 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:34.743369 25768 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:34.725142 25435 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:34.750735 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.731056 25399 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.757226 22704 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:34.740809 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:34.761286 25399 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:34.768380 22705 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:34.724694 25433 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:10:34.776232 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.777181 22706 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:34.755281 25435 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:34.781854 22750 leader_election.cc:336] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.540s (SENT)
W20260709 13:10:34.772441 25953 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:10:34.788655 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:34.777420 25902 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:34.804256 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.805176 22706 raft_consensus.cc:2368] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:34.726747 25766 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.784165 25768 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:34.833748 25949 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:10:34.835234 25902 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:34.835498 25954 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:10:34.836308 25902 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.808751 25764 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:34.868736 23417 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:34.877501 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.879999 25768 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.888657 25843 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:10:34.895260 25948 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:10:34.827500 25915 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:34.931020 22779 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:34.957935 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.962291 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.951499 25914 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:34.965433 25914 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:34.968479 25914 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:34.972589 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 194453us
I20260709 13:10:34.984169 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.985339 22701 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:34.984423 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.987212 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.987733 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.988497 22709 raft_consensus.cc:2410] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:34.988734 22703 raft_consensus.cc:2368] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 5.
I20260709 13:10:34.989652 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.989843 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.957237 25760 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.986539 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:34.991969 22701 raft_consensus.cc:2368] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:34.991365 25760 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:34.993160 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.994167 22701 raft_consensus.cc:2368] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 4. Current term is 6.
I20260709 13:10:34.952067 25766 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:34.995673 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:34.998229 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:34.999462 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.751s (SENT)
W20260709 13:10:35.001351 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.006884 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.019761 25915 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:35.021219 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46633.6us
W20260709 13:10:35.022470 22750 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.873s (SENT)
I20260709 13:10:35.023626 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.031877 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.032933 22709 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:35.063807 25848 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.071965 25848 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.065415 25958 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:10:35.076133 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.066988 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.064196 25768 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.083616 22707 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:10:35.093027 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.098768 23419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.005s	sys 0.009s
W20260709 13:10:35.099255 23419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.005s	sys 0.009s
W20260709 13:10:35.104200 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81752.1us
I20260709 13:10:35.065770 25915 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:35.080066 25848 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:35.108937 22750 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.138s (SENT)
I20260709 13:10:35.113488 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.083328 25914 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.114006 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.118068 22709 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:35.114895 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.119546 22704 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:35.131822 25764 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:10:35.132623 25618 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.004s
I20260709 13:10:35.133133 25618 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.134449 22631 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:35.134956 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.135885 23416 raft_consensus.cc:2410] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:10:35.136444 25618 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:35.102181 25760 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:35.124063 25848 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:35.142858 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.005s	sys 0.025s
I20260709 13:10:35.146214 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.148217 22705 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 3: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:35.149580 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.005s	sys 0.025s
W20260709 13:10:35.150004 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 646904us
W20260709 13:10:35.155416 22750 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.174s (SENT)
I20260709 13:10:35.157496 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.125831 25914 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.118566 25399 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.149250 25760 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.149329 25848 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.168339 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:35.169309 22750 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.203s (SENT)
W20260709 13:10:35.169622 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.169806 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.171948 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.172770 22704 raft_consensus.cc:2368] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:35.173020 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.173570 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.174010 22709 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:35.174304 22704 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:35.174953 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.175256 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.176115 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.176265 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.176780 22703 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7.
I20260709 13:10:35.177103 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.177628 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.177935 22707 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:10:35.178232 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:35.176792 22704 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:35.150753 25435 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.179402 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.179970 22705 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:35.180459 22705 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 10.
I20260709 13:10:35.181514 25950 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:35.181950 25950 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:35.158841 25767 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.193570 25767 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.195973 25767 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.163637 25812 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:35.201458 22706 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:35.200933 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.201682 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:35.203697 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.204375 22635 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:35.204630 25399 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:35.204941 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.205770 23417 raft_consensus.cc:2410] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:10:35.205741 25950 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.212267 23416 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:10:35.213131 25435 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.213616 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:35.214326 23417 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:35.216005 23421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 549) took 1535 ms (client timeout 1972 ms). Trace:
W20260709 13:10:35.216444 23421 rpcz_store.cc:269] 0709 13:10:33.680223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.689990 (+  9767us) service_pool.cc:224] Handling call
0709 13:10:35.215784 (+1525794us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54005028d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:35.215915 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1749671296}
I20260709 13:10:35.218801 23415 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:35.219633 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:35.188752 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.220635 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 693) took 1467 ms. Trace:
I20260709 13:10:35.182339 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.189771 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.221802 22708 raft_consensus.cc:2368] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7.
I20260709 13:10:35.220856 22705 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:35.198433 25760 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.229727 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.230185 25848 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:35.230508 22705 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:35.221040 23415 rpcz_store.cc:276] 0709 13:10:33.753428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.894039 (+140611us) service_pool.cc:224] Handling call
0709 13:10:35.218496 (+1324457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:35.219257 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:35.220431 (+  1174us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005028d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:35.220570 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2994296576}
I20260709 13:10:35.232172 25288 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:35.190171 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.239665 22701 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:35.191215 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.240917 22707 raft_consensus.cc:1813] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:35.241577 22707 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 10.
I20260709 13:10:35.191558 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.247443 22703 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:35.267181 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.272046 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52545.1us
I20260709 13:10:35.277549 23417 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:35.290731 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52760.4us
I20260709 13:10:35.293282 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.294483 22707 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:35.303334 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.312898 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:35.302465 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.323949 22635 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:10:35.271075 25627 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:35.271239 25853 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.290403 25915 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:35.339354 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.354417 22775 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:35.355893 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.332628 25435 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:35.325168 25914 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:35.366763 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 201930us
I20260709 13:10:35.367460 22705 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:10:35.367817 22750 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.234s (SENT)
I20260709 13:10:35.327885 25688 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.370586 22701 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:35.316349 25288 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:10:35.377667 22776 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:35.351751 25768 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.352138 25851 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.380712 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:35.388185 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.376242 25435 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:35.393146 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.285s (SENT)
I20260709 13:10:35.393667 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.393960 25851 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.396239 25971 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.099s	user 0.000s	sys 0.002s
I20260709 13:10:35.396657 25851 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:35.370355 25837 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.397691 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:35.398229 25837 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.400048 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:35.400519 25837 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.403053 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.403712 25767 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:10:35.404073 22705 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7.
I20260709 13:10:35.354948 25533 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.405365 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.406723 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.407855 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:35.408062 22750 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.321s (SENT)
I20260709 13:10:35.361440 25915 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:35.405323 25533 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.411404 25915 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.413868 22750 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.358s (SENT)
W20260709 13:10:35.415182 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 103203us
W20260709 13:10:35.421638 22750 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.577s (SENT)
I20260709 13:10:35.422269 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.425733 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.426911 22710 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:35.382432 25688 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.431005 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.431654 22705 raft_consensus.cc:2376] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8. Re-sending same reply.
I20260709 13:10:35.353633 25853 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.432566 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.433792 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.434731 22707 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:10:35.437338 22775 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:35.437662 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:35.443250 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.443831 22779 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:35.444739 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:35.446813 22701 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:35.447538 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:35.447918 25946 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.708s	user 0.011s	sys 0.002s
I20260709 13:10:35.448194 22776 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:35.393785 25435 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.430819 25760 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.403146 25768 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.469724 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.608s (SENT)
I20260709 13:10:35.472987 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.473980 22701 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:35.484429 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64112us
I20260709 13:10:35.481348 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:35.487706 22780 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:10:35.489634 25975 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:35.490983 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.458583 25768 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:35.489720 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68979.8us
I20260709 13:10:35.495540 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.508026 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.508529 22702 raft_consensus.cc:2376] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:10:35.497754 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:34.170147 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet f33fee9d64384b499e55f8bc58339f69 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:10:35.492952 25758 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.504102 25853 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:35.511650 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.456s (SENT)
I20260709 13:10:35.513221 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.517015 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.515043 22707 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:35.515751 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:35.516448 22519 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 1.34639e+06us
I20260709 13:10:35.521147 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.520710 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:35.515908 23385 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.665s (SENT)
I20260709 13:10:35.535602 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:35.538398 25767 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.539361 25767 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.538996 22750 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.521s (SENT)
W20260709 13:10:35.523283 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:35.516889 25758 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.551648 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.552637 22633 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:35.545820 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:35.554559 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:35.532691 25975 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:35.532501 25853 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.533551 25688 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.530984 25977 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:10:35.548880 25627 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:10:35.588378 22750 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.734s (SENT)
I20260709 13:10:35.589188 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.557845 25720 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:35.567835 25754 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:10:35.599431 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83560.7us
I20260709 13:10:35.587502 25767 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:35.627713 25966 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.431s	user 0.000s	sys 0.002s
I20260709 13:10:35.633343 22776 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:35.637285 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.638242 22633 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:35.639937 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.641397 25533 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.642678 25758 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.601866 25812 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:35.647073 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:35.647918 22778 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 2.
I20260709 13:10:35.595891 25754 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.655300 25754 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:35.655783 22783 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:35.659055 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2144) took 1150 ms. Trace:
I20260709 13:10:35.659565 22631 rpcz_store.cc:276] 0709 13:10:34.508642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:34.509125 (+   483us) service_pool.cc:224] Handling call
0709 13:10:34.525948 (+ 16823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:35.134885 (+608937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:35.625406 (+490521us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400127854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:35.658964 (+ 33558us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398912256}
I20260709 13:10:35.578562 25627 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:35.661201 25627 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:35.667191 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78853.2us
W20260709 13:10:35.668517 22750 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.748s (SENT)
I20260709 13:10:35.573113 25853 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.671854 25853 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:35.674371 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.677779 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.696426 22750 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.708s (SENT)
I20260709 13:10:35.697012 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.698181 22705 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:35.699599 23385 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.802s (SENT)
I20260709 13:10:35.700713 22701 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:35.700883 22780 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:35.706250 22631 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
W20260709 13:10:35.706292 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 151381us
I20260709 13:10:35.718019 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:35.719214 22776 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:35.718321 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.720100 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:35.729172 22709 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:10:35.696076 25853 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:35.733081 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49689.9us
I20260709 13:10:35.738158 25627 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.746956 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:10:35.748703 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.749317 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.762144 22706 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:35.750828 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 115932us
I20260709 13:10:35.753862 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:35.750242 22704 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:35.765125 22704 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:35.776140 22704 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:35.777346 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.719s (SENT)
I20260709 13:10:35.777515 22704 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:35.783378 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56387.6us
I20260709 13:10:35.787114 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.790982 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:35.790982 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.791672 22708 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:35.788570 25988 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.071s	user 0.001s	sys 0.018s
I20260709 13:10:35.792938 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.793025 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:35.782111 25288 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.798421 22709 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 3.
I20260709 13:10:35.809722 22704 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:10:35.786185 25804 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:35.812054 22704 pending_rounds.cc:85] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 3
I20260709 13:10:35.814462 22704 pending_rounds.cc:107] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:10:35.817037 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.817744 22709 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:35.818646 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:35.819190 22709 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:35.820174 22709 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:35.820447 22709 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:35.821190 22709 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.822141 22709 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:10:35.822755 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.824504 25837 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.814867 22704 raft_consensus.cc:2889] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:35.775635 25607 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:10:35.825042 25837 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:35.841315 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.854738 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.857120 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.825124 25890 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.865604 22706 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:35.789950 25843 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:10:35.779248 25853 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:35.878551 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.854981 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.549s (SENT)
I20260709 13:10:35.879101 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:35.833958 25856 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:35.877300 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:35.884688 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.813714 25990 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:10:35.866984 25758 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:35.827790 25288 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.900887 25853 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.901482 25288 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:35.898226 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 137635us
I20260709 13:10:35.902359 25754 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:35.902902 25627 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.904242 25627 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:35.904608 25627 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:35.904942 25754 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.882090 25750 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:35.906049 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:35.899104 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.975s (SENT)
I20260709 13:10:35.879315 25804 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:35.927248 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:35.928176 23415 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:35.924052 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:35.929281 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:35.930030 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.953435 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:35.880926 25890 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:35.966794 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.978952 25950 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:35.980595 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:35.982169 25837 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:35.926688 25856 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.983131 25618 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:35.982951 25856 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:35.983599 25618 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.984041 25618 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:35.985658 25856 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:35.996340 22709 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:10:35.997335 22709 pending_rounds.cc:85] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:10:35.997687 22709 pending_rounds.cc:107] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:10:35.998231 22709 raft_consensus.cc:2889] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:36.000562 25804 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.001572 25908 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:36.002211 25908 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:36.005877 22632 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:10:36.010061 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 227619us
I20260709 13:10:36.003619 25908 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:36.011065 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 200508us
I20260709 13:10:36.016445 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:35.997830 25977 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:36.019807 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:36.026232 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127170us
W20260709 13:10:36.027374 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.036976 23415 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:10:36.013160 25856 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:10:36.014698 25915 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:36.025403 25755 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:35.958154 25627 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.036332 25908 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:36.020031 25435 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:10:36.078981 23420 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:36.034705 25768 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:10:36.103428 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73723.1us
I20260709 13:10:36.109930 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:36.127743 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:36.073510 25755 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:36.073117 26001 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:36.069609 25758 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:10:36.067082 25915 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:36.157995 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123518us
I20260709 13:10:36.111681 25837 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:36.172252 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.165881 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:36.071697 25764 consensus_queue.cc:1175] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6a31ac6451ff4f73bee634487d84d001" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 8 } }
I20260709 13:10:36.148360 25804 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:36.186260 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.114931 25767 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:10:36.196607 22626 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:36.156999 25812 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:36.176995 25853 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:36.201311 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 167831us
I20260709 13:10:36.205193 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:36.206184 25853 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.199713 26001 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:36.210747 25915 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:36.210865 22632 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:36.201437 25399 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:10:36.202893 25890 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:36.217836 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:36.218784 23419 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:10:36.220728 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:36.221014 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.221674 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:36.222196 26000 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:10:36.228257 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97817.6us
W20260709 13:10:36.234470 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113082us
I20260709 13:10:36.236830 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.240919 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:36.192838 25804 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.252956 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.253196 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.256668 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.257962 22632 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:36.186730 25288 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:10:36.268992 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63900.7us
I20260709 13:10:36.187245 25837 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.278074 25399 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:36.283599 25399 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:36.283870 25994 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:10:36.284659 25399 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.222680 26000 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:36.287248 26000 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:36.291594 25825 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:36.293371 25750 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:10:36.293931 23385 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.544s (SENT)
W20260709 13:10:36.294770 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.295406 25843 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:36.296547 25618 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:36.302780 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.303638 22633 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:36.305550 25837 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:36.306201 25837 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:36.308604 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.278503 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:36.259877 25990 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:10:36.234968 25812 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.310911 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:36.316821 22776 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:10:36.322570 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 137713us
I20260709 13:10:36.269158 25915 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:36.271931 25804 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:36.306736 25837 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.351292 25837 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:36.258769 25767 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:10:36.357446 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:36.359887 25768 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:36.359710 25902 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:36.360369 25768 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:36.360603 25853 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:36.362926 26009 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:36.326874 25812 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:36.332923 25972 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:10:36.367568 25812 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:36.337885 25997 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.349030 25915 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:36.360826 25768 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:36.369073 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:36.378168 22776 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 3.
W20260709 13:10:36.379566 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:36.339951 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 155343us
I20260709 13:10:36.372022 26007 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:10:36.272801 25755 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:36.398792 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 115079us
I20260709 13:10:36.407157 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:36.409422 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.409499 22776 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 7. Ops: [5.3-5.3]
W20260709 13:10:36.410395 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.414691 22776 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:36.434070 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:36.444128 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.446521 22750 leader_election.cc:400] T f33fee9d64384b499e55f8bc58339f69 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 10.
W20260709 13:10:36.455694 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86703.4us
I20260709 13:10:36.460608 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.464226 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.401729 25618 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:36.370498 25997 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:36.469543 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86724.6us
I20260709 13:10:36.470449 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:36.463300 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:36.472122 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:36.473312 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 102578us
I20260709 13:10:36.481866 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.492108 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.495312 22776 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:36.496769 22778 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:36.496469 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:36.500842 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.502092 22780 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:36.470896 25961 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:36.509683 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:36.524154 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53816.9us
W20260709 13:10:36.532099 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.926s (SENT)
I20260709 13:10:36.532701 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:36.548903 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.551990 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.551864 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:36.512221 25961 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.557016 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:36.562175 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 82127.9us
I20260709 13:10:36.562561 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.564253 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.529157 25435 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:36.564960 22632 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:36.565989 22627 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:36.566674 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.571487 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:36.572564 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 109375us
I20260709 13:10:36.569275 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.550278 25950 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:36.570676 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:36.572625 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44191.2us
I20260709 13:10:36.481650 25915 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.577042 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.562745 25990 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:36.560839 25961 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:36.560498 25812 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:36.561918 26017 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.583405 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.586668 22627 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:36.591198 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:36.588116 25837 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.599179 25435 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:36.599520 26019 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:36.599790 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:36.603120 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.606544 22630 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
W20260709 13:10:36.611723 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.575033 25853 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.624549 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.628705 22630 raft_consensus.cc:1217] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-6.7]   Dedup: 6.7->[]
I20260709 13:10:36.629380 22630 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:10:36.589327 25908 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:36.636554 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.640821 25997 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:36.651068 23385 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.045s (SENT)
I20260709 13:10:36.600472 25435 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.615679 25399 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:36.656551 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.621006 25288 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:10:36.667133 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 110245us
I20260709 13:10:36.599294 25837 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.643101 25908 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.649091 25950 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.678575 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.630816 25607 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:10:36.602213 26017 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.634308 25853 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.696290 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.633885 26016 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:10:36.675320 25908 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:36.699414 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97969.6us
I20260709 13:10:36.702345 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.701574 22627 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:36.707144 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:36.717204 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.675256 25995 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:10:36.698521 25399 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:36.719852 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.685761 25950 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.733232 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.709551 26000 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.748941 22701 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:10:36.754966 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:36.756529 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:36.781442 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.773640 22781 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:36.809482 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:36.810194 22708 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:36.809054 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:36.820811 22705 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:36.822270 22710 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:10:36.822886 22710 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:10:36.823213 22710 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:10:36.823542 22710 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:36.842149 26024 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:10:36.845625 22708 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:36.846817 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.848261 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.848981 22708 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:36.802943 26012 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:36.861241 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.862048 22702 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:36.863236 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.864074 22704 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:10:36.874959 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.857733 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.885953 25997 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.820336 25804 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:36.889333 22708 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:36.858997 26012 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.896360 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.904404 22702 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:36.899364 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:36.901274 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.880008 25812 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.889956 26028 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.897449 26030 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.934126 22626 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:10:36.934518 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:36.938640 22702 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6. Re-sending same reply.
I20260709 13:10:36.936285 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:36.915139 26012 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:36.939817 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.957231 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.911636 26038 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:36.961082 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.961957 22702 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:36.856470 25399 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.891786 25997 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.973407 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.934158 25837 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:36.973373 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:36.977357 22627 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:36.974632 22702 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6. Re-sending same reply.
I20260709 13:10:36.971020 22704 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:36.984599 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.991526 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.988395 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:36.929404 25812 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.999122 22704 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:37.001972 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:37.003163 22707 raft_consensus.cc:2368] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
I20260709 13:10:37.005972 22704 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:36.990376 25975 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.010291 25975 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.019685 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:37.009070 26034 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:10:36.941272 26028 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.038302 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.022946 26000 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:36.972513 26038 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:36.963165 26030 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:37.045264 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:37.039501 26033 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:10:37.032743 25950 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.035637 25755 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:37.064066 23385 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.192s (SENT)
I20260709 13:10:37.091118 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:37.091130 25755 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.092630 25950 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:37.059227 26005 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.851s	user 0.007s	sys 0.000s
I20260709 13:10:37.094211 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:37.097092 22703 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:37.035704 25995 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:37.099068 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59053.2us
I20260709 13:10:37.100471 25995 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.102506 26027 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.103124 26027 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.116515 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.119686 22775 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:10:37.136234 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:37.137362 22703 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:10:37.138954 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74945.9us
I20260709 13:10:37.144448 22708 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:37.155660 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:37.122189 26000 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.176688 22703 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:10:37.146432 25804 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:37.187968 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:37.190986 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:37.192859 22709 raft_consensus.cc:2376] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:37.205422 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2299) took 1560 ms. Trace:
I20260709 13:10:37.206149 22782 rpcz_store.cc:276] 0709 13:10:35.644931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:35.645782 (+   851us) service_pool.cc:224] Handling call
0709 13:10:35.646273 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:36.393241 (+746968us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.393274 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:36.393897 (+   623us) log.cc:844] Serialized 52 byte log entry
0709 13:10:36.477487 (+ 83590us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:36.477533 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.199385 (+721852us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54001865d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:37.199446 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:37.199532 (+    86us) raft_consensus.cc:1685] finished
0709 13:10:37.199550 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:37.205314 (+  5764us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8124,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":24111,"spinlock_wait_cycles":1716790528,"thread_start_us":47002,"threads_started":1,"wal-append.queue_time_us":191853}
I20260709 13:10:37.223803 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:37.225190 22709 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:37.230253 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:37.231333 22703 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:37.226634 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 110752us
I20260709 13:10:37.196182 26000 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.215639 25997 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.215092 26036 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.227123 26027 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.263289 22709 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:10:37.242981 25995 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:37.275277 26039 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:10:37.299346 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:37.298000 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:37.309054 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:37.323115 26041 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
I20260709 13:10:37.276592 25997 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.340478 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.322994 25975 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.345675 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:37.334448 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98594us
I20260709 13:10:37.358120 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.290333 26028 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.366036 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:37.369855 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.382438 25975 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.386905 25975 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.389379 25975 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.390985 26046 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:37.391681 26046 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.395018 26049 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.395916 26049 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.399091 25764 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:37.404026 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.404364 26045 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.404969 26045 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.406181 26000 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.407310 25837 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.407943 25837 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.409875 26047 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:37.366632 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:37.410989 22709 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:37.411906 25837 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:37.412352 22709 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:10:37.412680 26060 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:10:37.266193 26036 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.382683 22633 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:37.363826 26040 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:10:37.416091 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.452515 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:37.431736 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:37.467263 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.468315 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.394464 26046 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:37.374845 26044 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:37.509801 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.342078 25997 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.513060 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:37.514299 23421 raft_consensus.cc:2410] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:10:37.454553 25837 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:37.520648 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:37.523980 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55771.7us
I20260709 13:10:37.526297 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.528188 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.529671 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:37.528221 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 285317us
I20260709 13:10:37.508941 26047 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:10:37.544174 26047 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:37.544307 26050 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:37.545395 26047 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:37.547297 26057 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:10:37.554564 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.524807 26046 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:37.527064 26049 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.560828 26050 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:10:37.517002 26044 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:37.573239 26059 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:10:37.583447 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:37.573431 26055 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.584954 26055 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.583514 26050 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:37.588312 26050 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:37.588860 25997 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.573939 26053 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:10:37.576916 26017 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:37.541870 25768 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.154s	user 0.007s	sys 0.011s
I20260709 13:10:37.541215 25627 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:37.587718 26055 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.610041 26045 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:37.613178 25908 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.900s	user 0.010s	sys 0.003s
I20260709 13:10:37.613755 25908 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:37.617825 25961 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.016s	user 0.013s	sys 0.002s
I20260709 13:10:37.585320 26044 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.593613 26046 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:37.644793 22632 raft_consensus.cc:2368] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8.
I20260709 13:10:37.604681 25837 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:37.625371 25997 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.605852 25768 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.618451 25961 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.589491 26050 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.668108 22634 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:37.665179 26067 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.689643 22629 raft_consensus.cc:2368] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4.
I20260709 13:10:37.653280 25997 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.705746 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1703) took 1269 ms. Trace:
I20260709 13:10:37.706066 22784 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:37.706420 22776 rpcz_store.cc:276] 0709 13:10:36.436798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.437059 (+   261us) service_pool.cc:224] Handling call
0709 13:10:36.437356 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:36.495952 (+ 58596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.705280 (+1209328us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54003403d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:37.705642 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132738304}
I20260709 13:10:37.667351 26028 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.711229 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.719915 25768 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.722324 25997 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:37.724164 26073 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:10:37.728124 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.728626 25812 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:37.731564 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:37.731986 26080 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:10:37.734262 25961 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:37.735950 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.739833 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.740782 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:37.740936 22777 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:37.742937 26028 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.746096 22779 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:37.749739 26030 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:37.750641 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.751857 25908 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.753283 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.754004 22776 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:37.754163 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.961s (SENT)
I20260709 13:10:37.742363 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.754655 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:37.713261 26050 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:37.715652 22775 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:37.775065 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.776160 22776 raft_consensus.cc:2410] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:10:37.746233 26028 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:37.782496 22782 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:10:37.786412 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 258277us
I20260709 13:10:37.745963 25853 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.787168 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:37.771864 26047 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.703228 26067 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.799222 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:37.812759 22702 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:37.802502 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.805217 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.815183 22635 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:37.806494 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.781637 25825 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.809231 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:37.791298 25977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.134s	user 0.000s	sys 0.010s
W20260709 13:10:37.817961 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 85886.4us
I20260709 13:10:37.823140 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.825127 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.792238 26030 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:37.791251 26046 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.802006 26085 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:37.822381 25977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.169s	user 0.000s	sys 0.011s
I20260709 13:10:37.844377 22779 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:37.845731 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.846460 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:37.851013 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:37.851588 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74946.3us
I20260709 13:10:37.852571 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.852777 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:37.854565 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:37.848501 26085 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.857219 26070 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:37.858189 26007 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:37.859633 22702 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:37.860195 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.860419 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.863929 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:37.849530 26012 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.884s	user 0.021s	sys 0.009s
I20260709 13:10:37.836727 26030 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:37.847159 25853 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:37.869719 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.860430 26045 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:37.818106 26047 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.874466 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:37.876631 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:37.863452 26028 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:37.827823 25825 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.866259 26049 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:37.886567 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.886866 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:37.870278 26017 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:37.898774 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.851331 26046 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:37.906659 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76483.3us
I20260709 13:10:37.930207 26086 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:37.871052 26012 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:37.935631 26082 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.936671 26012 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.936498 26082 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:37.937644 23419 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:10:37.878834 26093 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
I20260709 13:10:37.900965 26070 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:37.897266 25812 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:37.935752 26047 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:37.885391 25853 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:37.944389 26084 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:37.944895 26015 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.221s	user 0.005s	sys 0.004s
I20260709 13:10:37.895063 26045 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:37.957897 26085 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:37.947522 26091 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:10:37.279582 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:37.942338 26092 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:37.927428 26030 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:37.997175 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 110406us
W20260709 13:10:37.955011 26015 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.291s	user 0.005s	sys 0.004s
I20260709 13:10:37.995700 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.930537 25825 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.002966 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.009169 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.010499 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:37.994012 25767 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.012677 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:37.972297 26020 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.404s	user 0.004s	sys 0.000s
I20260709 13:10:38.016310 26007 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:38.014213 26045 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.018004 26067 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.018604 26082 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.019558 26038 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.019937 26046 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.014801 25853 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:38.026943 22781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 447) took 1790 ms (client timeout 1816 ms). Trace:
W20260709 13:10:38.027460 22781 rpcz_store.cc:269] 0709 13:10:36.236303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.236508 (+   205us) service_pool.cc:224] Handling call
0709 13:10:38.026740 (+1790232us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400186fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.026874 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1232761728}
W20260709 13:10:38.032893 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1695) took 1795 ms (client timeout 2223 ms). Trace:
W20260709 13:10:38.033422 22783 rpcz_store.cc:269] 0709 13:10:36.237782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.238007 (+   225us) service_pool.cc:224] Handling call
0709 13:10:38.032668 (+1794661us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b5400325ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.032821 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1014151552}
I20260709 13:10:38.035843 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.037151 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.038213 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.040787 22630 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:10:38.015419 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:38.046588 22708 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:10:38.047334 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 401) took 1013 ms. Trace:
I20260709 13:10:38.047631 22708 rpcz_store.cc:276] 0709 13:10:37.033953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.160995 (+127042us) service_pool.cc:224] Handling call
0709 13:10:38.045996 (+885001us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:10:38.046189 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.047085 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.047276 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6790400}
I20260709 13:10:38.050952 25853 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:38.051383 25767 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:38.052012 22750 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.606s (SENT)
I20260709 13:10:38.004607 25755 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.050701 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:37.955272 26070 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:38.057680 22705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.011s	sys 0.006s
W20260709 13:10:38.072113 22705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.011s	sys 0.007s
W20260709 13:10:38.058291 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1699) took 1498 ms (client timeout 1616 ms). Trace:
I20260709 13:10:38.058578 22628 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:38.068223 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:38.073668 22626 rpcz_store.cc:269] 0709 13:10:36.560328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.560585 (+   257us) service_pool.cc:224] Handling call
0709 13:10:38.058092 (+1497507us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b540006d6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.058202 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814527232}
I20260709 13:10:38.076391 22778 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:38.068296 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.069012 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.079116 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.080293 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.081307 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.082290 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.063361 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.067616 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.085197 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.086838 26067 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:38.087188 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.088306 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.089398 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.091619 22633 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:38.090322 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:38.099025 22750 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.071s (SENT)
I20260709 13:10:38.099613 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.102094 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.102998 22709 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:38.104058 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.105387 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.093047 22626 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:38.057582 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.061560 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.074855 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.062340 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.068852 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.143778 22708 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:38.141973 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:38.144835 22706 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:38.142534 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:38.144871 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45906.4us
I20260709 13:10:38.146833 22703 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:38.147431 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.443s (SENT)
I20260709 13:10:38.014451 26085 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:38.014554 26084 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.108011 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.174886 22783 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
W20260709 13:10:38.141024 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 119767us
I20260709 13:10:38.106487 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.176052 22709 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:38.169462 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:38.168545 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 111857us
I20260709 13:10:38.054603 25767 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:38.180780 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:38.216076 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.170580 26082 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:38.092376 26070 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.180670 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.169857 25768 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:38.061805 26069 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.107946 26043 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:38.286509 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107155us
I20260709 13:10:38.107735 25990 raft_consensus.cc:476] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Not starting pre-election -- already a leader
W20260709 13:10:38.288890 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 108543us
I20260709 13:10:38.140725 25627 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:10:38.289422 22778 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:38.186829 26085 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:38.299541 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.296823 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.319226 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.296167 25825 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:38.339910 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2303) took 1237 ms. Trace:
I20260709 13:10:38.340178 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:38.296542 25768 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.340647 22780 rpcz_store.cc:276] 0709 13:10:37.113287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.133747 (+ 20460us) service_pool.cc:224] Handling call
0709 13:10:37.157458 (+ 23711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.031483 (+874025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:38.031518 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:38.031571 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:38.031604 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.339526 (+307922us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b540015a9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:38.339604 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.339811 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2009575296}
W20260709 13:10:38.367170 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 353752us
I20260709 13:10:38.330447 26044 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:38.371645 22707 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:38.314082 26070 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:38.290822 26069 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.298725 26043 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.389560 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.394987 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.390405 22709 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:38.403474 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.373109 26044 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:38.405295 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.407109 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.403679 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.408553 22777 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:38.406456 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.413771 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.415655 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:38.403481 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.421298 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.423172 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:38.424111 22630 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:38.424407 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.424868 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.425905 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.426431 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.427838 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.428133 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.429014 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.425659 22776 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:38.432531 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.411738 26044 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:38.434576 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.434859 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.436897 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.416695 25825 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:38.441190 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.387991 26070 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.444631 25812 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:38.443552 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.451630 22780 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:38.447201 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.429803 25750 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:10:38.450129 26028 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.474507 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2310) took 1355 ms. Trace:
I20260709 13:10:38.447902 25825 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:38.482844 22632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1711) took 1809 ms (client timeout 1715 ms). Trace:
I20260709 13:10:38.408708 26085 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:38.486999 22632 rpcz_store.cc:269] 0709 13:10:36.673743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.673913 (+   170us) service_pool.cc:224] Handling call
0709 13:10:38.471127 (+1797214us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000b7cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.482686 (+ 11559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219531008}
I20260709 13:10:38.475368 22784 rpcz_store.cc:276] 0709 13:10:37.119136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.119732 (+   596us) service_pool.cc:224] Handling call
0709 13:10:37.705533 (+585801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:37.706714 (+  1181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.472382 (+765668us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b54003403d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.474409 (+  2027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1272470400}
W20260709 13:10:38.483928 26104 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:10:38.491578 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:38.484640 26106 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.059s	user 0.001s	sys 0.001s
W20260709 13:10:38.495430 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 198562us
I20260709 13:10:38.495223 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.496572 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:38.498883 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.499207 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2241) took 1674 ms. Trace:
I20260709 13:10:38.485857 25812 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:38.488230 25825 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:38.505914 26043 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.508805 26036 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:38.514829 22675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.616s (SENT) [suppressed 27 similar messages]
I20260709 13:10:38.500272 26065 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:38.541693 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131664us
I20260709 13:10:38.545534 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:38.522395 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.499969 25627 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:38.563694 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:38.564347 22709 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:38.566893 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44664.8us
I20260709 13:10:38.568353 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.570415 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.576571 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:38.532209 25990 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:10:38.581440 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66674us
I20260709 13:10:38.525151 26001 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:38.523553 22705 rpcz_store.cc:276] 0709 13:10:36.824495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.827276 (+  2781us) service_pool.cc:224] Handling call
0709 13:10:36.827632 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:36.827870 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:10:38.072554 (+1244684us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.491772 (+419218us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.498842 (+  7070us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:38.498903 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:38.498957 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.499026 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.499145 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7400,"prepare.queue_time_us":148019,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":60,"raft.queue_time_us":3440,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":22526,"thread_start_us":1657981,"threads_started":3,"wal-append.queue_time_us":415099}
I20260709 13:10:38.583683 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.584887 22705 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:38.561328 25760 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.589144 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.514242 26044 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.591189 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.592459 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.599509 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.606492 22709 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:38.573838 25767 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:38.586241 26027 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:38.625253 22707 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:10:38.625638 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.578368 25812 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:38.586807 26069 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.576005 26043 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:10:38.620945 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.630592 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.648502 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.620714 26084 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:38.657356 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.621769 26045 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:38.662720 22707 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:10:38.667495 26112 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:10:38.643626 25990 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:38.632793 26114 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:10:38.631327 26108 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:10:38.618939 25760 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.677577 26044 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.659472 26065 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:38.633000 25627 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:38.665994 26069 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:38.632081 26109 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:10:38.622196 26001 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:38.627494 25812 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.693080 22630 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:38.697942 26128 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:38.699549 26129 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:38.701923 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 109502us
I20260709 13:10:38.683772 26084 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.709478 25760 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.711877 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:38.710330 26065 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.701916 26123 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.719482 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:38.682737 26045 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.727103 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.733119 26124 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:38.733870 26124 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.734417 26124 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:38.735149 26023 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 2.081s	user 0.004s	sys 0.000s
I20260709 13:10:38.736735 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2166) took 2300 ms. Trace:
I20260709 13:10:38.737151 22631 rpcz_store.cc:276] 0709 13:10:36.436187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.436331 (+   144us) service_pool.cc:224] Handling call
0709 13:10:36.436750 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:36.437155 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.437187 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:36.439261 (+  2074us) log.cc:844] Serialized 52 byte log entry
0709 13:10:36.527020 (+ 87759us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:36.527072 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.527530 (+   458us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.734373 (+2206843us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:38.736343 (+  1970us) raft_consensus.cc:1685] finished
0709 13:10:38.736366 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.736652 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31473,"raft.queue_time_us":2318,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":10355,"spinlock_wait_cycles":26121600,"thread_start_us":77435,"threads_started":1,"wal-append.queue_time_us":133690}
I20260709 13:10:38.739955 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.729802 25804 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:38.741587 25804 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:38.741936 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.742538 25804 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.743485 26103 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:10:38.749214 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:38.730034 26045 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:38.746024 26044 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.746644 26012 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:38.746696 23419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 574) took 1727 ms (client timeout 2018 ms). Trace:
W20260709 13:10:38.759119 23419 rpcz_store.cc:269] 0709 13:10:37.019081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.019302 (+   221us) service_pool.cc:224] Handling call
0709 13:10:38.746345 (+1727043us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b5400455454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.746577 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2108838144}
I20260709 13:10:38.719672 25812 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:38.740010 26113 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 13:10:38.763816 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:38.751088 26070 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:38.727674 26065 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:38.728762 26084 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:38.780566 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:38.781649 23415 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:10:38.783070 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 134726us
I20260709 13:10:38.758234 26069 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:38.785048 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.767297 26128 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.791285 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:38.770679 26125 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:38.719111 26129 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.792150 22703 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:38.801080 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:38.806072 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:38.786832 26069 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.792125 25627 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.802074 26131 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.750381 26123 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.809724 26128 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:38.793884 26125 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.830891 26125 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:38.791587 26001 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.765985 26044 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.835729 26044 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.820371 26069 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:38.839145 26120 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:10:38.779506 26012 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.846123 22747 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:10:38.848347 22747 leader_election.cc:403] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:10:38.820609 25902 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:10:38.851886 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88240.1us
I20260709 13:10:38.853958 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:38.851100 25902 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.852825 25627 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
W20260709 13:10:38.889077 26074 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.153s	user 0.002s	sys 0.000s
I20260709 13:10:38.859092 25902 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:38.904774 25627 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.905486 25627 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.906914 26075 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:38.907553 26075 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:38.909351 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 442) took 1796 ms. Trace:
I20260709 13:10:38.862329 26129 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:38.913410 26062 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.587s	user 0.001s	sys 0.003s
I20260709 13:10:38.916405 26123 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:38.910017 26075 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:38.864248 26012 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.909785 22628 rpcz_store.cc:276] 0709 13:10:37.113287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.113442 (+   155us) service_pool.cc:224] Handling call
0709 13:10:38.058381 (+944939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.091949 (+ 33568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.909098 (+817149us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b540006d6d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:38.909278 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2172281600}
I20260709 13:10:38.923955 22710 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:38.925712 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2245) took 2058 ms. Trace:
I20260709 13:10:38.894030 25825 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:38.932369 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 78471.3us
I20260709 13:10:38.933836 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:38.934684 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:38.926178 22704 rpcz_store.cc:276] 0709 13:10:36.867677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.007404 (+139727us) service_pool.cc:224] Handling call
0709 13:10:37.007762 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:37.011191 (+  3429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:37.011224 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:37.025475 (+ 14251us) log.cc:844] Serialized 26 byte log entry
0709 13:10:37.165245 (+139770us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:37.196565 (+ 31320us) log.cc:844] Serialized 14 byte log entry
0709 13:10:37.196854 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.196939 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.896507 (+1699568us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b180043c960. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1
0709 13:10:38.912744 (+ 16237us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:38.920098 (+  7354us) spinlock_profiling.cc:243] Waited 1.99 ms on lock 0x7b180043c960. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:38.925490 (+  5392us) raft_consensus.cc:1685] finished
0709 13:10:38.925515 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.925646 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2980,"prepare.queue_time_us":32599,"raft.queue_time_us":4394,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":11073,"spinlock_wait_cycles":13257728,"thread_start_us":121149,"threads_started":1,"wal-append.queue_time_us":189943}
I20260709 13:10:38.945353 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:38.946012 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:38.948268 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:38.948318 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.954095 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.952605 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.956277 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.959385 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.960825 26072 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:38.961536 26072 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.961959 26072 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:38.964391 25825 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.965582 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:38.966339 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:38.968737 26126 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:38.969583 25627 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:38.951222 26132 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:38.954128 26129 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.960968 26127 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:38.924297 26008 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.974507 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:38.975405 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.976073 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:38.977494 23420 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:38.979612 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:38.980509 25627 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:38.990758 25902 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:38.991065 25990 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:38.993225 25902 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:38.998991 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:38.999969 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.019176 22780 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
W20260709 13:10:39.002041 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1701) took 2436 ms (client timeout 1989 ms). Trace:
W20260709 13:10:39.020268 22629 rpcz_store.cc:269] 0709 13:10:36.565675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.565843 (+   168us) service_pool.cc:224] Handling call
0709 13:10:39.001789 (+2435946us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002f5654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:39.001947 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2556014464}
I20260709 13:10:38.985466 25825 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:39.016906 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.019189 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.024330 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:38.876670 26045 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.038475 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2171) took 2357 ms. Trace:
I20260709 13:10:39.038802 25754 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:39.039409 25754 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.039669 25618 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:39.040654 25618 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:39.041069 25618 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:39.042068 25754 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.039084 22634 rpcz_store.cc:276] 0709 13:10:36.681356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.681519 (+   163us) service_pool.cc:224] Handling call
0709 13:10:36.681844 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:37.668788 (+986944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.038144 (+1369356us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b540006c7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:39.038381 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2267551616}
I20260709 13:10:39.041962 25618 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.053587 25618 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:39.054101 26126 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:39.055231 26127 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:39.064857 26101 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.065601 26101 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.053452 26022 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:10:39.079108 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48472.8us
I20260709 13:10:39.061280 26127 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:39.082170 26127 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.083897 26127 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:39.089569 26110 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.605s	user 0.000s	sys 0.001s
I20260709 13:10:39.015875 26008 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.092680 26008 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.094877 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2063) took 1024 ms. Trace:
I20260709 13:10:39.095386 22708 rpcz_store.cc:276] 0709 13:10:38.070924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.146506 (+ 75582us) service_pool.cc:224] Handling call
0709 13:10:38.178808 (+ 32302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:38.301505 (+122697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:38.301601 (+    96us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:38.309171 (+  7570us) log.cc:844] Serialized 26 byte log entry
0709 13:10:38.555180 (+246009us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:38.718182 (+163002us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.719489 (+  1307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.094332 (+374843us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b54002c38d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:39.094398 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:39.094466 (+    68us) raft_consensus.cc:1685] finished
0709 13:10:39.094565 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.094795 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29789,"prepare.queue_time_us":124225,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":11486,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":7946,"spinlock_wait_cycles":31603456,"thread_start_us":324540,"threads_started":2,"wal-append.queue_time_us":310002}
I20260709 13:10:39.096801 25812 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.098158 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.099795 26045 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.126729 26084 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.129498 26084 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.132104 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:39.132789 26072 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.133668 26067 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:39.135036 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.135272 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.135667 22630 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:39.135066 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.138208 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.103819 26124 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.108584 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.121402 26100 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:39.124070 22780 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:39.139230 22781 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:39.144405 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2067) took 1071 ms. Trace:
I20260709 13:10:39.144788 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:39.145035 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.145820 22782 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:39.146535 25815 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:39.144955 22701 rpcz_store.cc:276] 0709 13:10:38.073031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.150522 (+ 77491us) service_pool.cc:224] Handling call
0709 13:10:38.150996 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:38.151457 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:38.151489 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:38.174957 (+ 23468us) log.cc:844] Serialized 52 byte log entry
0709 13:10:38.378708 (+203751us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:38.561274 (+182566us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.585274 (+ 24000us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.589353 (+  4079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.589442 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.990688 (+401246us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b1800450ac0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:39.088869 (+ 98181us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:39.144048 (+ 55179us) raft_consensus.cc:1685] finished
0709 13:10:39.144086 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.144312 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45404,"prepare.queue_time_us":283514,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":78,"raft.queue_time_us":27497,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":38865,"spinlock_wait_cycles":89625984,"thread_start_us":223646,"threads_started":2,"wal-append.queue_time_us":181348}
I20260709 13:10:39.148015 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.151276 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:39.152084 22701 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:39.163722 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.165124 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:39.173532 25812 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:39.154003 26013 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.174754 26013 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:39.178858 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 231135us
I20260709 13:10:39.180269 22782 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:39.181160 22701 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:39.185192 22781 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:39.196791 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.197646 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:39.199007 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.200423 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.201857 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.208667 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.217512 26072 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:39.220657 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:39.222563 22779 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1716) took 1862 ms (client timeout 2159 ms). Trace:
W20260709 13:10:39.223057 22779 rpcz_store.cc:269] 0709 13:10:37.360293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.752980 (+392687us) service_pool.cc:224] Handling call
0709 13:10:39.222355 (+1469375us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54000cba54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:39.222494 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125232384}
I20260709 13:10:39.226758 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.231142 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:39.195575 26101 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.234489 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.235606 22632 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:10:39.235926 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.239202 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.240499 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.221707 25435 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:39.242277 22629 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 4: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:39.243805 26013 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:39.244407 26013 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.244946 26013 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.232466 26125 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.253989 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.254225 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.254953 22708 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:39.255000 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:39.250801 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.244822 26069 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.256594 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:39.222111 26111 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:10:39.259450 26126 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:39.242337 25435 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.262553 22707 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:39.275192 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56313.4us
I20260709 13:10:39.260725 25812 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:10:39.308312 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:39.288688 26082 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:39.318774 26124 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.318895 26070 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:39.321481 25435 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.264856 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:39.263854 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45102.1us
I20260709 13:10:39.385995 22703 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:10:39.400056 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.400838 22778 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:39.401249 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:39.406483 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 141722us
I20260709 13:10:39.384344 26126 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:39.410573 22707 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [7.8-8.9]
I20260709 13:10:39.395860 26030 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:39.398679 26065 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.612s	user 0.014s	sys 0.004s
I20260709 13:10:39.323226 26012 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.425601 22781 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 13:10:39.425165 26126 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.388063 26103 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:10:39.403265 26070 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.433288 26012 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.414059 26082 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:10:39.411804 26141 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:39.404023 26090 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:10:39.431735 26065 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.445495 26013 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:39.539776 26141 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.560599 26126 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.563575 26070 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.563740 26069 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.543951 26125 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:39.565518 26065 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:39.574054 22783 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:39.575963 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:39.578435 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:39.609787 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:39.610867 23421 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:39.546180 25768 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.620347 26141 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.621287 26069 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:39.621996 26069 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.629876 26132 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.630035 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.631755 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2271) took 1072 ms. Trace:
I20260709 13:10:39.632251 22710 rpcz_store.cc:276] 0709 13:10:38.559017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.934314 (+375297us) service_pool.cc:224] Handling call
0709 13:10:38.967476 (+ 33162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.967717 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:10:38.968059 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.631372 (+663313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:39.631441 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:39.631497 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.631570 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.631690 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":562817,"prepare.queue_time_us":390,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":99956,"threads_started":1,"wal-append.queue_time_us":172585}
I20260709 13:10:39.632257 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2270) took 1073 ms. Trace:
I20260709 13:10:39.634918 22706 rpcz_store.cc:276] 0709 13:10:38.558684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.917590 (+358906us) service_pool.cc:224] Handling call
0709 13:10:38.917973 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.918214 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:38.918567 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.631676 (+713109us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.631912 (+   236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:39.631963 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:39.632011 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.632084 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.632195 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":560482,"prepare.queue_time_us":369,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":149779,"threads_started":1,"wal-append.queue_time_us":263370}
I20260709 13:10:39.654170 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.656839 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.661301 22629 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:39.665831 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 416) took 1082 ms. Trace:
I20260709 13:10:39.666275 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 418) took 1082 ms. Trace:
I20260709 13:10:39.666239 22704 rpcz_store.cc:276] 0709 13:10:38.583318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.949564 (+366246us) service_pool.cc:224] Handling call
0709 13:10:38.949948 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:38.957450 (+  7502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:38.957502 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:38.958207 (+   705us) log.cc:844] Serialized 26 byte log entry
0709 13:10:39.078680 (+120473us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:39.192350 (+113670us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.665525 (+473175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.665604 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:39.665645 (+    41us) raft_consensus.cc:1685] finished
0709 13:10:39.665658 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.665775 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495771,"prepare.queue_time_us":119833,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":7110,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":13725,"thread_start_us":49647,"threads_started":1,"wal-append.queue_time_us":159670}
I20260709 13:10:39.668656 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:39.665271 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 125505us
I20260709 13:10:39.666757 22709 rpcz_store.cc:276] 0709 13:10:38.583947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.012008 (+428061us) service_pool.cc:224] Handling call
0709 13:10:39.012571 (+   563us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:39.030079 (+ 17508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.030127 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:39.031013 (+   886us) log.cc:844] Serialized 52 byte log entry
0709 13:10:39.167305 (+136292us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:39.198736 (+ 31431us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.665749 (+467013us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.665972 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.666046 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:39.666085 (+    39us) raft_consensus.cc:1685] finished
0709 13:10:39.666096 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.666218 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1105600,"prepare.queue_time_us":31309,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":13225,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":123,"thread_start_us":115077,"threads_started":1,"wal-append.queue_time_us":115494}
I20260709 13:10:39.624943 26069 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.671579 23421 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:39.680584 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.673652 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:39.685446 22778 raft_consensus.cc:2368] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
W20260709 13:10:39.687779 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 467163us
I20260709 13:10:39.688604 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:39.689640 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.695151 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.696240 22778 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:39.697592 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.698668 22778 raft_consensus.cc:2376] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13. Re-sending same reply.
I20260709 13:10:39.701686 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:39.702992 22629 raft_consensus.cc:2410] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:10:39.723850 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:39.695163 25750 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:39.686866 26128 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.732901 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.687539 26082 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:39.734162 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.721481 25837 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:39.745397 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.751205 22626 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:39.740725 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.756711 22631 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:39.721853 26085 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.755677 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.664757 26132 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.728171 26127 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.763334 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.765332 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.767135 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:39.764787 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.773836 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:39.781780 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.784799 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.785909 22709 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:39.788970 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.760120 26124 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:39.793107 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.766734 26131 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:39.772099 26084 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:39.808583 22709 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:10:39.806365 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:39.772478 26012 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:39.814781 26155 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:10:39.820137 26036 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:39.823226 26125 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.828936 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.803089 25837 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:39.835275 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:39.758284 26082 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.774691 26127 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.800859 26129 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:39.843884 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 422) took 1255 ms. Trace:
I20260709 13:10:39.809944 25750 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:39.799230 26124 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.752310 26128 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.799681 26162 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:39.767170 26085 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:39.820235 26149 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:10:39.841213 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.813630 26084 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.867564 22705 rpcz_store.cc:276] 0709 13:10:38.588338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.828581 (+1240243us) service_pool.cc:224] Handling call
0709 13:10:39.843804 (+ 15223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:39.833781 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:39.851038 26075 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:39.844874 26036 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.849143 25837 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.887676 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2277) took 1126 ms. Trace:
I20260709 13:10:39.892031 26132 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:39.892580 26084 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:39.870546 26124 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:39.850113 26125 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:39.886806 26162 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.903494 22708 rpcz_store.cc:276] 0709 13:10:38.761594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.395622 (+634028us) service_pool.cc:224] Handling call
0709 13:10:39.411264 (+ 15642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:39.411484 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:39.436740 (+ 25256us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.887235 (+450495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:39.887303 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:39.887357 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.887450 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.887606 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201693,"prepare.queue_time_us":15108,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":2276,"raft.queue_time_us":244104,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8259,"thread_start_us":248134,"threads_started":2,"wal-append.queue_time_us":180023}
I20260709 13:10:39.909003 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 582) took 1123 ms. Trace:
I20260709 13:10:39.910183 23415 rpcz_store.cc:276] 0709 13:10:38.785193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.785386 (+   193us) service_pool.cc:224] Handling call
0709 13:10:38.785747 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.851872 (+ 66125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:38.851907 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:38.851955 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:38.851987 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.908668 (+1056681us) spinlock_profiling.cc:243] Waited 65.9 ms on lock 0x7b5400503554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:39.908727 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.908929 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151550208}
I20260709 13:10:39.899580 25837 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:39.895079 26036 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:39.923794 22704 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:39.869861 26129 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.904064 26075 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:39.927054 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 242268us
I20260709 13:10:39.907814 26132 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:39.937649 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:39.908818 26162 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:39.928814 26129 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:39.955892 26132 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:39.990080 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:39.927959 25750 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:39.990677 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57218.1us
I20260709 13:10:39.988560 26075 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.014986 26082 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:40.016350 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:40.020370 26085 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:40.021731 26127 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:40.023849 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.030402 26132 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:40.030738 26164 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:10:40.032096 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2092) took 1003 ms. Trace:
I20260709 13:10:40.032415 26125 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.032542 22707 rpcz_store.cc:276] 0709 13:10:39.028355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.435124 (+406769us) service_pool.cc:224] Handling call
0709 13:10:39.435650 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.695457 (+259807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.695497 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.722096 (+ 26599us) log.cc:844] Serialized 26 byte log entry
0709 13:10:39.845716 (+123620us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:39.845764 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.845841 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.031823 (+185982us) raft_consensus.cc:1685] finished
0709 13:10:40.031850 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.032021 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":259237,"raft.queue_time_us":291178,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":11084,"thread_start_us":78158,"threads_started":1,"wal-append.queue_time_us":126934}
I20260709 13:10:40.044406 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:40.036249 26030 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.000s
I20260709 13:10:40.060035 26128 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.060869 26127 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.061622 26127 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.064147 26127 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:40.067122 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 73731.1us
I20260709 13:10:40.084956 22780 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:40.087893 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.091972 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.078838 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:40.088567 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.078475 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.087075 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.094375 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.092408 23384 leader_election.cc:400] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
I20260709 13:10:40.093577 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.096774 22631 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:40.096165 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.097263 23384 leader_election.cc:403] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:10:40.099305 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.093155 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.102190 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.066831 26125 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.104418 26125 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.107146 26125 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.108772 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:40.110832 26169 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
I20260709 13:10:40.111685 26169 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:40.112037 26169 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.112432 26169 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:40.112768 25837 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.115551 25990 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:40.117280 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.099613 26128 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:40.118813 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.119213 26128 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.120072 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.099825 26027 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:40.085278 26082 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:10:40.080633 26160 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.193s	user 0.003s	sys 0.001s
I20260709 13:10:40.102933 22779 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:10:40.071202 25627 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.126984 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.127686 22779 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:40.129493 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:40.176738 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73926.4us
I20260709 13:10:40.169639 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:40.186764 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.100451 26127 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:10:40.232455 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54668.1us
W20260709 13:10:40.236117 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57937.3us
I20260709 13:10:40.238382 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.233388 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.265211 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.268227 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:40.179903 26129 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.066466 26162 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.272303 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:40.283484 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50206.8us
I20260709 13:10:40.285517 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.224745 25837 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.180740 25627 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.286819 26027 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:40.286720 25990 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:40.255703 26132 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:40.226471 26028 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.288228 26168 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:40.290002 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.320434 26128 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.227579 26127 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.321585 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.290549 26069 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:40.326424 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2268) took 1768 ms. Trace:
I20260709 13:10:40.284498 26131 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:40.326920 22703 rpcz_store.cc:276] 0709 13:10:38.558017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.807458 (+249441us) service_pool.cc:224] Handling call
0709 13:10:38.807972 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.404657 (+596685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.326088 (+921431us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b5400314f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:40.326336 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074706688}
I20260709 13:10:40.293723 22781 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:40.329085 26162 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.329919 22706 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:40.227839 26085 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.321688 26127 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:40.332470 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2327) took 1226 ms. Trace:
I20260709 13:10:40.333669 22783 rpcz_store.cc:276] 0709 13:10:39.105786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.107492 (+  1706us) service_pool.cc:224] Handling call
0709 13:10:40.332271 (+1224779us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400341cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:40.332395 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775553280}
I20260709 13:10:40.340553 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:40.299153 25990 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.313719 26129 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.346678 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:40.347174 22780 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:40.311141 26132 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.333107 26126 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:40.399798 23421 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:10:40.396471 26069 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:40.345882 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 77757.9us
I20260709 13:10:40.309882 26084 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.348315 25837 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:40.408150 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.316860 26124 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.399580 26132 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:40.336930 26085 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.353634 26168 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.401324 26162 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.311652 26028 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.411484 26082 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.415230 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:40.418843 26129 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:40.424108 26131 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.443287 26128 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:40.501170 22702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2073) took 2408 ms (client timeout 1741 ms). Trace:
W20260709 13:10:40.501919 22702 rpcz_store.cc:269] 0709 13:10:38.077706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:38.089156 (+ 11450us) service_pool.cc:224] Handling call
0709 13:10:40.484865 (+2395709us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b5400086e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:40.486176 (+  1311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755890432}
W20260709 13:10:40.508687 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93573.2us
I20260709 13:10:40.509577 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.511883 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.472589 26082 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.451542 26162 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.518762 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.452198 26168 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.521279 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.520248 26129 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:40.522492 25837 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:40.539384 25837 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:40.540328 25837 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:40.540891 25837 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.543663 26084 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.512938 26123 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:40.513768 26166 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:40.552416 25990 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:40.561682 26036 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.622s	user 0.000s	sys 0.013s
I20260709 13:10:40.550199 26124 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.549873 26162 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.580055 26085 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.580363 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2326) took 1474 ms. Trace:
I20260709 13:10:40.581095 26028 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:40.580741 22777 rpcz_store.cc:276] 0709 13:10:39.105525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.106993 (+  1468us) service_pool.cc:224] Handling call
0709 13:10:39.222507 (+115514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:39.224316 (+  1809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.224346 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:39.224395 (+    49us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:39.224427 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.224487 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.580143 (+1355656us) spinlock_profiling.cc:243] Waited 97.9 ms on lock 0x7b54000cba50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:40.580288 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225152256}
W20260709 13:10:40.571537 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 334630us
I20260709 13:10:40.505057 26131 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:40.625429 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 104200us
I20260709 13:10:40.626480 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.629945 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.506242 26128 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.571650 26038 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:10:40.551070 26169 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.640831 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.640991 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.643730 22783 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:40.644014 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.644855 22780 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:40.645653 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.646302 22781 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:40.628227 26028 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.584775 26036 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.653652 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.654747 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.654958 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.655206 22781 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:40.655758 22776 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:40.656514 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.635774 26085 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:40.654501 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.658665 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:40.632056 26181 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:40.659368 22783 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:40.658046 26067 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:40.675971 26067 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:40.677152 26067 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.678256 26067 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:40.679819 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2203) took 1452 ms. Trace:
I20260709 13:10:40.680446 26036 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.638494 26162 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:10:40.664804 23419 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:40.653509 25990 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:40.664966 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:40.668357 26158 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.823s	user 0.010s	sys 0.013s
I20260709 13:10:40.672773 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:40.687288 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.688001 22704 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
W20260709 13:10:40.674269 26156 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.914s	user 0.009s	sys 0.015s
I20260709 13:10:40.681916 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2197) took 1459 ms. Trace:
I20260709 13:10:40.653152 26038 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:40.680284 22633 rpcz_store.cc:276] 0709 13:10:39.226911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.227644 (+   733us) service_pool.cc:224] Handling call
0709 13:10:39.227936 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:39.228146 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:39.411780 (+183634us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.418380 (+  6600us) log.cc:844] Serialized 14 byte log entry
0709 13:10:40.679214 (+1260834us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b18002fae80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:10:40.679406 (+   192us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:40.679467 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:40.679520 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.679604 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.679751 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20358,"prepare.queue_time_us":19903,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":119,"raft.queue_time_us":1280422,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":4542,"spinlock_wait_cycles":14835968,"thread_start_us":169488,"threads_started":1}
I20260709 13:10:40.689134 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.653313 26125 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.689020 22630 rpcz_store.cc:276] 0709 13:10:39.223020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.223658 (+   638us) service_pool.cc:224] Handling call
0709 13:10:39.223973 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:39.322440 (+ 98467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.322476 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:39.322525 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:39.322556 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.681654 (+1359098us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b54002f4c54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:40.681706 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.681848 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225874816}
W20260709 13:10:40.663335 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 46415.1us
I20260709 13:10:40.694110 26085 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:40.694945 26177 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:40.700285 22783 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:40.693207 26165 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:40.704398 25627 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:40.705209 22779 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:40.658756 26028 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.708081 25750 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:40.708516 25750 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:40.709215 26028 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:40.709544 25750 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.711696 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2330) took 1603 ms. Trace:
I20260709 13:10:40.712853 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:40.736882 22777 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:40.738817 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:40.739809 23421 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:40.740484 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.663699 25837 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:40.745270 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.06217e+06us
I20260709 13:10:40.712179 22775 rpcz_store.cc:276] 0709 13:10:39.108508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.109166 (+   658us) service_pool.cc:224] Handling call
0709 13:10:39.109503 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:40.008629 (+899126us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:40.018839 (+ 10210us) log.cc:844] Serialized 14 byte log entry
0709 13:10:40.237424 (+218585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:40.237498 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:40.237561 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.711436 (+473875us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b540033d454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:40.711489 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.711623 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106640,"prepare.queue_time_us":10573,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":139616,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8218,"spinlock_wait_cycles":2065084672,"thread_start_us":111434,"threads_started":1,"wal-append.queue_time_us":155569}
W20260709 13:10:40.747673 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58617.6us
I20260709 13:10:40.748517 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:40.745581 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:40.781998 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.754769 26085 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:40.754940 25755 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:40.787822 25755 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:40.714066 26141 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:40.717382 26125 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.691172 26038 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:40.759470 25837 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:40.750417 25853 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.760344 26012 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:40.761432 26082 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:40.763127 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2345) took 1315 ms. Trace:
I20260709 13:10:40.764238 26131 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.771178 26168 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.773350 26169 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:40.774837 26128 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:40.775185 26032 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:40.777261 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.781052 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.706123 26132 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.784313 22633 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:10:40.784938 26075 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.785733 26030 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.785549 26127 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.786831 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.792162 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.793562 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:40.793651 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.793922 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.843150 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:40.843502 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49987.4us
I20260709 13:10:40.794636 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.795331 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:40.764295 22532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 143) took 27716 ms (client timeout 15000 ms). Trace:
I20260709 13:10:40.841751 22635 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:40.781224 26028 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.805676 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.808209 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.750763 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:40.810775 22701 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
W20260709 13:10:40.849990 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72852.8us
W20260709 13:10:40.850273 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 99639us
I20260709 13:10:40.812968 22707 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:40.779426 26126 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.813972 22778 rpcz_store.cc:276] 0709 13:10:39.448061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.719098 (+271037us) service_pool.cc:224] Handling call
0709 13:10:39.719688 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.720303 (+   615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.720349 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.741151 (+ 20802us) log.cc:844] Serialized 26 byte log entry
0709 13:10:39.972387 (+231236us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:39.987688 (+ 15301us) log.cc:844] Serialized 14 byte log entry
0709 13:10:39.987979 (+   291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.988070 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.700817 (+712747us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b18002b0180. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:10:40.762785 (+ 61968us) raft_consensus.cc:1685] finished
0709 13:10:40.762902 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.763059 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15036,"prepare.run_cpu_time_us":830,"prepare.run_wall_time_us":15481,"raft.queue_time_us":2560,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":14735,"spinlock_wait_cycles":52781312,"thread_start_us":184937,"threads_started":1,"wal-append.queue_time_us":267289}
I20260709 13:10:40.788245 25755 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.789505 25750 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:40.826504 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:40.861562 22707 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:40.861871 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.890640 22702 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:40.833977 22630 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:40.892763 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.889987 22776 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
W20260709 13:10:40.835572 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:40.805161 26141 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.900395 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:40.846761 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:40.848227 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 155900us
I20260709 13:10:40.853281 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.901336 22775 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:40.902526 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.903821 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.853940 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:40.830085 26182 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:40.907275 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.908291 22709 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:10:40.908821 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.889780 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.822540 25853 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:40.914131 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.911876 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.906901 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:40.850977 26165 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:40.904997 22629 raft_consensus.cc:2376] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13. Re-sending same reply.
I20260709 13:10:40.910084 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.846048 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.912346 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.912858 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.916002 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.830096 25768 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:10:40.859207 26027 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:40.916052 22633 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:40.883328 26012 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:40.919320 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.882005 25803 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:40.848131 26030 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.893788 26128 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:40.883280 25755 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:40.985507 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 75491.2us
I20260709 13:10:40.918079 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:40.998260 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:40.840739 26111 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.143s
I20260709 13:10:41.003770 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.006479 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:40.860421 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:41.010283 22630 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:40.899593 26168 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:40.843443 26075 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.014181 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:41.065407 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51339.3us
I20260709 13:10:41.066152 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.913002 26131 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:40.917794 26085 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:40.917282 26162 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:41.015208 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 160323us
I20260709 13:10:41.078553 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:40.888927 26126 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.913249 25837 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:40.990702 26027 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:40.844852 22532 rpcz_store.cc:269] 0709 13:10:13.058959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.062797 (+  3838us) service_pool.cc:224] Handling call
0709 13:10:40.764028 (+27701231us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:31.870315 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:31.991891 (+121576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:31.998884 (+  6993us) write_op.cc:435] Acquired schema lock
0709 13:10:32.006141 (+  7257us) tablet.cc:662] Decoding operations
0709 13:10:32.168666 (+162525us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:32.168797 (+   131us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:32.168827 (+    30us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:10:32.169810 (+   983us) tablet.cc:701] Row locks acquired
0709 13:10:32.169870 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:10:32.170115 (+   245us) write_op.cc:270] Start()
0709 13:10:32.170321 (+   206us) write_op.cc:276] Timestamp: P: 1783602632170036 usec, L: 0
0709 13:10:32.170358 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:10:32.260393 (+ 90035us) log.cc:844] Serialized 3190 byte log entry
0709 13:10:33.358878 (+1098485us) op_driver.cc:464] REPLICATION: finished
0709 13:10:33.641384 (+282506us) write_op.cc:301] APPLY: starting
0709 13:10:33.646989 (+  5605us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:33.647264 (+   275us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:33.647408 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:33.648651 (+  1243us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:33.648954 (+   303us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:10:33.648999 (+    45us) write_op.cc:312] APPLY: finished
0709 13:10:33.649375 (+   376us) log.cc:844] Serialized 65 byte log entry
0709 13:10:33.649587 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:33.649742 (+   155us) write_op.cc:454] Released schema lock
0709 13:10:33.649811 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:33.650267 (+   456us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:33.650293 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":153337,"reported_tablets":24,"spinlock_wait_cycles":26240,"child_traces":[["op",{"apply.queue_time_us":240544,"apply.run_cpu_time_us":3522,"apply.run_wall_time_us":33981,"num_ops":8,"prepare.queue_time_us":39577,"prepare.run_cpu_time_us":27476,"prepare.run_wall_time_us":581997,"raft.queue_time_us":300398,"raft.run_cpu_time_us":14918,"raft.run_wall_time_us":284300,"replication_time_us":1095206,"thread_start_us":571552,"threads_started":4,"wal-append.queue_time_us":237523,"wal-append.run_cpu_time_us":97774,"wal-append.run_wall_time_us":2169835}]]}
I20260709 13:10:40.883584 26028 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:40.997416 25990 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:40.983147 25903 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:10:41.098176 25903 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:41.098712 25903 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.091176 26027 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:41.099763 25903 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.101404 25903 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:40.883581 26038 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.103338 22703 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:41.103691 26038 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:40.992710 26012 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:41.104576 26012 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:41.017911 26036 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:40.905797 26182 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:41.072813 26085 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:41.107864 26182 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.108908 26085 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:41.109712 26182 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:41.110950 26016 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:10:41.111173 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45128.9us
I20260709 13:10:41.111608 26016 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.112560 26016 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.016882 25803 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.070276 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.070636 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:41.090288 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:41.120221 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49696.2us
I20260709 13:10:41.121074 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:41.062435 25902 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.058588 26082 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.072432 26047 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:41.103513 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.019253 26128 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:41.125939 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:41.152976 22775 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:41.126526 25837 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:41.126173 25902 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.127179 26048 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:41.127686 26029 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:41.127923 26125 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.128427 26194 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.133644 26027 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.134859 26105 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:41.136211 26075 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.136653 25990 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:10:41.136843 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46665.7us
I20260709 13:10:41.137053 26196 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:41.158531 25837 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.140458 26127 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.141228 26132 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.142645 25803 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.089463 26069 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.145334 26036 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:41.078504 26165 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.073786 26101 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.177776 26180 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.235147 26101 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.086716 26184 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:41.153594 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.154016 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.237327 26028 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.238243 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.239181 22704 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:41.239647 26128 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.240242 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.241514 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.242111 22704 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:41.070996 26168 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.242738 26125 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.085666 26131 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.176496 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:41.178557 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.191141 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 275199us
I20260709 13:10:41.139137 26030 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.215566 23419 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:41.168568 26162 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.219560 25990 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.255326 26162 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.231511 25938 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:41.256035 25902 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.256417 25938 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.223071 25804 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:41.257037 26085 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.257828 26162 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.257535 25938 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.260470 25938 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:41.261083 26131 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.161777 22778 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:41.175336 25843 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:10:41.157526 26082 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.261788 22701 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:41.255446 25990 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.164965 26138 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:10:41.263912 26082 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.263985 26101 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:41.266912 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 113412us
I20260709 13:10:41.235363 26180 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:41.268489 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.266870 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:41.271531 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 366196us
I20260709 13:10:41.213594 25837 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:41.288558 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 110087us
I20260709 13:10:41.280967 22704 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:41.212749 26195 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.200752 26196 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.290012 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.239224 26027 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.294515 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.300613 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.223546 26036 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.320078 26036 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:41.321532 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.244534 26127 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.323109 26127 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.252810 26069 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.327855 25804 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:41.328505 25804 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:41.328868 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:41.328922 26105 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:41.331107 26105 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.243351 26125 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.332183 26105 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.332876 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.333477 26105 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:41.334530 26125 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.335274 25843 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:41.335716 25843 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.336621 25843 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.337908 25843 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.279915 26102 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:41.341912 22702 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:41.345767 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78016.2us
I20260709 13:10:41.330413 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.348145 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.349159 26180 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.349934 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.315934 26067 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.004s
I20260709 13:10:41.351191 25992 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:41.353045 25992 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:41.298254 26101 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:41.177767 26194 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.354351 26101 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.356254 25533 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:41.356581 26101 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.356762 25533 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.311892 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:41.357474 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63332.4us
W20260709 13:10:41.357788 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46004.8us
I20260709 13:10:41.266136 26141 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.360648 26012 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.316282 26030 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:41.362697 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.370425 22706 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:41.334425 22701 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:41.350193 22703 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:41.302287 26195 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.362816 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.378718 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:41.380012 22778 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:41.380580 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.299710 22704 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:41.331156 25853 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.367708 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:41.369428 22630 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:41.264071 25990 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.386523 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.387871 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:41.389757 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.392515 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.297987 26082 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:41.372426 25992 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:41.394845 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.399753 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.400861 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.394083 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.402455 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:41.404585 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.407094 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.406433 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.409602 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.410219 23421 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:41.301214 26196 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:41.393286 23419 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:41.422418 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.422860 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:41.423646 22631 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:41.425298 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.426867 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.376752 26127 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.427801 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.411176 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.302029 26188 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.396708 25803 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:41.441167 26162 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:10:41.416301 25990 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:41.396919 26067 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:41.416682 26195 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.393009 26180 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:41.418283 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.400296 26125 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.418341 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:41.430604 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.473301 26069 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.475164 26126 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.476838 26090 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:41.477465 26090 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.489781 26098 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:41.495046 22783 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:41.490564 26098 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.519181 26098 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.520685 26098 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:41.432406 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.432753 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.435508 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.436297 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.440465 26194 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.449749 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.397164 26203 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:41.457228 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.407280 26177 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:41.410486 26189 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:41.529757 26189 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.530438 26189 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.396961 26101 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:41.535220 26090 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.536615 26101 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.397689 25853 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.539345 25853 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.419428 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.464545 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.410885 26027 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.544351 26027 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.410656 26198 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:41.537235 26090 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:41.548331 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.443924 26208 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.559247 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:41.552775 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:41.560847 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 142592us
I20260709 13:10:41.561120 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.561901 22631 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:41.547067 26198 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:41.551600 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 157614us
I20260709 13:10:41.572005 22633 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:41.562453 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.563021 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.563658 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.563952 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:41.578665 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.357811 25533 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.568751 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.570117 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:41.584878 22708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.029s
W20260709 13:10:41.609338 22708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.004s	sys 0.029s
I20260709 13:10:41.587630 22673 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:41.567507 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:41.591948 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:41.462220 26180 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:41.605811 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54111.3us
W20260709 13:10:41.622891 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.615924 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.627050 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54432.6us
I20260709 13:10:41.555140 26031 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:41.559974 26082 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.628255 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:41.643893 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.572881 26085 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.569505 26069 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.576737 25902 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:41.653268 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.577034 26198 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:41.656920 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.651203 26190 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:10:41.661238 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.654649 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59874.9us
I20260709 13:10:41.670293 26138 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:41.672041 25435 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:41.672475 25435 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.585682 26162 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.674333 26162 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.666142 22707 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:41.682783 26069 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.687304 26027 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.622732 26168 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.637984 26180 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:41.673318 25435 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.688042 26030 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:41.642364 26031 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:41.629393 26032 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:10:41.691596 26199 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:41.743695 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.744725 22710 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:41.555297 26125 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.638134 26013 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:41.693800 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.763042 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69313.3us
I20260709 13:10:41.694134 26047 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:41.686275 26069 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:41.458750 26188 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.708174 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.688997 26127 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.651854 26072 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.660794 22630 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
W20260709 13:10:41.703085 25750 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.791s	user 0.011s	sys 0.002s
I20260709 13:10:41.652405 25990 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.809007 23420 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:41.658098 26126 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:10:41.658668 26194 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:41.648602 25627 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:10:41.699203 26197 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:41.757980 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.760710 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:41.707156 25533 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.775185 26030 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.764712 22749 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:41.775871 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.772017 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 371515us
I20260709 13:10:41.805332 22709 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:41.577572 26208 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.769570 26195 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:41.823004 22701 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:41.767472 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:41.867342 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 91641.3us
I20260709 13:10:41.856763 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:41.805652 25902 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:41.868603 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.819310 26181 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:41.838826 26212 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:10:41.871186 22634 raft_consensus.cc:2368] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
I20260709 13:10:41.902736 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:41.902225 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 314726us
I20260709 13:10:41.838136 25435 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:41.815567 26012 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:41.808736 26072 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.912606 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:41.858806 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94163.6us
I20260709 13:10:41.913314 23419 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:41.813921 26036 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.818027 25990 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:41.863663 25755 consensus_meta.cc:287] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.838s	user 0.012s	sys 0.003s
I20260709 13:10:41.813709 26194 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:41.874850 26141 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:41.809861 26127 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.904103 22673 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:41.870751 25902 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:41.915129 22781 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:41.864531 26030 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:41.874434 26126 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:41.873972 26199 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:10:41.825098 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.917971 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:41.919991 23384 leader_election.cc:400] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
I20260709 13:10:41.910471 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:41.976980 23384 leader_election.cc:403] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:10:41.977543 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67193.7us
I20260709 13:10:41.899276 26101 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:41.933467 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:41.914434 25627 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:41.921453 26194 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:41.997963 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.000358 22780 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:41.929333 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:41.973922 26209 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:10:42.006961 22778 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:10:42.007516 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1754) took 1140 ms. Trace:
I20260709 13:10:42.008116 22777 rpcz_store.cc:276] 0709 13:10:40.867126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:40.867296 (+   170us) service_pool.cc:224] Handling call
0709 13:10:40.867706 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:41.022400 (+154694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:41.022474 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:41.360557 (+338083us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:41.361123 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 13:10:41.818343 (+457220us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:41.818381 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:41.818464 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:41.990471 (+172007us) raft_consensus.cc:1685] finished
0709 13:10:41.990497 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.007423 (+ 16926us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35107,"raft.queue_time_us":146145,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":20387,"spinlock_wait_cycles":8713856,"thread_start_us":577668,"threads_started":2,"wal-append.queue_time_us":560098}
I20260709 13:10:42.010468 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:42.011065 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2292) took 1166 ms. Trace:
I20260709 13:10:42.011427 22705 rpcz_store.cc:276] 0709 13:10:40.844503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:40.844655 (+   152us) service_pool.cc:224] Handling call
0709 13:10:40.844967 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:40.845183 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:41.367862 (+522679us) log.cc:844] Serialized 14 byte log entry
0709 13:10:41.760048 (+392186us) log.cc:844] Serialized 14 byte log entry
0709 13:10:41.993575 (+233527us) spinlock_profiling.cc:243] Waited 4.89 ms on lock 0x7b180039d660. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:10:42.010718 (+ 17143us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:42.010772 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:42.010820 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.010894 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.011008 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44176,"prepare.queue_time_us":304388,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":97,"raft.queue_time_us":1973,"raft.run_wall_time_us":16993,"spinlock_wait_cycles":51142912,"thread_start_us":850354,"threads_started":2,"wal-append.queue_time_us":541865}
I20260709 13:10:41.972244 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.987545 26188 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:41.988708 26211 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:10:42.005880 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86045.5us
I20260709 13:10:42.012744 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:42.022804 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:42.027489 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:42.028683 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 124672us
I20260709 13:10:42.029755 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:42.031199 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59306.8us
I20260709 13:10:42.033282 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:41.990092 26190 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:42.038911 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:41.775460 26031 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:42.048110 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 64159.4us
W20260709 13:10:42.049563 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.049494 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:41.995746 26199 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:42.069235 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:41.927551 25755 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:41.908955 26191 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:10:42.070680 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:42.070010 26220 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.049750 26101 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:42.071753 26085 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:42.072348 26128 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:42.074787 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:42.051023 25837 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.004s
W20260709 13:10:42.068753 26210 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 13:10:42.045471 26168 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:42.095963 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:42.101218 23417 raft_consensus.cc:2368] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9.
W20260709 13:10:42.104863 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87987.9us
I20260709 13:10:42.044029 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:42.114388 22634 raft_consensus.cc:606] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:42.085548 26197 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:42.118254 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44782us
I20260709 13:10:42.089450 26141 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.121356 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:42.127555 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:42.095664 26132 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:42.072676 25760 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:10:42.099480 26222 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:42.099834 26168 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:42.080312 26221 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:10:42.125519 26214 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
W20260709 13:10:42.153811 26038 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.049s	user 0.008s	sys 0.004s
W20260709 13:10:42.155407 26206 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:10:42.147795 23419 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:42.094584 25837 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.133690 26085 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:42.161437 25750 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10.
W20260709 13:10:42.169445 26219 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:10:42.137629 26168 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:42.175904 26125 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:42.172926 26205 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 13:10:42.159632 25903 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.058s	user 0.009s	sys 0.009s
W20260709 13:10:42.171271 25843 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.005s
I20260709 13:10:42.137945 26141 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:42.187040 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81168.6us
I20260709 13:10:42.090194 26220 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.190814 25990 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.193135 26036 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.125864 26197 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.144739 26199 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.314437 26016 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:42.576407 26016 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:42.576918 23411 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:10:42.334867 26090 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.007s
I20260709 13:10:42.422750 26168 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:42.228724 26105 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.895s	user 0.014s	sys 0.000s
I20260709 13:10:42.409039 26194 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.584223 26105 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.585551 26029 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.194000 26072 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:42.254233 26182 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.144s	user 0.017s	sys 0.000s
I20260709 13:10:42.587350 26072 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.588235 26072 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.343324 26198 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.007s
I20260709 13:10:42.589718 26028 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.589855 26198 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.347025 26031 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:42.591223 26191 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:42.197055 26127 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.277657 26047 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:42.605755 26048 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.547621 26238 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:42.585386 26105 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.597827 26197 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.391961 26190 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:42.600281 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:42.223798 26224 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:10:42.247138 26196 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.803s	user 0.015s	sys 0.004s
I20260709 13:10:42.606981 26048 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.610769 26101 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.611763 26125 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.612838 26131 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.196538 25837 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:42.487433 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 296385us
I20260709 13:10:42.616314 26038 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.617525 23415 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:10:42.208282 26220 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.620198 26221 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:42.315289 26030 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.624058 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1767) took 1062 ms. Trace:
I20260709 13:10:42.624583 25853 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.629602 26194 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.630263 25399 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:42.550045 25755 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.229441 26141 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.558403 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.637959 26168 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.639135 26102 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.512040 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2362) took 1126 ms. Trace:
I20260709 13:10:42.560509 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.514189 22598 leader_election.cc:400] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9.
I20260709 13:10:42.694432 22598 leader_election.cc:403] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:10:42.491820 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 364172us
W20260709 13:10:42.488008 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 438795us
W20260709 13:10:42.506949 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 404846us
I20260709 13:10:42.676905 23415 pending_rounds.cc:85] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 8
W20260709 13:10:42.304787 26227 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:10:42.217905 26082 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.004s
W20260709 13:10:42.340317 26098 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.819s	user 0.010s	sys 0.007s
I20260709 13:10:42.661273 26198 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:42.400312 26180 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.002s
W20260709 13:10:42.198295 25804 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.869s	user 0.015s	sys 0.001s
I20260709 13:10:42.691592 26197 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:42.700677 26102 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:42.345216 26189 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.814s	user 0.010s	sys 0.003s
I20260709 13:10:42.690102 25399 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:42.277534 26207 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:42.714285 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.693665 22779 rpcz_store.cc:276] 0709 13:10:41.386187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.386316 (+   129us) service_pool.cc:224] Handling call
0709 13:10:41.386702 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:41.818578 (+431876us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:41.818645 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:41.818697 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:41.818730 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.511368 (+692638us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540033c054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:42.511455 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.511780 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992591104}
I20260709 13:10:42.685933 22630 rpcz_store.cc:276] 0709 13:10:41.561216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.567114 (+  5898us) service_pool.cc:224] Handling call
0709 13:10:42.623529 (+1056415us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b5400070654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:42.623783 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108934912}
I20260709 13:10:42.695537 23415 pending_rounds.cc:107] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:10:42.721360 23415 raft_consensus.cc:2889] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:42.225973 26162 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.413327 26126 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:42.657557 26028 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.393851 26181 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:42.578383 26016 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.663314 26196 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.249117 26208 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.731350 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:42.741242 22702 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:42.706255 26029 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.705457 26125 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.748689 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 235320us
I20260709 13:10:42.734789 26237 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.760952 23417 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:42.773805 26237 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.761973 25399 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.775205 25399 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:42.772262 22533 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 4, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:10:42.776528 26237 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.766898 26220 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:10:42.777408 22533 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.768504 26180 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.770164 26016 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:42.771685 26102 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.772065 25750 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.772354 26048 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.750676 26131 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.798899 26105 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.800251 26131 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.749286 26038 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.768049 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.781651 26125 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.757973 26221 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.786156 26180 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.745998 26101 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.794303 25750 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.748952 26197 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.811861 26216 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.787349 26189 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.814780 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.796475 26072 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.822400 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 748) took 1981 ms. Trace:
I20260709 13:10:42.826519 23415 rpcz_store.cc:276] 0709 13:10:40.834952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:40.835113 (+   161us) service_pool.cc:224] Handling call
0709 13:10:40.835716 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:42.749138 (+1913422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.815868 (+ 66730us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b54006fe854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:42.816184 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4040207104}
I20260709 13:10:42.815641 26207 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:42.835167 25750 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.845960 26191 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:42.847067 26191 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.848110 25768 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.848955 25768 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.849944 26191 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.847395 25938 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.850906 22702 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:42.851281 22708 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:10:42.851035 25938 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.815972 26216 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.852939 26230 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.853662 25938 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.853602 26230 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.856442 26230 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.856545 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.856804 26102 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.857137 25768 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.857524 22634 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:10:42.854125 26216 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.858364 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.857534 26102 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.859400 22630 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:42.845513 25804 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.860805 25533 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.012s	user 0.008s	sys 0.014s
I20260709 13:10:42.861332 22635 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:42.861864 26102 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.862994 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1773) took 1094 ms. Trace:
I20260709 13:10:42.863162 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:42.863471 22635 rpcz_store.cc:276] 0709 13:10:41.768264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.768439 (+   175us) service_pool.cc:224] Handling call
0709 13:10:41.804292 (+ 35853us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:42.861822 (+1057530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.862595 (+   773us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003d2f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:42.862911 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2429869312}
I20260709 13:10:42.865576 26101 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:42.869141 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 134960us
I20260709 13:10:42.871860 26029 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.872695 26029 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:42.873880 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:42.874557 26029 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:42.894496 26029 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:42.879050 25902 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.889s	user 0.013s	sys 0.008s
I20260709 13:10:42.883226 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.887089 25804 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.889657 26126 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:42.891497 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 80406.1us
I20260709 13:10:42.892097 26197 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.892691 26241 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.892959 26190 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.915928 26180 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.916116 26131 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.893749 26101 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:42.874598 26072 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.887584 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.895578 25902 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.900257 26126 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.901177 26194 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:42.903403 22702 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:42.905917 22673 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:42.911830 26038 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.935642 26029 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.913975 26082 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:42.937304 26029 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:42.940438 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.941532 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.943192 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.944087 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:42.944891 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.925200 26241 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.916160 26190 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.916929 26131 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:42.930367 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86500.7us
I20260709 13:10:42.931465 26072 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:42.944090 22635 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:42.932754 26126 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:42.973215 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 99703us
I20260709 13:10:42.946138 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:42.888947 26207 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.951133 26038 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:42.951349 26181 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:42.953598 26131 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.916862 26180 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:42.957792 26190 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.934348 26125 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.961210 25768 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:42.961686 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.961910 26102 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:10:42.964865 26196 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.965881 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.967644 22710 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:42.968446 25902 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:42.971225 22704 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:42.972566 26241 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:42.953441 26028 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:42.980957 26237 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.973524 26162 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.983192 26126 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:42.983718 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:42.984853 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:42.985265 22705 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:42.985714 22706 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:42.985967 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.986754 22702 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:10:42.986829 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:42.991523 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:42.991719 26082 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:42.995566 22779 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:10:43.020157 22701 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:43.021066 22707 raft_consensus.cc:2410] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:10:43.021600 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.022158 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:42.961251 26072 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.000381 22635 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:43.003752 26038 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:43.003790 22784 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:43.006839 26125 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.007952 26048 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:10:43.009683 26189 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:43.010370 26207 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:43.011729 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.015188 25837 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.015435 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.022545 22701 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:43.035768 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.024626 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.045706 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.051167 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.052591 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.052764 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.054014 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.054721 22779 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:43.035751 26048 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:43.059549 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 113639us
I20260709 13:10:43.060606 22673 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.062866 25837 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.066844 22635 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:43.072067 22627 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:10:43.072650 22627 pending_rounds.cc:85] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 8
I20260709 13:10:43.072922 22627 pending_rounds.cc:107] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:10:43.073396 22627 raft_consensus.cc:2889] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:43.073500 22632 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:43.074847 22701 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:43.074786 26162 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.076792 22784 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:43.072746 26125 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.078964 26038 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:43.079339 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.079571 26038 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.080174 26038 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:43.082590 22782 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:43.083053 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.084203 22707 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:43.035983 26028 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:43.087200 22701 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:43.087908 22780 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:43.088021 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.088894 22705 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:43.090025 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.090830 22705 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:43.091506 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.091943 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.123697 22704 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:43.092633 26082 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.095677 26048 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.138741 25837 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.139024 26048 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:43.140260 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.099066 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.141667 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.023540 26126 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.144192 26126 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:43.144330 26082 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:43.101246 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.023284 26237 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:43.146404 25435 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.236s	user 0.006s	sys 0.019s
I20260709 13:10:43.148345 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.148705 26237 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.150053 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.151410 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:43.153235 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:43.153620 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.154023 23414 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:10:43.156483 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.164294 22707 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:10:43.103508 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.166464 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.103816 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.104830 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.099066 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.110565 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.023092 22533 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 3 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:43.113008 26242 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:10:43.087402 26028 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.113623 26038 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.189718 26038 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.114277 26162 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.190507 26237 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:43.190698 26162 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.191918 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.192076 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.190533 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.192516 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.116973 22701 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:43.191208 26162 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.193202 26125 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.194021 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.194892 22701 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:43.195261 26162 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.046727 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.195952 26162 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:43.195676 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:43.196820 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.197300 26181 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:43.197559 22702 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:43.197768 26181 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.134387 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.198297 26181 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:43.198357 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.189996 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.200352 22779 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:43.195708 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.136775 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.202486 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.191409 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.138002 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:43.203476 22701 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:10:43.173933 26072 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.119701 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.204547 22710 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:43.204080 26072 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.175699 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.205412 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.179085 22708 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:43.206068 22704 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:43.179538 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.206923 22780 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:43.207306 22704 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:43.208421 22708 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:43.209059 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.180070 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.210454 22634 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:10:43.177964 22533 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 7 to 6, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:43.206068 22709 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:43.211895 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.184476 25768 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:43.212585 22776 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:43.212719 22709 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:43.191248 26237 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.114801 26208 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:43.193856 26125 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.213604 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.213686 26208 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.114533 26180 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.189847 26028 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.084096 23415 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:43.178987 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.175246 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.183581 25399 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:43.218529 26028 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:43.219201 26028 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:43.219719 26028 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:43.217808 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52374.9us
W20260709 13:10:43.170748 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 74022.1us
I20260709 13:10:43.240260 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:43.241102 23415 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:43.212272 22533 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 5 to 6, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:43.228047 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.246086 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.249152 22701 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:10:43.246259 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.239455 26072 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:43.257032 22704 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:43.257184 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.247864 22775 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:43.265920 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.267341 22782 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:43.272548 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:43.270388 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:43.276901 22784 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:43.281635 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.283900 22673 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.285053 26031 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:43.285600 26031 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:43.286968 25435 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:43.289533 25435 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:43.289956 25435 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:43.290975 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:43.292073 22632 raft_consensus.cc:606] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.292484 22632 raft_consensus.cc:614] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:43.293048 22779 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 3. Current term is 4. Ops: []
I20260709 13:10:43.286340 22533 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 4 to 6, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:43.295384 26105 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.296164 26105 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.296690 22533 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 1 to 2, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:43.298702 25938 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:43.299715 26237 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.300880 25938 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:43.303192 26102 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:43.303742 26102 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:43.304893 26102 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.309509 22784 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:43.317034 26221 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.317845 26221 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.320487 26221 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.329672 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:43.254271 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.335668 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:43.337112 22708 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:43.335726 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.306524 26102 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:43.339329 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73545.5us
W20260709 13:10:43.344915 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 90732.2us
I20260709 13:10:43.332895 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:43.348599 26126 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:43.353093 26126 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.357573 22627 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:43.302431 25938 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:10:43.359598 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:43.340982 26237 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:43.353494 26208 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.376575 22784 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:43.379509 22634 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:10:43.358124 26072 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:10:43.380817 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 106327us
I20260709 13:10:43.386942 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:43.371254 26220 consensus_meta.cc:287] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.594s	user 0.015s	sys 0.000s
I20260709 13:10:43.363327 26126 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:43.389717 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:43.366847 26221 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:43.390959 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61068.1us
I20260709 13:10:43.391252 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.349530 25902 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:10:43.381910 26180 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:43.357501 26029 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:10:43.431173 26141 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:43.431802 25533 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:43.433367 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.435480 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:43.435310 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.437109 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.370368 25768 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:10:43.388804 25435 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.401623 26162 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:43.444927 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 116794us
W20260709 13:10:43.445909 26194 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.511s	user 0.014s	sys 0.001s
I20260709 13:10:43.394555 26220 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9.
W20260709 13:10:43.399124 26016 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.006s
W20260709 13:10:43.417784 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.423511 26105 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.460911 26180 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:43.408367 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:43.416060 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:43.492816 22704 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:43.286962 26031 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.458261 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.439231 26221 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.453760 26028 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.475235 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.460224 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:43.479969 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 93587.8us
I20260709 13:10:43.475638 25533 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:43.481170 26030 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.481494 22706 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:10:43.481628 25938 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:10:43.484946 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85428.8us
I20260709 13:10:43.498706 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:43.447796 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.499956 22634 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:43.503031 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.508101 22784 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:43.462893 26141 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.388537 26072 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.464287 25399 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:10:43.489161 26194 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:43.495302 26180 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:43.524395 26162 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:43.512480 22626 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
W20260709 13:10:43.518802 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58702.6us
I20260709 13:10:43.527326 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.532904 23415 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:43.538584 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.506083 26241 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.517902 26141 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:43.526891 26072 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:43.534605 25768 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:43.527163 26048 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:43.500231 26030 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.540446 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:43.579504 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.546383 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.579628 22784 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:43.581955 26031 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:43.557394 26241 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.556896 22628 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:43.559636 22600 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.585592 22600 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:43.570148 22710 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:43.569078 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:43.539636 26199 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.015s	sys 0.086s
I20260709 13:10:43.540038 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:43.616129 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79621.5us
W20260709 13:10:43.568045 26252 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.328s	user 0.001s	sys 0.004s
I20260709 13:10:43.617493 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.540016 26082 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.567337 25768 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.546921 26237 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:43.569698 25399 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:10:43.628708 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.617851 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.583091 26030 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:43.576262 26250 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.334s	user 0.002s	sys 0.000s
W20260709 13:10:43.608615 26199 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.015s	sys 0.086s
I20260709 13:10:43.641700 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.607052 26016 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:43.569402 26208 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:43.589272 25804 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:43.661126 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.549489 26207 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:43.623127 26189 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:43.621392 26190 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:43.623483 26220 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:10:43.662034 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44649.8us
I20260709 13:10:43.630402 26101 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.634392 26237 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:43.674355 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:43.635653 25768 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:43.642952 26162 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:43.639082 25399 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.569496 26181 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:43.686317 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 117426us
I20260709 13:10:43.664705 26016 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.666913 25804 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:43.667253 26194 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:43.696235 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:43.677273 26220 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.672662 26189 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:43.638721 26082 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.709414 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.711406 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:43.712167 22632 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:10:43.706434 22779 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:43.677923 26196 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:43.715590 22703 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:43.685043 25399 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:43.721673 26030 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:43.722309 26030 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:43.722841 26030 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:43.704357 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.695019 26016 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:43.706459 26220 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:43.702988 26194 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.706554 26189 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:43.736007 26190 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:10:43.736709 26190 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:43.737761 26190 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.738708 26028 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:43.723022 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:43.690351 26101 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:43.752913 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 265151us
I20260709 13:10:43.733084 26194 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:43.751793 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.758046 22630 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:43.740216 26162 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:43.688817 26207 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.683176 26208 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.706557 26038 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:10:43.692798 25750 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:10:43.741050 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.775787 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:43.788537 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.789124 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.790048 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67168.5us
W20260709 13:10:43.789973 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85720.1us
I20260709 13:10:43.792923 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:43.786299 22632 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:43.787111 22778 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:10:43.803917 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107796us
I20260709 13:10:43.805142 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.806056 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.806768 22705 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:10:43.807991 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.810223 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.778594 26207 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.807327 22705 pending_rounds.cc:85] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:10:43.815475 22705 pending_rounds.cc:107] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:10:43.815828 22705 raft_consensus.cc:2889] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:43.823624 22710 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:43.824891 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:43.826167 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.828482 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.829248 22710 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:10:43.831837 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.831887 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.832976 22632 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:43.798643 26196 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:43.852180 26196 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.837124 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:43.854928 22709 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:43.855319 26181 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:43.847548 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 154562us
I20260709 13:10:43.856526 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:43.865726 26180 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:43.848249 26101 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:43.836597 26262 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.753995 25804 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:43.802217 26199 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:43.834741 26207 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:43.880993 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.885489 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:43.888710 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:43.889525 22632 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:43.874493 26180 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:43.927083 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:44.032853 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 175370us
I20260709 13:10:43.918812 26199 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:44.055308 26188 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:43.881963 26262 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.061832 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.063082 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.063562 22627 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:44.063848 22635 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:44.064978 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:44.066324 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:44.067205 22635 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:44.037479 25804 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:44.084863 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.084932 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.086344 26273 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:10:44.088150 26141 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.507s	user 0.013s	sys 0.000s
I20260709 13:10:44.088749 26141 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.090409 22634 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:44.098848 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.100795 22631 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:44.104888 26141 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:44.109014 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67766.6us
I20260709 13:10:44.110391 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.119282 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:44.127676 26082 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.145771 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:44.149473 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 264102us
W20260709 13:10:44.149593 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 71116.6us
W20260709 13:10:44.151515 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 572736us
I20260709 13:10:44.152323 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.149108 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.167328 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.141172 26180 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:44.195086 26265 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:44.172259 25750 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:10:44.196380 26101 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:44.198566 26272 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:44.215950 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:44.217403 22775 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 4: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:44.232460 26180 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.234112 26208 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.233454 26180 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.255455 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:44.258833 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.154724 25399 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.259943 23411 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:44.260732 22630 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:44.261677 26162 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.201453 26030 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:44.184463 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74237.5us
I20260709 13:10:44.185181 26191 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:44.155927 26082 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.171367 26256 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:44.148569 26141 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:10:44.194829 26254 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.001s
W20260709 13:10:44.244180 26189 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.508s	user 0.008s	sys 0.004s
W20260709 13:10:44.247004 26126 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.844s	user 0.007s	sys 0.008s
I20260709 13:10:44.199193 26272 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:44.154448 26262 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.180284 26125 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.139562 26221 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.250238 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.189798 26237 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.262478 26162 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.266443 26030 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.180658 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.300941 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.267892 22673 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:44.270408 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103204us
W20260709 13:10:44.223179 26072 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.641s	user 0.016s	sys 0.000s
I20260709 13:10:44.304576 26072 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.306794 26276 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:44.307439 26276 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.307853 26276 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:44.310057 26072 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.279708 22632 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:44.281107 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 216015us
I20260709 13:10:44.281126 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:44.311569 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 131011us
I20260709 13:10:44.312724 23417 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:44.313217 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45435.7us
W20260709 13:10:44.284476 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.320001 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:44.277966 25768 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.593s	user 0.016s	sys 0.000s
I20260709 13:10:44.267392 26082 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.302970 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.282472 26180 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:44.275408 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 484284us
I20260709 13:10:44.162750 26220 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.280145 26207 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:44.312424 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.327425 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.333999 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:44.298331 26016 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.006s
I20260709 13:10:44.279836 26189 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.340237 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.340518 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:44.312963 26237 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:44.326360 26125 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:44.343393 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.321154 25399 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.311486 26038 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.344369 23416 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:44.341239 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:44.338495 26256 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:44.341213 26180 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.346169 26272 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.349392 26241 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.351302 26220 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.351742 26255 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:44.329180 26265 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:44.353372 22631 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:44.353503 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.356483 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:44.380018 23415 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:44.343233 26181 raft_consensus.cc:2764] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:10:44.354151 26221 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:44.378700 26283 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.383450 26221 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.358024 26105 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:10:44.332403 25768 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.383674 26283 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.344163 26279 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.344002 26016 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.358299 26101 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:44.361555 26188 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:44.369997 26038 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.279402 26126 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.382141 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.383998 26221 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:44.355532 23384 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 2.198s (SENT) [suppressed 5 similar messages]
I20260709 13:10:44.387566 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:44.346666 26207 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.399906 26188 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:44.401793 26241 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:44.401988 26272 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:44.402585 26241 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:10:44.395306 26101 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.406226 26101 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:44.352506 26125 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.395308 26181 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:44.421066 26181 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.421525 26181 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:44.425797 22628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 473) took 1398 ms (client timeout 1859 ms). Trace:
W20260709 13:10:44.426365 22628 rpcz_store.cc:269] 0709 13:10:43.027213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.027399 (+   186us) service_pool.cc:224] Handling call
0709 13:10:44.425568 (+1398169us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54000b81d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.425716 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1039696768}
I20260709 13:10:44.427840 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.428836 22628 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:44.409031 22629 raft_consensus.cc:606] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.384943 26191 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:44.385344 26180 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.406912 26286 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:10:44.393731 26256 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:10:44.397081 26279 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.395277 26265 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:44.418248 25435 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:44.419415 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1777) took 1009 ms. Trace:
I20260709 13:10:44.420162 26125 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:44.469460 26191 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.470964 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.471866 22628 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:44.473403 22628 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:10:44.404506 26194 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.001s
I20260709 13:10:44.475723 26016 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.477237 26082 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.420531 26275 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.429363 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.453986 22629 raft_consensus.cc:614] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:44.479260 26275 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.480265 22634 raft_consensus.cc:2368] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9.
I20260709 13:10:44.404186 26038 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.458032 22777 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:44.486905 26279 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.487924 25768 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.489776 22628 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:44.427630 26162 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:44.461990 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 84032.2us
I20260709 13:10:44.466150 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:44.467110 22627 rpcz_store.cc:276] 0709 13:10:43.410112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.410408 (+   296us) service_pool.cc:224] Handling call
0709 13:10:44.419158 (+1008750us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400071a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.419333 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1311783680}
I20260709 13:10:44.427762 26101 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.479874 26126 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.504032 26216 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:44.489617 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 133090us
W20260709 13:10:44.447314 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52294.9us
W20260709 13:10:44.458242 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56951.3us
I20260709 13:10:44.499557 26230 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:44.500789 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 448) took 1390 ms. Trace:
I20260709 13:10:44.475749 26194 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.502527 26101 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.502015 25533 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.518113 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:44.517290 22703 rpcz_store.cc:276] 0709 13:10:43.110289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.190969 (+ 80680us) service_pool.cc:224] Handling call
0709 13:10:44.500554 (+1309585us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b540010e354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.500690 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1157681408}
I20260709 13:10:44.449002 26189 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.527328 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:44.528167 22705 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:44.505559 23416 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
W20260709 13:10:44.522567 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 178528us
I20260709 13:10:44.529693 23417 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:10:44.529029 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.530318 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:44.529476 22705 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:44.504067 26275 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.534029 22709 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:10:44.539158 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.542704 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:44.543435 22702 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:44.538192 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.545038 22709 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.546768 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.544443 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.547552 22701 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:44.548288 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:44.548815 22707 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:44.550773 26284 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:44.551352 26284 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.551800 26284 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:44.555502 26273 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.556249 26273 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.547499 22628 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:44.542405 26038 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:44.559449 26194 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.559638 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.524075 26162 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.560379 26162 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.560902 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.562036 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.562969 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.563588 22702 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.520215 26230 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:44.564009 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.565224 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:44.571583 22628 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:44.564436 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.572645 22708 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.547122 26101 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:44.575249 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.577667 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:44.579056 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.505898 26048 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:44.534550 26286 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:44.584259 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:44.583909 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54989.1us
I20260709 13:10:44.585764 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:44.586653 22710 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:44.586830 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.520282 26216 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.544003 26279 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.586440 22705 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 3.
I20260709 13:10:44.581665 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.593899 22628 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:44.596663 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:44.597112 22702 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:44.541736 26016 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 13:10:44.651438 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:44.651948 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 65235.2us
I20260709 13:10:44.654253 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:44.651707 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108169us
I20260709 13:10:44.665989 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:44.679041 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:44.687366 26216 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.580106 26275 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 13:10:44.688550 26162 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.590405 26291 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.690121 26291 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.690985 26283 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.692603 26291 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.584340 26048 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:44.701013 22710 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:44.713485 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:44.714421 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57340.5us
I20260709 13:10:44.588071 26189 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:44.715372 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:44.716605 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:44.719137 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.720242 23411 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:44.721874 22708 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:10:44.723979 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45069.6us
I20260709 13:10:44.725087 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:44.719050 26189 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.742421 26189 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:44.758605 22633 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:44.746897 26028 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:44.759121 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.759704 26028 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.760059 23416 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:44.762076 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.747743 26082 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:44.748088 22709 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:44.749167 26267 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:10:44.749619 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.749982 26016 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.588805 26286 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.783872 26048 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.784936 26267 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.785248 26286 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.785552 26048 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.750782 26280 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 13:10:44.677066 26279 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.695045 26194 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:44.752549 26029 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:44.753712 26276 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.716682 26265 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:44.728142 26199 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.622s	user 0.014s	sys 0.000s
I20260709 13:10:44.746724 26290 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:10:44.763854 26072 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:44.835426 26102 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:44.767591 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.836299 26194 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.836848 26194 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.839385 26290 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:10:44.836861 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69368us
I20260709 13:10:44.767037 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:44.842689 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 80757.3us
W20260709 13:10:44.844157 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 77216.6us
I20260709 13:10:44.818693 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.821575 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:44.715546 26237 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:44.850133 23419 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:44.850517 26237 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.762174 26028 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.785604 26267 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.863809 26267 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.833686 26125 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.867252 22707 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:44.868526 25435 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:44.871574 26125 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.886313 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1789) took 1053 ms. Trace:
I20260709 13:10:44.887183 22630 rpcz_store.cc:276] 0709 13:10:43.833197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.872894 (+ 39697us) service_pool.cc:224] Handling call
0709 13:10:44.885920 (+1013026us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b540022d154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.886187 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864404864}
I20260709 13:10:44.834815 26199 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:44.835937 26102 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:44.831094 26029 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.819321 26082 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:44.890347 26029 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:44.890018 26102 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.890834 26082 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.891947 26102 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:44.850801 26101 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:44.893388 26082 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.894081 26101 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.896787 26279 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.860487 26237 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.898643 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2143) took 1093 ms. Trace:
I20260709 13:10:44.899084 22701 rpcz_store.cc:276] 0709 13:10:43.805463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.805794 (+   331us) service_pool.cc:224] Handling call
0709 13:10:44.898434 (+1092640us) spinlock_profiling.cc:243] Waited 740 ms on lock 0x7b5400086e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.898553 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1703058816}
I20260709 13:10:44.890352 26291 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:44.900631 26291 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:44.901081 26291 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:44.839583 26188 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:44.828656 26016 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.904533 26016 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:44.840916 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:44.905901 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 65100.2us
I20260709 13:10:44.913247 26221 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:44.919116 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1779) took 1454 ms (client timeout 1918 ms). Trace:
I20260709 13:10:44.919235 26181 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.944689 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.922538 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1776) took 1489 ms. Trace:
I20260709 13:10:44.922821 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:44.923548 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 493) took 2994 ms (client timeout 2198 ms). Trace:
I20260709 13:10:44.926074 26180 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:44.927192 26259 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:44.927090 26291 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.928509 26237 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:44.917045 26029 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.972644 22780 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:44.930469 26038 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:10:44.931231 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 349676us
I20260709 13:10:44.932025 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:44.932931 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 363825us
W20260709 13:10:44.936147 22626 rpcz_store.cc:269] 0709 13:10:43.464911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.465083 (+   172us) service_pool.cc:224] Handling call
0709 13:10:44.918911 (+1453828us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b54003a7354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.919042 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36651776}
I20260709 13:10:44.970508 26259 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:44.938717 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:44.975916 22775 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.948192 25804 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:44.945554 22777 rpcz_store.cc:276] 0709 13:10:43.433270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.494573 (+ 61303us) service_pool.cc:224] Handling call
0709 13:10:44.922306 (+1427733us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b540019b254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.922446 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2116090624}
W20260709 13:10:44.947049 22783 rpcz_store.cc:269] 0709 13:10:41.928688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.933081 (+  4393us) service_pool.cc:224] Handling call
0709 13:10:44.923129 (+2990048us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b5400159fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:44.923336 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3930100608}
I20260709 13:10:44.969820 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:44.975178 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:44.921384 26194 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:44.770093 26284 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:44.975792 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.979897 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:44.968135 25399 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.981588 22784 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:44.967952 26241 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:44.984439 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.998117 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:44.991852 22779 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:10:44.995307 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:44.998263 22704 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:10:45.000303 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:44.974520 26290 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:44.976329 26105 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.017330 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.005627 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.018014 23418 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:45.006878 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.018908 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:44.988945 25902 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.028889 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.029126 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.017247 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.019057 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.018172 22778 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:45.026841 26180 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:45.022089 26105 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.032918 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.033432 22782 raft_consensus.cc:2410] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:10:44.993721 25804 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.023849 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.047749 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.031867 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.032119 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:45.027318 26282 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.008s	sys 0.010s
I20260709 13:10:45.049369 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.050328 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.003950 26194 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.055209 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:45.055918 22707 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:10:45.016994 26279 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:44.978761 26259 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.037895 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.041666 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.008054 26191 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.034130 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:45.053665 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.037559 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.056620 22783 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:45.057458 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.075623 22701 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:45.016901 26038 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:45.008904 26241 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:45.059538 22708 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:10:45.054881 26282 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.009s	sys 0.010s
I20260709 13:10:45.063203 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.028200 26016 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:45.066522 22632 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:45.074673 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.080607 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.085670 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.037734 26270 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:45.086522 22634 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:45.033005 26125 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:10:45.086448 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.001914 25399 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.033030 26180 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.083689 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.105288 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.106346 22626 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:45.095366 22777 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.095989 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.096457 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.072744 26207 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:45.089579 26275 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:45.106844 22784 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:45.072561 26196 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:45.072976 26190 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:45.054405 25902 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.090271 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.114217 22707 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:45.099064 26125 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.115051 25804 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:45.122483 26296 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:10:45.123821 26301 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.121519 26125 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:10:45.125345 26237 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.126205 26105 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.137756 26275 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.138211 26275 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:45.139216 22778 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:45.126338 26298 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:45.102896 26304 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:45.087157 26194 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.131155 26181 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.115495 26189 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.132757 26207 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.122570 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.108559 26180 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.135363 26190 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.135259 22632 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:45.159713 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.160544 22784 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:45.134388 26125 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:45.137275 26299 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:10:45.023181 26290 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.137492 22775 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:45.112931 26016 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.153573 22626 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:10:45.094262 26038 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:45.155488 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59132.7us
I20260709 13:10:45.103900 26259 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:45.158581 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:45.172536 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:45.173064 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.133632 25399 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.159657 26207 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.112710 25435 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.167906 22630 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:10:45.112485 26270 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:45.168382 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.178687 26270 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.179603 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:45.163791 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 219203us
I20260709 13:10:45.150667 26082 raft_consensus.cc:2764] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:10:45.140419 26273 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.183851 26241 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.163856 26105 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:45.160316 26221 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.164211 26190 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:45.187461 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.187542 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.170965 26208 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.148541 22779 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:10:45.195758 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 148420us
I20260709 13:10:45.196982 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.198578 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.201572 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:45.202345 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.203815 22779 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:45.206416 26310 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.202000 26315 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:10:45.207237 26310 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.210153 26105 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.210330 26048 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:45.211076 26048 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:45.211690 26305 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:10:45.211575 26105 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.202530 26102 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.218044 26102 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.206319 26181 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:45.229359 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:45.230362 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55329.3us
I20260709 13:10:45.231196 26255 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:45.235446 26255 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.235999 26189 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.236858 26255 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.236939 26125 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.194845 26291 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.274384 22708 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:45.251394 26230 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.252436 26290 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.278411 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:45.238991 26181 raft_consensus.cc:2764] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:10:45.256304 26284 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.237663 26016 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.287520 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.258798 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:45.290158 22701 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:45.259094 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:45.245456 26038 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.202732 26298 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:45.270220 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.295401 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.296502 22706 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.297626 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.298516 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:45.300034 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.301827 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:45.270947 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.303071 22710 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:45.304457 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.306037 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.271775 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.273005 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.273142 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.273460 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.309942 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.274128 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.311305 22632 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:10:45.312515 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.314015 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.198542 26208 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.316514 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.294013 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.317873 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.282390 26031 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:45.254593 25902 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.321015 22632 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:45.321161 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.323827 22704 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.324556 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.326838 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.327077 22630 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.327638 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.274755 26237 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:45.328404 22632 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:45.328374 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.329875 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.329953 22779 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:45.333236 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.330601 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:45.335603 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.337054 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.337916 22630 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:45.287528 26194 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.328341 26028 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:45.343624 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:45.323554 26181 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:45.351501 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.345213 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:45.356027 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.329068 26031 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.329131 26291 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:45.359819 22777 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:45.396579 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:45.364523 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46256us
I20260709 13:10:45.335134 26188 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:45.337203 26072 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:10:45.335440 26267 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:45.336378 26162 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.363085 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:45.365480 26030 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.329005 26125 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.332837 26237 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.334743 26199 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:10:45.344519 26298 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:45.443338 22632 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:45.443394 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.393350 26265 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:45.432071 26267 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:45.432018 26030 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.485437 26082 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.486159 26082 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.414556 26188 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.467828 26276 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.487344 26188 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:45.442785 26277 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.488325 26270 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.489223 26270 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.488375 26277 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.498056 26275 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.445133 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.419809 26101 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:45.444057 26031 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.449501 26208 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.451705 22704 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:45.455690 26180 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.460053 26048 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.467656 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 29) took 1794 ms. Trace:
I20260709 13:10:45.513485 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.415747 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.480181 26265 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.484885 26072 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.524946 22777 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:45.486253 26267 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.441450 26237 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:45.515695 22626 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:45.507930 22779 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:45.516727 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 47309.6us
I20260709 13:10:45.511816 22629 rpcz_store.cc:276] 0709 13:10:43.673602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.674169 (+   567us) service_pool.cc:224] Handling call
0709 13:10:45.467174 (+1793005us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400071554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:45.467407 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1664882432}
W20260709 13:10:45.528299 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.528910 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:45.529336 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:45.526005 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113565us
I20260709 13:10:45.530992 22600 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.533072 26277 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.534844 26162 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:45.534729 26072 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.535558 26162 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:45.535858 26270 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.536631 26270 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:45.537098 26270 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.536767 26162 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.537575 26270 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:45.537708 26072 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.481464 26291 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:45.539556 26291 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.541349 26310 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:45.542362 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.543869 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.544288 26180 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.545226 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.545987 22629 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:45.546905 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.547042 26310 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:45.547134 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.527455 26284 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 13:10:45.547950 26310 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.548666 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.524164 26259 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:10:45.549130 26310 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.549775 26259 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:45.549813 26189 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:45.550345 26259 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:45.550767 26189 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:10:45.551303 26259 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.552398 22749 leader_election.cc:400] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:10:45.554883 22749 leader_election.cc:403] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:45.552618 26259 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:45.556380 22709 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.555833 22704 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:45.557088 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.564527 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.568871 26082 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.569759 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.572410 22632 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:10:45.573777 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.569958 26207 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.525676 26265 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.576167 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.574867 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:45.574157 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.538460 26031 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:45.581043 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.583976 22704 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.584060 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.584980 22632 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:45.587467 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.593575 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.594310 22632 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.594765 22775 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:45.571254 26048 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:45.510430 26101 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.577221 26276 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:45.581655 26082 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:45.599121 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:45.600236 26329 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:45.598302 22629 raft_consensus.cc:2368] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
I20260709 13:10:45.575451 26072 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:10:45.602138 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.606541 22779 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:45.618193 22632 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:45.619830 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.621798 22777 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:45.627017 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.628491 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.629204 22632 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:45.634562 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.649493 22632 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:45.651042 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.632002 26038 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:45.652099 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:45.653410 26295 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:10:45.630075 26082 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:45.655108 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.660584 26276 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:45.662328 26302 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.575s	user 0.002s	sys 0.000s
I20260709 13:10:45.633641 26329 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.636785 26275 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.650931 26277 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:45.651186 26189 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:45.651409 26199 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:45.684319 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.671296 26125 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:45.682971 26189 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.694900 26189 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.696943 26270 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.698799 26199 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.701371 26190 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.716323 22781 raft_consensus.cc:2368] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5.
I20260709 13:10:45.694064 26237 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.676131 26276 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.681577 26329 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.724653 26125 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:45.745803 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.724344 26181 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:45.746788 22749 leader_election.cc:400] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:10:45.747617 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.747783 22705 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.748565 22749 leader_election.cc:403] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:10:45.750262 26277 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:45.750958 26101 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.753268 26237 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:45.746766 26207 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.754603 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.761168 22783 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:45.735328 26038 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:45.766579 22707 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:45.757548 26125 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.773645 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.775776 26190 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.776975 26038 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:45.779965 26199 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:45.790793 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.774513 22775 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:45.796124 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.794194 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.826130 26264 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:45.826673 26264 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:45.827527 26264 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.828713 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.830039 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.802349 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.803364 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.834944 22780 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:45.835925 25533 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:45.836112 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.837083 22780 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:45.838701 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.839118 22775 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:45.840091 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.841359 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.842175 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.842376 25902 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:45.842805 25902 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:45.843168 25902 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:45.843272 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2261) took 1730 ms (client timeout 1834 ms). Trace:
W20260709 13:10:45.843748 22631 rpcz_store.cc:269] 0709 13:10:44.113295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.113432 (+   137us) service_pool.cc:224] Handling call
0709 13:10:45.843053 (+1729621us) spinlock_profiling.cc:243] Waited 197 ms on lock 0x7b540022bd54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:45.843207 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":452577536}
I20260709 13:10:45.845011 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.846190 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:45.846930 26300 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.784s	user 0.004s	sys 0.000s
I20260709 13:10:45.847697 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.849063 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.849969 22784 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:45.851073 22627 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:10:45.853050 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1800) took 1327 ms. Trace:
I20260709 13:10:45.853387 22627 rpcz_store.cc:276] 0709 13:10:44.525221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.525374 (+   153us) service_pool.cc:224] Handling call
0709 13:10:45.850726 (+1325352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:45.851472 (+   746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.852805 (+  1333us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540022bd50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:45.852976 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3047131008}
I20260709 13:10:45.861140 22776 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:45.862916 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.863554 22776 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:45.866168 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.867161 26286 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:45.867776 22779 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:45.868309 26286 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:45.868686 26286 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:45.869590 26286 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:45.876554 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:45.877267 22634 raft_consensus.cc:606] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.879074 22634 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:45.860965 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:45.891589 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.892727 22784 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:45.894094 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:45.901475 22627 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:45.905529 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:45.916267 26286 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:45.917229 22628 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:45.919257 26141 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.919874 26141 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:45.920905 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:45.921612 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.925978 22777 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:45.926774 26230 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:45.928213 22631 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:45.898022 22779 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:45.953682 22777 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:45.954639 22782 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
W20260709 13:10:45.980412 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 139166us
W20260709 13:10:45.982844 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 245901us
I20260709 13:10:45.983420 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.984992 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:45.822518 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:45.985749 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:45.986790 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 240194us
W20260709 13:10:45.987500 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.989244 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.991480 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:45.995685 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.004335 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:46.005712 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:46.007350 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:46.007627 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:46.008347 22709 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:46.009088 22675 leader_election.cc:400] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 4 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5.
I20260709 13:10:46.015594 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:46.021698 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:46.019677 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:46.023622 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:46.024642 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.999873 26285 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:46.024616 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:45.990638 25750 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:46.034118 22702 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:10:46.041013 22707 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
W20260709 13:10:46.042641 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57694.9us
I20260709 13:10:46.003786 26102 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.031200 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:46.031164 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:46.011523 26196 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:46.053247 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.027526 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:46.006879 26126 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:46.032956 26273 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:46.005867 26028 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:10:46.032802 26275 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:46.059011 26029 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.059196 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:46.060381 26141 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:46.034157 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:46.062711 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:46.051532 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:46.053645 26196 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.055702 22630 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:46.068689 22705 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:46.032444 26237 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:46.079771 22707 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:46.081497 22701 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:10:46.073128 26275 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:46.072890 26221 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:10:46.057094 26284 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:46.075937 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:46.044351 26326 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:46.102939 22701 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:10:46.105175 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:46.099293 22630 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:46.072978 26196 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:46.073042 26181 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:46.080246 26237 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.126008 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.127746 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:10:46.027287 26339 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:10:46.098524 26275 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.074990 26270 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:46.100389 26326 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.080950 26216 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:10:46.186901 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131344us
W20260709 13:10:46.188628 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58043.2us
I20260709 13:10:46.121363 26295 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:46.200839 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:46.169011 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:46.054600 25750 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.140007 26016 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.139647 26125 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.203748 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:46.188191 26275 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:46.183667 26329 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.100425 26199 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
W20260709 13:10:46.204897 22626 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.009s
I20260709 13:10:46.172667 26237 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:46.197749 26326 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:46.226003 22626 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:46.226563 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:46.274188 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 47727.3us
I20260709 13:10:46.274260 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:46.275362 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:46.251636 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55641.6us
W20260709 13:10:46.266891 26031 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.674s	user 0.018s	sys 0.002s
I20260709 13:10:46.218035 26339 consensus_queue.cc:1243] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:10:46.214411 26016 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.279348 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:46.217978 26125 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.308480 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:46.301934 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.304937 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.299202 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.314224 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:46.302248 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.294869 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:46.319337 22626 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:46.305447 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.269317 26030 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:46.280740 26072 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.006s
I20260709 13:10:46.338864 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:46.342566 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 266119us
W20260709 13:10:46.350915 22633 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.005s
I20260709 13:10:46.353308 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:46.269279 26181 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:46.274760 26101 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:46.275091 26330 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.362298 22633 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:46.363560 22782 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:46.364020 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.363618 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.275388 26221 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:46.277144 26342 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:46.367498 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.397243 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:46.392385 23418 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:46.368276 23411 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:46.402220 26318 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:46.403172 26318 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:46.298982 26126 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:10:46.289822 26194 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:46.404996 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 208337us
I20260709 13:10:46.382282 22784 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:46.408438 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.408879 22784 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:10:46.383347 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 84253us
I20260709 13:10:46.297945 26338 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:46.409719 26273 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:46.298172 25750 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:46.410463 26273 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:46.410760 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:46.412134 26194 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:46.412551 26338 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:46.414072 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.414561 23415 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:10:46.416530 26126 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:46.419891 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84681.4us
I20260709 13:10:46.423100 26255 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:46.425434 26255 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.430709 22784 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:46.426120 26255 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.434507 26255 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.340703 26329 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:46.277711 26307 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:10:46.438112 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.439862 26307 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:46.439989 23415 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:46.440299 26307 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:46.390344 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 55261.3us
I20260709 13:10:46.441283 26307 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.443387 26307 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:46.442245 22747 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.444767 26329 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:46.632787 22782 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 13:10:46.636317 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:46.642020 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 174640us
I20260709 13:10:46.322526 26291 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:10:46.380636 26072 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:10:46.570995 22747 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:46.650512 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2427) took 1050 ms. Trace:
I20260709 13:10:46.322160 26310 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 13:10:46.435501 25804 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:10:46.696727 26188 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.209s	user 0.015s	sys 0.031s
I20260709 13:10:46.697113 26221 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:46.704195 26347 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.374s	user 0.000s	sys 0.004s
I20260709 13:10:46.651544 25750 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:10:46.570842 26330 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.706027 26125 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:46.462843 26102 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:46.672554 22634 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:10:46.763995 22634 pending_rounds.cc:85] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 8
I20260709 13:10:46.764340 22634 pending_rounds.cc:107] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:10:46.764614 22634 raft_consensus.cc:2889] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:46.648366 26298 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:46.745293 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:46.749723 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 298766us
I20260709 13:10:46.688221 26255 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.738566 26016 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:46.804075 22775 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:46.820324 22634 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:10:46.780009 26259 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.207s	user 0.021s	sys 0.000s
I20260709 13:10:46.812557 26255 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:46.825954 26352 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.272s	user 0.001s	sys 0.003s
I20260709 13:10:46.826826 26016 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.829187 25750 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:10:46.830796 26343 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.446s	user 0.000s	sys 0.005s
I20260709 13:10:46.831308 26363 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:46.741806 26188 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.605132 26342 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:46.880860 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.025s	sys 0.055s
W20260709 13:10:46.802126 26048 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.199s	user 0.019s	sys 0.010s
W20260709 13:10:46.847535 26196 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.006s
W20260709 13:10:46.852837 22626 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.532s	user 0.012s	sys 0.001s
W20260709 13:10:46.872187 22783 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.105s	user 0.016s	sys 0.006s
I20260709 13:10:46.886440 22626 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:46.877256 22776 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.013s	user 0.017s	sys 0.005s
I20260709 13:10:46.741451 26181 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.713011 22781 rpcz_store.cc:276] 0709 13:10:45.599820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.991819 (+391999us) service_pool.cc:224] Handling call
0709 13:10:46.017827 (+ 26008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:46.018042 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:46.207920 (+189878us) log.cc:844] Serialized 14 byte log entry
0709 13:10:46.650098 (+442178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:46.650158 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:46.650227 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.650303 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:46.650449 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14870,"prepare.queue_time_us":362296,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":81186,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":89,"thread_start_us":599008,"threads_started":3,"wal-append.queue_time_us":185879}
W20260709 13:10:46.881567 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.026s	sys 0.055s
I20260709 13:10:46.886534 22783 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:46.891479 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2420) took 1324 ms (client timeout 1579 ms). Trace:
W20260709 13:10:46.892050 22783 rpcz_store.cc:269] 0709 13:10:45.566626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.754195 (+187569us) service_pool.cc:224] Handling call
0709 13:10:46.891377 (+1137182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:46.887251 22776 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:46.893693 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1799) took 1563 ms. Trace:
I20260709 13:10:46.802604 26298 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.837656 26105 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:46.889735 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:46.835816 22630 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.004s
I20260709 13:10:46.894203 22776 rpcz_store.cc:276] 0709 13:10:45.330596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.862596 (+532000us) service_pool.cc:224] Handling call
0709 13:10:46.893597 (+1031001us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:46.870604 25750 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:46.899683 22780 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
W20260709 13:10:46.876391 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 233570us
I20260709 13:10:46.898358 22630 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:46.902573 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:46.879819 26125 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:46.903707 23416 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:10:46.641438 26295 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.910872 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:46.918591 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:46.919400 22631 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:46.869021 26016 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.899952 26307 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.898299 26331 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:46.906075 26363 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:10:46.922551 26189 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.227s	user 0.015s	sys 0.003s
I20260709 13:10:46.912679 26241 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:46.915815 26369 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:46.890483 26196 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:46.939023 26265 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.339s	user 0.021s	sys 0.007s
I20260709 13:10:46.936246 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:46.915140 26105 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.948572 26362 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:46.975719 26362 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.978077 26362 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:46.978739 26344 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.624s	user 0.000s	sys 0.006s
I20260709 13:10:46.986183 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.988145 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:46.992434 25750 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:46.999961 26361 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:10:47.001714 26342 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.916391 26125 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.002977 26259 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.003618 26259 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.006590 26259 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:46.948163 26369 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.009456 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 509) took 1727 ms. Trace:
I20260709 13:10:47.009666 26369 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.011822 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.016314 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:46.932147 26307 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.953881 26038 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.962354 26188 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.964030 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2430) took 1173 ms. Trace:
I20260709 13:10:46.922044 26339 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:10:47.015811 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:46.946887 26230 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.017s	user 0.023s	sys 0.026s
W20260709 13:10:46.948148 26286 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.006s
I20260709 13:10:46.962489 26270 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:47.100400 22629 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:47.100756 26270 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:47.103475 26270 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:47.106274 26369 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:47.029153 26265 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:47.095110 26277 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.344s	user 0.084s	sys 0.088s
I20260709 13:10:47.096524 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:47.095770 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 80070.5us
I20260709 13:10:46.941674 26372 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.166s
I20260709 13:10:46.974907 26371 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:46.951073 26341 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.277s	user 0.002s	sys 0.000s
I20260709 13:10:46.945226 26189 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:46.954267 26363 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:47.010059 22780 rpcz_store.cc:276] 0709 13:10:45.282173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.951585 (+669412us) service_pool.cc:224] Handling call
0709 13:10:45.951850 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:46.900217 (+948367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.009141 (+108924us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54003278d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.009390 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174840576}
I20260709 13:10:47.101016 22632 raft_consensus.cc:606] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.134356 22632 raft_consensus.cc:614] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:46.964736 26105 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.053027 22775 rpcz_store.cc:276] 0709 13:10:45.791048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.839852 (+ 48804us) service_pool.cc:224] Handling call
0709 13:10:46.963841 (+1123989us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b5400325fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:46.963961 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097794304}
I20260709 13:10:46.950449 26355 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:47.147541 25902 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.304s	user 0.013s	sys 0.010s
W20260709 13:10:47.147099 26349 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.741s	user 0.002s	sys 0.000s
I20260709 13:10:47.157121 26355 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:46.952322 26196 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.157125 25902 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:47.159759 26264 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.332s	user 0.014s	sys 0.004s
I20260709 13:10:47.160192 22634 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:47.161526 25902 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.162405 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1821) took 1089 ms. Trace:
I20260709 13:10:47.162832 22634 rpcz_store.cc:276] 0709 13:10:46.073421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.821603 (+748182us) service_pool.cc:224] Handling call
0709 13:10:46.836685 (+ 15082us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.160670 (+323985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.162132 (+  1462us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540022db54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.162335 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743495424}
I20260709 13:10:46.968046 26276 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:46.974949 26353 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.166759 26353 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.168393 23419 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:10:47.051393 26038 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.170326 22779 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:47.170799 26038 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.005556 26125 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.122336 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.173401 23420 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:47.174283 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 31) took 1050 ms. Trace:
I20260709 13:10:47.148739 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.182528 22631 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:46.935336 26331 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.155644 26363 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.185695 22709 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:10:47.156383 25804 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.189163 22705 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:47.160202 26264 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.167220 26355 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:47.170341 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.068079 26330 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.194665 22784 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:10:47.174999 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:47.106984 26369 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.138510 26241 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:47.178675 26162 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:47.120535 26277 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6.
W20260709 13:10:46.957422 25533 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.121s	user 0.017s	sys 0.005s
I20260709 13:10:47.180416 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.183521 26276 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.200325 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:47.187127 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.184293 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:47.135049 26286 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.135664 26221 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.141381 26290 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.224496 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.225181 22635 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:47.225476 22628 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:47.183524 26291 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:47.185009 26353 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:47.181289 22675 connection.cc:256] RPC call timeout handler was delayed by -1.021615880000013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25630000000000003s for a response
I20260709 13:10:47.168327 26189 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.193748 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.186730 26125 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:47.201220 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.182089 22632 rpcz_store.cc:276] 0709 13:10:46.123981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.127522 (+  3541us) service_pool.cc:224] Handling call
0709 13:10:47.174075 (+1046553us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b540006b3d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.174188 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147508480}
I20260709 13:10:47.191536 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.269241 22781 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:47.269454 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.270720 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.218485 22626 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:10:47.271687 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.202361 22775 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:10:47.228694 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.277446 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.278357 22775 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:47.226691 22706 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:47.240025 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1805) took 1082 ms. Trace:
I20260709 13:10:47.280005 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.280197 22779 rpcz_store.cc:276] 0709 13:10:46.157292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.305764 (+148472us) service_pool.cc:224] Handling call
0709 13:10:47.239770 (+934006us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54003273d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.239947 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968507648}
I20260709 13:10:47.192174 25804 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:47.244621 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:47.245543 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.226205 26220 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.291525 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:47.251617 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51403.7us
W20260709 13:10:47.293566 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.203889 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:47.242321 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2287) took 1606 ms (client timeout 2074 ms). Trace:
W20260709 13:10:47.295324 22629 rpcz_store.cc:269] 0709 13:10:45.636124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.684012 (+ 47888us) service_pool.cc:224] Handling call
0709 13:10:47.242103 (+1558091us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540006b3d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.242237 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3253404544}
I20260709 13:10:47.286203 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.297269 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.197404 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.261757 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.283810 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.286196 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.233417 26126 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:47.197561 26369 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:47.247032 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.288787 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:47.288798 22776 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:47.297392 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.306146 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.307126 22701 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:47.218839 26016 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.223893 26265 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.227440 26330 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:47.343144 26330 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.343645 26330 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:47.197250 26255 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:47.266368 26285 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.217s	user 0.015s	sys 0.004s
I20260709 13:10:47.198366 26162 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:47.347220 26285 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:47.348342 23417 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:47.347864 26162 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.349153 26162 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:47.349499 22635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1797) took 3286 ms (client timeout 2563 ms). Trace:
W20260709 13:10:47.349932 22635 rpcz_store.cc:269] 0709 13:10:44.063269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.067804 (+  4535us) service_pool.cc:224] Handling call
0709 13:10:47.349268 (+3281464us) spinlock_profiling.cc:243] Waited 3.14 s on lock 0x7b540022b854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.349399 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7220328320}
I20260709 13:10:47.350384 22634 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:10:47.265132 26291 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.353018 26291 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.353533 26291 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.353998 26291 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.354898 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:47.355872 26326 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.085s	user 0.009s	sys 0.008s
I20260709 13:10:47.356411 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.356339 26326 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.353031 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.359386 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.361271 22633 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:47.298489 22783 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:47.301604 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.302762 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.299934 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2175) took 1691 ms. Trace:
W20260709 13:10:47.366544 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63422.3us
I20260709 13:10:47.303794 26220 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.317123 25750 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:47.319170 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72232.8us
I20260709 13:10:47.320497 26105 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:47.321969 22775 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:10:47.322945 25768 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:47.231104 26286 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.325804 26270 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.434878 26326 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.435990 26181 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:47.289630 26125 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.442373 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 788) took 1118 ms. Trace:
W20260709 13:10:47.333894 26373 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:10:47.340786 25533 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:47.257695 26290 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.362429 22778 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:47.369583 22629 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:10:47.479740 25533 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:47.480273 25533 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:47.481257 25533 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.370332 22632 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:47.142935 26371 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:47.483592 26326 consensus_queue.cc:1175] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 13:10:47.484464 25533 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:47.485641 26371 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.274214 26376 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:47.371727 22776 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:47.486335 26376 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.365635 22704 rpcz_store.cc:276] 0709 13:10:45.596823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.638481 (+ 41658us) service_pool.cc:224] Handling call
0709 13:10:45.654735 (+ 16254us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:45.655144 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:45.655176 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:45.655677 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 13:10:46.017397 (+361720us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:46.017461 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.017566 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:47.288117 (+1270551us) raft_consensus.cc:1685] finished
0709 13:10:47.288142 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:47.288301 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2999,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4856,"thread_start_us":327488,"threads_started":1,"wal-append.queue_time_us":599481}
I20260709 13:10:47.438303 26028 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.486865 26376 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:47.487156 26028 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.487865 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.374203 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:47.489557 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:47.490016 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 115905us
I20260709 13:10:47.490185 26284 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:47.360065 22701 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:47.347860 26360 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.645s	user 0.005s	sys 0.000s
I20260709 13:10:47.490660 26284 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.218752 26241 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.491102 26284 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:47.491351 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.492259 26374 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.492563 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.492823 26241 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.493266 22701 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:47.385635 26265 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.293116 26298 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.391081 26331 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.400489 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:47.373579 26339 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:10:47.378933 25902 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:47.379160 26259 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:10:47.424867 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:47.426014 26237 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.160s	user 0.013s	sys 0.004s
W20260709 13:10:47.426643 26275 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.180s	user 0.013s	sys 0.003s
I20260709 13:10:47.288832 26221 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 13:10:47.431277 26286 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:47.431880 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51568.9us
I20260709 13:10:47.521404 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.436291 26270 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.438402 22703 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:10:47.528339 22703 pending_rounds.cc:85] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:10:47.528581 22703 pending_rounds.cc:107] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:10:47.528808 22703 raft_consensus.cc:2889] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:47.439962 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.489578 26028 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.445868 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.448881 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 496) took 1745 ms. Trace:
I20260709 13:10:47.456082 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.462395 26188 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.460464 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:47.468497 23417 rpcz_store.cc:276] 0709 13:10:46.323694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.323933 (+   239us) service_pool.cc:224] Handling call
0709 13:10:46.363591 (+ 39658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.348809 (+985218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.442018 (+ 93209us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54005d5554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.442284 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264191232}
W20260709 13:10:47.471771 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 74718.2us
I20260709 13:10:47.542502 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.500211 26329 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:47.477288 26048 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.372489 22630 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:10:47.495913 22701 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:10:47.517473 25902 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:47.500511 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 492) took 2013 ms. Trace:
I20260709 13:10:47.495958 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:10:47.510598 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 110239us
W20260709 13:10:47.583695 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 87835us
I20260709 13:10:47.584856 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2179) took 1590 ms. Trace:
I20260709 13:10:47.512341 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.519141 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.587250 22704 raft_consensus.cc:2410] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:10:47.522017 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:47.243548 26276 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.486555 26264 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.589236 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.476470 26291 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.517469 26342 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:47.496742 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 495) took 1918 ms. Trace:
I20260709 13:10:47.611733 22627 rpcz_store.cc:276] 0709 13:10:45.555896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.132775 (+576879us) service_pool.cc:224] Handling call
0709 13:10:46.133337 (+   562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:46.217741 (+ 84404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:46.217784 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:46.243090 (+ 25306us) log.cc:844] Serialized 26 byte log entry
0709 13:10:46.770560 (+527470us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:46.770595 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.770729 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:47.474366 (+703637us) raft_consensus.cc:1685] finished
0709 13:10:47.474396 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:47.474571 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100029,"raft.queue_time_us":88325,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":11399,"thread_start_us":442601,"threads_started":1,"wal-append.queue_time_us":825334}
I20260709 13:10:47.535238 22633 rpcz_store.cc:276] 0709 13:10:45.703612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.437044 (+733432us) service_pool.cc:224] Handling call
0709 13:10:46.437404 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.361658 (+924254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.448553 (+ 86895us) spinlock_profiling.cc:243] Waited 924 ms on lock 0x7b540006b8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.448797 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2124288384}
I20260709 13:10:47.617239 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:47.619238 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:47.526875 26220 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.531803 26190 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.492831 26374 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.582520 22628 rpcz_store.cc:276] 0709 13:10:45.486877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.918185 (+431308us) service_pool.cc:224] Handling call
0709 13:10:45.918493 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.225959 (+1307466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.500242 (+274283us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54001ca054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.500445 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3005343488}
I20260709 13:10:47.585253 22708 rpcz_store.cc:276] 0709 13:10:45.994729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.006757 (+ 12028us) service_pool.cc:224] Handling call
0709 13:10:47.584622 (+1577865us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b5400316354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.584786 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2910446848}
I20260709 13:10:47.522758 26237 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.519464 26338 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.629877 26298 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:10:47.663611 26338 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.664143 26298 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:47.664597 26298 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:47.665570 26298 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.666165 26338 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.609481 26330 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.667227 22702 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 10. Ops: []
I20260709 13:10:47.667680 22631 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:47.669438 26330 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.577934 26331 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:10:47.670764 26181 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:47.671929 26354 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:47.672988 26354 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:47.673503 26354 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:47.594447 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72510us
I20260709 13:10:47.676717 26307 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.677022 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:47.678085 26307 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:47.608515 26317 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:47.679034 26317 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:47.679477 26317 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:47.596825 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:47.606041 22780 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:47.682345 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.686120 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.674499 26354 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.689006 26369 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.568861 26126 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:47.693419 26126 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.693886 26126 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:47.564639 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 104349us
I20260709 13:10:47.500692 26265 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:47.590651 22778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1794) took 2368 ms (client timeout 2968 ms). Trace:
I20260709 13:10:47.576026 26180 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.592902 25902 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:47.698043 22778 rpcz_store.cc:269] 0709 13:10:45.222840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.841914 (+619074us) service_pool.cc:224] Handling call
0709 13:10:47.590423 (+1748509us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b540033d454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:47.590562 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3167294336}
W20260709 13:10:47.704509 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107790us
I20260709 13:10:47.571962 26188 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.527917 26275 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.708555 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.710019 23414 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:47.581530 26048 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.710436 26275 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.645047 23419 raft_consensus.cc:2410] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:10:47.712406 26048 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.713083 26048 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:47.575656 26371 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:10:47.646135 26374 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.650756 26342 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:47.715181 26342 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.715881 26285 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:47.716413 26285 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:47.717316 26285 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.654763 26276 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.718290 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:47.718873 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:47.708599 26188 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.720150 26220 raft_consensus.cc:993] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:10:47.720712 26031 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:47.721339 26369 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.722227 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.726506 26276 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:47.727073 26276 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.727478 26276 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:47.732086 26220 raft_consensus.cc:1081] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:10:47.732657 26290 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.734766 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.655766 26355 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:47.736492 26355 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.737807 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2379) took 1701 ms. Trace:
I20260709 13:10:47.656333 26162 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.739010 26162 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.742466 26355 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:47.658167 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:47.745299 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:47.659970 26384 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:10:47.662011 26237 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.581769 26329 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.662352 26376 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.534381 26230 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 13:10:47.606041 23412 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:10:47.614229 26286 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:47.690506 26270 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:47.690260 26298 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:47.691850 26354 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:47.791374 26376 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:47.787374 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:47.795104 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.795660 23411 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:10:47.692353 25804 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.698173 26265 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.625067 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:47.698555 26180 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.706041 26264 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:47.631711 26190 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.742765 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 608) took 1330 ms. Trace:
I20260709 13:10:47.656958 26125 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.810799 23414 rpcz_store.cc:276] 0709 13:10:46.412214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.412360 (+   146us) service_pool.cc:224] Handling call
0709 13:10:46.412705 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.710464 (+1297759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.742365 (+ 31901us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540020a154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.742642 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2983067904}
I20260709 13:10:47.748766 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.747596 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.738384 22702 rpcz_store.cc:276] 0709 13:10:46.036369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.036534 (+   165us) service_pool.cc:224] Handling call
0709 13:10:46.056132 (+ 19598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.667668 (+1611536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.737454 (+ 69786us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54002bedd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:47.737719 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3704680192}
I20260709 13:10:47.702106 26307 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:47.761682 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.763085 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.766305 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.767295 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:47.771203 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:47.819130 22707 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:47.771732 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.820241 22776 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:47.821235 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.771903 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.764182 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.773252 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.824837 22775 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:47.776034 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.777645 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.787372 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:47.825901 22784 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:47.826272 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:47.789435 26126 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:47.797811 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46911.9us
I20260709 13:10:47.760900 26369 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.830019 26369 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.830722 26329 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.800566 26286 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:47.832073 22708 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:47.824177 22703 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:47.832235 26286 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.833302 26180 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:47.806282 26264 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.834197 26264 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:47.834569 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.835461 22775 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:10:47.836210 22775 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:47.836591 22775 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:47.836764 25804 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:47.837494 25804 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.838768 26330 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:47.839819 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:47.830859 26126 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.841383 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:47.843274 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:47.845464 25902 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.846863 26220 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:47.847198 22631 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:47.847433 26220 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.847968 26220 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:47.848181 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.810420 26237 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:47.846014 25533 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:47.761797 26285 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:47.805538 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 107926us
I20260709 13:10:47.809787 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:47.819494 22782 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:47.800871 26031 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.802908 26255 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:10:47.840430 26371 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:47.826027 22779 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:47.750532 26291 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:47.850788 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.874673 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:47.875440 23419 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:47.874718 22703 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:10:47.803823 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 178898us
I20260709 13:10:47.878319 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:47.837996 25804 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.852620 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:47.862354 22777 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:47.862488 22775 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.881944 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:47.889930 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:47.886559 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.891764 22635 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:47.894454 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.895450 22629 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:47.896027 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.896543 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.897538 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:47.895641 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.860612 26048 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:47.895038 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.920946 22775 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.913080 26329 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.932722 26329 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.886493 26125 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:47.913959 26105 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:47.914553 26355 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:47.954497 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:47.955893 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:47.956137 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:47.956609 22776 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:47.913995 26338 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:47.960446 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:47.874127 26031 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.967391 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.925287 26371 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:47.874963 26237 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:10:47.906890 26363 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:10:47.972337 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.972311 26329 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:47.907030 26291 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:47.951575 26105 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.979610 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:47.980306 22776 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:48.001405 22633 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:47.982856 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:47.982883 23419 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:47.992081 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:47.975781 26307 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:47.978475 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:47.924595 26339 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:10:47.924862 26378 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:10:48.003758 22631 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:48.015175 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:48.003196 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:48.011356 22708 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:48.015456 22631 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:47.952523 26284 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:48.057744 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:48.062232 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:48.009922 26105 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:48.034149 22631 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:48.131776 22631 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:10:48.126747 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 123663us
I20260709 13:10:48.009740 26275 raft_consensus.cc:2764] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:10:48.130755 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:48.131062 23421 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:10:47.972427 26338 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:48.014461 26298 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:10:48.114144 26386 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:10:48.142967 22706 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:48.133771 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:48.133183 26407 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.184423 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:48.187837 22708 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:48.192243 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:48.198396 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:48.199321 22706 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:48.199455 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:48.199036 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:48.157704 26385 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.092s	user 0.000s	sys 0.008s
I20260709 13:10:48.153616 26298 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:48.205193 26396 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:10:48.206557 26162 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:48.195559 26407 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:48.208711 26338 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:48.209005 25804 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:48.201828 26125 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:48.145154 26342 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:10:48.201457 26378 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:48.202628 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:48.204670 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:48.187752 25902 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:48.191892 26369 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:48.221342 26029 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:48.269140 26318 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.014161 26318 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:52.015385 22748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.999s	user 0.014s	sys 0.019s
I20260709 13:10:52.015532 26429 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:10:52.016175 22748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.000s	user 0.014s	sys 0.019s
W20260709 13:10:48.291020 26386 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:10:48.388062 26393 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 13:10:48.389467 26291 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:48.390393 26405 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 13:10:48.390941 26031 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:48.292073 25804 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:48.475666 26264 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.002s
I20260709 13:10:52.023442 26264 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.024744 26190 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.026156 26264 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:48.392908 26397 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.382s	user 0.000s	sys 0.001s
W20260709 13:10:48.574559 26317 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.003s
I20260709 13:10:52.027009 26317 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.028085 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 3.82528e+06us
I20260709 13:10:52.029402 26317 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:48.615523 26329 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.010s	sys 0.017s
W20260709 13:10:52.030355 26329 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.969s	user 0.011s	sys 0.017s
W20260709 13:10:49.181234 26276 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.453s	user 0.008s	sys 0.008s
I20260709 13:10:52.017445 26318 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.031289 26429 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:10:49.450150 26398 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.313s	user 0.000s	sys 0.002s
W20260709 13:10:49.539448 26416 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:10:48.479616 26298 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:49.570060 26411 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.210s	user 0.001s	sys 0.000s
W20260709 13:10:49.571426 26406 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.912s	user 0.001s	sys 0.000s
I20260709 13:10:49.720232 26284 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:49.579468 26404 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
W20260709 13:10:49.725083 26220 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.877s	user 0.008s	sys 0.008s
I20260709 13:10:52.036288 26220 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.036783 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2450) took 4270 ms. Trace:
I20260709 13:10:52.037202 22778 rpcz_store.cc:276] 0709 13:10:47.766192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.766349 (+   157us) service_pool.cc:224] Handling call
0709 13:10:47.766748 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.767062 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.767091 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.911148 (+144057us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:10:47.920404 (+  9256us) log.cc:844] Serialized 26 byte log entry
0709 13:10:48.053084 (+132680us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:48.053153 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.053264 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.036404 (+3983140us) raft_consensus.cc:1685] finished
0709 13:10:52.036453 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.036689 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10951552,"thread_start_us":91406,"threads_started":1,"wal-append.queue_time_us":237538}
I20260709 13:10:49.840494 26407 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.038569 26220 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:50.535490 26307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.041s	sys 0.066s
W20260709 13:10:50.517817 26419 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.456s	user 0.000s	sys 0.000s
W20260709 13:10:51.052753 26105 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 2.904s	user 0.010s	sys 0.005s
I20260709 13:10:52.039033 26291 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:10:51.054831 26048 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 3.131s	user 0.015s	sys 0.003s
I20260709 13:10:52.039760 26105 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.040181 26048 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:51.863442 26259 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:52.040958 26259 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:52.042033 26105 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:51.848352 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.031409 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:51.950459 22781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.934s	user 0.027s	sys 0.175s
W20260709 13:10:51.969581 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.833s	user 0.020s	sys 0.034s
W20260709 13:10:52.044142 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.909s	user 0.021s	sys 0.035s
W20260709 13:10:52.044222 22781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.028s	user 0.028s	sys 0.175s
I20260709 13:10:52.045006 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2186) took 4108 ms. Trace:
I20260709 13:10:51.975903 26275 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.045239 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2458) took 4251 ms. Trace:
I20260709 13:10:52.045859 26275 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.045189 26429 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.045974 22781 rpcz_store.cc:276] 0709 13:10:47.794366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.794505 (+   139us) service_pool.cc:224] Handling call
0709 13:10:47.795055 (+   550us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:47.800535 (+  5480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.800570 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:47.813802 (+ 13232us) log.cc:844] Serialized 52 byte log entry
0709 13:10:47.936320 (+122518us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:48.007688 (+ 71368us) log.cc:844] Serialized 14 byte log entry
0709 13:10:52.044793 (+4037105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:52.044906 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.044970 (+    64us) raft_consensus.cc:1685] finished
0709 13:10:52.044990 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.045164 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7743,"prepare.queue_time_us":70694,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":47905,"raft.run_cpu_time_us":5575,"raft.run_wall_time_us":83913,"thread_start_us":4122025,"threads_started":2,"wal-append.queue_time_us":512504}
I20260709 13:10:52.048031 26429 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.049451 26347 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 4.136s	user 0.000s	sys 0.022s
W20260709 13:10:51.991230 22629 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 4.083s	user 0.026s	sys 0.039s
I20260709 13:10:52.050599 26429 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.050879 22629 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:52.051287 26429 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.051465 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2313) took 4157 ms (client timeout 1908 ms). Trace:
W20260709 13:10:52.051872 22629 rpcz_store.cc:269] 0709 13:10:47.894316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.896313 (+  1997us) service_pool.cc:224] Handling call
0709 13:10:52.051379 (+4155066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:52.051779 26429 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.054109 26429 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:51.992475 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 622) took 4310 ms. Trace:
I20260709 13:10:52.054772 23421 rpcz_store.cc:276] 0709 13:10:47.682168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.738729 (+ 56561us) service_pool.cc:224] Handling call
0709 13:10:47.739210 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.142567 (+403357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:51.992077 (+3849510us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54006ee454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:51.992355 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":900551424}
I20260709 13:10:52.031378 26276 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.056692 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 4.011s	user 0.011s	sys 0.015s
W20260709 13:10:52.057312 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 4.012s	user 0.012s	sys 0.015s
W20260709 13:10:52.057678 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 4.03497e+06us
I20260709 13:10:52.058230 26429 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:10:49.998929 26285 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 2.134s	user 0.016s	sys 0.000s
I20260709 13:10:52.041869 26259 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:51.918308 22703 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 4.012s	user 0.031s	sys 0.008s
I20260709 13:10:52.060176 22703 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:51.933717 22522 maintenance_manager.cc:419] P 71605ba8587a4b5193d8d69d9967c1af: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033583
I20260709 13:10:52.048442 26275 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.045403 22707 rpcz_store.cc:276] 0709 13:10:47.936573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.986536 (+ 49963us) service_pool.cc:224] Handling call
0709 13:10:47.987013 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.000643 (+ 13630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.000701 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:48.014278 (+ 13577us) log.cc:844] Serialized 26 byte log entry
0709 13:10:52.044578 (+4030300us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:52.044628 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:52.044719 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.044768 (+    49us) raft_consensus.cc:1685] finished
0709 13:10:52.044781 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.044914 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86260,"raft.queue_time_us":43544,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5577,"thread_start_us":3909683,"threads_started":1,"wal-append.queue_time_us":1358008}
I20260709 13:10:52.054339 26102 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:52.057750 26276 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:52.039330 26307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.315s	user 0.041s	sys 0.067s
W20260709 13:10:52.060235 22676 connection.cc:256] RPC call timeout handler was delayed by -2.788484498999992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 13:10:52.068370 22708 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 3.879s	user 0.021s	sys 0.059s
I20260709 13:10:52.071641 22708 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:52.067461 26428 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.072191 26291 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.072507 22708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2196) took 3923 ms (client timeout 1542 ms). Trace:
I20260709 13:10:52.072353 26428 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.072712 26291 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.072978 22708 rpcz_store.cc:269] 0709 13:10:48.152592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.152975 (+   383us) service_pool.cc:224] Handling call
0709 13:10:52.072383 (+3919408us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:51.994374 23384 connection.cc:256] RPC call timeout handler was delayed by -2.8349719510000568s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
W20260709 13:10:52.070099 22703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2424) took 4210 ms (client timeout 2123 ms). Trace:
W20260709 13:10:52.074045 22703 rpcz_store.cc:269] 0709 13:10:47.850245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.850414 (+   169us) service_pool.cc:224] Handling call
0709 13:10:52.060656 (+4210242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:52.075585 26329 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:52.076267 26329 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:52.079044 26399 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 3.811s	user 0.004s	sys 0.027s
I20260709 13:10:52.083660 26329 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.084527 26126 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.085146 26126 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.077207 26259 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:52.087790 26126 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:52.095304 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.929s	user 0.023s	sys 0.045s
W20260709 13:10:52.095880 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.929s	user 0.024s	sys 0.045s
W20260709 13:10:52.096534 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 3.96259e+06us
I20260709 13:10:52.090535 26126 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.068794 26429 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.101233 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:52.102325 26401 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 3.810s	user 0.005s	sys 0.031s
W20260709 13:10:52.102674 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.677s (SENT) [suppressed 7 similar messages]
W20260709 13:10:52.124501 22598 connection.cc:256] RPC call timeout handler was delayed by -3.5461245850000296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 13:10:52.106833 23384 connection.cc:256] RPC call timeout handler was delayed by -2.680567472000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:10:52.125448 23384 connection.cc:256] RPC call timeout handler was delayed by -2.6265103030000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:10:52.121620 26276 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:52.127956 22598 connection.cc:256] RPC call timeout handler was delayed by -2.3493946830000141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
I20260709 13:10:52.129428 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:52.110203 26259 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:52.112941 22750 connection.cc:256] RPC call timeout handler was delayed by -3.6455173949999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
W20260709 13:10:52.131495 22750 connection.cc:256] RPC call timeout handler was delayed by -3.3610262339999508s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3127s for a response
W20260709 13:10:52.131953 22750 connection.cc:256] RPC call timeout handler was delayed by -3.164879142000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 13:10:52.125952 23384 connection.cc:256] RPC call timeout handler was delayed by -2.5813791650000439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 13:10:52.136164 22750 connection.cc:256] RPC call timeout handler was delayed by -2.7455134690000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
W20260709 13:10:52.149739 22750 connection.cc:256] RPC call timeout handler was delayed by -2.7311429650000036s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
I20260709 13:10:52.148181 22629 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:52.124354 26307 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.149518 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:52.152228 22750 connection.cc:256] RPC call timeout handler was delayed by -2.6886300109999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1854s for a response
W20260709 13:10:52.137081 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:52.152292 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 120956us
I20260709 13:10:52.152624 22626 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:10:52.152905 22750 connection.cc:256] RPC call timeout handler was delayed by -2.6089016219999621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
I20260709 13:10:52.154536 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:52.155030 22750 connection.cc:256] RPC call timeout handler was delayed by -2.5097936579999214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
W20260709 13:10:52.156385 22750 connection.cc:256] RPC call timeout handler was delayed by -2.3223181189998741s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2848s for a response
I20260709 13:10:52.156729 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:52.157750 22750 connection.cc:256] RPC call timeout handler was delayed by -2.2089283689999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20870000000000002s for a response
I20260709 13:10:52.130453 26407 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.127358 26048 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:52.162943 22601 connection.cc:256] RPC call timeout handler was delayed by -2.3427560410000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21230000000000004s for a response
I20260709 13:10:52.163282 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:52.165503 26430 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.135898 26190 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.143086 26329 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.166109 22601 connection.cc:256] RPC call timeout handler was delayed by -1.8946624200000315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:10:52.146888 26264 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:52.127562 26126 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.171499 26430 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.180605 26126 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.180853 26430 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.181262 26126 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:52.181821 26126 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.168284 26284 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.183100 26284 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.181525 26430 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.184268 26430 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.172745 26433 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:52.173053 26431 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:52.145114 26285 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:10:52.213245 26285 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:52.213666 26285 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:52.214576 26285 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.215919 26285 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:52.162736 26307 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.168813 26048 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.135833 26276 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.167909 26434 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.185813 26284 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.164682 26180 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:52.166903 26407 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.187067 26329 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.164837 26432 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:52.164945 26317 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.144564 26259 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.188629 26190 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.130573 26429 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.156626 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:52.172096 26264 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:52.251520 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94976.6us
I20260709 13:10:52.254134 26438 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.254842 26438 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.256467 26190 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.256783 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.257094 26190 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.257124 26438 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.257462 22626 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:10:52.258512 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.244117 26180 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.212051 26431 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.164546 26436 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.224839 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 130252us
I20260709 13:10:52.228663 26440 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:52.164618 26437 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.204852 26126 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.259158 22629 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:52.211114 26433 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:52.260866 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:52.268914 26329 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.270694 26307 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.272251 26430 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.272917 26436 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.260345 26284 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:52.250705 26317 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.274889 26440 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.275923 26407 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.259617 26264 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:52.263672 23385 connection.cc:256] RPC call timeout handler was delayed by -3.528725449000035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 13:10:52.277987 26429 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.280085 26431 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.244995 26434 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.282281 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.246s	user 0.034s	sys 0.033s
I20260709 13:10:52.282739 26259 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.288008 26048 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.257815 26438 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.297559 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:52.257706 26190 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:52.357715 22705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.168s	user 0.100s	sys 0.174s
I20260709 13:10:52.298949 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.383895 26438 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.384837 22705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.195s	user 0.107s	sys 0.181s
I20260709 13:10:52.386077 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2191) took 4413 ms. Trace:
W20260709 13:10:52.339666 22631 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 4.207s	user 0.027s	sys 0.004s
W20260709 13:10:52.381016 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:52.334239 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107668us
I20260709 13:10:52.384414 22629 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:52.262099 26180 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:52.412515 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.386971 22631 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
W20260709 13:10:52.389408 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 91961.3us
I20260709 13:10:52.452533 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:52.452836 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:52.389160 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.353s	user 0.039s	sys 0.033s
W20260709 13:10:52.453900 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 4.41749e+06us
I20260709 13:10:52.454967 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:52.457253 26431 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:10:52.457515 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2315) took 4488 ms (client timeout 1542 ms). Trace:
I20260709 13:10:52.457765 26431 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.458278 26431 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:52.458077 22631 rpcz_store.cc:269] 0709 13:10:47.961587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.965292 (+  3705us) service_pool.cc:224] Handling call
0709 13:10:52.450187 (+4484895us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:52.460778 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2190) took 4491 ms. Trace:
I20260709 13:10:52.461232 22702 rpcz_store.cc:276] 0709 13:10:47.970229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.048187 (+ 77958us) service_pool.cc:224] Handling call
0709 13:10:48.053642 (+  5455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:48.053972 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:48.143641 (+ 89669us) log.cc:844] Serialized 14 byte log entry
0709 13:10:52.460338 (+4316697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:52.460402 (+    64us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:52.460470 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:52.460557 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.460711 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10662,"prepare.queue_time_us":135514,"thread_start_us":75946,"threads_started":1}
I20260709 13:10:52.431782 26048 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.386440 22705 rpcz_store.cc:276] 0709 13:10:47.972491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.058791 (+ 86300us) service_pool.cc:224] Handling call
0709 13:10:48.059219 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:48.059543 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:48.189175 (+129632us) log.cc:844] Serialized 14 byte log entry
0709 13:10:52.385646 (+4196471us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:52.385710 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:52.385763 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:52.385834 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.385978 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78473,"prepare.queue_time_us":233148,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":97,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":4195733,"threads_started":1,"wal-append.queue_time_us":1871894}
I20260709 13:10:52.386569 26438 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.426559 26434 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.431521 26307 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.390525 26275 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.300280 26439 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:52.495266 22628 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:52.485899 26189 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:52.498363 22628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 501) took 4784 ms (client timeout 1890 ms). Trace:
I20260709 13:10:52.499141 26189 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.499389 22628 rpcz_store.cc:269] 0709 13:10:47.713803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.721884 (+  8081us) service_pool.cc:224] Handling call
0709 13:10:52.498044 (+4776160us) spinlock_profiling.cc:243] Waited 4.77 s on lock 0x7b54003d3954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:52.498260 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10974871424}
I20260709 13:10:52.500540 26317 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.501490 26189 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.509660 26436 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.511391 26440 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.483382 26048 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.299707 26437 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.422945 26430 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.429001 26190 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.393427 26259 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:52.393421 26330 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.394321 26105 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.389139 26284 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.476050 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:52.488382 26438 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:52.533198 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57251.8us
I20260709 13:10:52.534391 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:52.492216 26439 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.521934 26285 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:52.539342 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.523514 26259 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.513940 26307 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.540639 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 342602us
I20260709 13:10:52.514384 26275 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.544463 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:52.533363 26284 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:52.536909 26190 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:52.548569 26190 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:52.549906 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.549690 26190 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.531442 26330 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.532857 26105 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.552594 26330 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.552772 26105 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.545305 26259 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:52.550812 22631 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:52.517935 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:52.523775 26430 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.560102 26275 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.561372 26430 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.565253 22634 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:52.537367 26438 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.568921 26443 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.569161 22628 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:52.569597 26438 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.569720 26443 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.571167 26189 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:52.571862 26189 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:52.572469 26443 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.576512 22634 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:52.579560 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.580415 26048 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.582177 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.582952 22632 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:52.585969 26307 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.601266 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.590793 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:52.603317 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.604223 22627 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:52.604856 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.543715 26285 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:52.606837 22782 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 4.743s	user 0.021s	sys 0.015s
I20260709 13:10:52.607334 22782 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:52.609357 26285 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:52.544203 26439 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:52.610803 22631 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:52.621168 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.621649 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.619082 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.599110 26440 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:52.622701 22782 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2451) took 4841 ms (client timeout 2194 ms). Trace:
I20260709 13:10:52.607901 26395 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:52.625880 22775 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:52.626104 26437 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.623267 26126 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:52.626983 26126 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.629017 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2415) took 5202 ms. Trace:
I20260709 13:10:52.629374 26126 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.572970 26189 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:52.624544 22782 rpcz_store.cc:269] 0709 13:10:47.766537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.767032 (+   495us) service_pool.cc:224] Handling call
0709 13:10:52.608197 (+4841165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:52.629514 22701 rpcz_store.cc:276] 0709 13:10:47.426729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.577430 (+150701us) service_pool.cc:224] Handling call
0709 13:10:47.577807 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.620550 (+ 42743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.620584 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.621288 (+   704us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.810914 (+189626us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:47.810986 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.811118 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.628750 (+4817632us) raft_consensus.cc:1685] finished
0709 13:10:52.628779 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.628952 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42207,"raft.queue_time_us":52114,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":61287,"spinlock_wait_cycles":326016,"thread_start_us":163745,"threads_started":1,"wal-append.queue_time_us":239046}
I20260709 13:10:52.631620 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.604995 26438 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.635288 22634 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:52.635871 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89878.5us
I20260709 13:10:52.609701 26442 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.646960 26442 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.649206 26237 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:52.649849 26237 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:52.640857 26438 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.652930 26438 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.653777 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.658654 26237 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.659531 26431 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.660540 26237 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:52.661973 26431 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.649777 26442 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.643945 26264 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.839733 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.835790 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2401) took 5520 ms. Trace:
I20260709 13:10:52.825547 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:52.818847 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.066000 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:52.824106 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.067965 22706 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:53.069254 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.069630 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:52.819610 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:52.812331 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.072939 22628 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:52.641319 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.077638 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:52.817831 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:52.928733 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:52.833389 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2405) took 5517 ms. Trace:
I20260709 13:10:52.834053 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.080291 22705 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:53.080675 22701 raft_consensus.cc:2376] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6. Re-sending same reply.
I20260709 13:10:53.080070 22710 rpcz_store.cc:276] 0709 13:10:47.316371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.569672 (+253301us) service_pool.cc:224] Handling call
0709 13:10:47.570090 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.570481 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.570512 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.575846 (+  5334us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.816974 (+241128us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:47.817020 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.817116 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.833074 (+5015958us) raft_consensus.cc:1685] finished
0709 13:10:52.833097 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.833302 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5842,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":25497,"thread_start_us":235294,"threads_started":1,"wal-append.queue_time_us":409550,"wal-append.run_cpu_time_us":4183,"wal-append.run_wall_time_us":1575126}
I20260709 13:10:52.819265 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:52.820145 22707 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:53.084443 22707 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:52.933328 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:52.842270 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 168589us
I20260709 13:10:53.087834 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:52.816078 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 178032us
I20260709 13:10:53.089123 22710 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
W20260709 13:10:52.936178 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.929627 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.091363 22633 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:52.697126 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:52.671562 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.096027 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.097252 22710 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:53.097434 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:52.835683 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.098508 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.099496 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:53.100546 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.100740 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.101477 22710 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:52.687474 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:53.104722 22707 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:52.685634 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.108732 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:52.669221 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:53.117529 22782 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:52.848555 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.119949 22707 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:10:52.875157 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.711040 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.697641 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.833220 23416 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:52.816023 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.813848 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.692273 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.821781 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:52.672547 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:52.827708 26428 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:52.670058 26430 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.147445 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.148054 26430 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.149713 22706 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:52.848838 26433 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:53.149196 26259 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.595s	user 0.021s	sys 0.005s
I20260709 13:10:53.151232 26259 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.149323 22776 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:53.151569 26433 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.153298 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:52.666869 26434 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.151252 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:52.838302 26439 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.159437 23415 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:53.159292 26434 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.161875 26439 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.634100 26440 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.165277 26440 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.670821 26443 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.166728 26443 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.167546 26433 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.168610 26433 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:53.169281 26433 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:53.169878 26443 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.852800 26445 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:53.170904 26433 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.171988 26445 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:53.162370 26434 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.820544 26446 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:53.173642 26446 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.843773 26455 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.175115 26430 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.175462 26455 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:52.687302 26447 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:53.177155 23417 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:10:53.177695 26447 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.180490 26447 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:52.876850 26457 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.180917 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.180555 26455 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.181844 26457 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.182435 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.184006 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.185154 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.185335 26457 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:52.974028 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.976384 26442 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.994724 26461 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:53.025110 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 367558us
W20260709 13:10:53.050357 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 5.137s	user 0.021s	sys 0.043s
I20260709 13:10:53.061286 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:53.071506 22631 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:53.074980 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.082165 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.088045 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.112349 22629 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:10:53.114125 26264 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:52.859305 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2455) took 5085 ms. Trace:
I20260709 13:10:53.229772 26457 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.115332 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.230721 22628 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.180493 26446 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:52.688957 23385 connection.cc:256] RPC call timeout handler was delayed by -3.3419645789999777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
I20260709 13:10:53.102320 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.120536 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.125536 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.131842 23418 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:10:53.130425 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 202241us
I20260709 13:10:53.067529 22709 rpcz_store.cc:276] 0709 13:10:47.315212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.534637 (+219425us) service_pool.cc:224] Handling call
0709 13:10:47.535002 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.535435 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.535469 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.544130 (+  8661us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.807608 (+263478us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:47.807685 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.807787 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.835511 (+5027724us) raft_consensus.cc:1685] finished
0709 13:10:52.835547 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.835725 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":192650,"threads_started":1,"wal-append.queue_time_us":347212,"wal-append.run_cpu_time_us":9410,"wal-append.run_wall_time_us":5109295}
I20260709 13:10:53.132926 23419 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:53.145239 22634 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.144482 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.148988 23420 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.150331 23414 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:52.636515 26437 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.239877 26447 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:52.838820 26438 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.240898 26291 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.172533 26445 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:53.241607 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.989573 26102 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.241961 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.242341 26102 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.242873 26445 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.243908 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.245158 26445 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:53.180804 23421 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:53.154567 22776 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:53.187637 23412 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:53.151441 22703 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:53.186414 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:53.226789 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 5.314s	user 0.024s	sys 0.049s
I20260709 13:10:53.227909 23411 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:53.082340 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.232506 22778 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:53.230998 22783 rpcz_store.cc:276] 0709 13:10:47.774187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.774361 (+   174us) service_pool.cc:224] Handling call
0709 13:10:47.774860 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.788794 (+ 13934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.788833 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.789404 (+   571us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.999349 (+209945us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:47.999472 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.999622 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:52.101707 (+4102085us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:52.849557 (+747850us) raft_consensus.cc:1685] finished
0709 13:10:52.849597 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:52.859187 (+  9590us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13383,"raft.queue_time_us":14739,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":101,"spinlock_wait_cycles":59610880,"thread_start_us":209281,"threads_started":1,"wal-append.queue_time_us":395006,"wal-append.run_cpu_time_us":39858,"wal-append.run_wall_time_us":4964994}
I20260709 13:10:53.239576 26437 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.252180 22705 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:53.245157 22709 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:53.215056 22706 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:53.240906 26447 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.256285 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.254683 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:53.236892 26439 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.254496 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.259445 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.259234 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.255112 22701 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:53.266726 22777 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:53.257755 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:53.267493 22781 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 5: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:53.264340 26440 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:10:53.257330 26291 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:10:53.270334 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:53.270566 26291 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:53.271092 26291 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:53.272441 26291 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.269038 22703 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:53.279930 26427 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:53.280531 26427 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.280927 26427 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:53.257908 23385 connection.cc:256] RPC call timeout handler was delayed by -2.9721460719999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15680000000000002s for a response
I20260709 13:10:53.283177 22710 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:10:53.283643 23385 connection.cc:256] RPC call timeout handler was delayed by -2.7963090259999035s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 13:10:53.284089 23385 connection.cc:256] RPC call timeout handler was delayed by -2.73737263299995s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
I20260709 13:10:53.284584 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.285501 22710 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:10:53.241657 26438 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.265762 26457 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.286875 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.294000 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.287372 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:53.297626 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.286578 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.287416 22776 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:53.274343 26291 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.300643 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.304407 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.303996 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.297338 22783 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:53.316300 22708 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:53.321069 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:10:53.306137 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 5.41598e+06us
I20260709 13:10:53.292774 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 524) took 5339 ms. Trace:
I20260709 13:10:53.289894 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2461) took 5466 ms. Trace:
I20260709 13:10:53.312285 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:53.264020 26259 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.311767 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:53.330438 23417 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:53.326001 26427 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.324780 22775 rpcz_store.cc:276] 0709 13:10:47.953157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.959619 (+  6462us) service_pool.cc:224] Handling call
0709 13:10:52.608005 (+4648386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:52.626320 (+ 18315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:52.977176 (+350856us) spinlock_profiling.cc:243] Waited 17.7 ms on lock 0x7b54002d5854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:53.292279 (+315103us) spinlock_profiling.cc:243] Waited 4.65 s on lock 0x7b54002d5850. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:53.292547 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10730347520}
I20260709 13:10:53.289831 26329 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.334854 26329 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.337119 26436 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:53.337687 26436 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:53.299890 26447 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.339293 22781 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:53.341920 23411 raft_consensus.cc:2410] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:10:53.302030 26457 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.290429 26439 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.345881 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.343152 26436 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.366758 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.367210 26436 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:53.367463 22780 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:53.368340 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.369441 22780 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:53.369619 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.351583 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.326550 22784 rpcz_store.cc:276] 0709 13:10:47.822986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.826752 (+  3766us) service_pool.cc:224] Handling call
0709 13:10:47.827168 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.839356 (+ 12188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:47.839392 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.839967 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:10:53.288561 (+5448594us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b18001f36c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1
0709 13:10:53.288949 (+   388us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:53.289032 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:53.289185 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:53.289249 (+    64us) raft_consensus.cc:1685] finished
0709 13:10:53.289272 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:53.289615 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":35150,"spinlock_wait_cycles":27977088,"thread_start_us":62505,"threads_started":1,"wal-append.queue_time_us":135552,"wal-append.run_cpu_time_us":2927,"wal-append.run_wall_time_us":1206251}
I20260709 13:10:53.371941 26378 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.372696 26378 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.262236 26461 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.375876 23418 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:53.376382 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.377305 26461 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:53.377967 26237 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.576s	user 0.010s	sys 0.016s
I20260709 13:10:53.351080 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.355583 26462 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:53.356365 26427 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.356204 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:53.359784 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.355122 26264 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.378842 22775 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:53.385040 22709 raft_consensus.cc:2410] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:10:53.386270 23414 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:10:53.388586 26378 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.390810 22704 raft_consensus.cc:2410] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:10:53.392060 26264 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.393330 22783 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:53.396612 26329 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:53.398687 26439 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.400568 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.401371 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.400568 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:53.386029 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89852.1us
W20260709 13:10:53.405547 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49557.8us
I20260709 13:10:53.407405 26105 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:53.407686 22629 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:53.408118 26105 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:53.409114 22781 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:10:53.409343 26105 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.261194 26442 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.410526 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.418427 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:53.418658 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.419654 22627 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:53.420141 26307 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:53.420478 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:53.420922 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.420634 26307 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.415557 22710 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:53.422173 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:53.425781 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69683.3us
I20260709 13:10:53.426571 22632 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:10:53.427423 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:10:53.412915 26264 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.430706 26462 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.433158 26437 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:53.433485 26462 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:53.434474 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 180634us
I20260709 13:10:53.436070 25533 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:53.436612 25533 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:53.437263 25533 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:53.436537 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.426386 26028 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:53.450450 22775 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:10:53.452919 26298 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:53.455220 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 168227us
I20260709 13:10:53.455403 26427 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:53.456316 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.458132 26438 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:53.458654 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.459053 26438 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:53.459539 26438 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.459896 26438 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:53.461436 26442 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.462185 22779 raft_consensus.cc:2376] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7. Re-sending same reply.
I20260709 13:10:53.464424 22776 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:53.468930 26378 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:53.469703 26378 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.472963 26378 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.427973 26307 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:53.473990 26429 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:53.474494 26429 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:53.475482 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:53.477669 22707 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:10:53.479306 22707 pending_rounds.cc:85] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 2
I20260709 13:10:53.446486 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:53.475607 26429 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:53.430013 26180 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.797s	user 0.005s	sys 0.025s
I20260709 13:10:53.481195 22782 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:53.482165 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:53.485330 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.487106 26189 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:53.487468 26276 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.491056 22780 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:53.496584 26427 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.497360 26427 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.498219 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.499423 26429 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:53.474015 26284 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.006s
I20260709 13:10:53.509058 22703 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:53.513087 22747 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.479734 22707 pending_rounds.cc:107] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:10:53.516108 22707 raft_consensus.cc:2889] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:53.520530 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.484791 26455 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:10:53.524469 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 84999.9us
W20260709 13:10:53.526806 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.523118 22783 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:10:53.527128 26437 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:53.527406 26230 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:53.530555 26462 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.541545 26430 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.542428 26430 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.545253 26430 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.546736 26189 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:53.509857 26180 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.550354 22701 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:53.536561 26230 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.553987 26230 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:53.554738 26180 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:53.559182 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 138927us
I20260709 13:10:53.561383 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.562593 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.563616 23419 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:53.566468 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:53.566958 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67040us
I20260709 13:10:53.568041 23419 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:53.568158 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:53.569905 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.570890 26436 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:53.572031 22708 raft_consensus.cc:2410] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:10:53.571805 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.572613 26462 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.511801 26439 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:53.571831 26436 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:53.478756 26102 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.575158 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.576341 23416 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:10:53.578420 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.537066 26437 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:53.510205 26284 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.580178 26437 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:53.586220 26291 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:53.579922 22635 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:53.587159 23421 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:53.588335 26284 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.588383 26102 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.574496 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:53.574949 26439 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.589789 26436 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.592549 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.575100 26462 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.579777 22706 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:53.596961 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.593743 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.625123 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:53.610569 26291 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:53.618487 26439 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:53.635378 23412 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:53.632818 25533 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.620052 26436 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.644683 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.647692 26189 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:53.648696 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:53.618386 26276 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:10:53.648919 26126 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:53.650978 26126 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.651479 26126 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.647492 26431 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:10:53.655620 26431 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:53.656044 26431 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:53.657284 26431 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:53.658793 26285 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.011s	user 0.011s	sys 0.018s
I20260709 13:10:53.688066 26285 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.645206 26462 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:53.646960 26438 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:53.681336 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107006us
I20260709 13:10:53.681025 22529 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 8, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:53.695950 22628 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
W20260709 13:10:53.682133 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72309.2us
I20260709 13:10:53.682894 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:53.683348 23442 debug-util.cc:398] Leaking SignalData structure 0x7b0800232ca0 after lost signal to thread 21529
I20260709 13:10:53.687282 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.689919 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:53.690320 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 215013us
I20260709 13:10:53.640318 26291 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:53.659968 26220 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:53.629835 26427 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.696347 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.646112 26440 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:53.704309 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:53.715123 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08002f11e0 after lost signal to thread 22567
W20260709 13:10:53.723553 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81417.2us
W20260709 13:10:53.724220 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.712163 26431 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:53.730798 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88510.1us
I20260709 13:10:53.736905 23420 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:53.712317 26440 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.746219 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.749475 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:53.747175 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.750113 22709 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:53.756562 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.759790 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.762965 22634 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:53.761389 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.760752 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.771261 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.763907 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:53.779183 26220 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:53.779899 26478 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:10:53.780771 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45874.2us
I20260709 13:10:53.781090 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.786782 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.754396 26440 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.784060 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.788769 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:53.789175 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb560 after lost signal to thread 22659
I20260709 13:10:53.790762 26180 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:53.792532 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.793313 22707 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:53.793851 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 148434us
I20260709 13:10:53.795066 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.795676 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:53.795884 22710 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:53.797096 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.799072 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.793183 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:53.799753 22701 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:53.800534 26285 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.801542 26180 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:53.801998 26180 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.802380 26180 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:53.803102 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.803758 22708 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:53.803701 26437 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.805475 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.806114 22703 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:10:53.806782 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:53.799310 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:53.808565 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.798415 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.811208 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.806320 26437 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.816355 22704 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:10:53.819512 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.820449 22633 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:10:53.820878 22631 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:53.821494 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.821677 26126 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.822346 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:53.822882 22702 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:53.823447 26298 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:53.825712 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.783614 26445 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:10:53.828570 26445 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:53.828989 26445 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:53.829828 26126 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.819512 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.831503 22626 raft_consensus.cc:2368] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
W20260709 13:10:53.835275 26467 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:10:53.836133 26438 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:53.837212 25533 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.852233 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:53.852937 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127501us
I20260709 13:10:53.852336 22707 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:53.855640 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:53.856616 23412 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:53.859476 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.860234 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 671) took 1022 ms. Trace:
I20260709 13:10:53.860694 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:53.789863 26189 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:53.863269 22627 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.863919 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.866546 23419 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:10:53.860152 22634 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:10:53.860616 23420 rpcz_store.cc:276] 0709 13:10:52.837315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:53.247413 (+410098us) service_pool.cc:224] Handling call
0709 13:10:53.247913 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:53.737673 (+489760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:53.859991 (+122318us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54000da7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:53.860172 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124106496}
I20260709 13:10:53.864872 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.868754 22626 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:53.871845 22709 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:53.873145 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:53.873833 23416 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.830045 26445 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.875875 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:53.869621 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55484.4us
I20260709 13:10:53.876901 22707 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:53.839643 26230 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:53.867699 26477 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:10:53.880666 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.881776 22628 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:53.816097 26028 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:53.852109 26478 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:53.880618 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.868204 26189 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:53.880908 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.891858 22704 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:53.893733 22632 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.891259 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.903177 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.904277 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.904804 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.905130 22707 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:53.905622 22628 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:53.906553 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.873188 26285 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:53.912043 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:53.917622 22635 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:53.878134 26188 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:53.926908 26298 raft_consensus.cc:2764] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:10:53.927881 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.930919 22627 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:53.780653 26220 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.935443 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.936420 22706 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:53.941025 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:53.941969 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:53.946138 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66177.9us
I20260709 13:10:53.884732 26028 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.947324 26478 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:53.948379 26440 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.911609 26189 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:53.975700 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.949363 26445 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:53.949405 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:53.954945 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:53.984211 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:53.948781 23416 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:53.970201 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:53.986109 22707 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:10:53.981940 22703 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:53.963423 23412 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:10:53.922642 26285 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.988783 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.992348 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:53.993132 22709 raft_consensus.cc:2376] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12. Re-sending same reply.
I20260709 13:10:53.969681 26028 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:53.983372 26448 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:53.994541 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:53.995157 26448 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.995584 26448 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:53.996124 26440 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:53.911223 26307 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.998834 26307 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:53.983771 22705 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:54.005467 22705 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:53.895695 26436 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:53.988543 26285 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:54.009181 26436 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:54.011735 22702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 486) took 1194 ms (client timeout 1568 ms). Trace:
W20260709 13:10:54.012213 22702 rpcz_store.cc:269] 0709 13:10:52.817764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:52.825276 (+  7512us) service_pool.cc:224] Handling call
0709 13:10:54.011386 (+1186110us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54000e7754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:54.011637 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653195648}
I20260709 13:10:54.015151 22629 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:10:54.016093 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.017118 22631 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:54.020041 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08002d0e80 after lost signal to thread 22734
I20260709 13:10:54.019577 26180 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.022775 26180 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:54.023471 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.015311 22706 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:54.024427 22707 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:54.024875 22709 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:54.025864 26285 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.029207 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:54.036075 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.037613 26285 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:54.038936 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:54.021802 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08001d31c0 after lost signal to thread 22808
I20260709 13:10:54.043138 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:54.044538 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08003f1940 after lost signal to thread 23445
I20260709 13:10:54.045255 22634 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:10:54.033829 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.034070 22784 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:54.052959 22703 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:54.002306 22701 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:54.054822 26237 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.055537 26237 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.055349 22529 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 7. New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.059544 22626 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
W20260709 13:10:54.061811 26429 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
I20260709 13:10:54.099673 22749 leader_election.cc:400] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:54.100122 22749 leader_election.cc:403] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:54.093977 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.074903 26434 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:54.114149 22781 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
W20260709 13:10:54.114779 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52096.2us
I20260709 13:10:54.115736 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:54.121308 23419 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.057665 22529 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 3 to 4, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.118673 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:54.137534 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.150703 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.116418 26436 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:10:54.128131 22529 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 6, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:54.161314 23384 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.568s (SENT)
I20260709 13:10:54.164055 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.167487 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:54.169699 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 266308us
I20260709 13:10:54.117689 26434 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:54.176457 22629 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:54.109426 26291 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.150383 26285 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:54.200184 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:54.201251 23420 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 13 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:54.136900 26188 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:10:54.165318 26307 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.204854 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.204377 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:54.180507 26434 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:54.209736 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.209813 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:54.145273 26439 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.243521 26285 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:54.245170 26439 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:54.246038 23420 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:54.247649 23414 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:10:54.249055 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.214964 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.215835 26180 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:54.255239 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.255542 22626 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:54.221266 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.255793 22710 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:10:54.179925 26237 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.262701 26481 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:54.265133 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.265931 22778 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:54.266892 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.215024 26329 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:54.267894 26329 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:54.268023 26395 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:54.268656 26395 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.268945 26329 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.270962 26190 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.271708 26190 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:54.273636 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 486966us
I20260709 13:10:54.275966 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.222975 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.212502 26189 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.237265 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.239428 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.211347 26438 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:10:54.250262 22709 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.220497 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.165436 26431 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:10:54.259932 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.266983 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.281816 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.289232 22783 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:54.285771 22708 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:54.228638 26448 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.298316 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:54.304387 23412 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:54.299832 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.274402 26190 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.302685 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.307946 22779 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:54.313997 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.317643 26429 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:54.318454 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.318771 26429 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:54.319254 26429 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:54.319550 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.320544 25399 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:54.321174 26472 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:54.321730 26472 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:54.322119 26472 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:54.320236 26429 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.294232 26481 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:54.306649 26230 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.323715 26481 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.320756 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.326122 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.330334 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:54.327339 22783 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8. Re-sending same reply.
I20260709 13:10:54.211927 26307 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.333024 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:54.293809 26478 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:10:54.244882 26285 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.321056 25768 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:54.357200 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.358033 22783 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:54.337083 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.335337 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.368026 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.351872 26102 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:54.357841 25768 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:54.383462 26493 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:10:54.385008 26395 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:54.386868 26276 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:54.359987 26126 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:54.388315 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:54.374867 26264 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:54.395102 26102 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.374537 26431 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:54.371327 22529 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:54.369830 26439 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:54.403774 26436 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:54.405154 26220 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:54.405319 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.423491 26462 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:54.433307 26462 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.437755 22779 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:10:54.406308 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.428056 26431 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.439150 26431 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:54.438827 26479 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:10:54.441264 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.443459 22629 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:10:54.413496 25768 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.373601 26028 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:54.410893 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77992.7us
I20260709 13:10:54.411784 22777 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:54.413873 22784 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:54.414299 22778 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:54.377216 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.375231 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:54.456396 22601 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.423967 22783 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:54.412036 26298 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.426533 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.386962 26448 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:54.424005 26126 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:54.445473 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57269.5us
I20260709 13:10:54.428717 26395 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:54.471225 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.409250 23419 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:54.459072 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.473568 22783 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:54.460791 22776 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:54.445055 26264 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.475114 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.475962 22779 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:54.476948 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.478242 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.458504 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.456769 26439 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:54.457264 26220 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:54.480027 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:54.481030 22782 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:54.485540 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.477663 22781 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:54.487075 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.488298 26102 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.489496 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.490870 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.491528 22779 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:54.492434 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.493000 26434 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.494006 26028 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:54.495697 26190 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:54.493410 22779 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:54.496870 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:54.501933 26264 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.507551 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.472110 26298 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.508971 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.510015 26298 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.510201 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.463871 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:54.514696 22784 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
W20260709 13:10:54.515581 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51818.8us
I20260709 13:10:54.515944 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.459926 26436 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 13:10:54.541903 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45127.4us
I20260709 13:10:54.496845 26220 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.519055 22783 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:54.517163 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:54.522214 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 161876us
I20260709 13:10:54.540464 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.547165 22779 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:54.540841 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.547508 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.511301 26276 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:54.549849 22784 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:54.483302 26291 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.517035 26438 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:54.539444 26284 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:54.542527 26298 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:54.553421 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.573562 22783 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:54.555147 22784 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
W20260709 13:10:54.559624 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74177.6us
I20260709 13:10:54.496146 26190 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:54.576009 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:54.551055 26220 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:54.548588 26189 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:54.582103 22709 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:54.591665 25399 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:54.592161 25399 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.580082 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:54.566772 26476 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:10:54.579429 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.594412 23412 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:10:54.594687 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.599797 22779 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:10:54.602246 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.567595 26438 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:54.610968 22780 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:10:54.609790 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.612788 22784 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:54.576779 26285 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:54.614666 22784 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:54.540211 26490 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:10:54.488313 26439 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.594044 26284 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:54.626264 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.605340 26291 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:54.658881 26501 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:54.644635 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:54.662081 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.663015 26291 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.613358 26434 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:54.679829 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.616480 26438 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:54.620229 26259 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:54.683694 26259 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.684082 26259 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:54.612287 26476 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.611223 26190 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.624809 25399 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:54.679359 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103608us
W20260709 13:10:54.664040 26499 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:54.682080 22782 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:54.682965 22780 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:10:54.683120 26498 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:10:54.685046 22781 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:54.615908 26436 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:54.622402 26276 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.616354 26502 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:54.687795 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 137687us
I20260709 13:10:54.644271 26448 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:54.692090 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:54.622017 26105 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
W20260709 13:10:54.720654 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 126091us
I20260709 13:10:54.721647 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.722047 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.668581 26307 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.693415 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.679378 26180 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.697368 25399 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.712949 23412 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:54.717658 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.722987 22708 raft_consensus.cc:2376] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8. Re-sending same reply.
I20260709 13:10:54.733588 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.734855 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.735715 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:10:54.737160 22781 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:54.737826 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.740245 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.744335 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.752306 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.753980 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.754951 22709 raft_consensus.cc:2376] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10. Re-sending same reply.
I20260709 13:10:54.757102 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.759177 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.759879 22707 raft_consensus.cc:2376] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9. Re-sending same reply.
W20260709 13:10:54.760601 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112766us
I20260709 13:10:54.735874 26490 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:54.761865 26490 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.763025 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.764521 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.745517 26298 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:54.745682 26395 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:54.769279 25768 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.769868 26395 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:54.770314 26395 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:54.762247 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.772544 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:10:54.764125 26126 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.734694 26285 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:54.725112 26502 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.740660 26430 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:54.780654 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.751663 26501 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.782862 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.735914 26180 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.785578 26126 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:54.815536 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.785737 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:54.786015 25768 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:54.745075 26448 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:54.781427 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.791285 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.797443 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:54.797456 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.806936 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.824741 22707 raft_consensus.cc:2376] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6. Re-sending same reply.
W20260709 13:10:54.784262 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.773816 26230 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:54.782253 26434 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:54.805124 26509 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:54.811173 26508 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:10:54.812803 22529 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 7, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:54.817157 22706 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:10:54.808429 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.763342 26490 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.874976 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:54.830593 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.845307 26434 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:54.857270 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.808087 26291 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.861927 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:54.864777 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:54.872073 25399 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:54.804397 26462 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:54.804242 26436 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:54.863071 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:54.824810 26190 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.884966 22777 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:54.807246 26506 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:54.833899 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:54.829406 26230 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.888842 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:54.890121 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:54.890796 22702 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:54.894145 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 85795.2us
I20260709 13:10:54.899286 22630 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:54.873198 26448 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:54.881771 26434 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:54.780169 26276 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:54.803939 26501 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.903292 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:54.807191 26284 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:54.906646 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 120987us
W20260709 13:10:54.928828 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:54.948560 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:54.906030 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 154369us
W20260709 13:10:54.928260 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97818.8us
I20260709 13:10:54.949575 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:54.917356 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 126159us
I20260709 13:10:54.950562 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:54.888347 26436 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.982106 26230 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:54.882669 26430 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:10:55.045936 26436 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:54.921692 26437 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:54.807240 26476 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.084044 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:55.081633 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131142us
I20260709 13:10:54.985299 26503 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:54.954053 26479 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:10:55.077135 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 127630us
W20260709 13:10:54.987794 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 84594.9us
W20260709 13:10:55.045557 26511 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:10:55.099490 26102 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:54.907413 26490 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:55.081061 26521 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:55.043058 26448 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.005789 26506 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:55.112500 26230 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:55.114562 23419 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:10:55.121687 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:55.096735 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:54.909019 26438 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:55.133473 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 235133us
W20260709 13:10:55.053333 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 105269us
I20260709 13:10:55.136704 22675 leader_election.cc:400] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
I20260709 13:10:55.109706 26437 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:55.149235 22675 leader_election.cc:403] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:55.122355 26503 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.985879 26285 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:55.165861 23385 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.677s (SENT)
I20260709 13:10:55.130318 26102 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:55.171541 23412 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:55.177737 22630 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:55.156826 26437 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:55.178241 22630 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:55.187878 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 140212us
W20260709 13:10:55.189772 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53310.1us
I20260709 13:10:55.129300 25768 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:10:55.167426 26462 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7.
W20260709 13:10:55.206856 22533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 248) took 37358 ms (client timeout 15000 ms). Trace:
I20260709 13:10:55.163239 26503 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:55.163538 26501 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.197496 22630 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.175884 26479 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:54.927480 22529 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 8, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:55.189368 26472 consensus_meta.cc:287] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.863s	user 0.006s	sys 0.008s
I20260709 13:10:55.178117 26102 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:55.227955 22776 raft_consensus.cc:2410] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:10:55.228127 26259 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:10:55.182264 26502 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.202033 26395 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.226114 26438 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:55.210378 22630 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:55.249480 23419 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:10:55.230777 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.209103 26180 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:55.251336 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 129752us
I20260709 13:10:55.251436 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:55.215970 26028 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:10:55.260234 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.243659 26478 raft_consensus.cc:2764] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:10:55.269412 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.270134 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:55.269867 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 120961us
I20260709 13:10:55.276433 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:55.279060 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 150587us
I20260709 13:10:55.243997 26472 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.228634 26259 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.260118 26028 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:55.284657 22626 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:55.290534 22775 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
W20260709 13:10:55.277411 26431 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.838s	user 0.012s	sys 0.003s
I20260709 13:10:55.275199 26502 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:10:55.280279 26515 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 13:10:55.295259 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113814us
I20260709 13:10:55.282408 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:55.297902 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.296918 26502 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.303153 26502 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:55.304708 26395 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:55.290108 26028 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:55.296666 26431 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.306749 23417 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:55.309505 22778 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:55.310091 26431 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:55.311153 26520 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:10:55.293210 22529 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:55.320757 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.548s (SENT)
W20260709 13:10:55.207451 22533 rpcz_store.cc:269] 0709 13:10:17.726278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.742780 (+ 16502us) service_pool.cc:224] Handling call
0709 13:10:55.080716 (+37337936us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:43.852006 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:43.853003 (+   997us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:43.853083 (+    80us) write_op.cc:435] Acquired schema lock
0709 13:10:43.853131 (+    48us) tablet.cc:662] Decoding operations
0709 13:10:43.854442 (+  1311us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:43.854588 (+   146us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:43.874325 (+ 19737us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:10:44.104662 (+230337us) tablet.cc:701] Row locks acquired
0709 13:10:44.104742 (+    80us) write_op.cc:260] PREPARE: finished
0709 13:10:44.105015 (+   273us) write_op.cc:270] Start()
0709 13:10:44.105254 (+   239us) write_op.cc:276] Timestamp: P: 1783602644104913 usec, L: 0
0709 13:10:44.105293 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:10:44.218834 (+113541us) log.cc:844] Serialized 2834 byte log entry
0709 13:10:44.978706 (+759872us) op_driver.cc:464] REPLICATION: finished
0709 13:10:45.222758 (+244052us) write_op.cc:301] APPLY: starting
0709 13:10:45.222937 (+   179us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:45.223302 (+   365us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:45.223468 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:45.245665 (+ 22197us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:45.250407 (+  4742us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:10:45.250475 (+    68us) write_op.cc:312] APPLY: finished
0709 13:10:45.250917 (+   442us) log.cc:844] Serialized 59 byte log entry
0709 13:10:45.251154 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:45.251299 (+   145us) write_op.cc:454] Released schema lock
0709 13:10:45.251383 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:45.251925 (+   542us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:45.251960 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":84383,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":211986,"apply.run_cpu_time_us":8629,"apply.run_wall_time_us":51401,"num_ops":7,"prepare.queue_time_us":366796,"prepare.run_cpu_time_us":40125,"prepare.run_wall_time_us":523072,"raft.queue_time_us":548229,"raft.run_cpu_time_us":27381,"raft.run_wall_time_us":113454,"replication_time_us":873174,"thread_start_us":1124780,"threads_started":4,"wal-append.queue_time_us":147636,"wal-append.run_cpu_time_us":23808,"wal-append.run_wall_time_us":1886777}]]}
W20260709 13:10:55.323666 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48681.8us
I20260709 13:10:55.316704 22529 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 7, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.324664 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.293682 26501 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:55.326932 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.327459 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.328794 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:55.322137 26504 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:55.330444 26504 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.331030 26482 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:55.331892 26482 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:55.332500 26482 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:55.332767 26504 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:55.285660 26479 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:55.336699 26523 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:55.331234 22710 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:55.341993 26462 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:55.342760 26462 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.344647 26259 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.345165 26462 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.292511 26105 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:55.346019 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 548) took 1080 ms. Trace:
I20260709 13:10:55.353642 22702 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:55.354337 26180 raft_consensus.cc:2764] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:10:55.355281 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:55.355940 23421 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:55.332808 26482 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:55.356623 26482 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:55.357553 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:55.358908 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:55.362962 26298 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:55.363533 26298 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:55.364048 26431 raft_consensus.cc:2764] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:10:55.364403 26298 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.365545 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:55.367044 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 84756.7us
I20260709 13:10:55.367365 26430 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:55.368151 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:55.369204 26430 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:55.369616 26430 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:55.371377 23414 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:55.374933 22630 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
W20260709 13:10:55.375276 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.375448 22630 pending_rounds.cc:85] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 9
W20260709 13:10:55.375123 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.375947 22630 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:10:55.376335 22630 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:55.346431 22778 rpcz_store.cc:276] 0709 13:10:54.265911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:54.475730 (+209819us) service_pool.cc:224] Handling call
0709 13:10:55.345797 (+870067us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54002d5854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:55.345947 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889139968}
I20260709 13:10:55.386216 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:55.325691 22529 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.412437 23416 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
W20260709 13:10:55.413986 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:55.416894 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.423815 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:10:55.427630 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 106911us
I20260709 13:10:55.388698 26462 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:55.342337 26028 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.388201 26431 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:55.432343 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:55.407070 26291 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:10:55.408438 26105 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:55.407337 26512 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.314602 26102 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.412564 26481 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:10:55.446022 23421 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:55.448323 23418 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 13:10:55.449234 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.568s (SENT)
I20260709 13:10:55.449882 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:55.451253 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:55.461438 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 79311.5us
I20260709 13:10:55.463709 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:55.466003 25399 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:55.466493 25399 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:55.467499 26472 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.470801 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.472709 26105 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:55.475274 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 177441us
W20260709 13:10:55.476255 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 110824us
I20260709 13:10:55.477051 26434 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.477583 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.419516 26506 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:10:55.478480 26431 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:10:55.479413 26506 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:55.482488 22784 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:55.485518 22775 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:55.467417 25399 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.370484 26430 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.489189 22634 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:55.494021 22779 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:55.458879 26478 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:10:55.455482 26439 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:55.479890 26506 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.495270 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:55.507532 22777 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:55.508288 26534 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.508963 26534 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.510739 26527 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
I20260709 13:10:55.512912 26102 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.505113 26429 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:55.490820 26462 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:55.502473 26478 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.515192 26462 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:55.515396 26478 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:55.516217 26462 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.517657 26462 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:55.525887 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.493073 26395 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:55.529114 26395 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:55.530771 26448 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:10:55.530437 26395 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.532577 26430 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:55.457109 26512 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.535746 26528 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:55.536832 26531 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:55.538463 26509 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:55.538970 26509 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:55.502943 25399 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:55.496685 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47490.8us
I20260709 13:10:55.505988 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:55.539914 26509 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.555488 26509 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:55.559335 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60529.8us
W20260709 13:10:55.550499 23385 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.667s (SENT)
I20260709 13:10:55.562201 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.562438 22779 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:10:55.541042 26506 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:55.568660 26434 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:55.570629 26437 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.573123 26531 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:55.582098 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.599354 22777 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:55.603492 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:55.613595 22704 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:55.604116 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.604188 26502 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:55.606721 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:55.567297 26028 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:55.626907 22710 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:55.602985 26437 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:55.503568 26276 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
W20260709 13:10:55.607555 26220 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.004s	user 0.013s	sys 0.005s
I20260709 13:10:55.507397 26521 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:10:55.659021 26502 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:55.660467 26540 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:10:55.662128 26102 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:55.651716 26298 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:55.608115 26533 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:55.663421 26533 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:55.663328 26298 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.664511 26533 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:55.666891 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116489us
I20260709 13:10:55.666740 26220 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:55.659237 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55221.2us
I20260709 13:10:55.670461 22629 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:55.688099 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:55.691246 23385 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.890s (SENT)
I20260709 13:10:55.696349 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.672438 26102 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.683018 26298 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:55.724534 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:55.527350 22529 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:10:55.729049 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59397.1us
I20260709 13:10:55.729594 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:55.730082 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:55.687969 26481 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:10:55.763785 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 91685.2us
I20260709 13:10:55.721591 26276 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:55.725286 26102 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:55.693193 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 321908us
W20260709 13:10:55.763911 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72709.9us
W20260709 13:10:55.769783 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:55.807323 26436 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.676s	user 0.017s	sys 0.005s
I20260709 13:10:55.757058 26431 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.757108 26259 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:55.823664 22710 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:10:55.826501 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:55.771041 26276 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:55.752600 26284 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 13:10:55.832172 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 102218us
I20260709 13:10:55.772472 26512 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:55.810371 26490 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:55.812937 26510 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:10:55.850108 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 175282us
I20260709 13:10:55.846783 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:55.818921 26437 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:55.852303 26490 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:55.865942 26501 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.003s
I20260709 13:10:55.833590 26431 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.914227 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 529) took 1429 ms. Trace:
I20260709 13:10:55.921056 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:55.922132 22782 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
W20260709 13:10:55.900566 26503 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.692s	user 0.007s	sys 0.006s
I20260709 13:10:55.921149 22629 rpcz_store.cc:276] 0709 13:10:54.455904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:54.456078 (+   174us) service_pool.cc:224] Handling call
0709 13:10:55.885151 (+1429073us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540006b8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:55.885386 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738818816}
I20260709 13:10:55.873478 26105 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:55.928519 22778 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:55.871887 26510 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:55.906981 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60509.8us
W20260709 13:10:55.922792 26520 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:55.912895 26490 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:10:55.951601 23385 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.931s (SENT)
I20260709 13:10:55.922580 26501 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:55.871168 26437 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:55.954308 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 200459us
I20260709 13:10:55.829972 26436 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:55.955941 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 147168us
I20260709 13:10:55.959937 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:55.967561 22708 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:55.972334 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:55.982434 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:55.983307 22706 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:55.944975 26503 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:55.989822 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:55.991163 26532 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:10:55.994738 26431 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:10:56.006104 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54541.7us
I20260709 13:10:56.013118 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.017248 22629 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:55.963132 26543 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:56.018787 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:56.021565 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.022495 22706 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:56.023108 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:55.977454 26504 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:56.026031 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.027153 22705 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:56.034384 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.036938 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:55.965281 26475 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.229s
I20260709 13:10:55.965561 26180 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:10:55.864153 22529 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:56.056237 22708 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:10:56.055374 26545 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 13:10:56.030253 22710 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12.
I20260709 13:10:56.079437 22704 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:56.080159 26549 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:10:56.080736 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:56.082962 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64443.8us
I20260709 13:10:56.083956 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:56.084466 26554 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:10:55.926074 26276 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.080680 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:56.086884 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.055684 26544 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.251s	user 0.001s	sys 0.001s
I20260709 13:10:56.106796 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.109572 22706 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:56.087253 26502 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.110584 26502 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.111377 26508 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:10:56.111927 26508 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:56.112823 26508 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:56.104803 26523 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.059s	user 0.000s	sys 0.003s
I20260709 13:10:56.033998 26476 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:10:56.108747 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.139596 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.147459 22709 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:56.142176 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.149487 22706 raft_consensus.cc:2410] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:10:56.152644 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 180405us
W20260709 13:10:56.154412 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70609.1us
W20260709 13:10:56.114567 26478 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.001s
W20260709 13:10:56.128208 26430 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.003s
W20260709 13:10:56.162097 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.033565 26543 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.161928 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:56.122799 26482 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.766s	user 0.015s	sys 0.000s
W20260709 13:10:56.167057 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.167008 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:56.168108 23414 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:56.125905 26431 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:56.114843 26567 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:56.161530 26436 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:56.174887 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:10:56.175334 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.177291 26553 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.138s	user 0.005s	sys 0.000s
W20260709 13:10:56.170142 26506 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.601s	user 0.013s	sys 0.000s
I20260709 13:10:56.162438 26264 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:56.195710 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.195326 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:10:56.199482 22709 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:56.201761 23414 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:10:56.203403 23418 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:10:56.203797 23418 pending_rounds.cc:85] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 8
I20260709 13:10:56.203734 26220 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.204186 23418 pending_rounds.cc:107] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:10:56.204478 23418 raft_consensus.cc:2889] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:56.207710 26562 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.152s	user 0.002s	sys 0.003s
I20260709 13:10:56.211722 26502 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:56.212167 26547 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 13:10:56.181017 26431 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.164386 26478 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.215328 26503 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.216221 26437 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.216372 26543 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.216840 26478 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:56.219901 26509 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.664s	user 0.011s	sys 0.002s
I20260709 13:10:56.181744 25768 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
W20260709 13:10:56.223729 26429 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.004s
I20260709 13:10:56.225652 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.227047 22707 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:10:56.095208 26510 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.231933 26564 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:56.232525 26564 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.232875 26564 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:56.181543 26567 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:56.250125 26567 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:56.255733 26102 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.177321 22529 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 7, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:56.180516 26479 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:10:56.261243 26442 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.261863 26442 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.171177 26482 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.259925 22529 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:56.257109 26431 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:56.243693 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.242823 26105 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:56.272140 26569 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:56.265074 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:56.275635 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.267381 26531 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:56.269163 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.278774 22709 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:56.271071 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.279486 22703 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:56.270191 22704 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:10:56.281513 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.282146 26482 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:56.271744 26538 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:10:56.277658 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77731.3us
I20260709 13:10:56.278208 22627 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:10:56.274582 26569 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.277223 22706 raft_consensus.cc:2393] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:56.291908 26569 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:56.292932 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.294165 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.283633 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.295522 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.296640 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.297773 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.304014 26482 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.304632 26482 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.307791 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:56.301285 22529 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 6, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:56.312068 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.312810 22705 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:56.313834 22704 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:56.312973 22703 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:56.312935 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:56.316397 22709 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:56.309711 22529 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:10:56.319489 26434 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.317505 26472 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:56.265935 26490 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.325660 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:56.340971 23414 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:56.325870 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:56.342684 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.343035 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.343357 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.344394 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.331971 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.333381 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.345705 22635 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:10:56.200098 26506 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.287884 26436 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.277946 26531 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:56.350843 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.352936 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:56.354161 23419 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:56.322748 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.337478 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:56.291136 26538 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:56.329546 22629 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:10:56.346670 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:56.347551 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:56.349928 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:56.349040 26478 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:56.336127 26437 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.358520 23418 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:56.287536 26264 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:10:56.333410 26475 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:56.355151 22705 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6. Re-sending same reply.
I20260709 13:10:56.367965 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.365872 22629 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:10:56.370256 22701 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:56.371038 22705 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:10:56.371803 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.372301 22675 connection.cc:256] RPC call timeout handler was delayed by -1.7475158960000954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
W20260709 13:10:56.372604 22675 connection.cc:256] RPC call timeout handler was delayed by -1.680830277000041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 13:10:56.372838 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5791302770001039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:10:56.373283 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0778302770000892s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
I20260709 13:10:56.375203 25768 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:56.375507 26264 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:56.375990 26264 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:56.376899 26264 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.325022 26105 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:10:56.382227 26442 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:56.358299 26220 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:10:56.383050 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.383735 22782 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:56.383942 26220 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:56.384342 26220 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:56.385274 26220 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.361194 26436 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.387724 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.390925 22784 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:56.371174 22747 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:56.392185 22626 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:56.394838 22779 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:56.395128 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.395447 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.395810 22783 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:56.395915 22630 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:56.396210 22784 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:56.397303 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.398100 22783 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:56.397322 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:56.403007 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.349714 26482 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:56.336460 26573 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:10:56.339933 26434 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.410154 22783 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [6.4-8.5]
W20260709 13:10:56.418576 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81217.3us
I20260709 13:10:56.410468 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.425118 22780 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:56.404798 22782 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:56.406493 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.408543 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.408946 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.428187 22705 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:56.391834 26429 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.436563 22776 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:56.429795 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.437475 22778 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:56.409209 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.430661 23417 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:56.378571 26264 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:56.439410 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:56.419951 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.431598 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.390580 26504 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:56.406841 26189 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:56.452955 22777 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:56.442891 26429 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.455432 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.422197 26482 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:10:56.369438 26437 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.450881 26535 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:56.417984 26462 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.900s	user 0.013s	sys 0.004s
I20260709 13:10:56.455803 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:56.447715 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.456444 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:56.461529 22783 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:10:56.429457 26220 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:56.486277 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.487215 22776 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:10:56.488889 22707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2480) took 1711 ms (client timeout 2034 ms). Trace:
I20260709 13:10:56.463491 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.490480 22782 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:10:56.482446 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:56.437139 26298 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:56.491529 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.484094 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.493134 22629 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:56.497537 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.482357 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.498457 22704 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:10:56.498769 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.499434 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.460136 26189 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:56.489331 22707 rpcz_store.cc:269] 0709 13:10:54.777926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:54.825594 (+ 47668us) service_pool.cc:224] Handling call
0709 13:10:56.483060 (+1657466us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540010e354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:56.488816 (+  5756us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3107153152}
I20260709 13:10:56.488723 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.417646 26509 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:10:56.519639 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.500537 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:56.494343 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.430258 26538 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.509440 26571 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:56.511211 26534 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.502455 22630 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:56.462744 26531 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:56.483587 26555 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:56.517516 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.492581 22775 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
W20260709 13:10:56.521962 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.472443 26533 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:56.524135 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.484279 26476 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.527279 25399 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:56.529156 22778 raft_consensus.cc:2410] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:10:56.484423 26503 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:56.533481 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.522377 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.524071 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:56.556600 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.562862 22630 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:10:56.567629 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51654.1us
I20260709 13:10:56.568581 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:56.571647 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47657.6us
I20260709 13:10:56.572857 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:56.519026 26189 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:56.575902 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.524587 26439 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:56.579593 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.491987 26521 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:10:56.490926 26535 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.583643 22705 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:56.529781 25768 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:10:56.589562 25399 consensus_queue.cc:1243] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:10:56.611336 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.596218 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.747023 22708 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:10:56.579680 26479 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.663269 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.670742 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:56.753736 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 297364us
I20260709 13:10:56.579012 26476 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.589233 26102 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:56.754282 22705 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:56.755458 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:10:56.756522 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.690796 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 117993us
I20260709 13:10:56.496733 26508 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:10:56.762013 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:56.758909 22628 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:56.763037 26482 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:56.763998 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.765447 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.766306 22708 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:10:56.771521 22705 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:10:56.772795 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.773408 22705 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:56.588012 26573 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:56.736512 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.589272 26534 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:10:56.796139 22626 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:56.798355 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.518797 26105 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:10:56.738133 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.590404 26291 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:56.593868 26502 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:10:56.745901 22631 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:10:56.839598 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:56.840409 22631 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:56.840178 26102 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:10:56.841005 26102 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:56.841347 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:56.842322 26548 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.919s	user 0.002s	sys 0.000s
I20260709 13:10:56.841986 26102 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.561496 26475 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:10:56.516798 26298 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.773542 26220 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:56.588418 26503 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.580773 26533 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.823731 22633 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:10:56.824434 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69523.8us
W20260709 13:10:56.826365 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:56.797524 26577 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:10:56.834789 26276 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:56.835861 26504 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.838430 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.842564 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:56.975872 22630 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:56.851843 22705 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:10:56.852697 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 116521us
I20260709 13:10:56.754709 26571 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.977730 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.823323 25768 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.749020 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:56.829675 26476 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.843248 26102 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:56.951917 26220 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:57.016762 22628 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:57.017688 26220 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.019567 26220 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:10:56.781678 26564 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.548s	user 0.009s	sys 0.010s
I20260709 13:10:56.848664 22707 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:10:56.953774 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:56.959852 26506 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:56.975617 26502 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:56.992167 25768 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.992934 22631 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:10:57.021266 22628 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:57.022691 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:56.994120 22704 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:56.981243 26291 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.046172 26533 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.046778 26189 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:56.955104 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:56.956264 26490 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:56.760668 26479 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.988229 22705 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:10:57.064041 22705 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:57.035987 23417 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:10:57.001386 26298 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.029462 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:57.030385 22775 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:57.031319 22707 pending_rounds.cc:85] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:10:57.072757 22707 pending_rounds.cc:107] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:10:57.073217 22707 raft_consensus.cc:2889] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:57.032305 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78658.9us
I20260709 13:10:57.074473 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:57.075341 22707 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:57.077741 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:57.078078 26430 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:57.040077 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.079136 22776 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:57.079659 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.080711 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.081782 22776 raft_consensus.cc:2368] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10.
I20260709 13:10:57.082017 22778 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.029078 26481 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.083487 26481 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.041925 26583 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:57.085812 26583 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.043802 26503 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.088006 26481 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.045337 22704 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:57.090268 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.091851 22704 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:10:57.045903 26291 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:56.748718 26567 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.094579 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.094885 26472 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:57.095486 26472 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.048283 22708 raft_consensus.cc:2376] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
I20260709 13:10:57.095943 26472 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.091753 22600 leader_election.cc:400] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10.
I20260709 13:10:57.097340 22600 leader_election.cc:403] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:56.976765 26573 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:57.099388 26573 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.029515 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.101352 26291 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.039258 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:57.101922 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72523.4us
I20260709 13:10:57.102362 22783 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.102746 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.104056 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.105175 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:57.087502 23417 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.106827 22778 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:56.985447 26504 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.115193 26291 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:57.117066 26291 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:57.088726 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.101713 26503 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:10:57.127072 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:56.755801 26569 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:57.135946 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.146528 22707 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:57.048637 26521 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:57.055924 26535 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.055460 26429 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.060421 26307 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:57.161144 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.176640 22783 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:57.066560 26437 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:57.181519 22632 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:57.132758 26517 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.047709 26502 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.036585 26564 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.158468 26291 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:57.190884 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.194458 23416 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:10:57.200729 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:57.202178 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.023348 26571 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.191152 26586 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.214244 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.217657 26533 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:10:57.182641 26307 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:57.215993 26264 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.008s
I20260709 13:10:57.215006 26476 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:57.251056 26556 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.135s	user 0.004s	sys 0.001s
I20260709 13:10:57.223413 26592 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:10:57.228204 26586 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.260671 26564 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.299477 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2497) took 1596 ms. Trace:
W20260709 13:10:57.249281 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45327.7us
I20260709 13:10:57.160549 22707 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:57.273478 26534 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:57.303382 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 198269us
I20260709 13:10:57.300040 22701 rpcz_store.cc:276] 0709 13:10:55.702735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.714663 (+ 11928us) service_pool.cc:224] Handling call
0709 13:10:57.299210 (+1584547us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b5400314554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:57.299397 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471323008}
I20260709 13:10:57.304273 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.308432 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.323648 26602 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:57.324754 26586 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:57.329414 26568 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.221s	user 0.004s	sys 0.005s
I20260709 13:10:57.332537 26291 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:57.333194 26291 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.333385 26307 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.334246 26479 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.334842 26569 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.334972 25768 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.312217 26429 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:57.336242 26502 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.315814 26437 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:10:57.316592 26189 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.342927 23417 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:57.338343 26567 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.341080 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.346757 26502 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:57.348909 22701 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:10:57.348075 26437 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.352965 26472 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.354974 26531 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:57.355141 26437 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:57.356820 26502 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:57.357317 26537 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:10:57.357772 26537 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:57.358945 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:57.360446 26436 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:57.361366 26594 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:57.343382 26429 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:57.371007 26538 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.368587 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.347460 26598 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 13:10:57.376677 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.376071 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:57.377485 22701 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:10:57.378656 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:57.381475 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 536) took 1368 ms. Trace:
I20260709 13:10:57.381983 22627 rpcz_store.cc:276] 0709 13:10:56.013364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.013546 (+   182us) service_pool.cc:224] Handling call
0709 13:10:56.013878 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:56.291790 (+277912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.381160 (+1089370us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540022e054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:57.381377 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":600554240}
I20260709 13:10:57.383288 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:57.384605 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:57.386341 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:57.408980 22703 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:57.387128 26434 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.375273 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:57.421247 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.422122 22626 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:57.368012 26594 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:57.394289 26504 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.404999 22781 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:10:57.407713 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.412989 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2500) took 1032 ms. Trace:
I20260709 13:10:57.414912 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.415685 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:57.438170 22778 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:57.438216 22783 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:57.416925 22780 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:10:57.437059 22777 rpcz_store.cc:276] 0709 13:10:56.380161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.397041 (+ 16880us) service_pool.cc:224] Handling call
0709 13:10:57.412708 (+1015667us) spinlock_profiling.cc:243] Waited 53.5 ms on lock 0x7b540033c554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:57.412917 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123011072}
I20260709 13:10:57.417476 22779 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:10:57.441262 22779 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:57.441663 22706 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:57.428551 22705 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:10:57.399133 26472 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.448520 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.445400 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:57.438863 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.449903 26429 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:57.449859 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:57.451634 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.502s (SENT) [suppressed 43 similar messages]
I20260709 13:10:57.450914 22635 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:57.442137 22779 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.409590 26502 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:57.451941 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.455062 22703 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:10:57.456387 22779 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:57.457818 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2239) took 1068 ms. Trace:
I20260709 13:10:57.457576 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.466959 22710 rpcz_store.cc:276] 0709 13:10:56.389517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.057953 (+668436us) service_pool.cc:224] Handling call
0709 13:10:57.130941 (+ 72988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.131228 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:10:57.304492 (+173264us) log.cc:844] Serialized 14 byte log entry
0709 13:10:57.457402 (+152910us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:57.457467 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:57.457518 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.457599 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:57.457751 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93103,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"thread_start_us":255819,"threads_started":2}
I20260709 13:10:57.468348 22778 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:57.469439 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.439605 26298 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:57.475188 22635 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:57.475950 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.474802 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.379904 26534 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.472802 26564 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:57.489818 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.526075 22635 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:10:57.522830 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48120.8us
I20260709 13:10:57.530519 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.479228 26503 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:10:57.577282 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46857.3us
I20260709 13:10:57.486291 26284 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:10:57.582568 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:57.567250 26599 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:10:57.580083 22779 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:10:57.578236 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.525538 26564 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.574899 26462 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:57.584395 26502 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:57.598371 22779 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:10:57.593684 26598 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:57.602634 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.594269 26564 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.593135 26517 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10.
I20260709 13:10:57.603629 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2373) took 1115 ms. Trace:
I20260709 13:10:57.592728 26284 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:57.607106 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2384) took 1051 ms. Trace:
I20260709 13:10:57.608459 22778 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:10:57.609555 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.612725 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.617799 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:57.610189 22628 rpcz_store.cc:276] 0709 13:10:56.488510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.036046 (+547536us) service_pool.cc:224] Handling call
0709 13:10:57.039320 (+  3274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:57.113082 (+ 73762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:57.113134 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:57.113808 (+   674us) log.cc:844] Serialized 26 byte log entry
0709 13:10:57.314076 (+200268us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:57.314119 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.376091 (+ 61972us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b5400071054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:57.376148 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:57.598739 (+222591us) raft_consensus.cc:1685] finished
0709 13:10:57.598764 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:57.603516 (+  4752us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16614,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":31753,"spinlock_wait_cycles":168483072,"thread_start_us":199866,"threads_started":1,"wal-append.queue_time_us":308086}
I20260709 13:10:57.619906 22784 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:10:57.599126 26502 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:57.608490 26598 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.612643 22627 rpcz_store.cc:276] 0709 13:10:56.555694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.384345 (+828651us) service_pool.cc:224] Handling call
0709 13:10:57.384640 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.384870 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:10:57.488243 (+103373us) log.cc:844] Serialized 14 byte log entry
0709 13:10:57.606693 (+118450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:57.606761 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:57.606817 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.606913 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:57.607045 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":181839,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":58,"raft.queue_time_us":113,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3062,"thread_start_us":219919,"threads_started":2}
I20260709 13:10:57.626859 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:57.630991 22781 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:57.632546 26504 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:57.633054 26504 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:57.633561 26504 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:57.634352 22628 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:10:57.639304 22779 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:57.639482 22626 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:57.632085 26598 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:57.649657 26504 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.650568 26537 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.650316 26284 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.657842 26566 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.626533 26462 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.658705 26566 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.659449 26462 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.661792 26462 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.627662 26298 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:57.662362 26462 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:57.664867 26298 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.664788 26462 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.665618 26298 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.628971 26472 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:10:57.667263 26462 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.670809 26102 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:10:57.671438 26298 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.672464 26230 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:10:57.630398 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.674306 26504 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.675283 22778 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:57.629060 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.625619 26502 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:57.714473 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2381) took 1216 ms. Trace:
I20260709 13:10:57.722448 22633 rpcz_store.cc:276] 0709 13:10:56.498057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.590167 (+ 92110us) service_pool.cc:224] Handling call
0709 13:10:57.714208 (+1124041us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b5400236ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:57.714380 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18894720}
I20260709 13:10:57.724149 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2505) took 1146 ms. Trace:
I20260709 13:10:57.724797 22708 rpcz_store.cc:276] 0709 13:10:56.559667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.098243 (+538576us) service_pool.cc:224] Handling call
0709 13:10:57.098518 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.207814 (+109296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:57.207848 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:57.207901 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:10:57.207942 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.706257 (+498315us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400314f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:57.706317 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:57.706521 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250794880}
I20260709 13:10:57.727792 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.729094 26434 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:57.729672 26434 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.730140 26434 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:57.734894 26566 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.736768 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:57.737687 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:57.738220 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:57.738858 23416 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.739008 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.739409 22775 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:10:57.740880 26438 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:10:57.741986 26436 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:57.742372 26436 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:57.693655 26571 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:57.701232 26606 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:10:57.702235 26598 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.743417 26436 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.703889 26613 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:10:57.704495 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 75554.1us
I20260709 13:10:57.747342 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.748049 22782 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:57.748152 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.748900 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.749652 22782 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:10:57.706722 26605 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 13:10:57.765223 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.765583 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:57.766543 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.709514 26583 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:57.713434 26126 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:10:57.674263 26230 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:10:57.714130 26291 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.719120 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1875) took 1368 ms. Trace:
I20260709 13:10:57.738469 22781 raft_consensus.cc:2368] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:10:57.590799 26594 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.751250 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.752966 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:57.755426 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:57.760026 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:57.755416 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:57.787498 22775 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.767570 23416 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:57.768173 26298 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.768460 26462 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:57.766315 26592 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.769268 26189 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:57.769591 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.769758 26481 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:57.770464 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:10:57.801604 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.801556 22702 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:57.805331 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:57.806247 22710 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:57.772506 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:57.775175 25399 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:57.775575 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:57.621801 26503 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:57.812623 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.745373 26598 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.777230 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.777451 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:57.774412 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:57.781339 22629 rpcz_store.cc:276] 0709 13:10:56.350217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.387189 (+ 36972us) service_pool.cc:224] Handling call
0709 13:10:56.392376 (+  5187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:56.498024 (+105648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.718809 (+1220785us) spinlock_profiling.cc:243] Waited 95 ms on lock 0x7b54002f4c54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:57.719019 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218389248}
I20260709 13:10:57.769827 26434 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.767027 26531 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:10:57.794086 22708 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
W20260709 13:10:57.781651 26276 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.000s
I20260709 13:10:57.815490 25399 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.821107 22635 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:10:57.864972 22635 pending_rounds.cc:85] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 8
I20260709 13:10:57.865396 22635 pending_rounds.cc:107] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:10:57.865749 22635 raft_consensus.cc:2889] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:57.766746 26506 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:10:57.830897 26560 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:57.791638 26613 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:10:57.801348 26481 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:10:57.869613 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57115us
I20260709 13:10:57.885772 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:10:57.870213 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 57397.2us
I20260709 13:10:57.887578 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:57.888249 22629 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:57.870774 26583 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:57.802850 26189 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.878878 22775 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:57.837359 26431 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:57.881965 22634 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:57.895533 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:57.898255 22633 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:10:57.899539 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 447919us
I20260709 13:10:57.895155 26593 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:10:57.904397 22628 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:57.894070 26481 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.903083 26517 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.912700 26517 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.811036 26592 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.916126 26592 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.916707 26429 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:57.917173 26429 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:57.859408 26503 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.919844 26503 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:57.928280 26434 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:57.930950 26502 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.885203 26564 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.896334 26285 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:10:57.936410 26285 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:57.936854 26285 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:57.937800 26285 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:57.895723 26189 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:57.940129 22706 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:10:57.896665 26276 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:10:57.954497 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52208.8us
I20260709 13:10:57.944736 22710 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:57.945695 25399 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:57.902470 26431 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.984669 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:57.994788 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51830.6us
I20260709 13:10:57.918155 26429 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:57.956434 26502 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:58.027313 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68762.9us
I20260709 13:10:57.991072 26431 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:57.982745 26617 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:10:58.031123 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.042789 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:57.993484 26285 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:58.030730 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:57.996374 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.054605 22710 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:10:58.045866 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:58.018803 26276 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:58.044507 26510 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:10:58.027154 26583 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:10:58.071336 26291 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:58.071501 26472 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:58.074009 26429 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:58.075137 26534 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:58.116760 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.082544 26564 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:58.107575 26534 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.131377 26534 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:58.136260 26619 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:10:58.137545 26434 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:10:58.141861 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 332894us
I20260709 13:10:58.148658 22628 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:10:58.108718 26472 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:10:58.162278 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 396750us
I20260709 13:10:58.164002 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:58.153676 26623 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:10:58.152163 26298 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:58.153275 26102 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:58.158649 26438 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:58.155333 26560 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:58.201890 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:58.164127 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 276629us
I20260709 13:10:58.210870 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:58.212177 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:58.215109 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 169370us
I20260709 13:10:58.143586 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:58.238130 22633 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:10:58.209803 26613 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:58.244179 26633 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:58.244896 26633 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:58.246127 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44377.1us
I20260709 13:10:58.194073 26586 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:10:58.155640 26276 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:58.258965 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 191224us
I20260709 13:10:58.196399 26481 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.218384 26438 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:58.273617 22632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1871) took 1941 ms (client timeout 1919 ms). Trace:
I20260709 13:10:58.264470 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:10:58.286202 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.247357 26633 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:58.288810 22632 rpcz_store.cc:269] 0709 13:10:56.331927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.332055 (+   128us) service_pool.cc:224] Handling call
0709 13:10:58.273324 (+1941269us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400070654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:58.273525 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917913344}
W20260709 13:10:58.300030 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87914.4us
I20260709 13:10:58.255513 26583 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:58.252033 26613 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:58.320912 22705 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:10:58.325043 22705 pending_rounds.cc:85] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:10:58.252211 26284 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:58.225131 26478 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:10:58.325332 22705 pending_rounds.cc:107] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:10:58.326000 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:58.326563 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:58.248059 26593 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:10:58.262085 26635 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:58.345175 22705 raft_consensus.cc:2889] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:58.208148 26472 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:58.350036 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 206548us
I20260709 13:10:58.351082 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.320222 26583 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.356797 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:58.358928 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.360723 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:58.326599 26502 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:10:58.361959 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.352195 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131662us
I20260709 13:10:58.324607 26462 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:58.287115 26291 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:10:58.350888 26614 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:10:58.358047 26618 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.262s	user 0.002s	sys 0.000s
I20260709 13:10:58.378901 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.378384 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.301131 26631 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:10:58.364050 26583 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:58.387627 26629 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:10:58.398823 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.399401 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:58.399768 22710 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:58.400249 22705 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:58.365681 26630 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:58.378935 26506 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:58.354480 26284 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.365403 26566 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:10:58.381052 26635 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:58.423226 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 118926us
I20260709 13:10:58.351755 26560 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:58.432029 26636 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:10:58.426329 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:58.427193 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.445323 22710 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:58.403261 26624 raft_consensus.cc:2764] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:10:58.427210 26630 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:58.447227 26637 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:10:58.447865 26649 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.450858 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.406473 26462 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:58.451344 26481 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:58.451342 26517 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:58.458003 26638 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:58.503259 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.503757 26638 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.459108 26635 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.462018 26102 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:58.463886 26284 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:58.400962 26438 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.466413 26490 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:58.469609 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.534838 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.535882 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:58.496505 26462 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:58.474161 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78193.6us
I20260709 13:10:58.544557 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.405649 26650 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:10:58.408594 26506 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:10:58.497485 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 232817us
I20260709 13:10:58.537720 22631 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:58.533567 22784 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:10:58.523102 26560 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:58.410108 26625 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:10:58.507146 26600 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.552249 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:58.469769 26651 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:58.547688 22705 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:10:58.549597 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:58.481854 26431 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.561525 22784 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:58.563825 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:58.529271 26566 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:58.497016 26189 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.570534 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.577272 22779 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:10:58.533231 26490 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:58.413110 26613 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:58.604406 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 258177us
W20260709 13:10:58.606130 23385 connection.cc:256] RPC call timeout handler was delayed by -1.0836959930001058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 13:10:58.606927 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.631s (SENT)
I20260709 13:10:58.609292 22600 leader_election.cc:400] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:10:58.608956 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:58.610587 22600 leader_election.cc:403] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:10:58.611879 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:58.612890 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.613260 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:58.614709 22783 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:10:58.618971 22784 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:10:58.647351 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:58.689790 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.506733 26649 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.503656 26630 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.682385 22784 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:10:58.670019 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 118737us
I20260709 13:10:58.692101 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.550765 26633 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.583140 26651 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:58.697321 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 90438.4us
I20260709 13:10:58.570371 26560 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.712563 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.703536 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:58.705721 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:58.709123 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.548s (SENT)
I20260709 13:10:58.710039 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:58.682278 26637 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:10:58.716260 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:58.718391 22779 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:58.721346 22781 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:10:58.690762 26632 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:58.693711 26586 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:58.707624 26285 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.608268 26462 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.583727 26566 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.621017 26644 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:10:58.775851 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:10:58.776754 26608 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.110s	user 0.005s	sys 0.002s
W20260709 13:10:58.777251 26667 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:10:58.693526 26284 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:58.694242 26102 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:58.725067 26517 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:10:58.709009 26638 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:58.570314 26600 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.795822 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:58.772934 26481 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:10:58.806655 26429 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.703s	user 0.015s	sys 0.001s
W20260709 13:10:58.778467 26640 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:10:58.698334 26506 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.765882 26490 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.859882 26431 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:58.802974 26662 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:10:58.778817 26655 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:58.861064 26655 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.863586 26655 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:58.863762 26600 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:58.841184 22707 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:58.869303 26506 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:58.838361 26102 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:58.874333 26102 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:58.800460 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 98575.9us
I20260709 13:10:58.829546 26517 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.835319 26462 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:58.842896 26653 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:58.876499 26517 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:58.876724 26653 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:10:58.877664 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:10:58.845512 26612 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.001s
I20260709 13:10:58.846357 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:10:58.879786 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 330273us
W20260709 13:10:58.848155 26659 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:10:58.849603 26631 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:58.877763 26653 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.831038 26651 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:58.897164 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:58.875836 26436 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:58.798805 26285 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:58.847229 26646 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:10:58.882594 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.884101 23421 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:10:58.792886 26586 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:58.858122 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2391) took 1466 ms. Trace:
I20260709 13:10:58.880487 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:58.898695 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:58.930392 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:58.929656 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.946112 22629 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:10:58.946811 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:58.892077 26429 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:58.952200 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 853) took 1199 ms. Trace:
I20260709 13:10:58.951493 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.944057 22635 rpcz_store.cc:276] 0709 13:10:57.391864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.581026 (+189162us) service_pool.cc:224] Handling call
0709 13:10:57.591947 (+ 10921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:57.866390 (+274443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.857784 (+991394us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54001ca054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:58.858039 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525645696}
W20260709 13:10:58.951768 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53204.2us
I20260709 13:10:58.955288 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:58.955744 22635 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:10:58.957746 22635 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:10:58.960549 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.952778 23421 rpcz_store.cc:276] 0709 13:10:57.736014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.743077 (+  7063us) service_pool.cc:224] Handling call
0709 13:10:58.877462 (+1134385us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5800003998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:10:58.935034 (+ 57572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29401344}
I20260709 13:10:58.928158 26284 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:10:58.929136 26656 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:58.971012 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:58.895659 26126 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:10:58.971390 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:58.968693 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.975365 22628 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
W20260709 13:10:58.977063 22599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.003s	sys 0.016s
W20260709 13:10:58.977504 22599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.003s	sys 0.016s
W20260709 13:10:58.979575 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 272563us
I20260709 13:10:58.898360 26583 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:58.963357 26653 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:58.959784 26639 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:10:58.960246 26594 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.001816 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:58.971110 26284 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:58.976084 26429 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:59.012233 26481 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:59.025787 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2533) took 1237 ms. Trace:
I20260709 13:10:59.014351 26560 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:58.983269 26656 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.016458 26284 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:59.008903 26633 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.026310 22709 rpcz_store.cc:276] 0709 13:10:57.784519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.784686 (+   167us) service_pool.cc:224] Handling call
0709 13:10:57.804469 (+ 19783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:58.084458 (+279989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:58.084493 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:58.097712 (+ 13219us) log.cc:844] Serialized 26 byte log entry
0709 13:10:58.214945 (+117233us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:10:58.214985 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.215062 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:59.021596 (+806534us) raft_consensus.cc:1685] finished
0709 13:10:59.021630 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:59.021824 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32151,"raft.queue_time_us":92790,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":19203,"thread_start_us":334859,"threads_started":2,"wal-append.queue_time_us":421667}
I20260709 13:10:59.006146 26650 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:10:59.018555 26649 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.047164 26583 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.020057 26472 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.020323 26573 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:59.024273 26654 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.031926 26600 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.042873 26560 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:59.057663 26672 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.068001 26672 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.070432 26285 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.071235 26285 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:59.071803 26285 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.074024 26285 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:59.080514 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.014775 26594 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:59.063274 26668 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:10:59.100636 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.087949 26631 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.083894 26429 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.114764 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.115638 22702 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:58.970528 22521 maintenance_manager.cc:643] P 71605ba8587a4b5193d8d69d9967c1af: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 6.894s	user 0.203s	sys 0.062s Metrics: {"bytes_written":29237,"cfile_init":1,"dirs.queue_time_us":354435,"dirs.run_cpu_time_us":472,"dirs.run_wall_time_us":9317,"drs_written":1,"lbm_read_time_us":481,"lbm_reads_lt_1ms":4,"lbm_write_time_us":286405,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":21,"peak_mem_usage":0,"rows_written":63,"thread_start_us":233767,"threads_started":1}
I20260709 13:10:59.154290 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:59.155100 22628 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:59.148092 26631 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.061026 26573 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.066887 26560 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.060650 26472 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:59.158192 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 277689us
I20260709 13:10:59.062299 26600 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:59.164564 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 148212us
I20260709 13:10:59.165529 22707 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:59.166821 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.168270 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.170747 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.174427 22704 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 4.
I20260709 13:10:59.114535 26638 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:59.172634 22628 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:59.174566 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:59.177138 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.177866 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:59.150626 26630 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:59.178737 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:10:59.181110 26613 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:59.184360 22522 maintenance_manager.cc:419] P 71605ba8587a4b5193d8d69d9967c1af: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 8302 bytes on disk
I20260709 13:10:59.061324 26654 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.186048 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:59.186487 26472 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.186825 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:10:59.188228 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.216820 22704 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:59.217864 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.219048 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.219643 22704 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:59.192619 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:59.193387 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.221885 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.222210 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.222810 23411 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:10:59.223443 22705 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:10:59.195523 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.227344 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.227658 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.228070 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 4.
I20260709 13:10:59.165320 26671 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.212095 23421 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:59.214923 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:59.215132 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.153878 26429 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.229810 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.294355 22703 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:10:59.257304 26671 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.205403 26630 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:59.258289 26676 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:10:59.260115 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.257190 22521 maintenance_manager.cc:643] P 71605ba8587a4b5193d8d69d9967c1af: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.027s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":356,"lbm_reads_lt_1ms":4}
I20260709 13:10:59.256711 26506 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:59.275161 26573 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.304517 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:59.305553 26651 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:59.305245 26506 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.191061 26298 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:10:59.273416 26503 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:10:59.203945 26672 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.228223 26635 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:59.296391 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:10:59.226485 26680 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:59.303254 26560 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.299496 26685 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.303654 26654 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.187423 26586 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.271834 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:10:59.307700 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47681.5us
I20260709 13:10:59.276752 26285 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:59.317348 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141766us
I20260709 13:10:59.320889 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:10:59.345655 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.346436 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:10:59.316387 26573 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:10:59.350471 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:59.342492 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:10:59.352005 22628 raft_consensus.cc:2376] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:10:59.317595 26503 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:59.318058 26695 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:10:59.342010 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:59.339236 26635 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.325203 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:59.357045 26594 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.340112 26298 raft_consensus.cc:2764] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:10:59.342869 26126 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:10:59.359125 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:59.373274 23421 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:59.353338 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:10:59.375803 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50722.5us
I20260709 13:10:59.353497 26672 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:59.365988 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:59.378571 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:10:59.349511 26680 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:59.349821 26504 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.467s	user 0.016s	sys 0.002s
I20260709 13:10:59.383203 25399 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:10:59.343421 26613 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:10:59.386008 22628 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:10:59.383715 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:59.353826 26586 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.364714 26635 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:59.391319 25399 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.379925 26654 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:59.349093 26285 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.349093 26685 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.382261 26651 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:10:59.397564 25399 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:59.384511 26672 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:59.392935 26504 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.522s	user 0.016s	sys 0.003s
I20260709 13:10:59.408420 26699 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:59.410751 26684 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.396138 26631 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:59.410529 26690 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:59.399117 26572 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:59.400919 26654 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.411028 26298 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:59.425916 26684 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.426645 26680 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:59.428369 26517 consensus_meta.cc:287] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.000s
I20260709 13:10:59.399029 26649 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:10:59.401381 26586 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.416709 26672 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:10:59.425145 26600 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:10:59.432926 26436 consensus_meta.cc:287] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.006s
W20260709 13:10:59.418362 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 189497us
I20260709 13:10:59.467170 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.424717 26651 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:10:59.428512 26661 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.648s	user 0.002s	sys 0.000s
I20260709 13:10:59.466410 23421 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:59.473392 22780 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:10:59.469542 26695 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:59.399060 26503 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.436806 26572 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.467856 25399 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.469323 26697 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:59.443238 26654 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:59.432232 26685 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:59.470053 26677 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:10:59.483774 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.443883 26298 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:10:59.497848 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.498628 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.503806 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.509920 22783 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:59.496104 26572 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:59.512801 25399 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.478883 26429 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.456588 26517 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:59.518177 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141095us
I20260709 13:10:59.492122 26506 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.467643 26284 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.516952 26503 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:10:59.528913 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 121226us
I20260709 13:10:59.523106 26701 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.530695 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2534) took 1070 ms. Trace:
I20260709 13:10:59.502818 26703 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:10:59.545046 22634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1865) took 3210 ms (client timeout 2051 ms). Trace:
W20260709 13:10:59.545459 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.540910 22782 rpcz_store.cc:276] 0709 13:10:58.460562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.462733 (+  2171us) service_pool.cc:224] Handling call
0709 13:10:58.463187 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:58.463591 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:58.463623 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:58.464163 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 13:10:58.527206 (+ 63043us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:58.527251 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:58.527338 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:59.427723 (+900385us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:59.520897 (+ 93174us) raft_consensus.cc:1685] finished
0709 13:10:59.520922 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:59.530530 (+  9608us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9002,"raft.queue_time_us":85521,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":33717,"spinlock_wait_cycles":10070784,"thread_start_us":62592,"threads_started":1,"wal-append.queue_time_us":230745}
W20260709 13:10:59.531275 26462 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.655s	user 0.015s	sys 0.000s
I20260709 13:10:59.543991 22629 raft_consensus.cc:2368] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9.
I20260709 13:10:59.499765 26653 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:10:59.545619 22634 rpcz_store.cc:269] 0709 13:10:56.327408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:56.327616 (+   208us) service_pool.cc:224] Handling call
0709 13:10:59.530664 (+3203048us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400071a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:59.537330 (+  6666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3524830464}
W20260709 13:10:59.551819 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.562144 22634 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:10:59.532503 26671 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:10:59.543560 26610 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.938s	user 0.006s	sys 0.008s
I20260709 13:10:59.520512 26685 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:10:59.529623 26697 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:10:59.575101 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 92954.2us
I20260709 13:10:59.580083 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.581001 22704 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:10:59.538378 26706 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:10:59.588886 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.548350 26701 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.577026 26429 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:59.618804 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 88648.3us
I20260709 13:10:59.620428 22704 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:10:59.621951 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.626070 22703 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:10:59.614603 26704 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:10:59.629055 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.588824 26285 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.561806 26651 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:10:59.630425 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89164.2us
I20260709 13:10:59.559038 26703 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.643731 26284 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.621763 26693 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:10:59.646098 26684 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.648061 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.635506 22704 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:59.570595 26298 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.638038 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 570) took 2223 ms. Trace:
W20260709 13:10:59.637539 26698 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:10:59.660923 26701 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.655622 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:59.653148 22776 rpcz_store.cc:276] 0709 13:10:57.407241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.407363 (+   122us) service_pool.cc:224] Handling call
0709 13:10:57.411682 (+  4319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:57.412111 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:57.412141 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:57.412720 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 13:10:57.482537 (+ 69817us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:57.482590 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:57.482687 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:59.438136 (+1955449us) spinlock_profiling.cc:243] Waited 16 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:10:59.627241 (+189105us) raft_consensus.cc:1685] finished
0709 13:10:59.627275 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:59.630746 (+  3471us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":111,"spinlock_wait_cycles":36817792,"thread_start_us":58352,"threads_started":1,"wal-append.queue_time_us":167353}
I20260709 13:10:59.671761 22784 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:59.667194 26697 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:10:59.672365 26705 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
I20260709 13:10:59.673611 26684 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.676151 26684 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.673281 22776 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:10:59.678524 26684 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:59.680416 26708 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:10:59.681504 26708 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:59.681864 26708 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:59.682696 22782 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:10:59.687888 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.690506 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.646384 26693 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.666930 22704 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:10:59.695781 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.696596 23411 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:10:59.626864 26429 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.662762 26586 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:59.699671 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.682926 26701 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:59.702209 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.701439 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:10:59.687472 26462 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.683266 26506 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:10:59.719646 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 106555us
W20260709 13:10:59.720008 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98616.3us
I20260709 13:10:59.711887 26701 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:59.725065 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.700425 26298 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:59.729120 22673 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.729647 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.730350 22703 raft_consensus.cc:2376] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13. Re-sending same reply.
I20260709 13:10:59.716643 26594 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:59.731307 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.735553 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.754501 26697 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.718940 26517 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.755211 26697 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.727602 26701 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:59.762045 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.762542 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.762372 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:10:59.769048 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.767290 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:10:59.762212 26709 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 13:10:59.770215 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:59.784193 22784 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:59.775166 26697 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:10:59.734078 26695 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:10:59.718931 26462 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.791994 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.734694 26594 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.793659 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.722580 26506 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.800035 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.801927 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:10:59.809939 26688 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:10:59.810778 26617 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:10:59.812242 26506 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.788761 26436 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:10:59.797518 26594 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:59.819564 22703 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:10:59.820640 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.821496 22703 raft_consensus.cc:2410] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:10:59.822284 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.825102 22703 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:10:59.829289 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.800946 26695 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.832175 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.831014 26703 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.841356 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:10:59.842545 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.811439 26617 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.749800 26630 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:10:59.851608 26722 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:59.852180 26722 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:10:59.852617 26722 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:59.727197 26708 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.824182 26706 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:59.850265 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.862581 22702 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:59.856230 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:10:59.844520 26462 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:10:59.859697 26703 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:10:59.869963 26537 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:10:59.871752 22701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2524) took 2519 ms (client timeout 2112 ms). Trace:
W20260709 13:10:59.872229 22701 rpcz_store.cc:269] 0709 13:10:57.351901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.384339 (+ 32438us) service_pool.cc:224] Handling call
0709 13:10:59.871423 (+2487084us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54000a8cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:10:59.871629 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3534394240}
I20260709 13:10:59.873628 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.874354 22701 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:10:59.875169 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.875919 22710 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:10:59.876770 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.878558 26592 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:10:59.879251 26592 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:10:59.879671 26708 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:59.880455 26592 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.881444 26706 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:10:59.885371 26462 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:59.882891 26572 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.886224 26703 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:10:59.845533 26695 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:10:59.890264 26506 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:10:59.892477 26658 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.053s	user 0.000s	sys 0.003s
I20260709 13:10:59.873577 22702 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:59.845827 26699 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:59.898119 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.959374 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.960060 22702 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:10:59.942693 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.960706 22781 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:10:59.945737 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.946872 26689 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.962420 26689 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.947302 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.947556 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:10:59.951093 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:10:59.965157 22776 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
W20260709 13:10:59.909852 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:59.908756 22701 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:10:59.939669 22782 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:10:59.968549 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.969288 22701 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:10:59.970866 22702 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:10:59.972676 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:10:59.974187 22702 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:10:59.958961 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:10:59.971320 26504 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.979379 26276 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:59.987932 26504 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.988209 26276 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.990856 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2532) took 2189 ms. Trace:
I20260709 13:10:59.996011 22710 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:59.991436 22706 rpcz_store.cc:276] 0709 13:10:57.782894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.783083 (+   189us) service_pool.cc:224] Handling call
0709 13:10:57.783521 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:57.949716 (+166195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:59.972246 (+2022530us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540010ca54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:10:59.972462 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359358720}
I20260709 13:10:59.997751 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.998514 22706 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:10:59.999423 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.960170 22784 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:10:59.948451 26617 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.007655 22780 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:00.011090 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:00.013972 22782 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.014515 22706 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:00.011401 26679 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.015571 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:10:59.904403 26503 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:11:00.026304 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.027486 22706 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:00.028643 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.032572 22706 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:10:59.902473 26635 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:10:59.995450 26291 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:59.990754 26653 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:00.033854 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.063791 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.064270 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.066017 22628 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:11:00.069262 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.036048 26677 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.036479 26298 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.078543 22705 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:00.081231 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.081492 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.089988 22701 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:00.045363 26291 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:00.096756 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.102124 22702 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:00.104425 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.105443 22784 raft_consensus.cc:2410] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:00.108261 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:00.079685 26654 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.006s
I20260709 13:11:00.115243 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.122493 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.129489 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.071377 26653 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.133316 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.067857 26644 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:11:00.138190 22782 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:00.139322 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.139248 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.139928 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:00.141150 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.141156 22778 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:00.143038 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:10:59.919489 26706 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.129740 26650 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:00.094684 26298 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.150667 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.135432 26572 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.166146 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.170631 26695 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.172278 26189 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.178208 22783 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:00.177073 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.177175 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:00.181206 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.144183 26644 consensus_queue.cc:1243] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:00.085402 26677 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.186187 26677 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.187103 26677 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:00.079183 26679 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.190603 26679 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.190985 22627 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:00.191987 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.195331 26635 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.182305 26291 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.145509 26264 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:00.201658 22782 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:00.194205 26722 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.199712 26481 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:00.200480 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:00.207419 22783 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:00.130805 26654 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.208653 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:00.209442 22778 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:00.204171 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.210572 22633 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:00.212016 22706 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 11. Ops: []
I20260709 13:11:00.212397 22630 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:00.182646 26504 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.222731 26635 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:00.223510 26635 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.224017 26722 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.225852 26635 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.163848 26650 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.212412 26481 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.228223 22634 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14.
W20260709 13:11:00.238771 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 88223.8us
I20260709 13:11:00.213212 26689 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.245558 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.249172 22630 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:00.249368 22783 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:00.261034 26653 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:00.262926 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:00.265254 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.271565 26481 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.273820 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:00.275195 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.276695 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.278434 22627 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:11:00.278724 22675 leader_election.cc:400] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9.
I20260709 13:11:00.280030 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:00.273041 26729 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:11:00.317469 26723 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:11:00.317739 22635 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:00.319332 26650 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.319386 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.319819 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2557) took 1108 ms. Trace:
I20260709 13:11:00.320508 23419 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:11:00.312175 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.312283 22675 leader_election.cc:403] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:00.313511 26298 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.279556 26653 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:00.323804 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:00.325979 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 47448.5us
I20260709 13:11:00.368366 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.320458 26654 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.327965 26503 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:00.331413 26656 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.332190 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.373452 22630 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:00.373924 22708 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:00.320271 22705 rpcz_store.cc:276] 0709 13:10:59.211850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.224465 (+ 12615us) service_pool.cc:224] Handling call
0709 13:10:59.224874 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:00.079076 (+854202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.319578 (+240502us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b5400314f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:00.319756 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1962774272}
I20260709 13:11:00.333184 26693 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.336104 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.338598 26429 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.353039 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:00.353245 26674 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.344s	user 0.005s	sys 0.000s
I20260709 13:11:00.356218 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.363456 23417 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:11:00.379766 23417 pending_rounds.cc:85] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 7
I20260709 13:11:00.380033 23417 pending_rounds.cc:107] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:11:00.366688 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.358800 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 862) took 1139 ms. Trace:
I20260709 13:11:00.382766 23419 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:11:00.370428 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.376700 22783 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:00.376885 22702 raft_consensus.cc:2410] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:11:00.377143 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.380263 23417 raft_consensus.cc:2889] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:00.381742 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.306777 22778 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:00.380721 23414 rpcz_store.cc:276] 0709 13:10:59.219603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.219760 (+   157us) service_pool.cc:224] Handling call
0709 13:10:59.220022 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:59.340005 (+119983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:59.340042 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:59.340094 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:59.340130 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.358317 (+1018187us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54006329d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:00.358407 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:00.358661 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275339008}
I20260709 13:11:00.392155 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.382354 26715 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:00.407758 26715 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:00.400871 26719 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:11:00.411286 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.415362 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 527) took 1223 ms. Trace:
I20260709 13:11:00.415993 22707 rpcz_store.cc:276] 0709 13:10:59.192331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.192463 (+   132us) service_pool.cc:224] Handling call
0709 13:10:59.192785 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:59.320283 (+127498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:59.320321 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:59.320368 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:59.320405 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.415031 (+1094626us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540010de54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:00.415098 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:00.415279 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292601728}
I20260709 13:11:00.367302 26653 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.440202 26653 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:00.443059 26672 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.964s	user 0.010s	sys 0.005s
I20260709 13:11:00.443837 26672 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.446875 26298 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.447515 26298 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.451629 26566 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:00.455135 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.455698 26298 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.400790 22783 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:00.417825 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:00.458076 26708 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.004s
I20260709 13:11:00.459533 23411 raft_consensus.cc:2410] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:11:00.460632 26592 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:00.458153 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.421207 22776 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:00.398139 26656 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:00.422163 26613 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:00.423035 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2377) took 3933 ms. Trace:
I20260709 13:11:00.423632 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.479297 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.400578 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.426939 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.480861 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.399046 26429 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:00.428845 26693 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:00.430980 26503 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9.
I20260709 13:11:00.431882 26689 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:00.432152 26537 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.002s
I20260709 13:11:00.377610 26703 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:00.433667 26701 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.000s
I20260709 13:11:00.436515 23412 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:00.436925 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.513098 22706 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:00.438978 23419 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:11:00.439591 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:00.472002 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.474061 22707 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:00.482519 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.514983 22702 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.425472 26631 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:00.479944 22631 rpcz_store.cc:276] 0709 13:10:56.489484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.040550 (+551066us) service_pool.cc:224] Handling call
0709 13:10:57.040824 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:58.551139 (+1510315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.422702 (+1871563us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000b77d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:00.422918 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442062720}
W20260709 13:11:00.419790 26594 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.582s	user 0.007s	sys 0.005s
I20260709 13:11:00.483908 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:00.484062 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83605.9us
I20260709 13:11:00.506273 22627 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:00.515134 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.532828 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:00.533311 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61475.8us
I20260709 13:11:00.551849 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.569252 22704 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 9.
I20260709 13:11:00.565614 22701 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:00.564705 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:00.566939 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.568054 22632 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:11:00.568518 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53478us
I20260709 13:11:00.571815 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.573060 22704 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:00.575296 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.575663 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.576457 22704 raft_consensus.cc:2368] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:00.577358 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.578186 22707 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:00.578554 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.579965 22704 raft_consensus.cc:2410] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:00.564000 22708 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:00.583640 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.584414 22630 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:11:00.586534 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.587634 25399 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:00.491674 26583 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:00.534147 26580 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:00.602340 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.604481 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.605170 22701 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:00.484500 26298 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:11:00.569327 26631 consensus_queue.cc:1243] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:11:00.614271 25399 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:00.567523 26592 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:00.615231 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:00.616870 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52302.3us
I20260709 13:11:00.560060 26701 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.588641 26510 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:00.595420 26632 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:00.590076 26654 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:00.621410 22706 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:00.625934 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:00.626670 22705 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:11:00.612006 26734 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:11:00.630833 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 244096us
I20260709 13:11:00.635545 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:00.600592 26276 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:00.645074 22750 connection.cc:256] RPC call timeout handler was delayed by -1.2200930090000384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
I20260709 13:11:00.651472 26534 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:00.652035 26534 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.652424 26534 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.652581 26503 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:00.653280 26503 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.616631 26703 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:00.654457 26703 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.654840 26102 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:00.656885 26593 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:00.657318 26503 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.657414 26593 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.663589 26593 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:00.664688 26701 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.630304 26517 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:00.665835 26517 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.666272 26517 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:00.648469 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:00.669874 26648 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:00.670671 26648 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:00.671877 26648 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.622104 26220 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:11:00.676082 26634 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:00.678089 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:00.591848 26537 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:11:00.618108 26583 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:00.679067 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 203986us
I20260709 13:11:00.617226 26580 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:00.679780 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 156909us
W20260709 13:11:00.680109 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:00.698760 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 317080us
I20260709 13:11:00.699616 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.701005 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.702201 22777 raft_consensus.cc:2410] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:00.563304 26689 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.579852 26594 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.638418 26672 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.715564 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.692340 22701 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:00.694432 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.703210 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.704036 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:00.718140 22781 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:00.705163 22779 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:11:00.657191 26703 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.729192 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.731428 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.716688 22778 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:00.732564 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.696097 26632 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:00.696063 26650 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.733320 22627 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:00.735453 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.738272 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.737972 26650 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:00.740440 26650 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.740736 26264 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:00.741772 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.652292 26478 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:11:00.740438 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:00.746460 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:00.746939 22701 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:11:00.747169 22776 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.734339 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:00.750739 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:00.751399 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.754060 22635 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:11:00.757974 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.765975 22675 leader_election.cc:400] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14.
I20260709 13:11:00.768548 22701 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:11:00.771315 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 121331us
I20260709 13:11:00.772855 22709 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:00.773234 22701 pending_rounds.cc:85] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:00.773622 22701 pending_rounds.cc:107] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:11:00.773874 22701 raft_consensus.cc:2889] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:00.773813 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.775120 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:00.775650 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.771502 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.780018 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:00.780685 22633 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:00.751745 26692 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.496s	user 0.014s	sys 0.029s
I20260709 13:11:00.784691 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.772509 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:00.749553 26715 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:00.809523 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:00.777279 26703 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:00.792640 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.789446 26672 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:00.733433 26706 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:00.746070 26632 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:00.804476 26692 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.540s	user 0.014s	sys 0.029s
I20260709 13:11:00.815702 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.793471 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:00.817165 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45796.3us
I20260709 13:11:00.828333 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.830600 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.791141 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:00.805300 26537 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:00.787693 26656 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:11:00.845602 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.852113 22776 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:00.830036 26680 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:00.811385 26102 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:11:00.822700 26703 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.816741 26583 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.875504 26703 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.814886 26592 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 13:11:00.848881 26189 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:00.879352 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:00.863015 26617 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:00.865819 26593 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:00.871184 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 95591.1us
I20260709 13:11:00.847661 26651 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:00.869719 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.866451 26436 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:00.888425 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:00.847128 26537 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:00.883759 26739 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:00.884883 26580 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.891687 26592 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:00.898120 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.886826 26617 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:00.911654 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:00.914155 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:00.928911 26649 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:00.930852 26593 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:00.932190 26594 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:00.932004 26617 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.932803 22632 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
W20260709 13:11:00.933485 26741 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:11:00.934954 26537 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.904505 26126 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:00.923576 26592 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:00.940959 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:00.974810 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:00.942143 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44144.8us
I20260709 13:11:00.948231 26434 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:00.977456 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:00.978065 26503 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:00.978544 26503 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:00.978960 26679 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:00.979163 26703 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:00.978971 26503 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:00.979990 26703 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:11:00.901300 26632 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.948819 26649 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:00.960932 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:00.985773 23417 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:00.897548 22633 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:00.901162 26706 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:00.987098 26706 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:00.987511 26706 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.991109 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:00.991518 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:00.992089 26630 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:00.992508 22710 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:00.985993 26649 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:00.993578 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:00.993803 26649 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:00.994158 22710 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:00.978964 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:00.996659 26679 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:00.997293 26679 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:00.998392 26679 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:01.003134 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 268183us
I20260709 13:11:01.003288 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:01.005174 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.005730 26436 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:01.006510 26436 raft_consensus.cc:2764] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:11:01.006680 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:01.007807 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:01.008905 26630 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:01.012048 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:01.018097 26630 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:01.016803 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:01.029681 26436 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:01.034166 22750 leader_election.cc:400] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:01.040772 22707 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:01.041970 22750 leader_election.cc:403] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:11:01.045694 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:01.047163 26679 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:01.048141 26679 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:01.048069 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.049099 26679 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:01.050338 26679 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.045962 22704 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:01.037461 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:01.053593 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:01.059749 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.073276 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:00.967533 26744 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:11:00.958503 26671 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:01.075882 26630 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:01.080246 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89251.2us
I20260709 13:11:00.962399 26689 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.089489 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:00.968485 26102 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:01.099313 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 401104us
W20260709 13:11:01.096587 26743 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:11:01.077291 26594 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:11:01.115132 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.090736 26617 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:00.974166 26708 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 13:11:01.104578 26632 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:01.104620 26671 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:01.144923 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.055294 26672 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:11:01.099536 26651 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:11:01.109268 26653 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.000s
I20260709 13:11:01.091777 26503 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.099304 26630 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.109513 26534 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.187003 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 721) took 1455 ms. Trace:
I20260709 13:11:01.098160 26706 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:01.194149 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 121088us
I20260709 13:11:01.110960 26102 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.167169 26632 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:01.256419 22629 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:01.052865 26436 raft_consensus.cc:2764] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:11:01.244388 26503 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.127028 26592 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.176944 26653 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.246798 22707 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:01.303339 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:01.223155 22750 leader_election.cc:400] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 9.
I20260709 13:11:01.187549 23420 rpcz_store.cc:276] 0709 13:10:59.731848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.732019 (+   171us) service_pool.cc:224] Handling call
0709 13:10:59.768948 (+ 36929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:59.785384 (+ 16436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:59.785419 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:00.538422 (+753003us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b2c00012a88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:00.539019 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:11:00.780344 (+241325us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:00.780408 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.780508 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:01.181557 (+401049us) raft_consensus.cc:1685] finished
0709 13:11:01.181589 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.186902 (+  5313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16005,"raft.queue_time_us":16469,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5544,"spinlock_wait_cycles":11518720,"thread_start_us":240839,"threads_started":1,"wal-append.queue_time_us":300578}
I20260709 13:11:01.176605 26126 raft_consensus.cc:2764] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 13:11:01.313549 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:01.254060 26102 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.261732 26298 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:01.248997 26517 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.582s	user 0.004s	sys 0.007s
I20260709 13:11:01.246587 26706 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:01.194535 26651 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:01.254922 26630 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.250555 26534 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:01.298799 26751 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:11:01.289242 26632 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:01.338786 22707 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:01.315057 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:01.339515 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.259248 26594 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:01.328051 22710 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:11:01.330080 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 130282us
I20260709 13:11:01.346028 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:01.346326 22783 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:01.346998 22782 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:01.346870 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.296221 26189 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:01.347915 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:01.348832 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:01.379891 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.349946 22781 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:11:01.380291 22704 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:01.387290 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.383091 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:01.388501 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.388897 22783 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:01.350656 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.352382 22780 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:11:01.308240 26634 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.312569 26436 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.324330 26703 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.359196 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:01.389976 22780 pending_rounds.cc:85] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 3
I20260709 13:11:01.409072 22780 pending_rounds.cc:107] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:01.409433 22780 raft_consensus.cc:2889] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:01.386093 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:01.381762 22628 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:01.385326 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.297787 26756 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:11:01.405877 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.406730 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:01.408872 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:01.393080 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:01.413326 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.413326 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:01.419317 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.420228 22778 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:01.421521 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.424461 22783 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:01.424523 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.425486 22776 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:01.427148 22750 leader_election.cc:400] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:01.427486 22750 leader_election.cc:403] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:11:01.427884 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:01.428457 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:01.428534 22776 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:01.336393 26770 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 13:11:01.378691 26754 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.402410 26503 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 13:11:01.352522 26517 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.437197 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:01.352843 26298 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.407824 26651 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:01.432340 26748 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:11:01.430493 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:01.423967 26126 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:01.425901 26739 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:01.343986 26671 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.434214 26580 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:01.423431 26679 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:01.441874 26715 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:01.454298 26648 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:01.462724 22635 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:01.455698 26739 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:01.463660 26739 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:01.453101 26766 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:11:01.409974 26634 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.416262 26436 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.452682 26651 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:01.423839 26703 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.422796 26534 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:11:01.461604 26583 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.457839 26776 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:01.488860 22707 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:01.467620 26715 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:01.475667 26747 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
W20260709 13:11:01.475909 26764 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 13:11:01.467916 26630 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.472640 26708 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:01.464788 26648 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:01.495944 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:01.496212 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:01.471829 26580 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:01.478271 26679 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:01.483062 26651 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:01.509554 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:01.467993 26706 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:11:01.453814 26754 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.497445 26715 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:01.529241 26759 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:11:01.558218 22707 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:01.497700 26583 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.506570 26708 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:01.532908 26771 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:11:01.535205 26634 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.465145 26594 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.510931 26648 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:01.531817 26773 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:11:01.558718 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:01.526304 26760 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.232s	user 0.000s	sys 0.002s
I20260709 13:11:01.520717 26593 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:01.542397 26653 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.509331 26630 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:01.620453 26749 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.272s	user 0.002s	sys 0.000s
I20260709 13:11:01.622732 26708 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:01.649394 26580 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.658579 26779 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:01.659029 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2570) took 1323 ms. Trace:
I20260709 13:11:01.659596 22702 rpcz_store.cc:276] 0709 13:11:00.335239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.643379 (+308140us) service_pool.cc:224] Handling call
0709 13:11:00.646043 (+  2664us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:00.658314 (+ 12271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:00.658349 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:00.658972 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 13:11:00.911999 (+253027us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:00.912048 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:00.912124 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:01.658706 (+746582us) raft_consensus.cc:1685] finished
0709 13:11:01.658741 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.658941 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11744,"raft.queue_time_us":13062,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":252625,"threads_started":1,"wal-append.queue_time_us":559285}
I20260709 13:11:01.660816 26517 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.661569 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:01.661849 22707 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:11:01.639412 26644 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:01.667142 26644 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.668322 26583 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.653028 26593 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:01.671939 26593 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.676885 22702 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:11:01.649642 26777 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:01.674944 26703 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.659081 26779 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:01.670414 26644 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.705785 26777 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:01.708976 22702 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
W20260709 13:11:01.693157 26680 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.001s
W20260709 13:11:01.721942 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.717113 26757 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:01.821552 26703 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:01.822072 26787 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:01.822921 26787 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:01.824783 26617 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.001s
I20260709 13:11:01.826323 26787 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.711935 26298 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.721042 26680 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:01.707217 26779 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:01.820194 26630 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.844044 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:01.899822 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 470799us
W20260709 13:11:01.838423 26761 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.654s	user 0.011s	sys 0.011s
I20260709 13:11:01.904263 22633 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:01.908058 22776 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:01.913467 26490 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:01.918480 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2560) took 1586 ms. Trace:
W20260709 13:11:01.928213 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 432069us
I20260709 13:11:01.822192 26703 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:01.930968 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.032s	sys 0.039s
W20260709 13:11:01.927129 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 500084us
I20260709 13:11:01.919060 22708 rpcz_store.cc:276] 0709 13:11:00.332521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.581125 (+248604us) service_pool.cc:224] Handling call
0709 13:11:00.581562 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:00.583647 (+  2085us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:00.907899 (+324252us) log.cc:844] Serialized 14 byte log entry
0709 13:11:01.384876 (+476977us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:01.384936 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:01.384989 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.918164 (+533175us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b54002bfcd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:01.918225 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:01.918403 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134133,"prepare.queue_time_us":69289,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":183671,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":20455,"spinlock_wait_cycles":35866240,"thread_start_us":643117,"threads_started":3,"wal-append.queue_time_us":425389}
W20260709 13:11:01.933974 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.033s	sys 0.039s
W20260709 13:11:01.935444 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 508424us
I20260709 13:11:01.934262 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:01.937592 22702 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:11:01.938120 22702 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:01.938416 22702 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:11:01.939570 22702 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:01.846397 26632 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:01.948632 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:01.961059 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:01.962182 22708 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:01.940958 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:01.941177 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:01.931928 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 498214us
I20260709 13:11:01.964905 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:01.966972 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:01.944151 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 488290us
I20260709 13:11:01.971331 22702 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:01.722078 26777 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.005491 26777 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.975286 23419 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:01.968026 26770 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:01.968449 26630 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:01.979365 26754 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:01.948060 26680 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:01.979431 26264 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:11:01.972796 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:01.987262 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:01.976692 26703 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:01.986478 26648 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.015151 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48317.8us
I20260709 13:11:02.008248 26730 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.001207 26739 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.003s
I20260709 13:11:02.014959 26770 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:01.994709 26715 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.059305 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86585.7us
I20260709 13:11:02.044121 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.018033 26777 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:02.065487 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63450.7us
I20260709 13:11:02.066638 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.008081 26780 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:11:02.090324 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 138459us
I20260709 13:11:02.018982 26481 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:11:02.081044 26594 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:02.081946 26630 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:02.096974 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:02.098027 23421 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:02.082116 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.099515 22708 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:02.057163 26770 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:02.114749 22705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2571) took 1779 ms (client timeout 1932 ms). Trace:
W20260709 13:11:02.115242 22705 rpcz_store.cc:269] 0709 13:11:00.335509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.410991 (+ 75482us) service_pool.cc:224] Handling call
0709 13:11:02.114506 (+1703515us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540010cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.114681 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384219904}
I20260709 13:11:02.115748 23411 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:11:02.120699 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.121126 26777 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:02.121954 26777 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:02.122130 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:02.123178 26787 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:02.123196 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:02.123729 26787 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.124140 26787 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:02.124300 26706 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:02.124622 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:02.124819 26706 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.125334 26706 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:02.126832 26617 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:02.128855 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:02.129468 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.026587 26680 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.130326 23419 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:11:02.129588 23412 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:02.139725 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 190581us
I20260709 13:11:02.152537 23417 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [8.10-10.11]
I20260709 13:11:02.087340 26478 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:02.089214 26703 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.130335 26758 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:02.039634 26651 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.114281 26780 consensus_queue.cc:1243] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:11:02.162232 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:02.140447 26594 raft_consensus.cc:2764] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:11:02.165694 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.115609 26298 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.056921 26739 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.057442 26730 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.171250 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.134351 26786 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.164083 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:02.086966 26781 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:11:02.182550 22706 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
W20260709 13:11:02.181196 22709 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 539) took 2478 ms (client timeout 1773 ms). Trace:
W20260709 13:11:02.183768 22709 rpcz_store.cc:269] 0709 13:10:59.702270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.897682 (+195412us) service_pool.cc:224] Handling call
0709 13:11:02.180869 (+2283187us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54004100d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.181084 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1814805760}
I20260709 13:11:02.158638 26478 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.159891 26653 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.139992 26580 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:11:02.211058 23412 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:11:02.172712 26126 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.714s	user 0.008s	sys 0.007s
I20260709 13:11:02.165395 26291 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:02.130048 26708 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.198863 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58124.3us
I20260709 13:11:02.203286 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.204025 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.204237 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.204980 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.204891 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.204834 23421 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:02.207197 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.208077 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.208788 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.209481 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.205060 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.219655 22779 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:02.158346 26617 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:02.211017 26594 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.220304 22631 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:02.220608 22627 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:02.220708 22633 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:02.219174 22776 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:02.194782 26478 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:02.219754 22778 raft_consensus.cc:2410] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:02.165848 26703 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.172109 26777 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:11:02.235178 23420 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:02.193547 26786 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.243125 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:02.238461 26651 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.223675 26126 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:02.179754 26298 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.226226 26756 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.251816 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:02.235256 26791 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:11:02.214802 26806 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:02.223358 26630 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:02.236140 26782 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 13:11:02.239404 26785 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:11:02.232302 26778 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.227711 26594 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.266536 22626 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:02.329963 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:02.248576 26739 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.214198 26653 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.279650 26793 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:11:02.255967 26806 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.252238 26805 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.279400 26648 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.342088 26580 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.376372 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1913) took 1620 ms. Trace:
I20260709 13:11:02.376533 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1892) took 1029 ms. Trace:
I20260709 13:11:02.378844 22635 rpcz_store.cc:276] 0709 13:11:00.755902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.756097 (+   195us) service_pool.cc:224] Handling call
0709 13:11:00.770849 (+ 14752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.463478 (+692629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.376103 (+912625us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b540022ea54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:02.376298 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590637952}
I20260709 13:11:02.380168 22775 rpcz_store.cc:276] 0709 13:11:01.346894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.347025 (+   131us) service_pool.cc:224] Handling call
0709 13:11:01.361217 (+ 14192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:01.361621 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.361651 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:01.383844 (+ 22193us) log.cc:844] Serialized 26 byte log entry
0709 13:11:01.715933 (+332089us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:01.716010 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:01.716140 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:02.376189 (+660049us) raft_consensus.cc:1685] finished
0709 13:11:02.376215 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.376373 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":16329,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":13799,"thread_start_us":267117,"threads_started":1,"wal-append.queue_time_us":637817}
W20260709 13:11:02.385339 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 218590us
I20260709 13:11:02.276595 26630 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.378999 26777 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.389042 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:02.393931 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.400749 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.355557 26806 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:02.257727 26756 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:02.371230 26803 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:11:02.370899 26701 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:02.375490 26503 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.336038 26778 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.429489 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.413722 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:02.377418 26708 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.429378 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:02.431186 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.432896 22784 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:02.389410 26715 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:02.436182 22632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1920) took 1557 ms (client timeout 1660 ms). Trace:
I20260709 13:11:02.347334 26617 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:02.439232 22632 rpcz_store.cc:269] 0709 13:11:00.878785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.879000 (+   215us) service_pool.cc:224] Handling call
0709 13:11:02.436006 (+1557006us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b540006e0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.436120 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120920832}
I20260709 13:11:02.435843 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:02.406286 26680 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.361470 26805 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.450429 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:02.447036 26778 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.451184 22709 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:02.447491 26298 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.417358 26701 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:02.387872 26580 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.414453 26801 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:02.456723 22708 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:11:02.457119 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43554.4us
I20260709 13:11:02.461398 25399 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.460350 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:02.467185 26799 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.228s	user 0.000s	sys 0.002s
I20260709 13:11:02.477974 25399 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.481782 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:02.406286 26777 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.468616 26790 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:11:02.456142 26814 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.433871 26503 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.487773 22708 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:02.467957 26685 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:02.509232 22709 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:11:02.508479 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.522135 25399 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.486542 23385 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.522892 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:02.526029 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:02.528719 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:02.532014 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:02.532472 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46029.4us
I20260709 13:11:02.536346 22709 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:02.556131 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:02.489313 26632 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.558897 22778 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:02.556751 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2297) took 2037 ms. Trace:
I20260709 13:11:02.524480 26814 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.491021 26703 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.575354 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.574002 22783 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:11:02.590251 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:02.521204 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:02.572512 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.591444 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.591655 22775 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:02.592228 22783 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:02.580376 23412 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:02.573201 22784 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:11:02.574894 22704 rpcz_store.cc:276] 0709 13:11:00.518892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.056700 (+537808us) service_pool.cc:224] Handling call
0709 13:11:01.380020 (+323320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.387056 (+  7036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.556529 (+1169473us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b5400314a50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.556684 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742870912}
I20260709 13:11:02.587257 22780 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:02.486665 26811 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.595530 22780 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:02.595953 22780 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:02.595903 26811 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.596892 22780 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:02.597664 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.597888 22780 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:02.593631 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:02.598989 23412 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:02.599524 22783 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:02.602653 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:02.562549 26810 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:11:02.606462 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1914) took 1849 ms (client timeout 1622 ms). Trace:
W20260709 13:11:02.564110 26809 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:11:02.611997 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.612754 22775 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:02.609021 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:02.610571 22778 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:02.610884 22629 rpcz_store.cc:269] 0709 13:11:00.757395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.757589 (+   194us) service_pool.cc:224] Handling call
0709 13:11:02.606223 (+1848634us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54002f4c54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.606381 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143550848}
I20260709 13:11:02.554983 26819 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.556172 26739 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:02.616966 22783 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:11:02.567226 26779 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:02.571837 26805 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.518152 26593 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:11:02.572547 26756 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:02.571389 26580 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.615922 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.490000 26801 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:11:02.571743 26708 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:02.591737 26594 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.564289 26786 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.573936 26715 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.629894 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.632220 23419 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:02.610026 26572 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:02.606805 26780 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.649590 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:02.617550 26648 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.657415 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2589) took 1773 ms. Trace:
I20260709 13:11:02.661528 22701 rpcz_store.cc:276] 0709 13:11:00.883688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.239602 (+355914us) service_pool.cc:224] Handling call
0709 13:11:01.239934 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:01.556061 (+316127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.556117 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:01.619090 (+ 62973us) log.cc:844] Serialized 52 byte log entry
0709 13:11:02.022801 (+403711us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:02.022846 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.022934 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:02.657145 (+634211us) raft_consensus.cc:1685] finished
0709 13:11:02.657170 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.657345 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21992,"raft.queue_time_us":107824,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":4596,"thread_start_us":696678,"threads_started":2,"wal-append.queue_time_us":542911}
I20260709 13:11:02.633819 26805 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:02.628880 26739 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:02.696506 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.642237 26572 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:02.591478 26701 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.624307 26817 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:02.701356 26800 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:02.670322 26648 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.702898 26816 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:02.703862 26812 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:02.706485 26814 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.721431 26685 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.728811 26567 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:02.717581 26739 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.653901 26715 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.726171 22703 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:11:02.671155 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:02.746807 22703 pending_rounds.cc:85] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 5
I20260709 13:11:02.747081 22703 pending_rounds.cc:107] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 9.6
W20260709 13:11:02.729918 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 208837us
I20260709 13:11:02.730913 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 1915) took 1970 ms. Trace:
I20260709 13:11:02.656394 26786 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:02.733714 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 210913us
I20260709 13:11:02.737715 22780 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:11:02.696841 26653 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.708087 26777 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.625921 26819 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.672612 26811 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.648097 26708 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.716393 26805 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:02.747334 22703 raft_consensus.cc:2889] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:02.728003 26823 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:11:02.732749 26478 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.753516 23417 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:11:02.749902 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:02.754187 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.747803 22630 rpcz_store.cc:276] 0709 13:11:00.760747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.805321 (+ 44574us) service_pool.cc:224] Handling call
0709 13:11:00.823104 (+ 17783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:02.132984 (+1309880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.133032 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:02.150492 (+ 17460us) log.cc:844] Serialized 26 byte log entry
0709 13:11:02.208014 (+ 57522us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:02.208059 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.605875 (+397816us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540022c254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:02.605936 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:02.697654 (+ 91718us) raft_consensus.cc:1685] finished
0709 13:11:02.697678 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:02.730811 (+ 33133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1670,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":838,"spinlock_wait_cycles":3011285376,"thread_start_us":26920,"threads_started":1,"wal-append.queue_time_us":258349}
W20260709 13:11:02.757815 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 86762.2us
I20260709 13:11:02.751078 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.772433 23419 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:11:02.733250 26593 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:11:02.741614 26701 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:02.761905 26776 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.762383 26784 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.764084 26708 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.765084 26805 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.766441 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1892) took 3834 ms (client timeout 2092 ms). Trace:
I20260709 13:11:02.751046 22780 pending_rounds.cc:85] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:11:02.771651 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:02.784039 26811 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:02.766949 26580 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:11:02.801358 22626 rpcz_store.cc:269] 0709 13:10:58.932469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:58.946364 (+ 13895us) service_pool.cc:224] Handling call
0709 13:11:02.766163 (+3819799us) spinlock_profiling.cc:243] Waited 3.27 s on lock 0x7b540022d154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:02.766329 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7509636480}
I20260709 13:11:02.774670 26572 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.806916 26593 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:02.807361 26572 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:02.809476 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.810072 26580 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:02.811208 26580 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.799028 26503 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:02.793069 26770 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.006s
I20260709 13:11:02.746502 26812 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.794548 26648 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.758814 26786 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.795193 26715 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.795446 26824 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:02.745527 26685 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.802735 22780 pending_rounds.cc:107] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:11:02.858105 22780 raft_consensus.cc:2889] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:02.760550 26567 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:02.736290 26298 consensus_queue.cc:1175] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "1f99ad34cc414233929e0a92f8c3d8db" responder_term: 10 status { last_received { term: 10 index: 11 } last_committed_idx: 9 last_received_current_leader { term: 10 index: 11 } }
I20260709 13:11:02.804560 22628 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:02.804152 26811 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:02.828289 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56745.2us
I20260709 13:11:02.807399 26593 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:02.833967 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.842381 26630 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.845909 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 880) took 1316 ms. Trace:
I20260709 13:11:02.868024 23420 rpcz_store.cc:276] 0709 13:11:01.529932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.530139 (+   207us) service_pool.cc:224] Handling call
0709 13:11:01.530528 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.241253 (+710725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:02.845524 (+604271us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54004e9654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:02.845809 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620013824}
I20260709 13:11:02.849416 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.657568 26780 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:02.855795 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 188974us
I20260709 13:11:02.799220 26776 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:02.879034 22702 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:02.866204 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.850607 26649 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:02.867403 22626 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:11:02.883185 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.882606 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:02.884491 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.850660 26503 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:02.871675 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.872684 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.888366 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.874200 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.889539 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:02.891345 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.892041 22634 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:11:02.891937 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.886870 26503 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:02.897460 26503 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:02.898015 26801 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:02.830667 26770 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.901137 26770 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:02.831043 26777 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:11:02.903822 26478 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.904520 26780 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:02.904673 23412 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:02.864926 26811 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:02.828547 22673 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:02.908572 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 80144us
I20260709 13:11:02.786016 26817 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:02.871320 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.886819 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.879753 26826 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:11:02.880764 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.914857 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.909847 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.910652 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.911625 22702 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:02.920418 22783 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:02.921254 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.913486 22708 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:02.913395 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.919872 22779 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:02.921978 22702 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:02.830370 26784 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.927109 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:02.930825 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:02.893064 26617 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.922794 22709 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:02.925661 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.893607 26798 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:02.931001 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:02.934134 22708 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:02.938196 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.929363 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.934180 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.936280 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.936609 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:02.907956 26802 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:02.942543 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.944115 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.906159 26567 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:02.945015 22779 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:02.943182 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.922472 26777 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:02.947508 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.948594 23417 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:02.942263 26787 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.818s	user 0.015s	sys 0.002s
I20260709 13:11:02.940296 26630 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:02.923846 26817 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:02.959784 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.975400 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.978339 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:02.979974 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.935583 26264 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:11:02.984310 26715 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:02.987069 26490 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:02.987473 26825 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:02.993624 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:02.952875 26777 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:02.999542 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:02.951478 26831 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:02.939925 26617 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.006440 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:02.927567 26593 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.008101 26685 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.008359 26593 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:03.009972 26767 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.792s	user 0.000s	sys 0.011s
I20260709 13:11:03.011571 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2575) took 2581 ms. Trace:
I20260709 13:11:03.012032 22703 rpcz_store.cc:276] 0709 13:11:00.429743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.727768 (+298025us) service_pool.cc:224] Handling call
0709 13:11:00.728121 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.760080 (+2031959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.011261 (+251181us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54004105d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:03.011494 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4579697408}
I20260709 13:11:03.013772 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 881) took 1170 ms. Trace:
I20260709 13:11:03.014235 23411 rpcz_store.cc:276] 0709 13:11:01.843194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.843403 (+   209us) service_pool.cc:224] Handling call
0709 13:11:01.843959 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:02.116260 (+272301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.013451 (+897191us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54005d5554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:03.013703 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624376064}
W20260709 13:11:03.015159 26806 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.593s	user 0.007s	sys 0.006s
I20260709 13:11:03.007369 22709 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:03.015654 26806 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.015866 26503 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.003315 26777 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:03.019151 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 587) took 2647 ms. Trace:
I20260709 13:11:03.019603 22782 rpcz_store.cc:276] 0709 13:11:00.371546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.371735 (+   189us) service_pool.cc:224] Handling call
0709 13:11:00.372178 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:01.383404 (+1011226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.018850 (+1635446us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b5400130e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:03.019081 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2241412992}
I20260709 13:11:03.024602 26617 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.026537 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:03.027992 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.029450 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 543) took 1628 ms. Trace:
I20260709 13:11:03.004629 26823 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:03.030990 26823 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:03.031032 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:02.948063 26276 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:11:03.036327 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.036518 26823 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.038048 26823 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:03.018347 26503 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:02.999822 26706 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.012s
I20260709 13:11:02.996471 26583 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:02.973600 26787 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.050753 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:03.064234 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62643.9us
I20260709 13:11:03.030007 22710 rpcz_store.cc:276] 0709 13:11:01.401574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.404989 (+  3415us) service_pool.cc:224] Handling call
0709 13:11:01.405342 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:01.424419 (+ 19077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:01.424455 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:01.437301 (+ 12846us) log.cc:844] Serialized 26 byte log entry
0709 13:11:01.921403 (+484102us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:03.028553 (+1107150us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b180021a000. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:11:03.028745 (+   192us) log.cc:844] Serialized 14 byte log entry
0709 13:11:03.029043 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.029126 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:03.029175 (+    49us) raft_consensus.cc:1685] finished
0709 13:11:03.029259 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.029389 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15731,"prepare.queue_time_us":314549,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":32005,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":12112,"spinlock_wait_cycles":17244416,"thread_start_us":590901,"threads_started":2,"wal-append.queue_time_us":614857}
I20260709 13:11:03.066414 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:02.997726 26630 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.045617 26812 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.005200 26825 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:11:03.095402 22704 raft_consensus.cc:2368] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:03.058097 26583 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:03.093744 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:03.059525 26650 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.077926 26624 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:03.086578 26572 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:03.084120 26806 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.118940 26787 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:03.119504 26833 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:11:03.059293 26706 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.086359 26778 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:11:03.104092 26831 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:03.101032 26786 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.133862 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:03.141847 22782 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:03.142997 26806 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:11:03.146759 26832 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:11:03.150833 26617 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.151546 26617 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.154757 26617 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.110055 26650 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.185122 26650 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:03.155695 26836 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:11:03.155107 26811 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.188374 26811 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.159209 26784 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:03.188550 26779 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.004s
I20260709 13:11:03.163259 26778 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:03.189419 26490 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:03.189894 26778 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:03.098697 26770 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:11:03.168025 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.168893 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:03.168915 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.134506 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:03.178479 26777 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.159628 22782 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:03.199086 22783 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:03.201560 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67156.9us
I20260709 13:11:03.176170 26630 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.203387 22779 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:03.113709 26583 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:03.207547 26811 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:03.214787 26811 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:03.215446 26811 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:03.174340 26817 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.218914 26617 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.198591 26778 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.220957 26777 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.223852 26778 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:03.220564 26617 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.221648 26817 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.203410 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:03.281600 22701 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
W20260709 13:11:03.282076 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 203063us
W20260709 13:11:03.282593 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:03.286286 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 194942us
W20260709 13:11:03.291719 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.298156 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05be2cb1839f447296223033d364a7cc"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:03.297405 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:03.305874 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 102376us
I20260709 13:11:03.308506 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 1909) took 1093 ms. Trace:
W20260709 13:11:03.348869 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57477.7us
W20260709 13:11:03.345371 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.344696 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:03.345830 26220 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:03.399602 26436 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:03.401403 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:03.406596 26784 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.407734 26784 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.394939 22782 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:03.330516 25399 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:11:03.423714 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1927) took 1217 ms. Trace:
I20260709 13:11:03.424232 22628 rpcz_store.cc:276] 0709 13:11:02.206349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.207768 (+  1419us) service_pool.cc:224] Handling call
0709 13:11:03.423196 (+1215428us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54003d3454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:03.423473 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1338338560}
I20260709 13:11:03.342007 26617 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.441013 26817 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.445339 26630 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.340025 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.450026 22635 raft_consensus.cc:606] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:03.453904 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.456208 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:03.459435 26592 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:03.345594 26624 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:03.309248 22781 rpcz_store.cc:276] 0709 13:11:02.215373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.218718 (+  3345us) service_pool.cc:224] Handling call
0709 13:11:02.233093 (+ 14375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:02.434211 (+201118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:02.434291 (+    80us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:02.436861 (+  2570us) log.cc:844] Serialized 26 byte log entry
0709 13:11:02.524763 (+ 87902us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:02.524817 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.307495 (+782678us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400131354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:03.307585 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:03.307678 (+    93us) raft_consensus.cc:1685] finished
0709 13:11:03.307697 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:03.308176 (+   479us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6131,"raft.queue_time_us":504,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":13679,"spinlock_wait_cycles":461517056,"thread_start_us":58599,"threads_started":1,"wal-append.queue_time_us":178630}
I20260709 13:11:03.463578 26784 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.465623 26708 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:03.475984 26780 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:03.477437 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.477522 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:03.480238 26816 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:11:03.500017 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98842.1us
I20260709 13:11:03.480579 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.483371 22784 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:03.484223 26819 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.336840 26291 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:03.487830 22703 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:03.516610 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:03.517776 22784 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:03.517678 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.398520 26220 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:03.330516 26481 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.182s
I20260709 13:11:03.499583 26780 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:03.525079 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.528123 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.556686 22782 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:03.503103 26816 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:03.565349 22779 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:03.565007 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:03.331864 26490 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.232s
I20260709 13:11:03.542476 26756 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:03.534001 26776 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.541373 26730 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.533368 26692 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:03.583052 22701 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:03.592418 26816 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.594424 26816 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:03.597030 26692 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:03.598654 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:03.599067 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.598264 26692 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.600041 23412 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:03.601020 26692 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:03.583966 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:03.602900 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:03.603672 22707 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:03.603902 22634 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:03.605218 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:03.606171 22779 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:03.541551 26701 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:03.611953 26189 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:03.612515 26189 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:03.613823 26189 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.616166 26189 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:03.507874 26630 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.541919 26819 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.625780 26701 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.626225 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.627547 23415 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:03.632089 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118798us
I20260709 13:11:03.581218 26756 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.634208 26828 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.638664 22634 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:03.652637 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:03.653623 22632 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:03.661150 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:03.668061 26706 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.671005 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:03.644707 26828 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.643827 22631 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:11:03.678890 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45846.3us
I20260709 13:11:03.608515 26817 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.646457 26756 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:03.710806 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:03.720706 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.682808 26264 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.711311 26580 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:11:03.721046 22627 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:03.722050 26264 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.725016 26264 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:03.711068 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.705590 26220 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.702940 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.728564 26788 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.729547 26788 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.734185 22708 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:03.735894 22627 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:03.692831 26436 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:03.737529 26436 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:03.737972 26436 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:03.743160 22629 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:11:03.740269 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 91350.3us
W20260709 13:11:03.750293 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63693.5us
I20260709 13:11:03.693034 26701 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:03.769577 26583 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.011s
I20260709 13:11:03.770248 26583 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.772850 26583 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:03.773656 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:03.776299 26630 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.776433 26817 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.684608 26819 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.801324 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:03.808666 22750 leader_election.cc:400] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:03.816346 22750 leader_election.cc:403] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:03.814205 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.829062 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.814066 26624 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:03.835186 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.839603 22627 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:03.844152 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:03.845850 26632 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:03.846647 26632 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.857234 26828 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:03.866410 26811 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.650s	user 0.023s	sys 0.002s
I20260709 13:11:03.867120 26811 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:03.846778 22783 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.637s	user 0.017s	sys 0.006s
I20260709 13:11:03.828568 26630 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:03.899036 22626 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:11:03.896561 26481 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:03.892791 22783 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:03.907232 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:03.907644 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 167523us
I20260709 13:11:03.908277 22627 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:03.899732 22626 pending_rounds.cc:85] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 6
I20260709 13:11:03.905404 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:03.910481 22626 pending_rounds.cc:107] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:11:03.915864 22626 raft_consensus.cc:2889] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:03.915381 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 221948us
I20260709 13:11:03.921217 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:03.924717 22627 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:03.931469 26811 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.934392 22627 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:03.941854 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:03.944036 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 135630us
I20260709 13:11:03.909953 26481 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:03.987445 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:03.990326 26706 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:03.997686 22632 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:04.010123 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.011243 22634 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:03.954685 26638 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
I20260709 13:11:04.012252 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.012430 22635 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:03.936379 26583 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:04.013338 26583 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:04.014595 26583 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:03.957345 26819 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:03.948190 22776 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:03.964426 26592 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:03.911514 26624 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.008582 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.008982 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:03.960088 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:03.930465 26780 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.066121 22626 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:04.076452 26291 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:04.077626 26434 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:04.037300 26816 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.068739 26220 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:04.082860 26780 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:04.049297 26481 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.080919 26756 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:04.133225 26189 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.514s	user 0.015s	sys 0.000s
I20260709 13:11:04.117343 26291 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.122889 26644 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:11:04.146773 26624 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:04.161821 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.165333 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 205439us
I20260709 13:11:04.167119 26291 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.125519 26816 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.171751 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:04.176074 26778 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.891s	user 0.013s	sys 0.007s
W20260709 13:11:04.176563 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 106775us
I20260709 13:11:04.177525 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.177963 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.179051 26819 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:04.180680 26816 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.177992 26291 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.180497 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.199635 26291 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:04.190397 26819 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:04.198235 26644 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:04.234820 22633 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:11:04.249305 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.245690 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:04.211977 26644 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:04.271391 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:04.246073 26849 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.391s	user 0.002s	sys 0.003s
I20260709 13:11:04.244256 26816 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:04.281426 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101056us
I20260709 13:11:04.243090 26630 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.219955 26481 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.242802 26811 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.273186 26644 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:04.240631 26434 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:04.286784 26816 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.291352 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:04.291467 26779 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.294477 26567 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:04.295053 26567 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:04.294853 26481 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.295496 26567 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:04.296624 26816 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.300477 26632 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.300833 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:04.301159 26632 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.302917 26819 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:04.303424 26819 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:04.303645 26866 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:11:04.303839 26819 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:04.309285 26867 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:11:04.294756 26779 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.309198 26816 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:04.301729 26632 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.244668 26592 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:11:04.306543 26434 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:04.316105 26436 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.578s	user 0.013s	sys 0.005s
I20260709 13:11:04.244402 26572 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:11:04.308413 26756 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:04.331126 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:04.331677 22777 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:11:04.332448 26632 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.333473 22783 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:04.313304 26816 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.336726 22777 pending_rounds.cc:85] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 6
I20260709 13:11:04.341038 26779 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:04.342077 26860 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.406s	user 0.000s	sys 0.003s
I20260709 13:11:04.342236 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:04.342697 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:04.343293 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:04.346402 22777 pending_rounds.cc:107] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 5.7
W20260709 13:11:04.350236 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48003.5us
I20260709 13:11:04.350292 22779 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:04.350541 22777 raft_consensus.cc:2889] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:04.350646 22778 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:04.344983 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.314131 26811 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.347165 26816 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:04.359851 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.365327 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:04.367853 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:04.359812 22701 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:04.320150 26630 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.360037 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2572) took 3987 ms. Trace:
I20260709 13:11:04.374408 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:04.376889 22781 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:04.375063 22706 rpcz_store.cc:276] 0709 13:11:00.372195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:00.645525 (+273330us) service_pool.cc:224] Handling call
0709 13:11:02.182257 (+1536732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:02.183258 (+  1001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.359582 (+2176324us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54004100d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:04.359816 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3533262720}
I20260709 13:11:04.382298 22703 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:04.386292 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:04.389555 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:04.394752 26189 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.395552 26189 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.338034 26436 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.407341 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 559) took 1318 ms. Trace:
W20260709 13:11:04.409936 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 166936us
I20260709 13:11:04.387074 26638 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.432693 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.433627 22701 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:04.434794 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:04.435560 22701 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:04.436525 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.440168 26630 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.442790 26671 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.443599 26671 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.446133 26671 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.447067 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:04.448069 23412 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:04.409403 22710 rpcz_store.cc:276] 0709 13:11:03.087828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.125204 (+ 37376us) service_pool.cc:224] Handling call
0709 13:11:03.125729 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:03.126865 (+  1136us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.126944 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:03.127933 (+   989us) log.cc:844] Serialized 26 byte log entry
0709 13:11:03.680010 (+552077us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:03.689817 (+  9807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.741730 (+ 51913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:04.193387 (+451657us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:04.403780 (+210393us) raft_consensus.cc:1685] finished
0709 13:11:04.403807 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:04.406217 (+  2410us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":199,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":16414,"spinlock_wait_cycles":5504384,"thread_start_us":382795,"threads_started":1,"wal-append.queue_time_us":643367}
I20260709 13:11:04.460062 26434 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:04.461112 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 101425us
I20260709 13:11:04.462965 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:04.467247 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 182480us
I20260709 13:11:04.478155 22706 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:04.417593 26816 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.490490 22781 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:04.426043 26638 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.478250 26630 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:04.490729 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:04.477059 26189 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.493047 26434 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:04.491272 26756 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:04.510918 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:04.520313 26873 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:04.523234 26873 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.523713 26873 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:04.526671 26672 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.527944 26816 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.526714 26875 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:04.510970 26436 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:04.537782 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:04.517297 26822 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.796s	user 0.005s	sys 0.000s
I20260709 13:11:04.589792 22631 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:04.541069 26756 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.546736 26819 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.507160 26630 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.551257 26872 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:04.551806 26811 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.515373 26434 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.552855 26481 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:04.554131 26638 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.555843 26800 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:04.559795 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2615) took 1073 ms. Trace:
W20260709 13:11:04.561710 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 802974us
W20260709 13:11:04.579150 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47303.1us
W20260709 13:11:04.580006 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.583961 26632 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:04.594323 26816 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:04.601624 26630 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.602713 26756 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.603293 26819 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.603530 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2604) took 2432 ms. Trace:
I20260709 13:11:04.596628 26872 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.570360 26788 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.595534 26780 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.615073 26436 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:04.585837 26644 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.611819 26816 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:04.611920 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:04.610160 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.609118 22708 rpcz_store.cc:276] 0709 13:11:03.486655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.507141 (+ 20486us) service_pool.cc:224] Handling call
0709 13:11:03.575040 (+ 67899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:03.734748 (+159708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.559425 (+824677us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540010cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:04.559705 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237904256}
I20260709 13:11:04.586490 26567 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.616008 26632 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.628400 26436 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.612797 26672 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.628535 26780 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.629603 26816 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:04.645187 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 182372us
I20260709 13:11:04.622833 26872 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:04.608917 26800 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:04.663025 23411 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:11:04.664636 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 127222us
W20260709 13:11:04.673167 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61389.4us
I20260709 13:11:04.675716 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:04.676584 23420 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:04.678030 26672 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:04.679661 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.666966 26436 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:04.684031 26481 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.684667 26481 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.619987 22705 rpcz_store.cc:276] 0709 13:11:02.170596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.170806 (+   210us) service_pool.cc:224] Handling call
0709 13:11:02.171206 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:02.480679 (+309473us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:02.515540 (+ 34861us) log.cc:844] Serialized 14 byte log entry
0709 13:11:02.601052 (+ 85512us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:02.601571 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:11:02.601715 (+   144us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:02.601751 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:03.295799 (+694048us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b54000a8cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:03.295892 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:04.357694 (+1061802us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:04.603081 (+245387us) raft_consensus.cc:1685] finished
0709 13:11:04.603105 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:04.603341 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12981,"prepare.queue_time_us":37506,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":24679,"raft.queue_time_us":156768,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":6424,"spinlock_wait_cycles":192227968,"thread_start_us":289788,"threads_started":2,"wal-append.queue_time_us":124379}
I20260709 13:11:04.686960 26788 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:04.687135 26481 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.673954 26572 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:04.688402 26644 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.689731 26800 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.690785 26632 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.690969 26800 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:04.693018 26780 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:04.695246 26567 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:04.696447 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86380.8us
W20260709 13:11:04.698426 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.703611 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:04.704622 26819 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:04.707327 26855 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.851s	user 0.000s	sys 0.004s
I20260709 13:11:04.711215 26873 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.712028 26434 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:04.712195 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:04.714293 26876 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:04.714551 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 600) took 1856 ms. Trace:
I20260709 13:11:04.716078 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2433) took 1192 ms. Trace:
I20260709 13:11:04.716332 26572 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:04.727540 23420 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:11:04.807283 22776 rpcz_store.cc:276] 0709 13:11:02.858600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:02.858987 (+   387us) service_pool.cc:224] Handling call
0709 13:11:02.859327 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:04.021191 (+1161864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.714232 (+693041us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003264d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:04.714485 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490176896}
I20260709 13:11:04.805855 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.815397 22705 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:11:04.817349 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:04.818315 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.819021 22705 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:04.808249 22633 rpcz_store.cc:276] 0709 13:11:03.524064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.526849 (+  2785us) service_pool.cc:224] Handling call
0709 13:11:03.527373 (+   524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:04.215382 (+688009us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b540005ada0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:11:04.267997 (+ 52615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.715741 (+447744us) spinlock_profiling.cc:243] Waited 85.4 ms on lock 0x7b540005ada0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0a02 00007f1806edcdd6 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:04.715988 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360970240}
I20260709 13:11:04.820350 26780 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.817442 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:04.829514 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 128222us
I20260709 13:11:04.830010 22705 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:04.826465 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.813804 22708 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:04.825306 26434 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.854002 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:04.851096 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 171870us
I20260709 13:11:04.835153 26291 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.831354 26876 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:04.861420 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:04.864454 26819 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:04.835646 26780 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.875285 26873 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:04.876214 26873 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:04.874820 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 161239us
W20260709 13:11:04.874378 22749 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.507s (SENT) [suppressed 22 similar messages]
W20260709 13:11:04.893210 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:04.862983 26434 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.895752 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:04.867023 26291 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.907055 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:04.893724 26882 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:04.928442 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:04.887532 26873 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.935570 26878 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:11:04.891014 26490 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:11:04.936959 26490 consensus_queue.cc:1243] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer 6a31ac6451ff4f73bee634487d84d001 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:11:04.943224 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.951200 26879 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:04.951853 26879 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:04.952255 26879 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:04.955680 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1945) took 1194 ms (client timeout 1586 ms). Trace:
W20260709 13:11:04.956435 22631 rpcz_store.cc:269] 0709 13:11:03.763664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.779074 (+ 15410us) service_pool.cc:224] Handling call
0709 13:11:04.955386 (+1176312us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54003197d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:04.955570 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776647424}
I20260709 13:11:04.958901 26638 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.959601 26638 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:04.963160 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:04.966006 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 91701.2us
I20260709 13:11:04.970167 22632 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:04.973908 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 96285.9us
I20260709 13:11:04.973592 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.961444 26828 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:04.981370 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:04.963315 26890 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:04.981914 26828 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:04.979183 26816 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:04.982163 26882 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.011377 22701 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:11:04.982832 26436 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.024161 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.029238 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:05.025786 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.033193 22784 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:05.028515 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.035831 22779 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:05.035015 22777 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:05.037977 22783 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:05.035080 22778 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:05.039098 22780 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:04.967089 26876 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.040987 26828 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.041090 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.050521 26828 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:05.050971 26873 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.020605 26291 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.056599 22628 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:05.018231 26890 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.071638 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.067947 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.072847 22631 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:05.074190 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:05.079463 26889 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:11:05.079624 26756 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.080299 26756 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.041067 26481 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.046929 26816 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.091854 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.095810 22784 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:05.095712 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:05.104485 26901 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.105425 26901 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.120283 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:05.121416 22710 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:11:05.122504 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.129479 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:05.130685 22710 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:11:05.153522 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.157407 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.157992 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.163918 22630 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:05.101052 26481 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.167435 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:05.170740 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.170503 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.172540 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.173633 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:05.173254 22633 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:05.185463 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.186424 22626 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:05.186839 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:05.144901 26872 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.189190 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:05.191429 22635 raft_consensus.cc:2368] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:05.171151 26873 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:11:05.204424 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.213696 22632 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
W20260709 13:11:05.215246 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45246.3us
I20260709 13:11:05.223209 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:05.222405 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:05.225477 22631 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:05.212540 26672 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:05.232842 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:05.234946 26672 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:05.235502 26672 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:05.239405 26739 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:05.239948 26739 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:05.240353 26739 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:05.256348 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.260440 26572 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:05.259228 26567 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:05.268194 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:05.276206 22775 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:05.267180 26572 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.278599 26572 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:05.278839 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:05.279578 22784 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:05.283221 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:05.283900 22704 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:05.287349 26882 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.288090 22778 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:11:05.288169 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:05.288578 22778 pending_rounds.cc:85] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:11:05.289029 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.289296 22778 pending_rounds.cc:107] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:11:05.289615 22778 raft_consensus.cc:2889] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:05.289023 22710 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:05.291909 26896 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 13:11:05.292181 26901 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:05.294150 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 405976us
I20260709 13:11:05.295536 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:05.297183 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:05.301594 26816 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:05.302218 26816 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:05.301635 26679 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.303802 26679 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.295097 26878 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:05.306653 26878 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:05.294432 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2633) took 1054 ms. Trace:
I20260709 13:11:05.307799 26878 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.308341 22706 rpcz_store.cc:276] 0709 13:11:04.239570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.479693 (+240123us) service_pool.cc:224] Handling call
0709 13:11:04.497308 (+ 17615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:04.508553 (+ 11245us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:11:04.809436 (+300883us) log.cc:844] Serialized 14 byte log entry
0709 13:11:04.896222 (+ 86786us) log.cc:844] Serialized 14 byte log entry
0709 13:11:05.285735 (+389513us) spinlock_profiling.cc:243] Waited 7.75 ms on lock 0x7b180018e200. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:05.293872 (+  8137us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b180018e200. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:05.294063 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:05.294117 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:05.294174 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:05.294254 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:05.294376 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8505,"prepare.queue_time_us":425521,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":2149,"raft.queue_time_us":402608,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":221,"spinlock_wait_cycles":25829632,"thread_start_us":315371,"threads_started":2,"wal-append.queue_time_us":107968}
I20260709 13:11:05.340896 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:05.347860 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:05.336998 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 63815.1us
I20260709 13:11:05.352372 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:05.352065 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:05.357519 22784 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:05.358732 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.359535 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:05.288328 26638 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.270777 26567 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:05.412256 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:05.468137 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:05.471829 22779 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:05.467562 26679 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:05.479584 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 138788us
W20260709 13:11:05.495127 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141332us
I20260709 13:11:05.495501 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:05.480587 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 128512us
I20260709 13:11:05.499199 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:05.501941 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.506067 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:05.505694 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.507187 22633 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:05.505620 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.476994 26567 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:05.511478 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:05.506206 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.513679 22631 raft_consensus.cc:2410] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:05.514508 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.350427 26264 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.148s
I20260709 13:11:05.347497 26632 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.157s
I20260709 13:11:05.541610 22784 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:05.496474 26887 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:05.498769 26644 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:05.501125 26220 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:05.507043 26910 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.504422 26679 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:05.553532 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52669.2us
I20260709 13:11:05.510197 26877 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:05.551952 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46861.5us
I20260709 13:11:05.573171 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.573206 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.540903 26291 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:05.584182 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:05.558351 26638 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.557309 26644 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.558010 26788 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:05.559515 26220 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:05.574494 26819 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:05.575855 26756 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:05.639427 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:05.639156 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66047.1us
I20260709 13:11:05.644804 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.632673 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.646561 23411 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:05.560685 26882 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:11:05.649981 23412 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:11:05.634800 26644 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:05.642450 26915 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.641407 26264 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:05.636848 26220 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:05.670079 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 170983us
I20260709 13:11:05.640373 26806 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.575392 26910 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.583441 26679 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.636214 26917 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:05.588707 26672 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.650677 26780 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.714121 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.715209 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:05.713325 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 362068us
I20260709 13:11:05.586562 26816 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.635649 26638 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.706199 26913 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.720556 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.723416 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.643146 26788 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:05.714744 26879 consensus_meta.cc:287] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.762s	user 0.013s	sys 0.000s
I20260709 13:11:05.726008 23420 raft_consensus.cc:2410] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
W20260709 13:11:05.707258 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51913.9us
W20260709 13:11:05.727079 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142994us
I20260709 13:11:05.711601 26917 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.715473 26756 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.729502 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.716444 26919 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:05.721493 26780 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:05.741544 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.728514 26264 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:05.748431 26879 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.755805 26264 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:05.757642 26879 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.758276 26890 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.758538 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:05.759270 26880 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.931s	user 0.005s	sys 0.000s
I20260709 13:11:05.759148 26927 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:05.761863 26291 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.762406 26291 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:05.764273 26638 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.766275 26679 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.768033 23411 pending_rounds.cc:85] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 8
I20260709 13:11:05.768465 23411 pending_rounds.cc:107] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:11:05.768724 23411 raft_consensus.cc:2889] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:05.769500 26788 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.770262 26919 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:05.772557 26912 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:11:05.772847 26919 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:05.773892 26914 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:11:05.776080 26672 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.776896 26672 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:05.777792 26672 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.754005 26780 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.778364 26672 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.779109 26780 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:05.704879 26915 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.781692 26672 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.783185 26915 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.725837 26882 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:05.784337 26882 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:05.784597 26592 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:05.778322 26291 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.784976 26882 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:05.785162 26592 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:05.726518 26926 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:05.788203 26756 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:05.783481 26921 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:11:05.729161 26632 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:05.642457 26819 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.769243 26572 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.683663 22704 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:05.808360 22633 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:05.804598 26291 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:05.810703 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59082.4us
I20260709 13:11:05.812747 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2616) took 2284 ms. Trace:
I20260709 13:11:05.806504 26817 raft_consensus.cc:2764] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:11:05.813607 23420 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:11:05.770934 26806 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.814777 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.786275 26592 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.793651 26632 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:05.730948 26913 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:05.824950 26909 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.581s	user 0.000s	sys 0.002s
I20260709 13:11:05.825094 26572 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.751147 26917 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:05.828753 26916 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:11:05.831167 26819 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:05.832728 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 112106us
I20260709 13:11:05.857954 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:05.857712 26632 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:05.834534 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 93187.3us
I20260709 13:11:05.859261 26632 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:05.860165 26592 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:05.844254 22709 rpcz_store.cc:276] 0709 13:11:03.510648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:03.515183 (+  4535us) service_pool.cc:224] Handling call
0709 13:11:03.523035 (+  7852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:03.523619 (+   584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:03.523657 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:04.533834 (+1010177us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b2c0006cb88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:04.534431 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:11:04.719442 (+185011us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:04.719507 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:04.719600 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:05.664130 (+944530us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:05.791690 (+127560us) raft_consensus.cc:1685] finished
0709 13:11:05.791714 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:05.791878 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7606,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":198,"spinlock_wait_cycles":84821632,"thread_start_us":184505,"threads_started":1,"wal-append.queue_time_us":292895}
I20260709 13:11:05.862709 26788 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:05.836805 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.836966 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.838722 26928 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:05.768074 26910 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.822223 23421 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:05.844970 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:05.858758 26806 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.865801 26264 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.873605 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:05.874027 26264 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:05.875068 26918 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:11:05.875780 26923 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:05.874856 26638 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:05.876760 26756 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.877859 26220 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.878890 26910 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:05.882714 22780 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.012s	sys 0.008s
W20260709 13:11:05.886489 22780 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.012s	sys 0.008s
I20260709 13:11:05.888165 26933 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:05.889189 26933 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:05.889564 26933 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:05.874024 26828 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.823s	user 0.010s	sys 0.003s
W20260709 13:11:05.892321 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 186287us
I20260709 13:11:05.898662 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:05.904657 22627 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:11:05.921744 26220 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.891975 26800 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:05.937579 26903 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.760s	user 0.000s	sys 0.015s
I20260709 13:11:05.940054 26780 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.941850 23414 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:05.944195 23418 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:11:05.889997 26644 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.921119 26919 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:11:05.921404 26679 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:05.948930 26679 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:05.949338 26644 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.949611 26922 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:11:05.950850 26780 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:05.951036 26922 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:05.951438 26922 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:05.953043 26739 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.712s	user 0.005s	sys 0.012s
I20260709 13:11:05.953677 26739 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.954501 26887 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:05.954587 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:05.957583 26436 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:05.959089 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.986439 22633 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:05.983776 22747 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:05.959972 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:05.983726 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 124211us
I20260709 13:11:05.961499 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:05.963032 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:05.949410 26679 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:05.965816 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.895221 26897 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:11:05.964440 26882 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.972656 26800 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:05.974428 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.917119 26915 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:11:05.994496 26915 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:05.994978 26915 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:05.995739 26913 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.996012 26915 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.997491 26882 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:05.964857 26933 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:05.976034 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:05.976610 26910 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:05.977571 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:05.978173 26919 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:05.978905 26931 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:11:06.006925 26739 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:05.978960 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 82753.3us
I20260709 13:11:05.984061 26887 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.007676 23415 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:06.007869 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:05.961110 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:05.988171 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:05.988598 22632 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:05.989048 22630 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:05.988865 26436 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:05.952482 26922 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.001335 23420 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:06.004329 22529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 144) took 35830 ms (client timeout 15000 ms). Trace:
I20260709 13:11:06.003733 26910 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:06.028273 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 100975us
I20260709 13:11:06.029121 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:06.035431 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46985.9us
I20260709 13:11:06.008399 26887 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.040234 26946 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:06.040715 22749 leader_election.cc:400] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:06.041708 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:06.047922 22749 leader_election.cc:403] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:06.019949 26264 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:06.035898 26919 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:06.048794 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:06.047380 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 117666us
W20260709 13:11:06.053826 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.054868 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:06.057829 26910 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:06.059824 26936 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:06.060415 26936 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.060497 26828 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:06.062315 26264 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.063473 26264 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.066715 26777 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:06.066807 22630 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:06.046300 26887 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.027361 26567 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.036300 26583 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:11:06.078558 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 433873us
I20260709 13:11:06.081269 26630 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:06.084738 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:06.051354 26919 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:06.068341 26936 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:06.092911 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:06.094740 26777 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:11:06.095292 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:06.095901 23412 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.096408 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.095773 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:06.098232 23415 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:06.101310 26946 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:06.083045 26877 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:06.104709 26291 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:06.104532 26877 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.094079 26819 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.106033 26819 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.108242 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.108929 26819 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.109995 22703 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:06.108963 22702 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:06.115458 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:06.118455 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:06.119058 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 74889.8us
I20260709 13:11:06.119714 22709 raft_consensus.cc:2368] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:06.132102 23420 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:06.137269 22632 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
W20260709 13:11:06.120922 26942 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:11:06.088997 26897 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
W20260709 13:11:06.146232 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 105686us
I20260709 13:11:06.151796 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:06.148028 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.152946 22704 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:06.147107 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.155194 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.156008 22701 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:06.156742 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.133304 26913 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:06.157423 22703 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.139106 26291 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:06.172742 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.179957 22706 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:11:06.178437 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:06.200695 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 49052.8us
I20260709 13:11:06.209033 22704 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:06.205128 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:06.215499 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:06.217799 26939 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:06.218196 22601 leader_election.cc:400] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:06.218240 22702 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:06.220265 22601 leader_election.cc:403] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:06.218804 26897 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:06.215538 22707 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:06.222075 26932 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.160290 26913 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.240197 22703 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:06.255939 26954 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:11:06.259586 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 135965us
W20260709 13:11:06.243743 22675 connection.cc:256] RPC call timeout handler was delayed by -1.6429308780000156s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:11:06.261400 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5556308780001018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16160000000000002s for a response
W20260709 13:11:06.262046 22675 connection.cc:256] RPC call timeout handler was delayed by -1.3633444810000128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
W20260709 13:11:06.262456 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2382444810000379s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 13:11:06.262900 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1389308780001102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 13:11:06.271349 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.273671 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.154112 26917 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.204912 26819 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.246706 26630 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.247908 26913 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:11:06.283022 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 191966us
I20260709 13:11:06.283433 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.284233 22703 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:06.296542 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:06.297336 22632 raft_consensus.cc:606] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:06.297721 22632 raft_consensus.cc:614] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:06.298115 22706 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:06.293905 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.299968 22776 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:06.307348 23418 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:06.339900 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.257685 26592 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:11:06.395622 22704 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:06.400004 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.400882 22780 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:06.194784 26919 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:06.401530 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.408897 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.413005 22706 raft_consensus.cc:2368] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:06.409932 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.413136 22703 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:06.423441 22704 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:06.425366 22707 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:06.429595 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.438397 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.440243 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.403581 26873 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:06.412060 26567 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.448751 22703 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:06.290916 26630 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.457096 26567 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:06.411988 26291 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.463788 26956 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:06.464263 26947 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:11:06.462031 26949 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:06.477244 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.449359 26436 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.487100 23412 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:06.311766 26819 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:06.491638 26918 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:11:06.502238 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 228644us
I20260709 13:11:06.508579 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.490041 26873 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.518672 22704 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:06.530328 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.531092 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.531939 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.532699 22709 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:06.533653 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.534770 26567 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.535362 22701 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:06.536710 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.537484 26567 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:06.538033 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.538764 26951 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:06.539661 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.540000 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:06.023614 22529 rpcz_store.cc:269] 0709 13:10:30.173483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.182825 (+  9342us) service_pool.cc:224] Handling call
0709 13:11:06.004007 (+35821182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:57.359255 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:57.360147 (+   892us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:57.360220 (+    73us) write_op.cc:435] Acquired schema lock
0709 13:10:57.360249 (+    29us) tablet.cc:662] Decoding operations
0709 13:10:57.456409 (+ 96160us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:57.481619 (+ 25210us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:57.485181 (+  3562us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:10:57.731542 (+246361us) tablet.cc:701] Row locks acquired
0709 13:10:57.731596 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:10:57.731832 (+   236us) write_op.cc:270] Start()
0709 13:10:57.732060 (+   228us) write_op.cc:276] Timestamp: P: 1783602657731722 usec, L: 0
0709 13:10:57.732109 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:10:57.733004 (+   895us) log.cc:844] Serialized 6292 byte log entry
0709 13:10:58.464560 (+731556us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b54003582a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806e546b8 00007f1806e542fa 00007f1806ed8965 00007f1806ed8e09 00007f180a913581 00007f180a912d58 00007f180a9179f1 00007f180a9179a9 00007f180a917939 00007f180a917901
0709 13:10:58.570889 (+106329us) op_driver.cc:464] REPLICATION: finished
0709 13:10:59.045971 (+475082us) write_op.cc:301] APPLY: starting
0709 13:10:59.046098 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:59.659772 (+613674us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:59.659823 (+    51us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:00.272645 (+612822us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:00.273061 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=0,mrs_lookups=0
0709 13:11:00.273120 (+    59us) write_op.cc:312] APPLY: finished
0709 13:11:00.430906 (+157786us) log.cc:844] Serialized 148 byte log entry
0709 13:11:00.810794 (+379888us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:00.826330 (+ 15536us) write_op.cc:454] Released schema lock
0709 13:11:00.843975 (+ 17645us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:00.856547 (+ 12572us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:00.856585 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7119471,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":470310,"apply.run_cpu_time_us":88559,"apply.run_wall_time_us":1803710,"cfile_cache_hit":62,"cfile_cache_hit_bytes":64914,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6221,"cfile_init":1,"lbm_read_time_us":18871,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":7,"num_ops":16,"prepare.queue_time_us":154704,"prepare.run_cpu_time_us":163067,"prepare.run_wall_time_us":1105583,"raft.queue_time_us":436723,"raft.run_cpu_time_us":31383,"raft.run_wall_time_us":446601,"replication_time_us":807602,"spinlock_wait_cycles":78331776,"thread_start_us":1139706,"threads_started":5,"wal-append.queue_time_us":702658,"wal-append.run_cpu_time_us":15435,"wal-append.run_wall_time_us":2646817}]]}
I20260709 13:11:06.540457 22710 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:11:06.541640 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.543449 22710 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:06.542501 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.500355 26949 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:06.531316 22705 raft_consensus.cc:2368] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:06.532205 22703 raft_consensus.cc:2410] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:11:06.534302 22709 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:06.544477 22708 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:06.512034 26940 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:06.565692 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.567229 22709 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:06.570884 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.572075 22710 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:06.576951 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.577760 22708 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:06.517880 26436 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:06.589375 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.556818 26951 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:06.595409 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.596310 22708 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:11:06.596803 26434 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:06.599602 26951 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.600943 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.625713 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2639) took 1373 ms. Trace:
I20260709 13:11:06.588611 26940 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.660431 22709 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:06.626366 22782 rpcz_store.cc:276] 0709 13:11:05.252282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.252457 (+   175us) service_pool.cc:224] Handling call
0709 13:11:05.264616 (+ 12159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:05.264852 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:05.276176 (+ 11324us) log.cc:844] Serialized 14 byte log entry
0709 13:11:06.624444 (+1348268us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b1800453ee0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287
0709 13:11:06.624936 (+   492us) log.cc:844] Serialized 14 byte log entry
0709 13:11:06.625174 (+   238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:06.625235 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:06.625304 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:06.625441 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:06.625625 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42371,"prepare.queue_time_us":37000,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":44,"raft.queue_time_us":139946,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4398,"spinlock_wait_cycles":40506624,"thread_start_us":338601,"threads_started":2,"wal-append.queue_time_us":361893}
I20260709 13:11:06.633225 26436 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:06.633395 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:06.634302 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.625527 26906 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.668579 22706 raft_consensus.cc:2410] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:11:06.637794 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.669777 22703 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:06.639268 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.640499 22710 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 7. Current term is 8. Ops: []
W20260709 13:11:06.641947 26679 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.652s	user 0.012s	sys 0.001s
I20260709 13:11:06.642488 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.624146 26917 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:06.644304 26630 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.671651 22708 raft_consensus.cc:2368] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:06.666934 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.672806 22701 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:11:06.643349 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.676007 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:06.659612 26632 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.800s	user 0.016s	sys 0.000s
I20260709 13:11:06.676766 22703 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:06.675050 22709 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:06.687222 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.687991 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.688153 22706 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:06.687243 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.689564 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.690780 26968 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.690945 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.691447 26968 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.692221 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.693503 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.680912 26932 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:06.704600 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.705454 22782 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:06.703862 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.677201 26679 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.660444 26873 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.725927 22775 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:06.726632 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:06.731786 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.732628 22782 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:06.733422 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.734315 22775 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:06.735174 22710 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:06.735459 22776 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:06.727121 22703 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:06.664479 26949 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.744314 26679 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:06.745503 22627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 583) took 1577 ms (client timeout 1990 ms). Trace:
W20260709 13:11:06.745971 22627 rpcz_store.cc:269] 0709 13:11:05.167934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.168079 (+   145us) service_pool.cc:224] Handling call
0709 13:11:06.745285 (+1577206us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b540006dbd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:06.745431 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685347072}
I20260709 13:11:06.734838 26968 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:06.747751 26907 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.536s	user 0.005s	sys 0.000s
I20260709 13:11:06.749191 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2624) took 1734 ms. Trace:
I20260709 13:11:06.749564 22781 rpcz_store.cc:276] 0709 13:11:05.015097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.015257 (+   160us) service_pool.cc:224] Handling call
0709 13:11:05.022481 (+  7224us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:05.022963 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:05.022998 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:05.049850 (+ 26852us) log.cc:844] Serialized 26 byte log entry
0709 13:11:05.211368 (+161518us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:05.211439 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:05.211556 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:06.746873 (+1535317us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:06.748858 (+  1985us) raft_consensus.cc:1685] finished
0709 13:11:06.748886 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:06.749127 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":16018,"spinlock_wait_cycles":120752000,"thread_start_us":129906,"threads_started":1,"wal-append.queue_time_us":142522}
W20260709 13:11:06.751134 26898 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.655s	user 0.000s	sys 0.007s
W20260709 13:11:06.754912 26936 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.632s	user 0.005s	sys 0.009s
I20260709 13:11:06.755721 26872 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.755927 23416 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:11:06.757768 26630 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.758483 26630 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.761081 26630 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.760694 26913 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.729641 26819 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:06.762954 26947 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:06.755481 26936 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.765504 26567 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:06.764955 26940 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.766199 26567 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.767211 26936 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:06.765144 26913 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.784226 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.677439 26906 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.785843 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.787642 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.788415 22783 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.790367 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:06.792500 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.793651 22776 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:06.769217 26567 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:06.794711 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.795527 22781 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:06.796283 26819 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.796893 26819 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:06.797691 26887 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.005s
I20260709 13:11:06.800092 26819 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:06.800326 26971 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:11:06.803934 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 80500.9us
I20260709 13:11:06.805939 26922 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:06.783324 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.809217 26819 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:06.809350 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.809700 26819 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:06.810077 26819 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:06.810956 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.811867 22775 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.733937 22782 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:06.825474 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 263741us
I20260709 13:11:06.839552 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.836980 22781 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:06.811620 26919 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:06.849723 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.851554 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.850025 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.859750 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.863444 22783 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:06.857106 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:06.872944 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.871572 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.874436 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.873765 22776 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:06.878355 22782 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:06.879179 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.842695 26630 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:06.883579 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.885125 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:06.884341 22782 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:06.887408 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:06.887315 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:06.893822 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:06.870023 26567 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:06.881614 26932 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:06.906003 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:06.845542 26777 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 13:11:06.912930 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 105172us
W20260709 13:11:06.913975 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.916836 22782 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:11:06.918747 22783 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:06.918633 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:06.919498 22630 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:06.913913 26291 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:11:06.887051 26976 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 13:11:06.895328 26630 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:06.902247 26913 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:06.931612 26630 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.911206 26567 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.914853 26932 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.940918 26932 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:06.941597 26934 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:06.942239 26934 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.943310 26934 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:06.946798 26961 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:06.947496 26961 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.948545 26961 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.920202 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:06.950814 26969 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:06.951429 26969 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:06.951853 26969 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:06.954676 26873 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:06.955417 26873 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:06.956506 26873 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:06.938426 26567 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:06.959157 26978 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:06.960747 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46188.6us
I20260709 13:11:06.971170 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.976670 22703 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:11:06.985597 26788 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:06.986873 22708 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:06.988148 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:06.988914 22782 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:11:06.992589 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:06.997596 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.998477 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:07.008327 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:06.993523 26567 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.011238 26567 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.031160 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.017383 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.043262 22704 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:07.050042 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.069197 26220 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.070025 26220 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.013798 26819 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.039996 26878 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:07.058027 22775 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:07.076151 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.014487 26567 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.084438 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:07.086171 22778 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:07.107331 22702 raft_consensus.cc:2368] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9.
W20260709 13:11:07.108705 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 220264us
I20260709 13:11:07.107263 22778 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
W20260709 13:11:07.109895 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62631.5us
I20260709 13:11:07.109411 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.077180 26962 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:07.079346 26878 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:07.116596 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.122422 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:07.149030 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:07.139817 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.156414 22778 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:07.159786 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48777.3us
W20260709 13:11:07.195546 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:07.203819 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91102.3us
I20260709 13:11:07.210043 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:07.234486 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.199200 26962 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:07.243638 26989 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:11:07.244295 26882 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:07.205072 26981 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:11:07.159269 26969 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:07.251152 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44186us
I20260709 13:11:07.203672 26940 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:07.273623 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.380672 22632 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:07.445452 26878 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.451786 22781 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
W20260709 13:11:07.464342 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 104951us
I20260709 13:11:07.466688 22782 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:07.487216 22775 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:07.367642 26922 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.561s	user 0.025s	sys 0.003s
W20260709 13:11:07.545928 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.312873 26991 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:11:07.489979 26934 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.023s	sys 0.034s
I20260709 13:11:07.358774 26913 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:07.570817 22782 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:07.470674 26878 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:07.570901 26934 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.023s	sys 0.034s
I20260709 13:11:07.567286 26978 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:07.575743 26980 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
W20260709 13:11:07.612447 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.574048 26991 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:07.632479 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 522729us
I20260709 13:11:07.637434 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:07.645398 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 178059us
I20260709 13:11:07.646557 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:07.648463 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:07.644667 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 347931us
I20260709 13:11:07.576977 26913 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.651924 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:07.633787 26919 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:07.655905 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.636270 26978 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.665107 26788 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:07.665715 26788 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.666208 26788 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:07.666028 26991 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.668334 26886 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:07.668939 26886 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:07.669826 26886 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.665715 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 619) took 1154 ms. Trace:
I20260709 13:11:07.659970 26934 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:07.657719 26969 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.660432 26999 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:07.652683 26976 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:11:07.661550 26962 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:07.675956 22780 rpcz_store.cc:276] 0709 13:11:06.511228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.522221 (+ 10993us) service_pool.cc:224] Handling call
0709 13:11:06.552087 (+ 29866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:06.557900 (+  5813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:06.557955 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:06.578856 (+ 20901us) log.cc:844] Serialized 26 byte log entry
0709 13:11:06.843603 (+264747us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:06.843648 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:06.843721 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:07.665427 (+821706us) raft_consensus.cc:1685] finished
0709 13:11:07.665459 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:07.665638 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5256,"raft.queue_time_us":17959,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":2958,"thread_start_us":221779,"threads_started":1,"wal-append.queue_time_us":474012}
I20260709 13:11:07.661872 26919 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.691152 26934 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:07.704862 26800 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:07.708205 26969 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.732311 26978 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:07.745224 26982 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.745872 26913 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:07.746167 26982 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.749097 26982 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.751730 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.751850 26934 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.755131 26969 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.755702 26938 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:07.756250 26938 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:07.757246 26938 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.757781 26969 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.760036 26919 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.769814 26220 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.778501 26933 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:07.784593 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.785331 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.785574 22631 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:07.786161 26966 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:07.786692 26966 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:07.790531 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.791468 26940 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:07.794385 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.797735 26938 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:07.798398 26944 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:07.798938 26944 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:07.799968 26944 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.801222 26944 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:07.802505 26963 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.803319 26963 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.804543 23421 raft_consensus.cc:2410] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:07.805195 26780 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.805791 26963 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.805732 26780 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.807255 22709 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:07.810544 26780 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.810820 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.812440 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.813212 22709 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:07.787794 26966 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.815603 26966 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:07.818192 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:07.820544 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:07.821105 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:07.822232 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.822707 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:07.823040 22782 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:07.823673 26756 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:07.825488 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.826179 22780 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:07.826381 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.827445 26919 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:07.830732 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:07.835434 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:07.839509 22750 connection.cc:256] RPC call timeout handler was delayed by -1.5899639709999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:11:07.836459 23415 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:07.824810 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:07.856310 26220 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.857014 26220 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:07.864837 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:07.865043 22750 connection.cc:256] RPC call timeout handler was delayed by -1.3337992729999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260709 13:11:07.866655 22750 connection.cc:256] RPC call timeout handler was delayed by -1.3158652689999144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2217s for a response
W20260709 13:11:07.867125 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1917221059999292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
I20260709 13:11:07.876742 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.865674 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:07.887488 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:07.888586 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:07.889498 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:07.890228 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:07.891571 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:07.894330 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:07.896220 22709 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:11:07.889132 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 251819us
I20260709 13:11:07.901510 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.902159 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.902840 22633 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:07.834673 26919 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:07.906476 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:07.907990 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:07.908692 23420 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:07.909915 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:07.910574 22629 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:07.912391 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:07.914574 23416 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:07.916145 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.916219 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.920331 26220 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:07.922031 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.865100 26436 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.925417 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.911617 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.926748 26919 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:07.927049 23417 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:07.898308 26756 raft_consensus.cc:2764] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:11:07.892899 26959 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:07.957991 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46497us
I20260709 13:11:07.958668 26220 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:07.963768 22628 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:07.901109 26917 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:07.913941 26933 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:07.917287 26978 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:07.967476 22634 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:11:07.977770 22628 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:07.987231 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.964632 26816 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:07.966159 26959 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.970000 26922 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.992288 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:07.998421 26756 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:07.999303 26919 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:07.970279 26933 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:08.007886 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.952s (SENT)
I20260709 13:11:08.009696 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:07.972798 26978 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:07.975574 26890 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:08.008559 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:07.969748 26917 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:08.020395 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2660) took 1041 ms. Trace:
I20260709 13:11:07.979866 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:08.026177 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46413.1us
I20260709 13:11:08.024796 22781 rpcz_store.cc:276] 0709 13:11:06.978825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.469967 (+491142us) service_pool.cc:224] Handling call
0709 13:11:07.554660 (+ 84693us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:07.574773 (+ 20113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:07.574806 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:07.576014 (+  1208us) log.cc:844] Serialized 52 byte log entry
0709 13:11:07.714448 (+138434us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:07.985731 (+271283us) log.cc:844] Serialized 14 byte log entry
0709 13:11:08.019526 (+ 33795us) log.cc:844] Serialized 14 byte log entry
0709 13:11:08.020080 (+   554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.020156 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.020198 (+    42us) raft_consensus.cc:1685] finished
0709 13:11:08.020210 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.020331 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31981,"prepare.queue_time_us":49102,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":150,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5863,"thread_start_us":323060,"threads_started":2,"wal-append.queue_time_us":256689}
W20260709 13:11:08.015285 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.501s (SENT)
I20260709 13:11:08.027398 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.029208 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:07.966632 26436 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:08.029146 22750 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.596s (SENT)
I20260709 13:11:08.032109 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.004067 26959 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.030167 22709 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:08.024034 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.015919 26933 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:08.044845 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.022114 26978 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:08.016149 26877 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:08.141613 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2690) took 1089 ms. Trace:
I20260709 13:11:08.189880 26220 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:08.189775 22706 rpcz_store.cc:276] 0709 13:11:07.052534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:07.070121 (+ 17587us) service_pool.cc:224] Handling call
0709 13:11:07.086988 (+ 16867us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:07.107406 (+ 20418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:07.107444 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:07.198373 (+ 90929us) log.cc:844] Serialized 26 byte log entry
0709 13:11:07.834507 (+636134us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b180018df00. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:11:07.977431 (+142924us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:07.977492 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:07.977594 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.141248 (+163654us) raft_consensus.cc:1685] finished
0709 13:11:08.141271 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.141531 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8393,"raft.queue_time_us":2539,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":49328,"spinlock_wait_cycles":2791808,"thread_start_us":142659,"threads_started":1,"wal-append.queue_time_us":838326}
I20260709 13:11:08.122023 22629 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:08.021412 26890 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:08.018383 26756 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.004626 26788 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.022979 26917 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:08.152463 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.130877 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2313) took 1846 ms. Trace:
I20260709 13:11:08.036410 26886 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:11:08.120060 27016 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:08.128450 26933 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.113862 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.131955 26939 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:08.129791 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2657) took 1317 ms. Trace:
I20260709 13:11:08.045396 26816 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:08.010530 26922 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.035216 26969 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:08.039263 26934 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:08.185014 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.184671 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:08.187433 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.188082 22776 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:11:08.237653 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53098us
I20260709 13:11:08.091512 26910 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:11:08.188894 22775 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:08.017515 26954 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.192732 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.242071 22706 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:08.246218 22779 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:08.216251 26939 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:08.157159 27015 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:08.202636 22707 rpcz_store.cc:276] 0709 13:11:06.284798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.285385 (+   587us) service_pool.cc:224] Handling call
0709 13:11:06.425079 (+139694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:06.432265 (+  7186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.130556 (+1698291us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54000a87d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:08.130791 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274124160}
I20260709 13:11:08.206550 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.205735 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.210834 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.251585 22782 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:08.211378 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:08.252362 22629 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.212240 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:08.156602 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127513us
I20260709 13:11:08.253600 22633 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
W20260709 13:11:08.253754 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.672s (SENT)
I20260709 13:11:08.243561 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:08.214084 23421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 758) took 2234 ms (client timeout 1929 ms). Trace:
I20260709 13:11:08.198218 26890 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.219661 26788 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.220136 22630 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:08.228514 26816 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.216225 22783 rpcz_store.cc:276] 0709 13:11:06.812630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.892702 (+ 80072us) service_pool.cc:224] Handling call
0709 13:11:06.917959 (+ 25257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:06.921040 (+  3081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.129423 (+1208383us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400341cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:08.129684 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53378432}
I20260709 13:11:08.205278 26630 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:08.207741 27003 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.243758 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:08.268909 22703 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:11:08.234431 26934 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.273682 26934 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:08.274253 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c015ab1d14c443258576403d9d39c485"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:08.274899 22629 raft_consensus.cc:606] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.275262 22629 raft_consensus.cc:614] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:08.275521 22776 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:08.276743 26630 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.199501 26436 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:08.255676 23421 rpcz_store.cc:269] 0709 13:11:05.965441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:05.965562 (+   121us) service_pool.cc:224] Handling call
0709 13:11:08.200079 (+2234517us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54005f7654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:08.200265 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4172495232}
W20260709 13:11:08.265853 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60211us
I20260709 13:11:08.266412 22709 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:08.279386 22709 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:08.279422 26630 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.279619 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:08.280375 22709 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.281579 22709 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.282047 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.217784 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.271751 22707 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:08.283238 26291 raft_consensus.cc:2764] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:11:08.249135 26954 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.284274 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:08.285280 26291 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:08.285691 26291 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.286036 26291 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:08.286445 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:08.286854 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.242848 26949 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:08.278772 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:08.288224 22634 raft_consensus.cc:606] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.242686 26583 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:08.321368 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.323603 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.293344 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:08.327864 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:08.334199 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.337471 22703 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:08.290077 26877 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:08.340301 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.340601 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.343396 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:08.344312 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.362987 23411 raft_consensus.cc:2393] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:08.344949 26890 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.345996 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.347548 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.351373 26481 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.370263 26917 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.377970 26817 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.383401 26817 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.383334 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.387826 22626 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:08.389336 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.386461 26817 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.397784 26481 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.398339 23421 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:08.398520 26481 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.400601 23417 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
W20260709 13:11:08.353101 22702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2675) took 1390 ms (client timeout 1525 ms). Trace:
I20260709 13:11:08.280395 27003 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.269131 26816 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:08.359020 26906 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.333475 26583 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.385921 26978 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.353149 22634 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:08.364456 26933 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.320731 26788 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:11:08.365428 22701 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:08.366108 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.328960 26973 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:11:08.366755 26922 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.366820 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.367319 26879 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:08.367396 26878 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:08.374161 26969 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:08.362212 26952 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:08.391191 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.404330 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:08.404390 22702 rpcz_store.cc:269] 0709 13:11:06.962280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.962427 (+   147us) service_pool.cc:224] Handling call
0709 13:11:08.352802 (+1390375us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54001ea854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:08.352986 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310941440}
I20260709 13:11:08.412225 22709 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:11:08.493516 22709 pending_rounds.cc:85] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:11:08.493880 22709 pending_rounds.cc:107] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:11:08.494158 22709 raft_consensus.cc:2889] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:08.416198 23414 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:08.361666 26630 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:08.395112 26917 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.421149 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.366529 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.435119 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.434736 22706 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:08.445971 22707 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:08.410984 26951 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.466310 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.472704 27021 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.420884 26817 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.473893 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.507157 22783 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:08.474534 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.508397 22708 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:08.475369 26481 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.432327 26583 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:08.477979 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56595.4us
W20260709 13:11:08.478268 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.467507 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:08.499361 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 95167.8us
I20260709 13:11:08.515983 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.509325 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:08.507186 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:08.516820 23417 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:08.516950 22708 raft_consensus.cc:2368] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:08.422071 26828 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:11:08.511433 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.518266 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:08.520641 22708 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:08.521090 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.521847 22709 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:08.522234 22634 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:08.375182 26939 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.524931 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:08.525086 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.525701 22629 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:08.422573 26959 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.528157 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.532068 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.528777 22709 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:08.533634 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.534786 22709 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:08.536285 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:08.540632 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73235us
I20260709 13:11:08.523207 26481 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:08.543962 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:08.544430 26481 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:08.544653 26436 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:08.514426 26630 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:08.546963 26436 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:08.546897 26630 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.547488 26436 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:08.548647 26436 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.550901 27015 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:08.551051 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.494460 26922 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:11:08.515389 26952 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:08.545826 26481 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:08.519153 26972 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:08.551452 27015 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:08.556110 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.568657 22629 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:11:08.555096 26993 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.099s	user 0.005s	sys 0.000s
I20260709 13:11:08.558054 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.573797 22783 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:08.558962 26879 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.550068 26436 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:08.562975 27016 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:08.514460 26817 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.553540 26638 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.566156 26481 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.514489 27021 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.585734 27015 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.586750 27021 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.511979 26969 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.587787 27022 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:08.588204 27022 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:08.589447 26969 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.589128 27022 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.536788 26923 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:08.590878 27021 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.591435 27021 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.554591 26291 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.593641 27021 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.594076 26291 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.594906 26978 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.595208 27022 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:08.596781 26959 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.597700 26915 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:08.599563 27016 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:08.600145 27016 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:08.603169 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2676) took 1631 ms. Trace:
I20260709 13:11:08.603668 22705 rpcz_store.cc:276] 0709 13:11:06.971507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.984332 (+ 12825us) service_pool.cc:224] Handling call
0709 13:11:06.984766 (+   434us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:06.985206 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:06.985241 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:06.985903 (+   662us) log.cc:844] Serialized 52 byte log entry
0709 13:11:07.252217 (+266314us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:07.252290 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:07.252406 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:08.370567 (+1118161us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:08.602775 (+232208us) raft_consensus.cc:1685] finished
0709 13:11:08.602800 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.603091 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1857,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5275,"spinlock_wait_cycles":38772096,"thread_start_us":265817,"threads_started":1,"wal-append.queue_time_us":468535}
I20260709 13:11:08.605839 26819 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.563550 22708 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:08.607409 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.567968 26952 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.608246 22708 raft_consensus.cc:2368] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:08.609167 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.609448 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.584645 26481 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:08.611471 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.612218 22782 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:08.614238 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:08.614864 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.616773 26638 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.587280 27015 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.514578 26951 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:08.619999 26996 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.048s	user 0.001s	sys 0.000s
I20260709 13:11:08.621543 26951 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.571534 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.626194 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:08.581404 26879 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.636660 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.610355 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:08.631703 22784 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 618) took 2133 ms (client timeout 2162 ms). Trace:
I20260709 13:11:08.631529 26934 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.637692 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.608042 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.644459 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:08.560262 27003 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.629807 26756 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.605784 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:08.610144 22781 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:08.643640 22775 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:08.642154 22784 rpcz_store.cc:269] 0709 13:11:06.498818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.499118 (+   300us) service_pool.cc:224] Handling call
0709 13:11:08.631431 (+2132313us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540015a4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:08.631597 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4298405120}
I20260709 13:11:08.675048 22708 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:08.682163 22634 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:11:08.683439 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 46887.7us
I20260709 13:11:08.687505 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.688506 22783 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:08.692086 23417 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:08.610428 26952 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.702512 22705 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 6: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:08.687410 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.704310 22632 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:11:08.705358 22634 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:08.710031 22634 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:08.711300 22629 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:08.712422 26220 raft_consensus.cc:2764] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:08.712883 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.713084 26882 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:08.714095 26882 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:08.714641 26882 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:08.715649 26882 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.717814 26877 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:08.718793 22706 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:08.719830 26583 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.720964 23417 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:08.722484 26583 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.724584 27024 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.725261 27024 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.727079 22629 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:08.729259 22634 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:08.731338 26788 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.732311 22629 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:08.732040 26788 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.737268 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.738281 23417 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:08.740974 26879 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.741163 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2468) took 1889 ms. Trace:
I20260709 13:11:08.741325 22781 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:08.743985 26925 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:08.744483 26925 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:08.743610 22635 rpcz_store.cc:276] 0709 13:11:06.851769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.852751 (+   982us) service_pool.cc:224] Handling call
0709 13:11:06.879137 (+ 26386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:06.964604 (+ 85467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:06.964657 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:06.964723 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:06.964757 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:08.740920 (+1776163us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b54003a7354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:08.740974 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:08.741098 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195820672}
I20260709 13:11:08.734293 26816 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.745635 26925 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.746994 26982 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:08.747354 22701 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:08.728087 27024 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.749141 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.687742 26973 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:08.734330 26788 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.755618 26481 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.758297 22784 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:08.747064 26816 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.742602 26817 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.779111 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.791528 23415 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:08.745996 26638 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.786144 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.801429 22635 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:08.786281 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.787073 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:08.784435 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.742923 26954 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.758963 26934 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.790045 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:08.808216 23417 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:08.808704 22775 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:11:08.811512 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.797113 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.783515 26888 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:08.798080 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 282649us
I20260709 13:11:08.804237 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.783972 27024 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.812827 22629 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:08.816947 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:08.817595 22776 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:08.818704 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.819437 22635 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:08.818638 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.820647 23411 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:08.824950 26817 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:08.801290 26968 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:08.827347 22626 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:11:08.827090 26968 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.829425 22626 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:08.834264 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:08.835176 22779 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:08.835680 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.840149 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.807708 26739 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:08.850164 26481 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:08.831430 26978 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:08.836660 27024 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.815467 26888 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.851635 26888 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.852946 27024 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:08.842324 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.854522 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.841511 26969 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:08.830523 26968 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:08.855465 26969 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:08.856093 27018 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:08.856484 26969 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:08.843395 26672 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:08.858512 26672 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:08.858186 27016 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.858999 26672 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:08.859701 27016 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:08.860884 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.856515 27018 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:08.863487 27018 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:08.864140 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.864568 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.864850 27018 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:08.844444 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:08.868337 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:08.807711 26638 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.864193 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.880829 23417 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:08.884449 26982 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:08.887431 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.888770 26940 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:08.889489 26940 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.889894 26940 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:08.879923 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:08.891685 26915 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:08.892127 26915 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.892552 26915 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:08.892416 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:08.899904 22777 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:08.900827 26800 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.901366 26800 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:08.935220 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.934156 22629 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:08.936470 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.914913 22776 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:08.937558 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:08.931990 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:08.906109 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:08.950053 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:08.950912 22629 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:08.950217 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:08.956067 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50048.5us
W20260709 13:11:08.904539 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 409795us
I20260709 13:11:08.965323 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.905256 26890 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:08.957326 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:08.907624 27022 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:08.994380 22779 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:08.996162 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:08.964933 26739 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:09.004624 22629 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:09.015089 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.014391 22776 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:09.015038 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:09.016852 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:08.958696 27031 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:08.983724 26890 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.907472 26638 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.005712 26968 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:09.029873 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.039261 22778 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.037282 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.040424 22776 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:09.031258 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:08.997843 27022 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:09.007958 27028 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.979133 26934 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:11:09.036055 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.043486 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:09.048197 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.049023 22629 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:09.050031 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:09.051424 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:09.054174 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.052932 26969 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:09.061859 22601 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.005s	sys 0.016s
W20260709 13:11:09.062353 22601 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.005s	sys 0.016s
W20260709 13:11:09.065372 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 533187us
I20260709 13:11:09.066723 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:09.070477 22778 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:09.036886 26890 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:09.072296 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.071790 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.089321 22778 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:09.091418 22750 leader_election.cc:400] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:09.093204 26906 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:09.093721 26906 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:09.095148 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.095896 22629 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:09.096865 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:09.101500 27038 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.050s	user 0.003s	sys 0.000s
I20260709 13:11:09.054674 27028 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.108887 26800 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.109668 27028 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.112303 26638 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:09.115720 26481 raft_consensus.cc:2764] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:11:09.089617 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.098224 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.045581 27022 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.049995 26739 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:09.124320 22703 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:09.128093 22778 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:09.064940 26978 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:09.053586 27031 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:09.131168 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:09.134829 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.134979 22703 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:09.135440 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.131556 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.136798 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.065701 26968 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:09.127604 26882 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:09.170405 26968 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.170552 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.176251 22635 raft_consensus.cc:2376] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12. Re-sending same reply.
I20260709 13:11:09.146994 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.148010 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.170989 22747 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.178143 22704 raft_consensus.cc:2410] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:11:09.149014 26816 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:09.172063 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:09.180078 23417 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:09.172907 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:09.150774 27027 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:09.151897 26817 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:09.152756 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.192407 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:09.153811 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.193301 22629 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:09.110208 26915 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.155162 22778 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:09.119282 26969 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:11:09.158663 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.159376 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.160249 26940 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.162158 22781 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:09.162587 26938 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:11:09.167938 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.172293 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.149524 27031 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.188095 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:09.201610 22783 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:09.202353 22779 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:09.193367 27016 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:09.171577 26882 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:09.199643 22776 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:09.209259 22632 raft_consensus.cc:2368] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:11:09.213698 22628 raft_consensus.cc:2368] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:11:09.213732 23417 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:09.215991 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.224301 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.223675 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.227223 22629 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
W20260709 13:11:09.227707 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 159413us
I20260709 13:11:09.206492 22703 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:09.253623 22702 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:09.256261 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.257766 22629 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:11:09.195722 27003 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:11:09.264856 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.540s (SENT)
I20260709 13:11:09.268862 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.201504 26969 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:09.234828 26567 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:11:09.284569 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.285583 22704 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:09.287173 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.288972 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.290885 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.292603 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.276096 26969 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.303315 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.308655 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.222378 26481 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:09.207852 27030 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:09.261359 27016 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.314738 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.318051 26940 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:09.320163 27026 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:09.321218 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.276497 27012 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:09.325312 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.187449 26906 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.326232 22707 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:11:09.327236 22777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 615) took 2906 ms (client timeout 1851 ms). Trace:
I20260709 13:11:09.327431 26915 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:09.332950 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.323261 26882 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.334818 22702 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:09.342335 22777 rpcz_store.cc:269] 0709 13:11:06.420690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:06.429720 (+  9030us) service_pool.cc:224] Handling call
0709 13:11:09.327017 (+2897297us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54002d5854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:09.327152 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5558268672}
I20260709 13:11:09.335783 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.213559 26934 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 13:11:09.340399 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.355261 22634 raft_consensus.cc:2376] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12. Re-sending same reply.
I20260709 13:11:09.356518 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.208869 26828 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:11:09.209916 26630 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:11:09.351703 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.334300 27016 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:09.339031 27012 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:09.341663 26788 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:09.361011 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.363619 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.359279 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.329888 27030 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:09.367648 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.373162 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:09.355125 27018 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:09.329459 26888 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.358310 26934 consensus_queue.cc:1243] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:11:09.396382 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.406739 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.386180 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.408071 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.408913 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.341100 26481 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.411269 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.371517 27012 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:09.412250 22632 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
W20260709 13:11:09.412456 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 200455us
I20260709 13:11:09.380285 27030 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:09.370116 27027 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:09.414512 22704 raft_consensus.cc:2410] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:09.426224 22627 raft_consensus.cc:2368] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12.
I20260709 13:11:09.386933 26915 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.405120 26906 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:11:09.430361 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 202165us
I20260709 13:11:09.433919 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.439667 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.440794 22704 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:09.442070 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.411626 26817 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:09.443311 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.451375 22702 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:11:09.444397 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 168912us
I20260709 13:11:09.453238 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.453933 22777 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.444839 22704 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:09.429319 26981 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:09.431219 27018 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:09.472903 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.476926 22777 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:09.473886 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:09.486635 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.487571 22708 raft_consensus.cc:2368] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9.
I20260709 13:11:09.431761 26940 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:11:09.496845 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.502238 22777 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:11:09.502923 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:09.499604 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61108.7us
I20260709 13:11:09.504086 22632 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:09.436404 26915 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.467054 26981 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:09.436856 27022 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:09.515888 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66667.6us
I20260709 13:11:09.520057 22704 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:09.502442 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:11:09.521576 22782 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.521978 22782 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:09.526126 22701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 585) took 1179 ms (client timeout 1540 ms). Trace:
I20260709 13:11:09.527010 22630 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:09.526594 22701 rpcz_store.cc:269] 0709 13:11:08.346918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.465932 (+119014us) service_pool.cc:224] Handling call
0709 13:11:09.525927 (+1059995us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54003d13d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:09.526050 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558925696}
W20260709 13:11:09.533299 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 124516us
I20260709 13:11:09.534447 26888 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.535156 26888 raft_consensus.cc:993] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:09.537101 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:09.538368 26976 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:09.540122 26976 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:09.540405 26888 raft_consensus.cc:1081] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:09.548915 26817 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:09.550267 27027 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:09.559489 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:09.560626 22675 leader_election.cc:400] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:11:09.562569 27027 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:09.559186 26630 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:11:09.577059 26817 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:09.577934 26817 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:09.579344 26817 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:09.582878 27047 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:11:09.579660 27030 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.585436 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:09.585947 26890 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.597909 22782 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:09.560957 22675 leader_election.cc:403] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:11:09.597386 27030 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.611307 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:09.612298 23415 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:09.614192 26672 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.755s	user 0.021s	sys 0.000s
I20260709 13:11:09.638978 26672 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.640532 23416 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:11:09.606457 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45982.5us
I20260709 13:11:09.643270 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2684) took 1167 ms. Trace:
I20260709 13:11:09.643656 22781 rpcz_store.cc:276] 0709 13:11:08.476160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.744220 (+268060us) service_pool.cc:224] Handling call
0709 13:11:08.744593 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:09.202186 (+457593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.643026 (+440840us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540015a9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:09.643201 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959516928}
I20260709 13:11:09.647383 26672 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:09.616000 27059 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.651098 22675 leader_election.cc:400] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:11:09.524360 27052 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
W20260709 13:11:09.656479 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:09.657788 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.659075 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.659380 26939 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:09.660116 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.661335 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.662209 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.662853 27018 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:09.663029 26978 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:09.663627 26978 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:09.665091 27048 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:09.665354 26978 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.617383 26816 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:09.679157 26816 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.681855 26816 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.687107 26931 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:09.688308 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:09.629609 27016 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.694633 27016 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.619524 26925 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.603278 26981 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:09.606079 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69082.7us
I20260709 13:11:09.702504 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.626720 26481 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:09.703329 22779 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:09.628047 22706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 589) took 1280 ms (client timeout 1674 ms). Trace:
W20260709 13:11:09.704367 22706 rpcz_store.cc:269] 0709 13:11:08.347981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.513700 (+165719us) service_pool.cc:224] Handling call
0709 13:11:09.627876 (+1114176us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54002c4cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:09.627983 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427538688}
I20260709 13:11:09.705693 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.707060 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.708463 26981 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.709983 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.628760 26890 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:09.637219 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2682) took 1161 ms. Trace:
I20260709 13:11:09.711980 22707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:09.666090 27048 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:09.712666 22707 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:09.713209 27048 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:09.713205 22707 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.714258 27048 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.716418 22702 pending_rounds.cc:85] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:09.716849 22702 pending_rounds.cc:107] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:11:09.712442 22784 rpcz_store.cc:276] 0709 13:11:08.475576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.733190 (+257614us) service_pool.cc:224] Handling call
0709 13:11:08.733469 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:08.784009 (+ 50540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.636905 (+852896us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b540033cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:09.637141 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55527680}
I20260709 13:11:09.717093 22702 raft_consensus.cc:2889] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:09.619216 27022 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.616824 26915 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:09.614395 27003 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:09.755278 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.672257 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 592) took 1318 ms. Trace:
I20260709 13:11:09.673184 26969 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.673688 22704 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:09.695385 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.695560 22782 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:09.696390 27016 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:11:09.759642 22630 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:09.764341 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.765147 22775 raft_consensus.cc:2368] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:09.666579 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:09.721450 22779 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:09.736364 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.738319 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.756300 22705 rpcz_store.cc:276] 0709 13:11:08.353473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.605383 (+251910us) service_pool.cc:224] Handling call
0709 13:11:09.671985 (+1066602us) spinlock_profiling.cc:243] Waited 34.5 ms on lock 0x7b54002be8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:09.672172 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79240576}
I20260709 13:11:09.760097 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:09.761034 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.755149 22707 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:09.767815 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 101326us
I20260709 13:11:09.769160 22629 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:09.770381 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.783696 22708 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:11:09.784394 22629 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:11:09.785101 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 140956us
I20260709 13:11:09.787683 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:09.788375 23417 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:09.723222 27059 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:11:09.789621 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:09.790517 22707 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:09.800740 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:09.802920 23421 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:09.807140 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.809859 23416 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:09.760798 26583 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
W20260709 13:11:09.812038 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52482.9us
I20260709 13:11:09.777344 26816 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:09.831277 22628 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:11:09.824615 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.824960 22629 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:11:09.825973 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:09.829159 22627 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:09.833349 23420 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:09.830061 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.833900 22782 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:09.834844 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.835526 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.836508 22780 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:09.837433 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.838284 23384 leader_election.cc:400] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12.
I20260709 13:11:09.840608 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.841784 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.843351 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.844170 22780 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:09.844951 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:09.845296 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:09.846057 22780 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:11:09.848057 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.850595 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:09.850931 26968 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:09.862294 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.863145 22706 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:09.864975 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.865686 22777 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:09.758983 27012 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:09.815696 26630 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:09.879513 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:09.809301 27003 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:09.871018 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:09.872821 23417 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:09.815414 26982 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:09.809239 27027 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:09.876327 22776 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:09.876444 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:09.905941 23420 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:09.889740 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.878583 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.827066 26938 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:09.900144 22782 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:09.914849 23411 raft_consensus.cc:2368] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:09.900410 22706 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:11:09.877787 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49138.9us
W20260709 13:11:09.916373 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 106197us
I20260709 13:11:09.842681 26915 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:09.910494 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:09.916016 23415 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:11:09.908810 23417 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:09.917538 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:09.863494 27065 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:09.918964 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.916989 22747 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:09.929468 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.903517 26481 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:09.931869 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.863639 27048 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:11:09.928668 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:09.938426 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:09.944211 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:09.949854 22705 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8. Re-sending same reply.
I20260709 13:11:09.915566 26968 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:09.951356 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.953109 22780 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:09.865574 26739 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:11:10.011788 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:10.016088 22631 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:10.017509 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:10.018246 22632 raft_consensus.cc:2376] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7. Re-sending same reply.
I20260709 13:11:09.926826 26816 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:10.019590 22626 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:09.923132 26817 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:10.012267 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:10.022850 22629 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:09.927345 27065 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:10.025573 22706 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:11:09.924592 27016 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:09.988348 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108904us
I20260709 13:11:10.069161 22705 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 13:11:10.069087 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:10.043445 22626 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:09.931586 27073 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:09.941345 26897 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:09.942507 27059 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 13:11:09.911789 27071 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.124s
I20260709 13:11:09.927484 26915 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.069650 22705 pending_rounds.cc:85] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 10
I20260709 13:11:10.096732 22705 pending_rounds.cc:107] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 13:11:10.097116 22705 raft_consensus.cc:2889] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:10.070451 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:09.955129 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:10.071301 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:10.066234 27065 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:10.061223 26973 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:10.072587 26982 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:10.078866 22632 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:10.080281 27077 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:11:10.080957 27090 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:10.081454 27067 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
W20260709 13:11:10.082187 27068 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:11:10.073407 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 118184us
I20260709 13:11:10.070415 22626 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.101130 22635 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:10.068111 26817 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:10.211817 27018 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:10.063025 26966 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.143086 22630 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:11:10.185515 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.068684 27016 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:10.069006 26816 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:10.040297 27051 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.274s
I20260709 13:11:10.247599 22629 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:11:10.222252 27073 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:10.271653 26739 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.267647 26969 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:10.351207 22635 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:11:10.237649 27090 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:10.146663 27019 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.208s
I20260709 13:11:10.267169 26934 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.418287 22708 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:11:10.403096 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 524570us
I20260709 13:11:10.853344 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:10.395757 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:10.068012 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 150556us
W20260709 13:11:10.425231 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 356221us
I20260709 13:11:10.854919 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:10.819461 26672 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.815793 26925 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.114s	user 0.009s	sys 0.004s
I20260709 13:11:10.393302 27016 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:10.094161 26897 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.667115 26915 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:10.428952 26982 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:10.584169 27091 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:11:10.641422 27079 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:11:10.663494 27084 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:10.550092 22630 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:11:10.593775 27070 log.cc:927] Time spent T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.488s	user 0.000s	sys 0.001s
I20260709 13:11:10.543634 26816 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:10.356397 26981 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.647s	user 0.009s	sys 0.005s
I20260709 13:11:10.427356 27018 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.649818 26906 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:11:10.592959 26481 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.122s	sys 0.226s
W20260709 13:11:10.816782 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:10.605362 27071 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.642943 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.741s (SENT)
I20260709 13:11:10.943338 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:10.531067 26888 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:10.644380 27078 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
W20260709 13:11:10.643764 27082 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.570s	user 0.001s	sys 0.000s
I20260709 13:11:10.392172 26966 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:10.569290 27012 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:10.658991 27081 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.391s	user 0.001s	sys 0.001s
I20260709 13:11:10.531692 26976 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:10.710023 26817 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:10.833032 26938 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.014s	sys 0.009s
I20260709 13:11:10.890229 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:10.896435 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.013s	sys 0.005s
W20260709 13:11:11.009274 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.014s	sys 0.005s
I20260709 13:11:10.889843 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:10.856771 26939 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.197s	user 0.018s	sys 0.001s
W20260709 13:11:10.896487 22703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.017s	sys 0.001s
I20260709 13:11:10.852260 27090 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:10.861578 26630 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.034303 26630 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.034906 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:10.928725 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73955.7us
I20260709 13:11:10.960768 27066 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:11:11.036387 27018 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.036679 22600 leader_election.cc:400] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:11.036996 22600 leader_election.cc:403] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:11:11.029361 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2738) took 1296 ms. Trace:
I20260709 13:11:11.038681 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:11.039227 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:10.900115 27088 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:10.927099 22775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.973s	user 0.070s	sys 0.107s
I20260709 13:11:10.914074 27091 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.879524 26934 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:10.922183 27084 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:10.992390 27104 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:10.892073 26672 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:10.922138 27093 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:11.020766 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 130605us
I20260709 13:11:10.959918 26969 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:10.941609 27073 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:11.021696 27092 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:11:11.030957 22703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.018s	sys 0.001s
I20260709 13:11:11.023587 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:11.037007 27018 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.037793 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:10.837046 26739 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.051306 27084 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:11.040987 22775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.070s	sys 0.107s
I20260709 13:11:11.004437 26915 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:11:11.043293 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 153542us
W20260709 13:11:11.008862 26938 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.017s	sys 0.013s
W20260709 13:11:10.944442 26481 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.125s	sys 0.250s
I20260709 13:11:11.048578 22708 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:11.049769 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:11.038390 22701 rpcz_store.cc:276] 0709 13:11:09.733140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.734139 (+   999us) service_pool.cc:224] Handling call
0709 13:11:09.778165 (+ 44026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:09.783274 (+  5109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:09.783309 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:09.795339 (+ 12030us) log.cc:844] Serialized 26 byte log entry
0709 13:11:09.936070 (+140731us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:09.936114 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.936188 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.029069 (+1092881us) raft_consensus.cc:1685] finished
0709 13:11:11.029094 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.029288 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4884,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":11670,"thread_start_us":106532,"threads_started":1,"wal-append.queue_time_us":263785}
I20260709 13:11:10.939142 26906 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:10.960427 27071 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.054255 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:11.069509 27102 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:11.069813 26968 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 1.017s	user 0.012s	sys 0.004s
I20260709 13:11:11.050701 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.055104 22707 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:11.056979 23420 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:11.057538 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:10.965167 26888 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:11.046293 27065 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.836s	user 0.014s	sys 0.000s
I20260709 13:11:11.009783 26976 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.060721 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2736) took 1352 ms. Trace:
I20260709 13:11:11.045892 27026 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:11.062388 23411 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:10.947850 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 304931us
I20260709 13:11:11.071903 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.049959 27088 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.086133 27102 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.057684 27059 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.081756 22705 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:11.100009 22709 rpcz_store.cc:276] 0709 13:11:09.708506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.713899 (+  5393us) service_pool.cc:224] Handling call
0709 13:11:09.714211 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:09.714467 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:09.714493 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:09.726071 (+ 11578us) log.cc:844] Serialized 26 byte log entry
0709 13:11:09.890826 (+164755us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:09.890902 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:09.891020 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.060421 (+1169401us) raft_consensus.cc:1685] finished
0709 13:11:11.060450 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.060633 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":164186,"threads_started":1,"wal-append.queue_time_us":420664}
I20260709 13:11:11.099272 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2732) took 1391 ms. Trace:
I20260709 13:11:11.079784 26481 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:11.078198 26817 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:11.071018 27018 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.166728 26888 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.167735 27064 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.168450 27064 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.168942 27018 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.171885 26481 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.180881 26739 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.154785 26981 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.184391 26981 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.080652 26938 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:11.094377 26978 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:11.077945 27091 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.098906 27092 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.153345 27097 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:11.187840 26817 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.074931 27066 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:11.115857 27073 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.118774 26976 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.187875 26978 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.191066 27066 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:11.192297 27066 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.194813 26817 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.083520 26906 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.081106 27090 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.066879 26939 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:11.091287 27086 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:11.120708 26897 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.065721 27104 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.086567 26968 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:11.085028 27071 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.141669 22702 rpcz_store.cc:276] 0709 13:11:09.707384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.715832 (+  8448us) service_pool.cc:224] Handling call
0709 13:11:09.716174 (+   342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:09.717873 (+  1699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:09.717911 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:09.729740 (+ 11829us) spinlock_profiling.cc:243] Waited 8.72 ms on lock 0x7b2c0006cb88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:09.730553 (+   813us) log.cc:844] Serialized 52 byte log entry
0709 13:11:10.044627 (+314074us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:10.079142 (+ 34515us) log.cc:844] Serialized 14 byte log entry
0709 13:11:10.079416 (+   274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.079500 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.091523 (+1012023us) raft_consensus.cc:1685] finished
0709 13:11:11.098943 (+  7420us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.099160 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15636,"prepare.queue_time_us":35111,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":59,"raft.queue_time_us":253,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":187,"spinlock_wait_cycles":20057472,"thread_start_us":210857,"threads_started":1,"wal-append.queue_time_us":1326568}
W20260709 13:11:11.126346 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.182s (SENT) [suppressed 51 similar messages]
I20260709 13:11:11.129110 22629 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:11.131320 22598 leader_election.cc:400] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:11.110206 22633 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:11.055629 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:11.137107 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:11.230208 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 99322.2us
I20260709 13:11:11.080735 26915 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:11.143234 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71440.8us
I20260709 13:11:11.195647 26978 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.142036 26925 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.062036 27093 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.158842 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.246740 22630 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:11.159122 22705 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:11.160974 26672 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.186955 27102 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.212924 26817 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.189652 27054 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:11.188421 27092 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.135109 27065 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.236836 22780 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
W20260709 13:11:11.239955 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102981us
W20260709 13:11:11.263756 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 388283us
I20260709 13:11:11.264916 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.249481 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 611) took 1320 ms. Trace:
I20260709 13:11:11.266808 22627 rpcz_store.cc:276] 0709 13:11:09.928632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.928768 (+   136us) service_pool.cc:224] Handling call
0709 13:11:09.937334 (+  8566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:10.059533 (+122199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:10.059572 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:10.091318 (+ 31746us) log.cc:844] Serialized 26 byte log entry
0709 13:11:10.424319 (+333001us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:10.424391 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.424506 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:10.669309 (+244803us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b4800230058. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:11:11.231007 (+561698us) raft_consensus.cc:1685] finished
0709 13:11:11.231042 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.249383 (+ 18341us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228066,"raft.queue_time_us":79317,"raft.run_cpu_time_us":10838,"raft.run_wall_time_us":628505,"spinlock_wait_cycles":12389120,"thread_start_us":273203,"threads_started":2,"wal-append.queue_time_us":947874}
I20260709 13:11:11.245733 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.254938 22783 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:11.188608 27097 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.158370 27088 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.254315 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:11.246263 27103 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:11:11.257905 22776 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:11.231945 26567 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.239198 26915 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:11.188318 26938 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.232115 27003 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.270835 23415 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:11.147053 27059 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.247185 27018 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.272393 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.291538 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.085557 27016 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.227726 26939 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.287256 23411 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:11.291395 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:11.293022 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.263443 27071 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:11.300526 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.287719 26567 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.316063 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:11.324177 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 231685us
I20260709 13:11:11.273643 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.247745 27091 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 13:11:11.319499 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54928.6us
I20260709 13:11:11.310814 27018 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.273962 27097 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:11.329334 22784 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:11.251482 26925 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.326659 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2709) took 1506 ms. Trace:
I20260709 13:11:11.327941 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.296581 26981 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.292201 27054 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:11.336710 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.337795 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2744) took 1404 ms. Trace:
I20260709 13:11:11.311607 26816 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.334547 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 595) took 1516 ms. Trace:
I20260709 13:11:11.297255 27064 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.338316 22703 rpcz_store.cc:276] 0709 13:11:09.933297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.066468 (+133171us) service_pool.cc:224] Handling call
0709 13:11:10.066813 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:10.074502 (+  7689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:10.074550 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:10.075389 (+   839us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.062304 (+986915us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:11.062347 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.062426 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.337531 (+275105us) raft_consensus.cc:1685] finished
0709 13:11:11.337557 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.337715 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2446,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":850985,"threads_started":1,"wal-append.queue_time_us":1058576}
I20260709 13:11:11.332731 22775 rpcz_store.cc:276] 0709 13:11:09.820098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.820279 (+   181us) service_pool.cc:224] Handling call
0709 13:11:09.820571 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:09.820796 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:09.953770 (+132974us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.326263 (+1372493us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.326323 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:11.326374 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.326450 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.326574 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8492,"prepare.queue_time_us":789402,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":166200,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":8483,"thread_start_us":1456805,"threads_started":3,"wal-append.queue_time_us":1133029}
W20260709 13:11:11.345727 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 271283us
I20260709 13:11:11.347021 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.351197 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:11.292524 27088 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:11:11.282469 26817 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.351380 23415 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:11.298486 27003 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.340745 22704 rpcz_store.cc:276] 0709 13:11:09.818710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.818848 (+   138us) service_pool.cc:224] Handling call
0709 13:11:09.819257 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:09.824184 (+  4927us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:09.824219 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:09.878973 (+ 54754us) log.cc:844] Serialized 26 byte log entry
0709 13:11:10.058658 (+179685us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:10.081211 (+ 22553us) log.cc:844] Serialized 14 byte log entry
0709 13:11:10.083315 (+  2104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:10.083421 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.334265 (+1250844us) raft_consensus.cc:1685] finished
0709 13:11:11.334299 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.334466 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24736,"prepare.queue_time_us":21402,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":46,"raft.queue_time_us":4897,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":34053,"thread_start_us":119327,"threads_started":1,"wal-append.queue_time_us":1025403}
I20260709 13:11:11.336330 27024 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.333758 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:11.355973 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:11.340307 27071 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:11.340973 27106 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:11.366370 27090 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.377300 22631 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:11.366227 27071 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:11.368127 27106 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:11.390957 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:11.395864 23415 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:11.355083 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:11.406257 22775 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:11.424100 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.354259 27064 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:11:11.348286 26816 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.433701 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.434562 23415 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:11.437464 26978 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:11.438253 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86690.4us
I20260709 13:11:11.438362 26978 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:11.410060 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 283716us
W20260709 13:11:11.426131 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.381676 27090 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:11.446014 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.382983 27065 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.419284 27012 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:11.441334 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:11.420043 27088 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:11.460436 26939 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.425661 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2708) took 1613 ms. Trace:
I20260709 13:11:11.384346 27003 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.461773 27092 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.379694 27112 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:11.368348 27024 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.462674 27018 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.463860 27115 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:11.463954 27093 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:11.465533 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 115695us
I20260709 13:11:11.423224 27106 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:11.466138 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.477128 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.481640 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.444276 26925 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.464716 22779 rpcz_store.cc:276] 0709 13:11:09.812134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.812403 (+   269us) service_pool.cc:224] Handling call
0709 13:11:09.867220 (+ 54817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:09.867613 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:09.867646 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:09.893474 (+ 25828us) log.cc:844] Serialized 26 byte log entry
0709 13:11:10.083667 (+190193us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:10.559428 (+475761us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.203689 (+644261us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18004536a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:11.401219 (+197530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.401344 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.401415 (+    71us) raft_consensus.cc:1685] finished
0709 13:11:11.425174 (+ 23759us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.425539 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13400,"prepare.queue_time_us":558218,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":50,"raft.queue_time_us":380250,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":13622,"spinlock_wait_cycles":26573056,"thread_start_us":667038,"threads_started":3,"wal-append.queue_time_us":620143}
W20260709 13:11:11.450531 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 394957us
I20260709 13:11:11.428355 27111 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:11.492825 22704 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:11.493377 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.494920 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.493618 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:11.502012 22628 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:11.450039 27099 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:11:11.509611 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 175949us
I20260709 13:11:11.521203 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.522708 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.523190 27066 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:11.523550 23415 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:11.488668 27104 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.515527 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.527238 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.539002 22709 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:11.491418 27024 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:11.554319 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.498883 27111 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.559289 22709 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:11:11.559634 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44243.4us
I20260709 13:11:11.562918 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.564908 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.454175 26969 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:11:11.526374 27112 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:11.580116 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:11.580998 22779 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:11.582077 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:11.520547 26982 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.518874 27018 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.491101 27092 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:11:11.442957 27054 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.560694 26978 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:11.610146 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 133128us
I20260709 13:11:11.589097 22779 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:11.620837 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.614113 22777 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:11.621717 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:11.622350 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:11.623163 22777 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:11.625372 22778 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:11.551967 27016 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.628652 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.629557 22707 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:11.610154 27104 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.631393 27104 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.634203 27104 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.575873 27003 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:11.639748 22777 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:11.566323 27111 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:11.606396 27090 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.647143 22778 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:11.652855 22777 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
W20260709 13:11:11.641901 27113 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 13:11:11.657526 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:11.656908 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 340926us
I20260709 13:11:11.661340 22707 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:11.662843 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:11.663673 27118 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:11:11.691493 22779 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:11.691860 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:11.692193 22778 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
I20260709 13:11:11.664043 27139 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:11.620555 26982 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:11.699703 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 251303us
I20260709 13:11:11.701057 22779 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:11.711544 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:11.715408 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52709.9us
I20260709 13:11:11.716339 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:11.690327 26817 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.648085 27018 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.698321 27003 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:11.726543 22750 leader_election.cc:400] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9.
I20260709 13:11:11.658356 27090 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:11.729291 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 164448us
I20260709 13:11:11.697224 27059 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:11.717489 27026 consensus_meta.cc:287] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.577s	user 0.015s	sys 0.000s
I20260709 13:11:11.729506 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.731606 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.731652 22750 leader_election.cc:403] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:11.713873 27104 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:11.727882 27073 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:11.735885 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.697767 26978 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:11:11.746727 27122 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:11:11.741107 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.726744 26816 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.749799 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.731269 27016 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:11.769764 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:11.773273 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 150426us
I20260709 13:11:11.774940 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:11.776383 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:11.773607 27084 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.674s	user 0.010s	sys 0.009s
I20260709 13:11:11.778445 27084 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.779963 22775 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:11.787453 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:11.736887 27116 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:11.732499 27119 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:11.784925 27092 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:11.750805 27104 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.749624 27093 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9.
I20260709 13:11:11.793017 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.742395 27026 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12.
I20260709 13:11:11.740233 27112 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.743522 27073 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:11.785414 27081 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:11:11.744606 27059 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.748443 26978 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:11:11.798843 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.813578 23415 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:11.749202 27003 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:11.785593 27016 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:11.746757 27121 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:11.819059 27119 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:11.819494 27101 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.815s	user 0.006s	sys 0.000s
I20260709 13:11:11.819700 27116 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:11.820245 27117 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 13:11:11.822537 27134 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:11.824414 27135 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:11:11.878046 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2009) took 1943 ms. Trace:
I20260709 13:11:11.878580 22632 rpcz_store.cc:276] 0709 13:11:09.935061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.152488 (+217427us) service_pool.cc:224] Handling call
0709 13:11:10.261725 (+109237us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:10.887235 (+625510us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:11.345054 (+457819us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.728482 (+383428us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.735419 (+  6937us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.735618 (+   199us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:11.735656 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.735730 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.877761 (+142031us) raft_consensus.cc:1685] finished
0709 13:11:11.877790 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.877964 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":761766,"prepare.queue_time_us":559992,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":8886,"raft.queue_time_us":744519,"raft.run_cpu_time_us":390,"raft.run_wall_time_us":24867,"thread_start_us":587395,"threads_started":2,"wal-append.queue_time_us":321742}
W20260709 13:11:11.881228 26915 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.560s	user 0.013s	sys 0.000s
I20260709 13:11:11.881696 26915 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:11.810112 27108 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 13:11:11.827904 27136 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:11.884569 27136 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.885097 27136 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:11.828094 27090 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.829147 27061 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:11.886634 27061 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:11.887751 27018 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.881042 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.790395 27084 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.840124 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.811674 27073 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:11.904196 26982 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.904772 26567 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.835886 27016 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:11.916436 27116 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.849274 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.849983 27128 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:11.856985 27126 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 13:11:11.859041 27123 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:11:11.861006 26978 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:11.938910 26978 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:11.939436 26978 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:11.862910 27104 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.811530 27112 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.944078 27112 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:11.944679 27112 raft_consensus.cc:2764] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:11.864925 27003 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.834503 27059 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.869421 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2742) took 1949 ms. Trace:
I20260709 13:11:11.834414 26816 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:11.836314 27121 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.895035 22777 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:11.895632 27134 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.896543 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.938098 22627 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:11.897773 27119 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:11.920037 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.923257 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:11.887670 27061 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:11.925827 27018 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:11.927559 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.935149 27152 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:11.937469 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.990422 22632 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:11.952179 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:11.946316 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.961598 27059 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:11.956939 22706 rpcz_store.cc:276] 0709 13:11:09.919951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.065127 (+145176us) service_pool.cc:224] Handling call
0709 13:11:10.065381 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:10.426265 (+360884us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:10.564486 (+138221us) log.cc:844] Serialized 14 byte log entry
0709 13:11:10.897667 (+333181us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.027543 (+129876us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18003e5a80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:11.115461 (+ 87918us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b18003e5a80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:11.251365 (+135904us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:11.252124 (+   759us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:11.252196 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.807776 (+555580us) spinlock_profiling.cc:243] Waited 20.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:11.869060 (+ 61284us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b54000e6d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:11.869131 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:11.869339 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":104413,"prepare.queue_time_us":143051,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":5464,"raft.queue_time_us":140541,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":15678,"spinlock_wait_cycles":906134144,"thread_start_us":555317,"threads_started":2,"wal-append.queue_time_us":363342}
I20260709 13:11:11.965812 27128 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:11.966528 26915 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:11.967046 27121 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:11.897042 27147 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:11.893203 27071 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:11:11.981000 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:11.981312 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:11.980577 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57475.4us
I20260709 13:11:11.986797 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:11.993083 23411 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:12.022380 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:12.024667 22629 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:12.025363 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:12.013864 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:11.982276 26644 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:12.019160 27126 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:11:11.984716 26816 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:12.048854 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:12.040184 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:12.026676 26981 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:12.050482 26592 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
I20260709 13:11:12.063941 26934 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.041914 27061 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:12.051401 27147 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:12.076699 27147 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:12.072311 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:12.083458 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:12.085955 26982 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:12.096459 22629 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:12.066149 26981 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:12.077829 26890 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:12.123067 27134 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:12.125743 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:12.140069 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2704) took 2376 ms. Trace:
I20260709 13:11:12.142663 22781 rpcz_store.cc:276] 0709 13:11:09.732760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.732941 (+   181us) service_pool.cc:224] Handling call
0709 13:11:09.761169 (+ 28228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:09.782024 (+ 20855us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:10.903652 (+1121628us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b1800453be0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:11:11.262806 (+359154us) log.cc:844] Serialized 14 byte log entry
0709 13:11:11.531490 (+268684us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.556188 (+ 24698us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.556356 (+   168us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:11.556394 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:11.556474 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:11.856134 (+299660us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:12.108883 (+252749us) raft_consensus.cc:1685] finished
0709 13:11:12.108910 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:12.109066 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89233,"prepare.queue_time_us":663546,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":69,"raft.queue_time_us":67313,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":35963,"spinlock_wait_cycles":18560000,"thread_start_us":554267,"threads_started":2,"wal-append.queue_time_us":357248}
W20260709 13:11:12.154078 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 377732us
I20260709 13:11:12.165253 27139 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:12.021296 27152 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:12.192030 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 404655us
I20260709 13:11:12.084471 26644 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:12.099457 26982 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.203593 23416 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:11:12.205302 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:12.106662 26981 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:12.212007 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.212767 22776 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:12.213866 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.108211 26890 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.217272 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.081836 27016 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.217958 22779 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
W20260709 13:11:12.190563 27106 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.687s	user 0.014s	sys 0.006s
I20260709 13:11:12.218461 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.218991 26890 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:12.221750 22775 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:12.219383 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.237807 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.240360 27100 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:12.240896 27100 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:12.213140 27018 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:11:12.247772 27018 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:11:12.248665 27018 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:12.212735 27059 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:11:12.252169 26966 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.252381 27059 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:12.253005 27100 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.253976 27059 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:12.255239 27100 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:12.206534 26644 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.254421 27059 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:12.221107 27111 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.565s	user 0.007s	sys 0.005s
I20260709 13:11:12.266012 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.282871 22776 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:12.268734 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.219336 26816 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.300570 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.304178 22781 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:12.214879 26982 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.264487 26644 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:12.285214 27059 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:12.316488 27139 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:12.319504 26890 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.320768 26630 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.379509 26630 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:12.380082 26630 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:12.381228 26630 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:12.324566 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.308245 27102 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.386469 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2714) took 1132 ms. Trace:
I20260709 13:11:12.387121 27102 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.342970 27138 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:11:12.331976 27097 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.985s	user 0.014s	sys 0.001s
I20260709 13:11:12.309197 27018 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:12.351750 22528 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.204658 27147 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.296248 26978 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.374915 27152 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:12.292784 27111 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.415062 22632 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:12.398752 27153 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:12.417151 27153 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:12.419296 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2005) took 2485 ms. Trace:
I20260709 13:11:12.419888 22631 rpcz_store.cc:276] 0709 13:11:09.933507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.064879 (+131372us) service_pool.cc:224] Handling call
0709 13:11:10.073068 (+  8189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.381476 (+1308408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.418868 (+1037392us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540034a154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:12.419163 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2998868224}
I20260709 13:11:12.387161 22784 rpcz_store.cc:276] 0709 13:11:11.254432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.254679 (+   247us) service_pool.cc:224] Handling call
0709 13:11:11.256275 (+  1596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.329753 (+ 73478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.386068 (+1056315us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54000cbf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:12.386373 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167471488}
W20260709 13:11:12.261063 27098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.002s	sys 0.012s
I20260709 13:11:12.259423 27106 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:12.435305 27149 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:11:12.438769 26978 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.439924 27106 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:12.404592 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:12.442471 22528 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 8, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:12.446712 26828 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:12.447674 26828 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.406335 27147 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:12.398931 27018 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:12.394460 27097 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.426793 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:12.217149 27128 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.376261 27139 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.404153 26630 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:12.453536 22629 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:11:12.453737 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:12.457913 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:12.460985 22630 raft_consensus.cc:2410] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:12.462862 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:12.465293 27097 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:12.466097 27145 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:12.414434 27111 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:12.432515 27098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.003s	sys 0.013s
I20260709 13:11:12.457396 27024 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:12.468725 27063 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.451213 27106 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.467011 27084 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:12.448016 22528 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.471410 27018 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.473004 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:12.463699 22749 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:12.475795 22635 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:12.475920 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:12.494467 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.497036 22778 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:12.478305 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.499583 22775 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:12.488760 27063 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.485641 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.487849 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.505024 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:12.505013 22779 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:12.507776 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.508046 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:12.508863 22630 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:12.471829 26982 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.503206 26981 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.515401 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:12.516140 22777 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:11:12.516935 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.517902 22779 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:12.488776 27089 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:12.510361 27084 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:11:12.535876 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.418314 27153 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.536677 22778 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:12.538439 27016 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:12.533427 27018 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:12.548519 27121 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 13:11:12.512159 27071 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:12.486948 27024 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.492033 27106 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.552624 27145 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:12.555148 22633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 1963) took 4273 ms (client timeout 2242 ms). Trace:
W20260709 13:11:12.565671 22633 rpcz_store.cc:269] 0709 13:11:08.281293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:08.321040 (+ 39747us) service_pool.cc:224] Handling call
0709 13:11:12.554894 (+4233854us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b540022cc54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:12.555023 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6237458816}
I20260709 13:11:12.556978 22629 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:11:12.546273 27073 consensus_meta.cc:287] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.000s
I20260709 13:11:12.555702 22528 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 11, leader changed from <none> to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:12.570658 27016 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:12.575765 26890 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:12.576622 26828 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:12.577371 22628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.020s	sys 0.046s
W20260709 13:11:12.589457 22628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.021s	sys 0.046s
I20260709 13:11:12.589916 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:12.577070 27054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.022s	sys 0.018s
I20260709 13:11:12.595423 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2494) took 1120 ms. Trace:
I20260709 13:11:12.595976 22635 rpcz_store.cc:276] 0709 13:11:11.475325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.896122 (+420797us) service_pool.cc:224] Handling call
0709 13:11:12.595194 (+699072us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54002ac954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:12.595350 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986408704}
W20260709 13:11:12.579744 27003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.011s	sys 0.028s
I20260709 13:11:12.597640 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:12.599288 22631 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:12.588944 27116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.017s	sys 0.021s
I20260709 13:11:12.576416 27145 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:12.595403 27054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.022s	sys 0.018s
I20260709 13:11:12.606784 22777 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:12.610144 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:12.589555 27071 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:12.619417 22674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.013s	sys 0.028s
I20260709 13:11:12.623075 22778 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:12.640094 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 616) took 1388 ms. Trace:
I20260709 13:11:12.643091 22632 rpcz_store.cc:276] 0709 13:11:11.251224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.022763 (+771539us) service_pool.cc:224] Handling call
0709 13:11:12.639730 (+616967us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540006ccd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:12.639987 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839094528}
I20260709 13:11:12.581274 27073 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:12.605973 27003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.011s	sys 0.028s
W20260709 13:11:12.609756 27116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.018s	sys 0.021s
I20260709 13:11:12.615309 27084 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:12.549067 27089 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:12.641196 22674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.015s	sys 0.029s
W20260709 13:11:12.666335 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:12.622099 27051 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.018s	sys 0.025s
W20260709 13:11:12.669891 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.616566 27139 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:12.675069 22781 raft_consensus.cc:1240] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:12.676654 26739 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:12.579267 27121 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:12.679143 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.019s	sys 0.041s
W20260709 13:11:12.712450 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.019s	sys 0.042s
I20260709 13:11:12.683838 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.714259 22703 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:12.716693 22631 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:11:12.669350 22676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.038s	sys 0.045s
W20260709 13:11:12.717909 22676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.038s	sys 0.046s
I20260709 13:11:12.681890 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.683009 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.720073 22706 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:12.706712 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:12.718828 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:12.732378 22706 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:12.732849 22706 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:12.741457 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.744745 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:12.745926 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.011s	sys 0.017s
W20260709 13:11:12.746444 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.011s	sys 0.017s
W20260709 13:11:12.678643 27136 consensus_meta.cc:287] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.793s	user 0.016s	sys 0.000s
I20260709 13:11:12.673043 27152 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:12.752441 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.768998 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:12.746944 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 551699us
I20260709 13:11:12.753530 22706 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:12.746471 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.935s	user 0.019s	sys 0.075s
W20260709 13:11:12.771085 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.020s	sys 0.075s
I20260709 13:11:12.769491 22710 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:11:12.701324 27051 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.018s	sys 0.029s
I20260709 13:11:12.775285 22706 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:12.677536 27154 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:12.778828 26892 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:12.738392 27003 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:12.780043 26906 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:12.780627 26939 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:12.781424 27143 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.008s	sys 0.030s
I20260709 13:11:12.729952 27099 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:12.805259 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.723533 26739 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.779909 27145 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:12.788944 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:12.787588 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:12.812536 22710 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:12.783041 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:12.749783 27019 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:12.713876 26882 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:11:12.714035 27111 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 13:11:12.798928 22675 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.014s
W20260709 13:11:12.802397 22780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.027s	sys 0.030s
W20260709 13:11:12.778558 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 978462us
I20260709 13:11:12.771628 27136 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.767212 22528 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:12.812187 27143 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.008s	sys 0.030s
W20260709 13:11:12.820189 22675 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.008s	sys 0.014s
I20260709 13:11:12.823369 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2507) took 1067 ms. Trace:
W20260709 13:11:12.820399 22780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.027s	sys 0.030s
W20260709 13:11:12.830426 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47542us
I20260709 13:11:12.829317 22710 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:11:12.826908 27155 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:11:12.786343 27139 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:11:12.830868 22630 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:12.809031 26939 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.795006 27154 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.784262 27152 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.803071 27003 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:12.830005 27063 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.898428 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:12.839764 27087 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:12.913151 27087 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:12.864871 27171 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:12.887452 22706 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:11:12.929747 22706 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 5
I20260709 13:11:12.930087 22706 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:11:12.930409 22706 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:12.838280 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:12.782965 27071 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:12.821358 27024 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.827140 27106 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.835284 27019 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:11:12.836331 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 566455us
I20260709 13:11:12.887432 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2721) took 1565 ms. Trace:
I20260709 13:11:12.918354 23416 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:12.927079 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2720) took 1611 ms. Trace:
I20260709 13:11:12.872394 27102 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:12.903424 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:12.830912 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:12.980528 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142382us
I20260709 13:11:13.034965 23416 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:12.913692 27087 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:12.829777 27099 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.012351 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:13.037688 22706 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:13.038936 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:13.040516 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:13.013948 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:12.836385 22628 rpcz_store.cc:276] 0709 13:11:11.755993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.028366 (+272373us) service_pool.cc:224] Handling call
0709 13:11:12.028745 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.029142 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.029177 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.041461 (+ 12284us) log.cc:844] Serialized 26 byte log entry
0709 13:11:12.589929 (+548468us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:12.589965 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.590029 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:12.823096 (+233067us) raft_consensus.cc:1685] finished
0709 13:11:12.823135 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:12.823299 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":507,"raft.queue_time_us":9091,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":6085,"thread_start_us":536976,"threads_started":1,"wal-append.queue_time_us":768630}
I20260709 13:11:13.041743 22710 raft_consensus.cc:2410] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:11:13.005242 22780 rpcz_store.cc:276] 0709 13:11:11.322243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.322502 (+   259us) service_pool.cc:224] Handling call
0709 13:11:11.323060 (+   558us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.534128 (+211068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.534166 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.534858 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 13:11:11.982085 (+447227us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:12.884052 (+901967us) log.cc:844] Serialized 14 byte log entry
0709 13:11:12.885834 (+  1782us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.887105 (+  1271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:12.887167 (+    62us) raft_consensus.cc:1685] finished
0709 13:11:12.887180 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:12.887352 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136957,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":12595,"thread_start_us":1435062,"threads_started":3,"wal-append.queue_time_us":612978}
I20260709 13:11:13.013468 22783 rpcz_store.cc:276] 0709 13:11:11.315623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.315763 (+   140us) service_pool.cc:224] Handling call
0709 13:11:11.316137 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:11.742500 (+426363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:11.742542 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:11.755341 (+ 12799us) log.cc:844] Serialized 26 byte log entry
0709 13:11:12.174475 (+419134us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:12.174546 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:12.194818 (+ 20272us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:12.926747 (+731929us) raft_consensus.cc:1685] finished
0709 13:11:12.926776 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:12.926982 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31531,"raft.queue_time_us":117077,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":14256,"thread_start_us":792430,"threads_started":2,"wal-append.queue_time_us":787886}
I20260709 13:11:13.045419 22706 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:12.890523 27139 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:11:12.915702 27166 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:11:13.050206 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:13.036403 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:13.051360 23414 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:12.825901 27136 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:12.889739 27088 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:13.036238 27148 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:11:13.016364 27073 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:13.056341 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:13.058679 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:13.098374 22705 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:13.015547 27024 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:13.035135 26816 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.014s	sys 0.020s
I20260709 13:11:13.105023 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:13.054515 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:13.061735 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:13.111439 22710 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:13.062789 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:13.112751 22708 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:13.067283 27121 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.088924 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:13.049638 27139 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:13.095337 22701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.009s	sys 0.038s
I20260709 13:11:13.049634 27089 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:13.171825 22701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.009s	sys 0.038s
W20260709 13:11:13.083736 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 252883us
I20260709 13:11:13.096414 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:13.064622 27028 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:13.109582 26816 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.016s	sys 0.026s
I20260709 13:11:13.186978 27028 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.105162 27024 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:13.032004 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.151131 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:13.188953 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:13.190238 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:13.190663 27024 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:13.095113 26882 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:13.130434 22706 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:13.169690 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:13.102794 27119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.032s	sys 0.084s
I20260709 13:11:13.173017 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:13.173090 23414 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:13.231319 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58453.5us
W20260709 13:11:13.172968 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 84133.4us
W20260709 13:11:13.105333 26938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.007s	sys 0.032s
I20260709 13:11:13.066850 27088 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.243737 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:13.178938 27159 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:11:13.143977 26630 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.682s	user 0.014s	sys 0.000s
I20260709 13:11:13.187508 27028 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:12.828855 22528 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:13.142829 22782 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.016s	sys 0.050s
I20260709 13:11:13.196017 22708 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:11:13.251801 22782 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.018s	sys 0.052s
W20260709 13:11:13.202505 27133 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.836s	user 0.006s	sys 0.040s
I20260709 13:11:13.011215 27171 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:13.239089 23411 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.009s	sys 0.025s
W20260709 13:11:13.290947 23411 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.010s	sys 0.026s
W20260709 13:11:13.291347 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47730.4us
I20260709 13:11:13.234017 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:13.094082 27169 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:13.243757 22703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.011s	sys 0.020s
W20260709 13:11:13.297458 22703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.011s	sys 0.021s
W20260709 13:11:13.201267 26897 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.017s	sys 0.028s
I20260709 13:11:13.103171 27019 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:13.253003 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2713) took 2012 ms. Trace:
I20260709 13:11:13.302415 22782 rpcz_store.cc:276] 0709 13:11:11.240860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.241065 (+   205us) service_pool.cc:224] Handling call
0709 13:11:11.241490 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:11.241814 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:11.256693 (+ 14879us) log.cc:844] Serialized 14 byte log entry
0709 13:11:12.533981 (+1277288us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b18004535e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:11:13.252482 (+718501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:13.252572 (+    90us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:13.252640 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.252744 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:13.252922 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14127,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":1158,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":19223,"spinlock_wait_cycles":7627008,"thread_start_us":718149,"threads_started":1}
W20260709 13:11:13.199812 22627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.027s	sys 0.043s
W20260709 13:11:13.305225 22627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.027s	sys 0.044s
W20260709 13:11:13.272665 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.200412 26882 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.275197 26816 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
W20260709 13:11:13.292565 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58638.8us
W20260709 13:11:13.231019 27119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.032s	sys 0.085s
W20260709 13:11:13.311381 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:13.292824 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:13.239224 26938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.012s	sys 0.032s
I20260709 13:11:13.243851 26592 raft_consensus.cc:2764] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:11:13.201926 26939 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:13.309999 26630 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:11:13.301962 26897 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.918s	user 0.017s	sys 0.028s
W20260709 13:11:13.285698 27133 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.006s	sys 0.043s
W20260709 13:11:13.306897 27147 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.004s
I20260709 13:11:13.315414 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:13.304399 27063 raft_consensus.cc:2764] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:13.353068 22634 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:11:13.289983 27159 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:13.276345 27121 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:13.311388 26882 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:13.313763 27143 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:11:13.360888 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45611.7us
I20260709 13:11:13.315622 26739 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:13.342804 27052 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:11:13.353366 26630 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:13.343802 27019 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:13.347611 27162 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:13.365723 27018 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.002s
W20260709 13:11:13.274006 27084 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.016s	sys 0.014s
W20260709 13:11:13.372176 23385 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.023s	sys 0.030s
I20260709 13:11:13.372360 27158 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:13.316253 27139 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.275226 27163 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.100s
W20260709 13:11:13.397964 22809 heartbeater.cc:646] Failed to heartbeat to 127.21.5.190:44025 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.5.190:44025 timed out after 15.000s (SENT)
W20260709 13:11:13.398294 27164 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:11:13.385378 27168 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.334s	user 0.000s	sys 0.002s
I20260709 13:11:13.386368 27143 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:13.399956 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2766) took 1701 ms. Trace:
I20260709 13:11:13.429824 22701 rpcz_store.cc:276] 0709 13:11:11.698275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:11.702926 (+  4651us) service_pool.cc:224] Handling call
0709 13:11:11.728470 (+ 25544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.191657 (+463187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.191696 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.219525 (+ 27829us) log.cc:844] Serialized 26 byte log entry
0709 13:11:13.172426 (+952901us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:13.172480 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.172570 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:13.399635 (+227065us) raft_consensus.cc:1685] finished
0709 13:11:13.399666 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:13.399873 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58150,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":17955,"thread_start_us":1356917,"threads_started":2,"wal-append.queue_time_us":1015499}
W20260709 13:11:13.413456 23385 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.024s	sys 0.030s
W20260709 13:11:13.432123 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 709440us
W20260709 13:11:13.404735 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 479668us
I20260709 13:11:13.383433 27121 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.403647 26630 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:13.411645 27154 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:13.436455 22599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.015s	sys 0.024s
W20260709 13:11:13.534439 22599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.015s	sys 0.024s
W20260709 13:11:13.436678 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.014s	sys 0.025s
W20260709 13:11:13.468835 22709 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.045s	sys 0.015s
W20260709 13:11:13.536422 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.014s	sys 0.025s
W20260709 13:11:13.536525 22709 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.045s	sys 0.015s
W20260709 13:11:13.419088 27084 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.017s	sys 0.014s
I20260709 13:11:13.396951 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:13.546052 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 149200us
I20260709 13:11:13.556062 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:13.557176 22782 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:13.416216 27158 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:13.573035 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:13.584527 22782 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:11:13.537938 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 491658us
I20260709 13:11:13.593526 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:13.652030 22778 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:13.596312 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:13.423928 27003 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:13.678215 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:13.433866 27128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.021s	sys 0.037s
I20260709 13:11:13.682117 22780 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:13.699225 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:13.707084 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.708516 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.701610 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.702437 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.703199 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.703855 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.434003 27143 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:13.700367 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.726919 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:13.727505 22628 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:13.712882 22630 raft_consensus.cc:2410] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:11:13.713508 22631 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:13.723258 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:13.730221 22781 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:13.681492 27071 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.010s	sys 0.025s
I20260709 13:11:13.745085 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.746088 22635 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:13.736820 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.415012 27162 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.737596 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.748508 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2778) took 1040 ms. Trace:
I20260709 13:11:13.549485 27018 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:13.749804 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:11:13.705797 27128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.023s	sys 0.042s
I20260709 13:11:13.683542 27158 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:13.751835 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:13.681715 27147 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.736488 27099 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:13.775058 27147 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.776794 27099 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:13.778106 27099 consensus_queue.cc:1243] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:11:13.547632 27084 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:13.779371 27175 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.415s	user 0.000s	sys 0.001s
I20260709 13:11:13.779768 27084 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.691928 26817 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:13.782960 26817 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:13.783677 26817 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:13.755028 27018 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:13.788739 27018 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.761515 22632 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:13.725657 27159 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:13.802883 23414 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.572s	user 0.014s	sys 0.000s
I20260709 13:11:13.803709 23414 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:13.763389 27128 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:13.765074 27150 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:13.769605 27184 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:13.723838 27019 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.725292 27139 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:13.692425 27163 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.111s
I20260709 13:11:13.778255 27147 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:13.815454 27184 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:13.802273 27189 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:13.822467 27061 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:13.812717 27003 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:13.815666 27136 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.020s	sys 0.034s
W20260709 13:11:13.848320 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 315046us
W20260709 13:11:13.860386 23411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.016s	sys 0.047s
W20260709 13:11:13.863384 23411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.016s	sys 0.047s
I20260709 13:11:13.870153 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:13.871057 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:13.832547 27184 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:13.815363 27128 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.892589 23420 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:13.835239 27061 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:13.922148 27024 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.731s	user 0.010s	sys 0.000s
W20260709 13:11:13.857036 27136 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.021s	sys 0.034s
I20260709 13:11:13.875121 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 783) took 1190 ms. Trace:
I20260709 13:11:13.844398 27189 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.868218 26969 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:13.926369 23411 rpcz_store.cc:276] 0709 13:11:12.677502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.678171 (+   669us) service_pool.cc:224] Handling call
0709 13:11:12.678530 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:12.678786 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:13.291532 (+612746us) log.cc:844] Serialized 14 byte log entry
0709 13:11:13.866964 (+575432us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:13.867055 (+    91us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:13.867132 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.867240 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:13.867449 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1091439,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":50,"raft.queue_time_us":1084,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":11533,"thread_start_us":1134699,"threads_started":2,"wal-append.queue_time_us":627685}
I20260709 13:11:13.859777 27003 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:13.870258 27170 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.014s	sys 0.017s
W20260709 13:11:13.939517 27170 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.015s	sys 0.018s
W20260709 13:11:13.952260 27059 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.553s	user 0.017s	sys 0.000s
I20260709 13:11:13.952869 27059 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.926437 27171 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.929018 27024 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.958078 27024 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.959686 27121 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.973202 27003 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:13.940213 26969 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:13.983668 26817 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:13.984763 27088 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:14.021029 27191 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 13:11:14.028210 27179 log.cc:927] Time spent T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
W20260709 13:11:14.033458 27161 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.158s	user 0.005s	sys 0.002s
I20260709 13:11:14.040954 27152 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:14.045671 27160 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.006s	user 0.006s	sys 0.006s
I20260709 13:11:14.052134 27147 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:14.051842 27158 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.052834 27147 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:14.054054 27147 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.016592 27162 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.036924 27153 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:14.040179 27196 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:11:14.040910 27128 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.069056 27189 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:14.072482 27087 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.017s	user 0.014s	sys 0.000s
I20260709 13:11:14.073278 27087 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.080919 27158 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.412336 22528 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 5, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.126448 22528 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 8, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.131142 22528 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:13.766534 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 60967.9us
W20260709 13:11:14.137635 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.148687 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2514) took 1614 ms. Trace:
I20260709 13:11:14.151098 22626 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:14.149518 22627 rpcz_store.cc:276] 0709 13:11:12.533721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.533961 (+   240us) service_pool.cc:224] Handling call
0709 13:11:12.547406 (+ 13445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.547913 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.547958 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.548660 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:11:13.305771 (+757111us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:13.305835 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.305945 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.147771 (+841826us) raft_consensus.cc:1685] finished
0709 13:11:14.147885 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.148333 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1572,"raft.queue_time_us":1639,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":10371,"thread_start_us":756662,"threads_started":1,"wal-append.queue_time_us":1269821}
I20260709 13:11:14.154726 22528 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 7, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 7 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:14.157945 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2008) took 4223 ms. Trace:
I20260709 13:11:14.167938 22634 rpcz_store.cc:276] 0709 13:11:09.934618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:10.099648 (+165030us) service_pool.cc:224] Handling call
0709 13:11:10.100037 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:13.363785 (+3263748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.156983 (+793198us) spinlock_profiling.cc:243] Waited 3.25 s on lock 0x7b540003bbd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:14.157633 (+   650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7472659840}
W20260709 13:11:13.805027 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.013s	sys 0.041s
W20260709 13:11:14.166230 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:14.169981 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.014s	sys 0.041s
W20260709 13:11:14.169267 22709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.006s	sys 0.009s
W20260709 13:11:14.170917 22709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.006s	sys 0.009s
I20260709 13:11:14.172524 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2769) took 1496 ms. Trace:
I20260709 13:11:14.171370 22528 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.175884 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2774) took 1491 ms. Trace:
I20260709 13:11:14.176522 22707 rpcz_store.cc:276] 0709 13:11:12.684202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.684359 (+   157us) service_pool.cc:224] Handling call
0709 13:11:12.706728 (+ 22369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:12.707275 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:12.707316 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:12.724513 (+ 17197us) log.cc:844] Serialized 26 byte log entry
0709 13:11:13.188221 (+463708us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b18001de120. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:11:14.174993 (+986772us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:14.175133 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.175292 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:14.175406 (+   114us) raft_consensus.cc:1685] finished
0709 13:11:14.175426 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.175809 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7140,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":18203,"spinlock_wait_cycles":16003968,"thread_start_us":982013,"threads_started":1,"wal-append.queue_time_us":1275733}
I20260709 13:11:13.749389 22703 rpcz_store.cc:276] 0709 13:11:12.708264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.715315 (+  7051us) service_pool.cc:224] Handling call
0709 13:11:12.715800 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:12.716124 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:13.298010 (+581886us) log.cc:844] Serialized 14 byte log entry
0709 13:11:13.747936 (+449926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:13.748045 (+   109us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:13.748122 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:13.748232 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:13.748415 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12686,"prepare.queue_time_us":1087253,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":137,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":38354,"spinlock_wait_cycles":9946880,"thread_start_us":1015363,"threads_started":2,"wal-append.queue_time_us":614436}
I20260709 13:11:14.172981 22709 rpcz_store.cc:276] 0709 13:11:12.675603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.676726 (+  1123us) service_pool.cc:224] Handling call
0709 13:11:12.677141 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:12.677357 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:13.537239 (+859882us) log.cc:844] Serialized 14 byte log entry
0709 13:11:14.171752 (+634513us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:14.171890 (+   138us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:14.172048 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.172199 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.172421 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44385,"prepare.queue_time_us":958177,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":19981,"spinlock_wait_cycles":28544,"thread_start_us":1442389,"threads_started":2,"wal-append.queue_time_us":455160}
I20260709 13:11:14.174559 22528 catalog_manager.cc:5697] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.189349 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:14.190897 22782 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:14.193481 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:14.194584 22778 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:11:14.201208 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:14.202983 22635 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:14.224319 22626 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:14.224828 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:14.225607 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2013) took 4280 ms (client timeout 1607 ms). Trace:
W20260709 13:11:14.226188 22626 rpcz_store.cc:269] 0709 13:11:09.944813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:09.950954 (+  6141us) service_pool.cc:224] Handling call
0709 13:11:12.208565 (+2257611us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b180012d080. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806f18646 00007f1806e5258e 00007f1806ed3f9e 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:11:12.440455 (+231890us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b58006f2cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:11:14.149763 (+1709308us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b58006f2fd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:11:14.225472 (+ 75709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1883851776}
I20260709 13:11:14.227625 22628 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:14.236150 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:14.206943 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:14.208060 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:14.240785 22776 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:14.244961 22528 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.248359 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:13.991179 26644 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.631s	user 0.014s	sys 0.004s
I20260709 13:11:14.264915 26817 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.264724 26644 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.266069 26817 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.267500 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:14.271791 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 954) took 1373 ms. Trace:
I20260709 13:11:14.272346 23415 rpcz_store.cc:276] 0709 13:11:12.897945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.898094 (+   149us) service_pool.cc:224] Handling call
0709 13:11:12.898451 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:13.314908 (+416457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.314941 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:13.314984 (+    43us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:13.315013 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.271128 (+956115us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54005a3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:14.271227 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.271534 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957226240}
I20260709 13:11:14.248359 23414 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:14.248443 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:14.259408 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:14.280370 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:14.263907 22528 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:14.267768 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 961) took 1210 ms. Trace:
I20260709 13:11:14.282817 23421 rpcz_store.cc:276] 0709 13:11:13.057193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.057521 (+   328us) service_pool.cc:224] Handling call
0709 13:11:13.088460 (+ 30939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:13.346004 (+257544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.346040 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:13.346089 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:13.346123 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.267184 (+921061us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b5400514ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:14.267268 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.267552 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588835584}
I20260709 13:11:14.259265 23417 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 12. (term mismatch)
I20260709 13:11:14.287262 23417 pending_rounds.cc:85] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 11
I20260709 13:11:14.285957 22528 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 8, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.287554 23417 pending_rounds.cc:107] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 11.12
I20260709 13:11:14.297773 23417 raft_consensus.cc:2889] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:14.296491 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:14.301473 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 957) took 1363 ms. Trace:
I20260709 13:11:14.302135 23417 rpcz_store.cc:276] 0709 13:11:12.938580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.938783 (+   203us) service_pool.cc:224] Handling call
0709 13:11:12.939224 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:14.298580 (+1359356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.301089 (+  2509us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54005028d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:14.301389 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3011044608}
I20260709 13:11:14.304468 26644 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.283994 26817 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.309468 26817 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.309583 26264 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:14.316279 22528 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.317971 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:14.319424 23415 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:14.320554 26906 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.514s	user 0.011s	sys 0.005s
I20260709 13:11:14.323102 22777 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:14.324868 26906 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:14.325994 26906 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:14.327042 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.332520 22776 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:14.333796 26906 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:14.314270 26630 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.008s
I20260709 13:11:14.340605 26968 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.341261 27003 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:14.346064 22528 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.348018 27059 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:13.993896 27061 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.350713 27064 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.351516 27073 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.352319 27084 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.355378 27087 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.388540 27121 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:14.389810 27138 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.389474 26630 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.393865 27195 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.473186 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.399369 27153 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.399441 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:14.350409 26882 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.963s	user 0.011s	sys 0.007s
I20260709 13:11:14.461385 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.311641 26817 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.472545 27169 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.388554 27133 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:14.479758 22703 raft_consensus.cc:2410] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:11:14.354456 27018 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:14.472877 22528 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 8, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.469940 26981 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.467201 27138 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.086429 27184 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.549793 22707 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:14.473625 27195 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.471230 27064 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.523330 27133 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:14.520512 26882 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.600001 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.601099 22705 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:14.602728 27028 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.225s	user 0.012s	sys 0.011s
I20260709 13:11:14.603327 27028 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:14.605647 26672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.026s	sys 0.026s
W20260709 13:11:14.606334 26672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.026s	sys 0.026s
W20260709 13:11:14.607124 26739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.019s	sys 0.026s
W20260709 13:11:14.607614 26739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.020s	sys 0.026s
I20260709 13:11:14.608767 27061 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:14.609556 27184 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.547672 27169 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.613025 27169 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:14.615017 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.617157 27162 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:14.618027 27162 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.619109 27133 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.620790 27162 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.625052 26882 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.605777 27028 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.519402 27073 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.542634 27003 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.547153 26968 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.562227 26817 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.542284 27153 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:14.643764 27089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.051s	sys 0.086s
I20260709 13:11:14.627827 22528 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:14.666110 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:14.580540 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 333971us
I20260709 13:11:14.577735 27084 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.682830 22703 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:14.671579 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.671888 26969 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:14.677100 22747 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.852s	user 0.005s	sys 0.013s
W20260709 13:11:14.697006 22747 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.006s	sys 0.013s
W20260709 13:11:14.701521 27143 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.950s	user 0.013s	sys 0.003s
I20260709 13:11:14.704797 27003 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.706810 27138 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:14.681041 27089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.051s	sys 0.087s
W20260709 13:11:14.710680 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.849s	user 0.005s	sys 0.012s
W20260709 13:11:14.711279 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.005s	sys 0.012s
I20260709 13:11:14.713066 26788 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:14.714951 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 479378us
I20260709 13:11:14.716149 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:14.718753 27159 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.001s	sys 0.014s
W20260709 13:11:14.719326 27159 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.001s	sys 0.015s
I20260709 13:11:14.721167 27065 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:14.721778 27065 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:14.721998 27195 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.722524 26968 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:14.724798 27153 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.724175 22528 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.726668 27065 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.720213 26981 consensus_queue.cc:1059] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:11:14.731551 27064 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.731922 26630 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.732578 27224 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:14.733043 27224 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:14.734069 27224 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.735344 27224 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:14.737924 27073 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:14.713841 26788 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:14.747655 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61130.3us
I20260709 13:11:14.676964 26817 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:14.752744 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:14.754136 22707 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:14.755021 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 77023.6us
I20260709 13:11:14.755566 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:14.745443 22528 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:14.769193 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:14.702255 27143 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:14.801348 22782 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:14.803308 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:14.804582 22780 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:14.806938 22709 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:14.807344 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:14.808439 22779 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:14.808558 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:14.809273 22775 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:14.809417 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:14.810222 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:14.811013 27088 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:14.811542 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:14.787427 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 1.06057e+06us
I20260709 13:11:14.824952 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:14.836894 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 261398us
I20260709 13:11:14.841360 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:14.842311 22709 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:14.847941 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.131s	user 0.010s	sys 0.021s
W20260709 13:11:14.848515 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.301s	user 0.010s	sys 0.021s
W20260709 13:11:14.848979 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 1.30174e+06us
W20260709 13:11:14.850685 22702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.001s	sys 0.035s
W20260709 13:11:14.851238 22702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.002s	sys 0.035s
I20260709 13:11:14.852293 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2768) took 2177 ms. Trace:
I20260709 13:11:14.853446 22709 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:14.861938 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:14.853444 22632 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:14.791121 27184 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:14.867373 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43840.1us
I20260709 13:11:14.808583 27143 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:14.774717 26906 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:14.854867 23412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.011s	sys 0.023s
I20260709 13:11:14.876962 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:14.877717 27220 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:11:14.878355 22779 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:14.852738 22702 rpcz_store.cc:276] 0709 13:11:12.675296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:12.676765 (+  1469us) service_pool.cc:224] Handling call
0709 13:11:13.190182 (+513417us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:13.190352 (+   170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:13.190600 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:13.681974 (+491374us) log.cc:844] Serialized 14 byte log entry
0709 13:11:14.851780 (+1169806us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:14.851862 (+    82us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:14.851971 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.852076 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:14.852218 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6617,"prepare.queue_time_us":615082,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"spinlock_wait_cycles":5427840,"thread_start_us":1631863,"threads_started":2}
I20260709 13:11:14.894099 27208 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:14.895069 27061 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:11:14.901311 23412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.012s	sys 0.024s
W20260709 13:11:14.905387 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 149935us
I20260709 13:11:14.909157 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:14.910079 22783 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:14.770287 22528 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:14.917461 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:14.899881 27061 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:14.920719 27215 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.237s	user 0.002s	sys 0.002s
I20260709 13:11:14.930748 22707 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:14.924196 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:14.922706 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:14.927891 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:14.915979 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.827263 26968 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:11:14.921401 26906 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:14.949983 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:14.952351 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:14.954582 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 92889.8us
I20260709 13:11:14.951547 27195 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:14.965988 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:14.945209 27061 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:14.983565 22626 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:14.956447 27195 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:14.971735 26969 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:14.948859 26981 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:11:15.019002 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.019883 22626 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:15.020918 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.008706 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:14.958119 27088 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:11:15.029011 22705 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:14.998272 27195 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:15.042124 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48035.1us
W20260709 13:11:15.043895 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.023991 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.011713 26969 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:15.042841 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.182s (SENT)
I20260709 13:11:15.070415 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.074834 22704 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:15.075773 27217 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:11:15.076821 22704 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:11:15.079310 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.080674 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.081333 22705 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:15.040246 27084 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:15.069918 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 202470us
I20260709 13:11:15.090994 22707 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:11:15.086041 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:15.061218 27073 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:15.091729 22707 pending_rounds.cc:85] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:11:15.097589 22707 pending_rounds.cc:107] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:11:15.098150 22707 raft_consensus.cc:2889] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:15.098124 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.099903 22703 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8. Re-sending same reply.
I20260709 13:11:15.100065 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:15.102262 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.103212 22631 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:15.065223 26969 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:15.122187 22707 pending_rounds.cc:107] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:11:15.122654 22707 raft_consensus.cc:2889] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:15.125692 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.124636 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.109076 27073 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.129952 23415 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:15.129709 22630 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:15.128413 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.138391 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:15.145393 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.146235 22706 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:15.123907 26788 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.150235 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:15.160041 27230 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:11:15.160555 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 116883us
W20260709 13:11:15.161931 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 167362us
W20260709 13:11:15.163022 27221 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.174s	user 0.002s	sys 0.000s
I20260709 13:11:15.164157 27153 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:15.165012 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2746) took 1567 ms. Trace:
I20260709 13:11:15.165437 22784 rpcz_store.cc:276] 0709 13:11:13.597678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.679083 (+ 81405us) service_pool.cc:224] Handling call
0709 13:11:13.679595 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:13.735468 (+ 55873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:13.735506 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:13.736088 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:11:14.203660 (+467572us) spinlock_profiling.cc:243] Waited 37.1 ms on lock 0x7b1800453760. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:11:14.616590 (+412930us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:14.883922 (+267332us) log.cc:844] Serialized 14 byte log entry
0709 13:11:14.904304 (+ 20382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.913324 (+  9020us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:15.106730 (+193406us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:15.164552 (+ 57822us) raft_consensus.cc:1685] finished
0709 13:11:15.164576 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.164767 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31976,"prepare.queue_time_us":244453,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":42037,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":32442,"spinlock_wait_cycles":92873216,"thread_start_us":594521,"threads_started":2,"wal-append.queue_time_us":1052231}
W20260709 13:11:15.168138 27210 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:11:15.165017 27153 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:15.170029 27223 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:11:15.173981 27153 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:15.181171 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:15.180799 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46788.5us
I20260709 13:11:15.135275 27222 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.195s
I20260709 13:11:15.194631 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.201745 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2747) took 1023 ms. Trace:
I20260709 13:11:15.202433 22781 rpcz_store.cc:276] 0709 13:11:14.178045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.185411 (+  7366us) service_pool.cc:224] Handling call
0709 13:11:14.186354 (+   943us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:14.187408 (+  1054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:14.187439 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:14.188631 (+  1192us) log.cc:844] Serialized 26 byte log entry
0709 13:11:14.543441 (+354810us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:14.543499 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.543602 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:15.201263 (+657661us) raft_consensus.cc:1685] finished
0709 13:11:15.201291 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.201489 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136831,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":277,"thread_start_us":354117,"threads_started":1,"wal-append.queue_time_us":1010099}
W20260709 13:11:15.207394 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.139763 27073 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:15.213584 22750 connection.cc:256] RPC call timeout handler was delayed by -1.9510721910000939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:11:15.216140 22750 connection.cc:256] RPC call timeout handler was delayed by -1.91777219100004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 13:11:15.216883 22750 connection.cc:256] RPC call timeout handler was delayed by -1.7458721910001032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1766s for a response
W20260709 13:11:15.217417 22750 connection.cc:256] RPC call timeout handler was delayed by -1.6918721910000158s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
W20260709 13:11:15.217960 22750 connection.cc:256] RPC call timeout handler was delayed by -1.5730721910000511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1958s for a response
W20260709 13:11:15.218475 22750 connection.cc:256] RPC call timeout handler was delayed by -1.5559721910000235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 13:11:15.218933 22750 connection.cc:256] RPC call timeout handler was delayed by -1.5559721910000235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 13:11:15.219430 22750 connection.cc:256] RPC call timeout handler was delayed by -1.4758721910000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:11:15.219873 22750 connection.cc:256] RPC call timeout handler was delayed by -1.4551721910000879s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
I20260709 13:11:15.149554 27088 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:15.269268 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:15.275004 22781 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:15.283411 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 141088us
I20260709 13:11:15.284837 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:15.289399 27138 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:15.290383 27071 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:15.291145 27071 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.293891 27071 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:15.299536 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:15.301249 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46476.2us
I20260709 13:11:15.309996 27220 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.290180 27222 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.316311 27222 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:15.321933 27219 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:11:15.121680 22528 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 3, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 3 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:15.330313 27147 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:15.331601 22528 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:15.335451 26906 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.339355 23419 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:11:15.341464 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.343174 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 709) took 1493 ms. Trace:
I20260709 13:11:15.343421 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.343636 23412 rpcz_store.cc:276] 0709 13:11:13.850595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:13.856837 (+  6242us) service_pool.cc:224] Handling call
0709 13:11:13.881592 (+ 24755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:13.881861 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:11:13.911364 (+ 29503us) log.cc:844] Serialized 14 byte log entry
0709 13:11:15.210995 (+1299631us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b18003ebcc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:11:15.270465 (+ 59470us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:15.278927 (+  8462us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:15.342767 (+ 63840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.342903 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.343078 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26434,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":7422,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3714,"spinlock_wait_cycles":11447552,"thread_start_us":990078,"threads_started":1,"wal-append.queue_time_us":972524}
I20260709 13:11:15.344995 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:15.347782 23420 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:15.290169 27138 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:15.354799 22783 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:15.346410 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:15.358286 22706 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:15.360194 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:15.360714 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.364995 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.366433 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.370270 23415 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:15.376068 27184 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.379994 27184 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.382966 27128 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.383662 27128 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.384792 22705 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:15.386129 27128 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:15.386291 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:15.376263 26672 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.387674 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:15.387843 26672 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.389222 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:15.389354 27195 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.390229 26672 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.391037 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2516) took 1200 ms. Trace:
I20260709 13:11:15.363494 27220 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.394481 23417 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:11:15.343495 27139 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:15.391393 22634 rpcz_store.cc:276] 0709 13:11:14.190964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.193753 (+  2789us) service_pool.cc:224] Handling call
0709 13:11:14.194356 (+   603us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:14.195236 (+   880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:14.195289 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:14.199471 (+  4182us) log.cc:844] Serialized 52 byte log entry
0709 13:11:14.539080 (+339609us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:14.539188 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:14.539307 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:15.320792 (+781485us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:15.390688 (+ 69896us) raft_consensus.cc:1685] finished
0709 13:11:15.390710 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.390967 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126276,"raft.queue_time_us":111231,"raft.run_cpu_time_us":423,"raft.run_wall_time_us":22654,"spinlock_wait_cycles":7604992,"thread_start_us":338873,"threads_started":1,"wal-append.queue_time_us":774360}
I20260709 13:11:15.397094 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.398883 26968 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:15.399454 26968 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.399904 26968 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:15.387703 26906 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.402863 27064 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.403615 27064 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.404685 27133 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.405352 27133 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.407006 27084 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:15.413559 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:15.409737 27153 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:15.415822 26672 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:15.448326 27133 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:15.382591 27184 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.451078 26672 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:15.451933 26672 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:15.452674 27220 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.452533 26672 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.455235 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.457137 26672 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:15.471066 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.334111 22528 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.476709 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.485813 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.486956 23420 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:15.491173 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.493472 22630 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:15.497956 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 87322us
I20260709 13:11:15.501729 27222 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:15.469247 26906 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.507058 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.513286 22626 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:15.507361 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:15.493021 27220 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:15.503800 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:15.532577 27018 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.985s	user 0.008s	sys 0.009s
I20260709 13:11:15.550803 22708 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:15.553691 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:15.553941 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.563480 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:15.547327 26981 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:15.569239 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:15.502991 27138 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.549711 27133 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:11:15.596735 27133 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:15.597162 27133 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:15.587458 22707 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:15.564682 27224 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 13:11:15.607008 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45002.1us
I20260709 13:11:15.608160 27224 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:11:15.609829 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.590566 27163 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:15.591964 27237 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:11:15.593019 27234 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:11:15.580637 22528 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.602224 27195 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.631808 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.633778 22630 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:15.634986 27107 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:15.638939 27138 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:15.639149 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 2787) took 1240 ms. Trace:
I20260709 13:11:15.615584 27064 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.639770 22708 rpcz_store.cc:276] 0709 13:11:14.401803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.460835 (+ 59032us) service_pool.cc:224] Handling call
0709 13:11:15.638860 (+1178025us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400315e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:15.639029 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2373113088}
I20260709 13:11:15.644616 27195 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:15.645481 22710 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:11:15.646248 22710 pending_rounds.cc:85] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 5
I20260709 13:11:15.645602 27195 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.646581 22710 pending_rounds.cc:107] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:11:15.647033 22710 raft_consensus.cc:2889] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:15.649987 23419 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:15.658221 27195 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.662873 27019 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:15.663672 27019 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:15.665158 27019 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.671063 27139 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:15.671895 27139 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:15.672109 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.675946 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:15.637094 22528 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:15.703449 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:15.714782 27158 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.714912 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.715802 22708 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:11:15.715595 27158 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.716126 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.717051 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:15.723757 27236 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:11:15.598260 27133 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.732311 27133 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:15.747982 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.745960 27019 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:15.770028 26969 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.641s	user 0.013s	sys 0.004s
W20260709 13:11:15.777815 27073 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.007s
I20260709 13:11:15.735932 27064 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.803650 27139 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.791131 26906 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:15.791867 27219 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:11:15.798121 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:15.808126 26906 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:15.808785 26906 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:15.801263 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.714030 22528 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 1 to 4, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 4 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:15.809999 26906 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.812359 22528 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:15.804013 27064 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.814649 27158 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.817240 27064 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.616910 27163 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:15.821249 27163 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:15.798055 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:15.821879 27061 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.812s	user 0.015s	sys 0.001s
I20260709 13:11:15.829954 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.831997 23416 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:15.836079 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:15.833566 22632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.025s	sys 0.041s
I20260709 13:11:15.839974 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:15.814783 22528 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:15.841740 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 436604us
W20260709 13:11:15.843469 22701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.020s	sys 0.016s
W20260709 13:11:15.863009 22701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.020s	sys 0.016s
I20260709 13:11:15.823638 26969 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.824589 27073 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:15.843233 22708 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:11:15.860718 22598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.018s	sys 0.046s
W20260709 13:11:15.869625 22598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.019s	sys 0.046s
I20260709 13:11:15.870393 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:15.863606 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.024s	sys 0.016s
W20260709 13:11:15.866540 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:15.871933 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.024s	sys 0.016s
W20260709 13:11:15.872617 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 570652us
W20260709 13:11:15.852559 22632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.026s	sys 0.041s
I20260709 13:11:15.878293 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.827749 27061 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:15.859367 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 470179us
I20260709 13:11:15.911939 23412 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:15.864248 27128 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:15.927430 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56549us
I20260709 13:11:15.944723 22635 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:15.944828 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:15.909600 26264 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:15.958747 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:15.913046 27187 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:15.915011 27071 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:15.919950 27128 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:15.973363 27184 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.004647 27184 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.005282 22528 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:16.007496 26264 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:16.008266 26264 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:16.007925 22528 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:15.997069 27065 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:11:15.977066 26968 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.577s	user 0.016s	sys 0.000s
I20260709 13:11:16.013623 26968 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.016693 23420 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:11:16.018162 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2793) took 1265 ms. Trace:
I20260709 13:11:16.019025 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:16.020871 27106 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:16.021459 27106 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:16.021920 27106 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:16.018760 22701 rpcz_store.cc:276] 0709 13:11:14.752832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.756862 (+  4030us) service_pool.cc:224] Handling call
0709 13:11:14.792045 (+ 35183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:14.829309 (+ 37264us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:15.254822 (+425513us) log.cc:844] Serialized 14 byte log entry
0709 13:11:15.864455 (+609633us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:15.879900 (+ 15445us) log.cc:844] Serialized 26 byte log entry
0709 13:11:15.880051 (+   151us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:15.880089 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.880156 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:16.017791 (+137635us) raft_consensus.cc:1685] finished
0709 13:11:16.017825 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:16.018072 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":464509,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":17737,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":24042,"thread_start_us":815525,"threads_started":2,"wal-append.queue_time_us":683049}
I20260709 13:11:16.024268 27061 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:15.996330 26969 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:16.003247 27139 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.028792 27184 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:15.981263 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:15.982301 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 623) took 1075 ms. Trace:
I20260709 13:11:15.982640 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:16.041889 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:16.048743 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.049696 23417 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:16.031948 22635 rpcz_store.cc:276] 0709 13:11:14.907103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.907300 (+   197us) service_pool.cc:224] Handling call
0709 13:11:14.927245 (+ 19945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:15.945489 (+1018244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.981936 (+ 36447us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b5400127854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:15.982217 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2305595776}
I20260709 13:11:16.049806 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:15.970368 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2801) took 1038 ms. Trace:
W20260709 13:11:16.052199 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 353383us
W20260709 13:11:16.055335 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 246588us
W20260709 13:11:16.053236 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.033725 27128 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:16.050449 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94848us
I20260709 13:11:15.973390 27187 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.000303 27073 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:16.052883 22702 rpcz_store.cc:276] 0709 13:11:14.932401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.944852 (+ 12451us) service_pool.cc:224] Handling call
0709 13:11:15.015987 (+ 71135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:15.107886 (+ 91899us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:15.613526 (+505640us) log.cc:844] Serialized 14 byte log entry
0709 13:11:15.969827 (+356301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:15.969912 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:15.969996 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.970097 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:15.970281 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":213811,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":31758,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":163,"thread_start_us":710649,"threads_started":2}
I20260709 13:11:16.074769 23416 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:16.076474 22778 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:11:16.080765 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.083617 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.084599 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:16.084934 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.085331 22783 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:15.987124 27248 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.111s
W20260709 13:11:16.094167 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 485388us
I20260709 13:11:16.096453 23420 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:16.077271 27187 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:15.987545 27242 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 13:11:16.097841 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.039067 27071 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.116056 22635 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:16.116652 22633 raft_consensus.cc:2393] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:16.082798 26968 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.091262 26981 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:16.063691 27224 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:16.081507 27158 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:11:16.148548 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:16.144585 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:16.155537 27224 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.159036 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.160219 22776 raft_consensus.cc:2410] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:11:16.161960 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.165239 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.161723 22781 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:11:16.168815 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.168823 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:16.169848 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.170603 22784 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:11:16.163064 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.958s (SENT) [suppressed 24 similar messages]
I20260709 13:11:16.174801 23420 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:16.146317 27064 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:16.189000 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.189756 22776 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:16.146513 27147 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.192596 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.139329 27018 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:11:16.174705 27248 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:16.200812 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.201360 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.102978 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:16.180307 26968 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.204882 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:16.209312 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:16.209697 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.193638 27064 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:16.197551 27019 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:16.197875 27158 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:16.218071 22631 raft_consensus.cc:2368] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8.
I20260709 13:11:16.234272 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.236541 22780 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:16.222386 23416 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:16.224736 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:16.232010 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.239876 22782 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:16.211360 27222 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.244634 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.232996 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:16.245568 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.880s (SENT)
I20260709 13:11:16.271006 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.245952 22775 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:16.246533 27073 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:16.258414 22780 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:16.223641 27147 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.273588 22777 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:11:16.276368 27138 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.012s
W20260709 13:11:16.277676 27258 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:11:16.260196 27252 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:16.281567 26981 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:16.283188 27257 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:11:16.290710 27267 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:11:16.291970 27267 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:16.292384 27267 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:16.272845 22779 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:16.274304 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:16.261441 27251 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:11:16.261200 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:16.237998 27065 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:11:16.209481 27242 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:11:16.237679 27018 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:16.235230 27019 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:16.331048 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 102473us
I20260709 13:11:16.297355 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:16.315032 22783 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:16.330950 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.298388 22778 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:16.233258 27158 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:16.318576 22777 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:11:16.336062 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.340283 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.218348 26968 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.318024 27139 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:16.292970 27073 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:16.277676 27147 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:16.347786 27248 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:16.331439 27018 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:16.333869 27019 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:16.293174 27222 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.369529 22776 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
W20260709 13:11:16.370905 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 129305us
I20260709 13:11:16.375382 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:16.352699 22784 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
W20260709 13:11:16.382973 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 137472us
I20260709 13:11:16.319057 27264 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:11:16.342583 27138 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:16.391263 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2530) took 1096 ms. Trace:
I20260709 13:11:16.394985 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.412030 22627 rpcz_store.cc:276] 0709 13:11:15.350748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:15.357837 (+  7089us) service_pool.cc:224] Handling call
0709 13:11:15.376968 (+ 19131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:15.460511 (+ 83543us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:15.460568 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:15.473217 (+ 12649us) log.cc:844] Serialized 26 byte log entry
0709 13:11:15.917435 (+444218us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:15.917503 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:15.917593 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:16.390932 (+473339us) raft_consensus.cc:1685] finished
0709 13:11:16.390963 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:16.391173 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":43745,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":11660,"thread_start_us":411951,"threads_started":1,"wal-append.queue_time_us":576144}
I20260709 13:11:16.396873 27158 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.400861 27018 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.402912 26264 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.352212 27073 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:16.426787 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.432207 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57804.5us
I20260709 13:11:16.441474 26264 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:16.443445 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 192671us
I20260709 13:11:16.359318 27252 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.072s
I20260709 13:11:16.448801 26264 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:16.449507 26264 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:16.451800 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.379972 27065 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:11:16.472031 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.493265 23419 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:11:16.515580 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:16.436069 27138 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:16.516700 23420 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:16.520958 26264 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.473047 27018 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:16.536183 23385 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.521s (SENT)
I20260709 13:11:16.536208 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.539366 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:16.519259 27158 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:16.544822 27266 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.546945 27222 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.384229 27064 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.417006 27267 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.554798 27187 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:16.562902 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 151245us
I20260709 13:11:16.510457 27248 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:11:16.572254 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:16.574295 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.593811 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.582023 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 134276us
W20260709 13:11:16.583374 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 295486us
W20260709 13:11:16.595396 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.595757 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:16.540786 27071 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:16.628544 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.632890 22631 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:16.647637 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 272092us
W20260709 13:11:16.648939 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 154834us
I20260709 13:11:16.649236 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:16.589062 27266 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:16.671088 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.675998 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 80236.5us
I20260709 13:11:16.677112 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:16.684903 23419 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:16.688766 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:16.690657 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.619485 27071 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:11:16.679340 27128 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.005s
I20260709 13:11:16.664866 27064 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:16.645478 27278 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.707312 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.711647 22781 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:16.716499 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118194us
I20260709 13:11:16.700009 27290 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:11:16.717692 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.760293 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:16.763813 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52564.1us
W20260709 13:11:16.771864 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94878us
I20260709 13:11:16.711248 27065 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:16.775115 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:16.775497 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.659754 27138 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.782114 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.784389 27292 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:11:16.791558 27280 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:11:16.791980 27292 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:16.792428 27292 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:16.793360 27292 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.713357 27128 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:16.805202 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132543us
I20260709 13:11:16.797616 27107 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:11:16.814437 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:16.786947 27248 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:16.768872 27278 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:16.847589 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72603us
I20260709 13:11:16.817507 27107 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:16.863848 27107 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.826440 27248 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:16.867594 27248 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.868350 27284 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.869118 27284 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.869604 27248 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:16.795477 27285 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:11:16.876343 27266 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.876969 27128 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:16.879166 27294 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:16.881753 27278 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.882824 22781 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:11:16.883461 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.884425 22779 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:11:16.884724 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:16.885308 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.887216 27187 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:16.887620 27291 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:16.886279 27267 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:16.860096 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:16.879726 27106 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.829s	user 0.011s	sys 0.002s
I20260709 13:11:16.885351 23420 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:16.895056 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.896854 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.854882 27292 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:16.888846 27275 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 13:11:16.905825 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68747.9us
I20260709 13:11:16.892692 26969 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:11:16.919034 27274 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
I20260709 13:11:16.935884 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.942055 22783 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:16.943154 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.944481 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.945653 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:16.935945 23420 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:16.947857 22747 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:16.947510 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89989.1us
W20260709 13:11:16.949182 22675 connection.cc:256] RPC call timeout handler was delayed by -2.19474538399993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17470000000000002s for a response
W20260709 13:11:16.949638 22675 connection.cc:256] RPC call timeout handler was delayed by -2.167745384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:11:16.949934 22675 connection.cc:256] RPC call timeout handler was delayed by -2.1659453839999969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1779s for a response
W20260709 13:11:16.950245 22675 connection.cc:256] RPC call timeout handler was delayed by -2.154245383999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
W20260709 13:11:16.950537 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0813453839999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18730000000000002s for a response
W20260709 13:11:16.950850 22675 connection.cc:256] RPC call timeout handler was delayed by -1.9832453839999289s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:11:16.951140 22675 connection.cc:256] RPC call timeout handler was delayed by -1.9067453839999189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20670000000000002s for a response
W20260709 13:11:16.951521 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8671453839999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
W20260709 13:11:16.951805 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8662453839999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
I20260709 13:11:16.951977 22627 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:16.952152 22675 connection.cc:256] RPC call timeout handler was delayed by -1.7510453840000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
I20260709 13:11:16.953394 27276 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.953871 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:16.953974 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:16.954236 27297 raft_consensus.cc:2764] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:11:16.954842 27256 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.955461 27256 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.957557 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:16.958540 27289 raft_consensus.cc:2764] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:11:16.959299 22630 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:16.959453 27289 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:16.960635 27222 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:16.961333 27222 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:16.954061 27276 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.962548 27222 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:16.963672 27284 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:16.964356 27284 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:16.964859 27284 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:16.965698 27276 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:16.958236 27256 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:16.969563 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.969732 23385 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:16.989408 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:16.970742 22635 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:16.971791 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:16.974821 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:16.990381 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:16.980201 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:16.984596 22750 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.849s (SENT)
I20260709 13:11:16.921746 27147 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.924811 26969 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:16.998689 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:16.988204 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:16.991380 22778 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:17.001988 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:17.004244 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:16.896915 27106 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:17.006866 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:17.007395 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.008172 23417 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:17.011107 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61999.2us
I20260709 13:11:17.011698 22626 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:17.006191 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:16.987437 27073 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.547s	user 0.011s	sys 0.002s
I20260709 13:11:17.005939 27266 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:17.021415 23414 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:17.022819 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:17.024775 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.024621 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.025578 22775 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:17.032305 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.035936 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.036864 22778 raft_consensus.cc:2393] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:17.030172 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:17.040117 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.040993 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:16.997907 27293 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:17.006592 26981 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:16.998139 27184 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:17.010231 27287 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:17.017269 27289 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:17.035033 27276 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:17.044778 22627 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:17.055621 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.058907 27302 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:17.060626 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.064839 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.065593 22707 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:11:17.066395 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.073643 27220 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.082149 27220 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.085147 27220 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.096719 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.098117 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:17.098924 22710 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:17.102908 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118417us
I20260709 13:11:17.106422 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.044778 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.007565 27147 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.062281 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:17.049974 27293 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:17.109416 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 155542us
I20260709 13:11:17.078243 22701 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:17.052709 27187 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.115100 22710 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:17.118441 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:17.121460 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.031191 27073 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.094043 22626 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:11:17.068316 27284 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.161418 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.162322 22775 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:17.163429 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:17.164252 22777 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:17.061024 26981 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.165197 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.117025 27293 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:17.166630 26981 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.168581 27147 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.108379 26788 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:17.169349 26788 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.172165 26788 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.146710 22632 raft_consensus.cc:606] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.183804 22632 raft_consensus.cc:614] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:17.149063 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.149627 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.159289 22779 raft_consensus.cc:2368] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10.
I20260709 13:11:17.160488 27073 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:17.112375 27276 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:17.173589 27019 consensus_meta.cc:287] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.775s	user 0.009s	sys 0.005s
I20260709 13:11:17.134759 27287 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.146065 27187 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.175369 22708 raft_consensus.cc:2410] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:11:17.175782 26981 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.179019 27184 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:17.166347 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.160406 27106 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.185635 22630 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
W20260709 13:11:17.192996 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.190850 22777 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.195885 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.197568 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.187147 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:17.199507 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:17.200186 22710 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:17.200891 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.201170 22633 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:17.202423 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:17.203708 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.115860 26969 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.204759 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.205125 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:17.191514 27267 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:17.192427 27019 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.199875 27184 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.208556 22775 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:17.223240 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.211263 27061 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:17.212641 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.212913 22706 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:17.231293 22776 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:11:17.213737 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.214193 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.216156 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.233037 22701 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.216410 22707 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:17.217037 22703 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:17.217483 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.234601 22705 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.218745 26981 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.221169 27187 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.229314 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.224884 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 37) took 1143 ms. Trace:
I20260709 13:11:17.229003 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.232064 22634 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:17.239686 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.222585 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.241537 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:17.242452 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 92950.4us
I20260709 13:11:17.192615 27276 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.243395 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.259979 22780 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:17.262681 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.264554 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.222980 27287 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.250157 27019 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.251017 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.252379 26981 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.254932 22704 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:17.255106 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.230808 27061 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:17.274161 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.279879 22778 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:17.255669 26788 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:17.256963 27220 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:17.257841 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.238955 22632 rpcz_store.cc:276] 0709 13:11:16.090588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.090904 (+   316us) service_pool.cc:224] Handling call
0709 13:11:17.224623 (+1133719us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b540003b1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:17.224764 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153762944}
I20260709 13:11:17.245128 27064 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:17.265614 22628 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:17.288776 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2033) took 1105 ms. Trace:
I20260709 13:11:17.265664 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:17.271023 22601 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.268515 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.258922 27107 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:17.289251 22628 rpcz_store.cc:276] 0709 13:11:16.183131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.187486 (+  4355us) service_pool.cc:224] Handling call
0709 13:11:17.265077 (+1077591us) spinlock_profiling.cc:243] Waited 994 us on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:17.265332 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.288342 (+ 23010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.288689 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316032}
I20260709 13:11:17.296116 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.291731 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:17.297091 22628 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:17.298686 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.269956 27267 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:17.301156 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.281682 27147 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:17.303951 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.287678 27220 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:17.310163 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52396.1us
I20260709 13:11:17.297811 27290 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:17.306133 27019 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:11:17.317941 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53540.9us
I20260709 13:11:17.305984 27294 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:17.306696 27106 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:17.327235 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.327805 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 639) took 1128 ms. Trace:
I20260709 13:11:17.334282 22779 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:11:17.305506 27293 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.310177 27220 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:17.318842 27255 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:17.344184 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:17.344959 22631 rpcz_store.cc:276] 0709 13:11:16.199474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.524965 (+325491us) service_pool.cc:224] Handling call
0709 13:11:17.327508 (+802543us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b540006b8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:17.327687 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174049152}
I20260709 13:11:17.298862 26788 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.356261 27248 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:17.357537 22779 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [6.4-10.6]
W20260709 13:11:17.358429 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 247710us
W20260709 13:11:17.382541 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:17.383625 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56549.9us
I20260709 13:11:17.385620 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2779) took 1053 ms. Trace:
I20260709 13:11:17.353288 27019 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:17.357407 27147 raft_consensus.cc:2764] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:11:17.387055 22776 rpcz_store.cc:276] 0709 13:11:16.332728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.255931 (+923203us) service_pool.cc:224] Handling call
0709 13:11:17.256221 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.256436 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:17.257263 (+   827us) log.cc:844] Serialized 14 byte log entry
0709 13:11:17.385083 (+127820us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:17.385176 (+    93us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:17.385245 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.385350 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:17.385538 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":156,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":262,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":120,"thread_start_us":127145,"threads_started":1}
I20260709 13:11:17.393401 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.359442 27255 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:17.389307 22750 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.987s (SENT)
W20260709 13:11:17.394143 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50074us
I20260709 13:11:17.400805 22600 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:17.355654 27290 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.408617 22776 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:17.411355 22776 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:17.394459 27294 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:17.382419 27248 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.415921 27267 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:17.430191 27293 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.435233 27255 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.437315 27255 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:17.439335 27019 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.445623 26788 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.451051 27253 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:17.451610 27253 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.451977 27253 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:17.453258 22775 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:17.454555 27264 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:11:17.455297 27264 consensus_queue.cc:1243] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:11:17.455749 27297 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:17.456120 27297 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:17.457438 27278 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.457058 27297 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.458487 22778 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:17.457998 27278 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.411530 27317 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:17.465653 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81247us
W20260709 13:11:17.439378 27319 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:11:17.470072 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 80855.7us
I20260709 13:11:17.472657 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.476559 22635 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:17.477411 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 637) took 1279 ms (client timeout 1593 ms). Trace:
I20260709 13:11:17.477576 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:17.478186 22635 raft_consensus.cc:606] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:17.477862 22626 rpcz_store.cc:269] 0709 13:11:16.193403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.200455 (+  7052us) service_pool.cc:224] Handling call
0709 13:11:17.472809 (+1272354us) spinlock_profiling.cc:243] Waited 810 ms on lock 0x7b5400071054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:17.472947 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862427520}
I20260709 13:11:17.474921 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.467128 27317 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.484284 27317 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:17.485545 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.493707 27292 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.493995 27278 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:17.494472 27292 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.495644 27267 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:17.496181 27267 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:17.496415 27248 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.496615 27267 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:17.431352 27294 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.473327 27073 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:17.506531 27073 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.508116 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:17.508846 22631 raft_consensus.cc:606] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.510514 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.511816 27147 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:17.512229 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:17.513990 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:17.515897 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:17.516618 23416 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:17.514098 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:17.517992 23420 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:17.519723 22675 leader_election.cc:400] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8.
I20260709 13:11:17.520309 22675 leader_election.cc:403] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 13:11:17.522497 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.522656 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.526893 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:17.528021 22778 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:17.496791 27292 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.542747 23414 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:17.542856 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:17.543635 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.543779 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:17.557027 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.556689 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:17.556864 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.560309 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.563272 22598 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:17.563526 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:17.563885 22598 leader_election.cc:403] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:11:17.566128 26264 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
W20260709 13:11:17.568455 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.541621 27073 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.559859 27292 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:17.566757 27147 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:17.542155 27289 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:11:17.544312 27248 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:11:17.577250 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:17.569610 27273 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:17.577280 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.567961 27138 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:17.587930 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.589660 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.601205 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:17.604087 23416 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.605258 22776 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:17.606828 27256 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:17.608440 27253 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.609001 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:17.611061 27284 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:17.611202 27253 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.611445 27292 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:17.601984 22631 raft_consensus.cc:606] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.612630 27292 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:17.613802 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.573280 27139 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:17.616417 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:17.616696 27139 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:17.601008 27138 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:17.617905 27138 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:17.618083 27139 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.570729 27285 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:11:17.620851 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:17.622108 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:17.588949 27289 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:17.623332 27248 consensus_queue.cc:1243] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:11:17.638701 22778 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:17.640581 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:17.626859 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.634862 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:17.626974 27256 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:17.643112 22627 raft_consensus.cc:606] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.575961 27302 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:11:17.592461 27267 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.635842 27253 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:11:17.650643 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.645991 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:17.635931 27289 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:17.653079 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:17.639681 27284 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:17.644104 27256 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.660243 27283 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:17.647097 27248 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:11:17.692238 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51732.3us
I20260709 13:11:17.694485 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:17.666708 27293 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:17.663403 26906 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:17.702464 27330 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:11:17.704808 26969 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:17.708912 27184 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:17.711261 22777 raft_consensus.cc:2368] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:17.669765 27253 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:17.717728 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:17.682443 27256 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:17.717273 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:17.734179 23417 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:17.644780 27317 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.680198 27283 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:17.738348 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:17.717651 27163 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:17.758832 27163 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:17.759518 27163 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:17.759778 27061 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.760710 26968 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.761258 27061 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:17.763224 27283 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:17.749301 27338 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:17.772557 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:17.783010 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 1986) took 1144 ms. Trace:
I20260709 13:11:17.783541 22781 rpcz_store.cc:276] 0709 13:11:16.634010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.764522 (+130512us) service_pool.cc:224] Handling call
0709 13:11:17.778105 (+1013583us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:17.778299 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228837632}
I20260709 13:11:17.732918 27293 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:17.780277 27184 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:17.773949 27338 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:17.818763 27338 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.820250 27297 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:17.802147 27302 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:11:17.821421 27253 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:17.813410 27339 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 13:11:17.826156 22750 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.538s (SENT)
I20260709 13:11:17.827389 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:17.833838 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:17.729365 26906 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:17.850453 27334 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:11:17.851575 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.853826 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.856714 22709 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:17.854506 22702 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:17.860966 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.861142 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.861789 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.855846 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:17.866348 22707 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:17.809199 27184 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:17.880121 22710 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:17.862041 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.767664 27289 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.820255 27338 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:17.892993 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:17.847939 27284 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:17.860605 27269 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:17.883700 27187 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:17.911906 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 104615us
I20260709 13:11:17.912925 22706 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:17.905121 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:17.887499 27184 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:11:17.893680 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 195395us
W20260709 13:11:17.913317 22750 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.575s (SENT)
I20260709 13:11:17.913354 22706 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:17.919349 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:17.921756 22702 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:17.921976 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.889814 27147 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8.
I20260709 13:11:17.941969 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.886094 27019 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:17.945132 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:17.946317 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 170973us
W20260709 13:11:17.946797 23385 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.542s (SENT)
I20260709 13:11:17.908080 27267 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:17.919890 27289 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:17.912636 27187 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:17.967125 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:17.952075 22706 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:17.944434 27264 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:17.972987 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59733.1us
W20260709 13:11:17.982378 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:17.989212 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.568s (SENT)
I20260709 13:11:17.991243 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:17.954538 27269 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:18.007308 22706 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:17.969103 27187 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:18.012967 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57267.4us
W20260709 13:11:18.014884 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69808.9us
I20260709 13:11:18.022539 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:18.023643 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:17.908423 27293 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:17.955240 27255 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:11:18.034850 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.041712 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:17.992290 27267 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.035029 22709 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:18.006352 27317 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.046643 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.046039 22701 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:11:18.104372 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44123.8us
W20260709 13:11:18.041229 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.658s (SENT)
W20260709 13:11:18.106496 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.106540 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.105266 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:18.033078 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43885.2us
W20260709 13:11:18.109261 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.711s (SENT)
I20260709 13:11:18.110774 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.111605 22709 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:18.035099 27292 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.112713 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:18.027537 27220 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.002s
I20260709 13:11:18.024411 27343 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:18.034935 27291 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:18.127825 22709 raft_consensus.cc:2368] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8.
W20260709 13:11:18.129982 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.103381 22701 pending_rounds.cc:85] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:11:18.131750 22701 pending_rounds.cc:107] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:11:18.132622 22701 raft_consensus.cc:2889] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:18.134502 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93308.6us
W20260709 13:11:18.135739 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.766s (SENT)
I20260709 13:11:18.141304 26264 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:18.103839 27290 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:18.143266 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.147342 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.148101 22707 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:18.148319 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.149161 22704 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:18.024861 27254 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.278s
I20260709 13:11:18.154783 27272 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:11:18.155318 27345 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:11:18.155484 27272 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:18.159313 27341 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:11:18.160121 27276 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:18.162600 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.165328 27073 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:18.165961 27073 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.168340 27073 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:18.168455 27147 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:18.168967 27147 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:18.173816 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2550) took 1191 ms. Trace:
I20260709 13:11:18.173995 27290 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:18.174561 27290 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:18.129212 27269 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:18.154650 23412 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:18.141927 26264 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.035636 27071 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:18.047188 27346 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.186266 22706 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:11:18.155948 27272 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:18.192057 22706 pending_rounds.cc:85] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:18.194973 22706 pending_rounds.cc:107] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 13:11:18.193725 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.195959 22706 raft_consensus.cc:2889] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:18.137835 27343 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:18.106135 27267 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.174312 22632 rpcz_store.cc:276] 0709 13:11:16.982761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.305858 (+323097us) service_pool.cc:224] Handling call
0709 13:11:17.306328 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.391143 (+ 84815us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:17.420993 (+ 29850us) log.cc:844] Serialized 14 byte log entry
0709 13:11:17.474266 (+ 53273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:17.474325 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:17.474382 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.173560 (+699178us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b54000b86d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:18.173620 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:18.173753 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12095,"prepare.queue_time_us":24938,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":968,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":11642,"spinlock_wait_cycles":194386176,"thread_start_us":52859,"threads_started":1,"wal-append.queue_time_us":177207}
I20260709 13:11:18.042999 27242 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.149s
I20260709 13:11:18.182701 26906 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:18.206184 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70483.8us
I20260709 13:11:18.136602 27220 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.038838 27284 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.201385 27163 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.219381 26264 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.169991 27147 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.202867 27343 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:18.224066 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.230687 22702 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:18.241526 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.242690 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.234401 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
W20260709 13:11:18.237581 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.826s (SENT)
I20260709 13:11:18.134932 27292 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.244510 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.242281 22701 raft_consensus.cc:2376] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:11:18.244029 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:18.249505 22628 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:18.245707 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.263901 22703 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:18.266865 22778 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:18.196110 27071 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.277141 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.274997 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.278955 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.280441 22706 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:11:18.289216 23385 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.798s (SENT)
I20260709 13:11:18.289815 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.292905 27292 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.293666 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.298833 27128 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:18.300016 27292 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:18.301677 27220 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.263499 27317 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:11:18.305147 27267 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.306531 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:18.307394 23420 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:18.298879 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.308831 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:18.309808 27267 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:18.310205 27267 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:18.310545 27267 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:18.314109 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 92515.4us
I20260709 13:11:18.310220 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.315724 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.317479 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.318349 22707 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:18.319123 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.319624 22707 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:11:18.320348 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.321380 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.322099 22707 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:11:18.324972 22781 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:11:18.325366 22781 pending_rounds.cc:85] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 5
I20260709 13:11:18.325560 22781 pending_rounds.cc:107] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:11:18.325770 22781 raft_consensus.cc:2889] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:18.328235 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.198220 27346 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.258654 27107 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:11:18.286765 27283 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.004s
I20260709 13:11:18.343199 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.343962 22701 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:18.345769 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.345907 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.346514 22784 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:18.339286 27138 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.004s
I20260709 13:11:18.357537 27220 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.358242 27107 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:18.371022 27267 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.369532 22784 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:18.371868 27107 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.369103 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.399561 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.402096 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:18.352707 27256 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 13:11:18.349545 27283 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.408170 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.443223 22707 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:18.444406 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.445133 22707 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:11:18.411185 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.445343 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2548) took 1463 ms. Trace:
I20260709 13:11:18.383231 27220 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.432497 27346 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.439788 27071 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.441202 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:18.379722 27138 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.450508 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.452152 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:18.446349 22634 rpcz_store.cc:276] 0709 13:11:16.982193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.300304 (+318111us) service_pool.cc:224] Handling call
0709 13:11:17.300634 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.300865 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:17.315903 (+ 15038us) log.cc:844] Serialized 14 byte log entry
0709 13:11:18.172030 (+856127us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b1800444b80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:11:18.444945 (+272915us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:18.445005 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:18.445057 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.445145 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:18.445281 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14818,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":99,"spinlock_wait_cycles":8038784,"thread_start_us":272569,"threads_started":1}
I20260709 13:11:18.448148 27252 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:18.466002 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.466817 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:18.455605 27297 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.635s	user 0.013s	sys 0.000s
I20260709 13:11:18.483363 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.484010 22704 raft_consensus.cc:2376] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:11:18.473513 22703 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:18.477866 27184 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 13:11:18.487959 22630 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:11:18.488451 22630 pending_rounds.cc:85] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 9
I20260709 13:11:18.488674 22630 pending_rounds.cc:107] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:11:18.488903 22630 raft_consensus.cc:2889] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:18.480173 27138 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:18.491174 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 163848us
I20260709 13:11:18.482522 22707 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:18.487316 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.492846 22704 raft_consensus.cc:2376] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
I20260709 13:11:18.480569 27283 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.496508 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:18.446820 27256 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:18.490244 27019 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.008s
W20260709 13:11:18.497573 27338 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.000s
W20260709 13:11:18.509811 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.860s (SENT)
I20260709 13:11:18.518190 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.522485 22710 raft_consensus.cc:2368] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10.
I20260709 13:11:18.516945 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:18.521965 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.537107 22779 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:18.514307 27019 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:18.543190 22702 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:11:18.490244 27184 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:18.560539 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1916148589999693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 13:11:18.561096 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1817148589999533s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 13:11:18.561393 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1259148589998631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:11:18.561700 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1259148589998631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16010000000000002s for a response
W20260709 13:11:18.562002 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1079148589999477s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16210000000000002s for a response
W20260709 13:11:18.562300 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1061148589999448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:11:18.562592 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0620148589998735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:11:18.563056 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0098148589999028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 13:11:18.563548 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62111.4us
I20260709 13:11:18.564436 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:18.565019 22750 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.958s (SENT)
I20260709 13:11:18.565351 27284 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:18.565572 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.567040 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:18.571952 27284 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:18.566602 27128 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:18.514948 27290 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.551824 27273 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:18.583811 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2036) took 1615 ms. Trace:
I20260709 13:11:18.585144 22633 rpcz_store.cc:276] 0709 13:11:16.968549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.968751 (+   202us) service_pool.cc:224] Handling call
0709 13:11:16.969088 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.201532 (+232444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.583517 (+1381985us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54002f5654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:18.583739 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533208960}
I20260709 13:11:18.586541 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.542222 27187 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.591634 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:18.592410 23419 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:18.593180 22706 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:18.581696 27267 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.598157 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:18.602661 27356 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:11:18.606559 27352 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:11:18.608741 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.599644 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.611248 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.612161 22706 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:11:18.613620 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48633us
I20260709 13:11:18.591818 27343 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.617053 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.626127 22705 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
W20260709 13:11:18.619038 27358 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:11:18.619632 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:18.619956 27360 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:11:18.629112 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:18.623041 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 987) took 1112 ms. Trace:
W20260709 13:11:18.623677 27289 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.003s
I20260709 13:11:18.623945 22628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2551) took 1640 ms. Trace:
W20260709 13:11:18.632203 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:18.624872 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:18.633495 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.627578 22706 raft_consensus.cc:2376] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13. Re-sending same reply.
I20260709 13:11:18.631435 22628 rpcz_store.cc:276] 0709 13:11:16.983066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.307208 (+324142us) service_pool.cc:224] Handling call
0709 13:11:17.308165 (+   957us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.249964 (+941799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.623700 (+373736us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b540022c754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:18.623879 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164401408}
I20260709 13:11:18.624068 27220 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.636272 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.648051 22709 raft_consensus.cc:2368] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:18.638131 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.648756 22634 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:18.639088 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2788) took 1467 ms. Trace:
I20260709 13:11:18.640488 22627 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:18.641752 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.652637 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.653254 22705 raft_consensus.cc:2376] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7. Re-sending same reply.
I20260709 13:11:18.643882 23417 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:11:18.654042 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:18.644167 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.655542 22776 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:18.645787 22778 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:18.646134 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.656929 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.622922 27256 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.631232 23412 rpcz_store.cc:276] 0709 13:11:17.511060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.517540 (+  6480us) service_pool.cc:224] Handling call
0709 13:11:17.558051 (+ 40511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.179565 (+621514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.622744 (+443179us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54005e2254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:18.622965 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1370984832}
I20260709 13:11:18.654081 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.658500 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:18.659040 22705 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:18.659178 22634 raft_consensus.cc:2368] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:18.660336 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.660478 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.661669 22705 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:18.662177 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.662587 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.663777 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.664561 22705 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:11:18.664649 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.650143 22777 rpcz_store.cc:276] 0709 13:11:17.171305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.204886 (+ 33581us) service_pool.cc:224] Handling call
0709 13:11:18.638880 (+1433994us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b5400186fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:18.639027 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153146240}
I20260709 13:11:18.666639 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.667300 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.668658 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.666651 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.670143 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:18.641145 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:18.671417 22706 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:18.676049 27278 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:18.677745 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.678948 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.679715 22775 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:18.681782 27220 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:18.682543 27220 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:18.683710 27220 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.686111 27264 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.686865 27264 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.687299 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.689649 27264 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.635123 27284 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:18.673203 27338 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:18.641232 27252 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.692413 27338 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.692868 27252 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.695389 27252 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.696760 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:18.696960 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 63379.9us
I20260709 13:11:18.699218 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.690510 27272 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:18.703775 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:18.704934 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.701308 22633 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:18.705678 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:18.707784 22627 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:18.708765 23419 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:18.711057 22635 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 13:11:18.711562 22635 pending_rounds.cc:85] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 11
I20260709 13:11:18.711808 22635 pending_rounds.cc:107] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:11:18.712049 22635 raft_consensus.cc:2889] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:18.712560 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.713407 27128 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:18.723210 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.732489 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.735538 22705 raft_consensus.cc:2410] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:11:18.737097 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.727837 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:18.738606 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:18.749480 22628 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:18.742300 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.743507 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.666832 27071 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.747752 22632 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:11:18.750541 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 153653us
I20260709 13:11:18.714123 27276 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:18.756974 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:18.751912 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:18.771827 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.750567 22706 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:18.776114 22703 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:11:18.777285 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:18.777848 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:18.781790 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:18.750214 27338 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.758091 27284 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:18.788944 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:18.789958 22703 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:18.800674 22703 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:11:18.800978 22705 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:18.802757 22703 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:18.775784 27139 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.798436 22701 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:18.783989 27353 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:11:18.822052 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.771341 27071 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.800174 27338 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:18.790777 27184 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.836489 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:18.837041 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 124593us
I20260709 13:11:18.828488 23419 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:18.803526 27284 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:18.839507 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:18.849193 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.811616 27290 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.815431 27289 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:11:18.876205 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98987.1us
I20260709 13:11:18.811028 27291 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:11:18.810954 27264 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:11:18.844691 27187 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.825907 27019 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:11:18.886121 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:18.818661 27139 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.881562 27184 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:18.894256 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:18.906097 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 230813us
W20260709 13:11:18.909662 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 71524us
I20260709 13:11:18.913164 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.880864 27367 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:18.916013 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:18.917969 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:18.905444 27343 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.919189 27272 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:18.939960 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.940833 22779 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:18.917304 27184 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:18.943120 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.943953 22784 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:18.943573 27184 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.944545 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:18.945703 22704 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:11:18.945976 22775 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:18.947081 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.949638 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.950345 22778 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:18.951128 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2787) took 1817 ms. Trace:
W20260709 13:11:18.953550 27344 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.933s	user 0.005s	sys 0.000s
I20260709 13:11:18.956980 27071 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:18.960950 27139 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:18.927928 27284 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.962481 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:18.950390 27269 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:18.951514 22780 rpcz_store.cc:276] 0709 13:11:17.133351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.455073 (+321722us) service_pool.cc:224] Handling call
0709 13:11:17.778416 (+323343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:17.778658 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:17.778698 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:17.778770 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:17.778821 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:17.778911 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:18.950966 (+1172055us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54003417d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:18.951071 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742665472}
W20260709 13:11:18.976464 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:18.976764 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.980792 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.978232 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:18.921136 27128 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:18.967103 27367 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:18.982239 22778 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:18.993187 27372 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:18.994045 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.991043 27367 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:18.995764 27367 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:18.974110 27064 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.997506 27064 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:18.993101 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:18.994323 27224 raft_consensus.cc:993] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:19.000262 27355 raft_consensus.cc:1081] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:18.919577 27276 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:11:19.007371 22781 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:11:18.976663 27353 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:18.980509 27139 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:19.002539 27355 raft_consensus.cc:993] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:18.983904 26981 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:18.986541 27362 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.413s	user 0.003s	sys 0.003s
I20260709 13:11:19.001184 27073 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:19.008083 27338 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:18.983596 27272 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:11:18.991751 27220 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:11:19.050467 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132598us
W20260709 13:11:19.059463 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.063531 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:19.065399 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.043992 26981 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:19.078500 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 184356us
W20260709 13:11:19.047377 27300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.623s	user 0.011s	sys 0.019s
I20260709 13:11:19.017658 27317 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:11:19.080883 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:19.039825 27073 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:19.086848 22776 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:11:19.047003 27289 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:19.088194 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:19.090039 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 174155us
I20260709 13:11:19.090817 22776 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
W20260709 13:11:19.082504 27300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.658s	user 0.011s	sys 0.019s
I20260709 13:11:19.095752 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:19.035769 27139 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.083783 27369 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:19.096764 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:19.100816 27289 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.095310 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:19.113059 22775 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:11:19.118242 27353 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:19.118773 27353 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:19.119306 27353 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:19.113567 22632 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:19.114024 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:19.121260 22629 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:19.121506 22775 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:19.114593 22783 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:11:19.092165 27273 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:19.115597 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 95686.3us
I20260709 13:11:19.088922 27019 raft_consensus.cc:1081] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:19.110394 27107 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.057432 27338 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.091235 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:19.095021 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:19.078207 27276 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:11:19.125697 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:19.087280 27224 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:11:19.118862 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.108031 27364 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:11:19.135383 27064 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:19.142107 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.142858 22600 leader_election.cc:400] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:19.148897 27139 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.152513 22779 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:19.123786 27273 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:19.158067 27266 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:19.160578 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69468.3us
I20260709 13:11:19.157140 22780 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:19.157449 27107 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.171195 27139 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:19.171356 27338 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:19.185026 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66258.6us
I20260709 13:11:19.164909 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.175740 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:19.154814 27367 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.248924 27367 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:19.188782 26981 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:19.174871 27266 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.250914 27266 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.253212 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2568) took 1469 ms. Trace:
I20260709 13:11:19.253666 22630 rpcz_store.cc:276] 0709 13:11:17.784156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.784351 (+   195us) service_pool.cc:224] Handling call
0709 13:11:17.784692 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:18.489345 (+704653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.252893 (+763548us) spinlock_profiling.cc:243] Waited 703 ms on lock 0x7b540006ccd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:19.253139 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1616538368}
I20260709 13:11:19.194835 27139 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.254925 27139 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.204807 22780 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:19.209265 27147 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:19.269377 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:19.269994 23419 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:19.270442 23421 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:19.274736 23420 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:19.278141 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:19.210057 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.287806 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.213414 27107 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.213934 27289 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:19.293455 27184 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:19.294102 27184 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:19.219262 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43636.7us
I20260709 13:11:19.297197 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:19.298112 22776 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:19.300104 27276 raft_consensus.cc:2764] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:19.302192 27184 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.303260 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:19.304200 22710 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:11:19.304232 27184 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:19.224766 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.302359 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:19.309630 27338 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:19.310374 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:19.311692 27379 raft_consensus.cc:2764] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:11:19.312366 27269 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:19.317283 27369 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.318068 27369 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.304863 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2405) took 1154 ms. Trace:
I20260709 13:11:19.175468 27273 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.230415 27128 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:19.319249 22710 rpcz_store.cc:276] 0709 13:11:18.150926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.619293 (+468367us) service_pool.cc:224] Handling call
0709 13:11:19.302949 (+683656us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:11:19.304784 (+  1835us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20310400}
I20260709 13:11:19.320240 27128 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:19.321019 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:19.239014 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:19.323287 22708 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:19.205996 27073 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.326509 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2399) took 1284 ms. Trace:
I20260709 13:11:19.327126 22709 rpcz_store.cc:276] 0709 13:11:18.042497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.142958 (+100461us) service_pool.cc:224] Handling call
0709 13:11:19.326236 (+1183278us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b540010cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:19.326428 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1158802176}
I20260709 13:11:19.327666 27073 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.328296 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:19.328934 22709 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:19.239662 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:19.330870 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2433) took 1090 ms. Trace:
I20260709 13:11:19.330930 22706 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:19.331307 22707 rpcz_store.cc:276] 0709 13:11:18.240673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.498933 (+258260us) service_pool.cc:224] Handling call
0709 13:11:19.239497 (+740564us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:11:19.330792 (+ 91295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16966016}
I20260709 13:11:19.333300 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2404) took 1193 ms. Trace:
I20260709 13:11:19.333699 22706 rpcz_store.cc:276] 0709 13:11:18.139768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.775741 (+635973us) service_pool.cc:224] Handling call
0709 13:11:19.330724 (+554983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:19.331450 (+   726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.333088 (+  1638us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b540010cf50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:19.333240 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275584384}
I20260709 13:11:19.335562 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:19.240021 27163 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.242015 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2409) took 1089 ms. Trace:
I20260709 13:11:19.337028 22702 rpcz_store.cc:276] 0709 13:11:18.152389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.242383 (+ 89994us) service_pool.cc:224] Handling call
0709 13:11:19.241753 (+999370us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b5400316354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:19.241938 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688255744}
I20260709 13:11:19.337950 27253 raft_consensus.cc:993] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:19.242861 22703 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:11:19.338505 22703 pending_rounds.cc:85] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 3
I20260709 13:11:19.338783 22703 pending_rounds.cc:107] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:11:19.338827 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:19.339109 22703 raft_consensus.cc:2889] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:19.339241 22702 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:19.174070 27374 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:19.220083 26906 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:19.340163 26906 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:19.340654 26906 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:19.342044 22702 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:11:19.321635 27128 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:19.245739 27365 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 13:11:19.343950 27128 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:19.344703 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 981) took 2775 ms. Trace:
I20260709 13:11:19.345124 23415 rpcz_store.cc:276] 0709 13:11:16.568932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:16.569163 (+   231us) service_pool.cc:224] Handling call
0709 13:11:16.569689 (+   526us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:16.681326 (+111637us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:16.682285 (+   959us) log.cc:844] Serialized 14 byte log entry
0709 13:11:16.972785 (+290500us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b18000bffe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:11:17.078318 (+105533us) log.cc:844] Serialized 14 byte log entry
0709 13:11:17.146503 (+ 68185us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b18000bffe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:18.321848 (+1175345us) spinlock_profiling.cc:243] Waited 6.16 ms on lock 0x7b18000bffe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:18.457865 (+136017us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b18000bffe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:18.593231 (+135366us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:18.593913 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 13:11:18.808064 (+214151us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:18.808126 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:18.808415 (+   289us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:19.245107 (+436692us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:19.344426 (+ 99319us) raft_consensus.cc:1685] finished
0709 13:11:19.344463 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:19.344611 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13715,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":3803,"raft.queue_time_us":150509,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":11914,"spinlock_wait_cycles":193153152,"thread_start_us":546522,"threads_started":4,"wal-append.queue_time_us":645801,"wal-append.run_cpu_time_us":1694,"wal-append.run_wall_time_us":1001714}
I20260709 13:11:19.343106 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2860) took 1121 ms. Trace:
I20260709 13:11:19.347973 22703 rpcz_store.cc:276] 0709 13:11:18.221361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:18.803611 (+582250us) service_pool.cc:224] Handling call
0709 13:11:19.242407 (+438796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:19.341948 (+ 99541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.342885 (+   937us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b5400316350. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:19.343046 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999403264}
I20260709 13:11:19.336020 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:19.351222 22706 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:19.334436 22707 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:11:19.353468 27367 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:19.354233 27367 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.354115 27379 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:19.355517 27379 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:19.356004 27379 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:19.357198 27379 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.359052 27367 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:19.361110 27379 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:19.364308 27297 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.351809 22706 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:19.374588 27253 raft_consensus.cc:1081] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:11:19.375903 27269 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:19.380089 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:19.311350 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86677us
W20260709 13:11:19.382642 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.338009 27369 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:19.385752 26906 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.388933 27061 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.389492 27061 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.390817 27252 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:19.391235 27252 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.391548 27252 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:19.395208 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:19.396764 27107 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:19.397342 27107 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:19.398382 27107 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.382412 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:19.403447 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.404256 22709 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:19.404309 22705 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:11:19.406905 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:19.410393 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 245563us
W20260709 13:11:19.412081 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.413120 22776 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:19.405203 27256 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.417945 27256 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.405526 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2872) took 1044 ms. Trace:
I20260709 13:11:19.420550 22705 rpcz_store.cc:276] 0709 13:11:18.360839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.185044 (+824205us) service_pool.cc:224] Handling call
0709 13:11:19.185485 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:19.404989 (+219504us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:19.405437 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:19.426429 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:19.442672 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:19.444695 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:19.231950 27353 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.457489 22632 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:19.453253 27139 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:19.468148 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.472021 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 62726.6us
I20260709 13:11:19.473155 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:19.475709 23385 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.656s (SENT)
I20260709 13:11:19.476421 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:19.481545 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.481663 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 69882.5us
I20260709 13:11:19.487339 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:19.491807 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 109516us
I20260709 13:11:19.456410 26906 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:19.462376 27374 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:19.496276 27061 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:19.501639 27266 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.439380 27267 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:19.508265 22675 leader_election.cc:400] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:19.531797 22675 leader_election.cc:403] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:11:19.510726 22750 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.961s (SENT)
I20260709 13:11:19.534230 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:19.511705 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.740s (SENT)
I20260709 13:11:19.537706 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:19.539022 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 64702.7us
W20260709 13:11:19.539045 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45949.6us
I20260709 13:11:19.540565 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.543105 27128 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:19.486603 26981 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:11:19.536055 27147 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:19.546031 27147 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:19.551739 27297 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:19.533854 27303 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:11:19.554656 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 459383us
I20260709 13:11:19.488471 27289 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:19.506476 27317 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:11:19.557130 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45484.9us
I20260709 13:11:19.562175 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:19.553085 27303 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.564218 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:19.557911 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:19.481223 27269 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:11:19.573071 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.576264 22635 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:19.506204 27272 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:11:19.562934 26981 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:11:19.590059 26981 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:19.562240 27317 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.592432 27317 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:19.565922 27364 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:19.600703 22706 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.602756 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:19.612916 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.539018 27267 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.603927 23414 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:19.618067 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.624786 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:19.627118 23420 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:11:19.656790 22628 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:19.586772 27284 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:11:19.670362 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 128151us
I20260709 13:11:19.671769 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.590579 26981 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:19.682121 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:19.680243 27256 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:19.680850 27147 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:19.694031 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.659859 27224 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:19.700836 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:19.702116 27256 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:19.702840 27256 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:19.703325 27256 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:19.704602 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 196386us
I20260709 13:11:19.664002 27378 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:11:19.718562 22706 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:19.703208 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 220028us
W20260709 13:11:19.701015 27355 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.033s	sys 0.038s
W20260709 13:11:19.710374 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.669921 27364 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:11:19.712630 22629 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:11:19.682502 26981 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:19.746881 26981 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:19.734355 27355 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.034s	sys 0.038s
I20260709 13:11:19.732393 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:19.751185 22532 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 12 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:19.757200 27353 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.707055 27071 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:19.762228 27071 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:19.763651 27071 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.768393 27139 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.771253 27266 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:19.771483 27379 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:19.772123 27379 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:19.773063 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:19.773280 27379 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.776157 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:19.778517 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.707042 27224 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:19.694765 27291 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:11:19.794893 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62663.3us
W20260709 13:11:19.796270 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57632.9us
I20260709 13:11:19.797168 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:19.759423 27303 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:19.812529 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 301896us
W20260709 13:11:19.800323 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 237515us
I20260709 13:11:19.778524 27379 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:19.814446 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.970s (SENT)
I20260709 13:11:19.815268 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:19.817111 22675 leader_election.cc:400] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10.
I20260709 13:11:19.795873 27224 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:19.826898 27269 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:19.827725 27269 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.819772 27266 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:19.805105 27287 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:19.833576 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:19.836998 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 249409us
I20260709 13:11:19.839226 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:19.863358 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53688.7us
I20260709 13:11:19.865746 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:19.838438 27073 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.002s
I20260709 13:11:19.807179 27276 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:19.868131 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71067.5us
I20260709 13:11:19.970341 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:19.837469 27287 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:19.967860 27303 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:19.896252 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:19.994714 27303 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:19.964753 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 95464.7us
I20260709 13:11:19.995337 27303 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:19.845400 27252 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.963351 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:19.987069 27267 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:19.868247 22532 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:19.997248 27276 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:19.998334 27273 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.633s	user 0.004s	sys 0.008s
I20260709 13:11:19.998895 27273 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:19.999025 22532 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 9 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 12 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:20.000908 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:20.001528 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.003707 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:19.964255 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127703us
I20260709 13:11:19.991675 27353 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:11:19.993960 27287 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:19.990823 27387 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 13:11:20.007460 22776 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
W20260709 13:11:20.010339 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47106.9us
I20260709 13:11:20.027640 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:20.032315 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:11:20.033038 22782 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.033409 22782 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:20.050221 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:20.005645 27364 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:11:20.085670 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 242677us
W20260709 13:11:20.090895 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 269273us
I20260709 13:11:20.014135 27107 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:20.090226 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:20.000945 27276 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:20.098062 22784 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:11:20.098666 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.037s (SENT)
W20260709 13:11:20.100664 22528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 235) took 43373 ms (client timeout 15000 ms). Trace:
I20260709 13:11:20.108620 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:20.112421 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98665.9us
I20260709 13:11:20.112721 22784 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:11:20.113497 22784 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:20.113898 22784 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:20.049196 27147 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:20.108224 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:20.117681 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 90149.7us
I20260709 13:11:20.118827 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:20.128917 27284 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:20.129568 27284 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.130043 27284 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:20.111266 22781 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:20.132966 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:20.135813 27386 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:20.136351 27266 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:20.138036 27386 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:11:20.138316 22750 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.066s (SENT)
I20260709 13:11:20.138797 27353 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.140094 27064 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:20.140882 26981 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:20.140529 22784 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.145802 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:20.146596 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:20.173666 22628 raft_consensus.cc:2368] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:20.147408 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:20.173470 22532 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 10, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 10 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:20.021536 27355 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.085s
I20260709 13:11:20.135288 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:20.128975 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.154349 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:20.159348 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:20.171141 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 3.160s (SENT)
I20260709 13:11:20.177243 22781 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:20.187928 22780 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:11:20.191293 22532 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 7, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:20.196211 27369 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:11:20.237201 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.149910 27405 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:20.209527 22782 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.179371 27273 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:20.172756 27253 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
W20260709 13:11:20.303407 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 270255us
I20260709 13:11:20.153883 27353 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.312304 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:20.107604 22528 rpcz_store.cc:269] 0709 13:10:36.727362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.772251 (+ 44889us) service_pool.cc:224] Handling call
0709 13:10:37.751881 (+979630us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:11:20.100450 (+42348569us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:16.804626 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:16.895388 (+ 90762us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:16.895452 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:11:16.895497 (+    45us) tablet.cc:662] Decoding operations
0709 13:11:17.090270 (+194773us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:17.090407 (+   137us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:17.090445 (+    38us) tablet.cc:685] Acquiring locks for 32 operations
0709 13:11:17.175056 (+ 84611us) tablet.cc:701] Row locks acquired
0709 13:11:17.175106 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:11:17.175328 (+   222us) write_op.cc:270] Start()
0709 13:11:17.175503 (+   175us) write_op.cc:276] Timestamp: P: 1783602677175250 usec, L: 0
0709 13:11:17.175540 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:11:17.176542 (+  1002us) log.cc:844] Serialized 12558 byte log entry
0709 13:11:17.463308 (+286766us) op_driver.cc:464] REPLICATION: finished
0709 13:11:17.633126 (+169818us) write_op.cc:301] APPLY: starting
0709 13:11:17.633241 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:17.820993 (+187752us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:17.823469 (+  2476us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:18.616750 (+793281us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:18.617267 (+   517us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=0,mrs_lookups=0
0709 13:11:18.617344 (+    77us) write_op.cc:312] APPLY: finished
0709 13:11:18.618016 (+   672us) log.cc:844] Serialized 276 byte log entry
0709 13:11:18.665419 (+ 47403us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:18.665876 (+   457us) write_op.cc:454] Released schema lock
0709 13:11:18.720229 (+ 54353us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:18.763168 (+ 42939us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:18.763218 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2739910,"reported_tablets":47,"spinlock_wait_cycles":66447616,"child_traces":[["op",{"apply.queue_time_us":157999,"apply.run_cpu_time_us":68772,"apply.run_wall_time_us":1130003,"cfile_cache_hit":130,"cfile_cache_hit_bytes":138143,"num_ops":32,"prepare.queue_time_us":162098,"prepare.run_cpu_time_us":20360,"prepare.run_wall_time_us":472974,"raft.queue_time_us":73101,"raft.run_cpu_time_us":42660,"raft.run_wall_time_us":156392,"replication_time_us":237187,"spinlock_wait_cycles":26368,"thread_start_us":270942,"threads_started":5,"wal-append.queue_time_us":189689,"wal-append.run_cpu_time_us":16089,"wal-append.run_wall_time_us":2074804}]]}
I20260709 13:11:20.305132 22782 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:20.306274 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 167988us
W20260709 13:11:20.317153 22750 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.089s (SENT)
I20260709 13:11:20.284421 27147 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:20.274451 27269 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:20.322000 22784 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:20.316167 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:20.240792 27220 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 13:11:20.297608 27405 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:20.330957 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.311105 27264 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:20.331813 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:20.364127 27252 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:20.370863 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53740.3us
I20260709 13:11:20.325842 27147 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:20.392277 22750 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.180s (SENT)
W20260709 13:11:20.393615 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61933.8us
I20260709 13:11:20.394120 22780 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:11:20.330186 27387 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:20.451815 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:20.401448 27317 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.808s	user 0.013s	sys 0.006s
W20260709 13:11:20.463100 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70867.5us
W20260709 13:11:20.464603 22750 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.210s (SENT)
I20260709 13:11:20.465307 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:20.467175 22750 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.097s (SENT)
I20260709 13:11:20.467975 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:20.333348 27297 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:20.335937 27364 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:20.332690 27272 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
W20260709 13:11:20.484439 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 283993us
I20260709 13:11:20.406524 27264 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:20.424862 27365 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.094s	user 0.002s	sys 0.010s
W20260709 13:11:20.402913 27400 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.301s	user 0.009s	sys 0.003s
W20260709 13:11:20.419127 27407 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:11:20.435781 27367 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.046s	user 0.014s	sys 0.009s
W20260709 13:11:20.499771 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.501780 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:20.514098 22629 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:20.517026 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:20.518113 22781 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:11:20.521203 22784 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:20.401340 27252 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:20.399770 27147 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:20.503993 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:11:20.525524 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58399.1us
W20260709 13:11:20.530400 22750 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.190s (SENT)
I20260709 13:11:20.531075 22750 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:20.469534 27276 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:20.514518 22629 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:20.469342 26981 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:20.546063 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:20.547055 22632 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:20.547461 22632 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:20.454453 27107 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:11:20.561463 22632 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.568944 22632 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:20.514098 27398 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:20.523769 27252 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:20.573227 22629 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.585563 22629 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:20.586735 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.476054 27317 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:20.604315 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73972.5us
I20260709 13:11:20.587158 27252 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:20.538699 27367 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.608057 27395 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.622692 27424 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:20.636994 27403 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.326s	user 0.002s	sys 0.000s
I20260709 13:11:20.556398 27405 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.627741 27264 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.641161 27395 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.725260 27428 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.726052 27428 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:20.726454 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:20.642247 26981 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.729054 27428 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:20.729796 27428 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:20.690431 27429 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:20.741520 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:20.749701 27402 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 13:11:20.607438 27364 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:20.702064 27408 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
W20260709 13:11:20.715148 27416 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:11:20.766129 27367 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:20.771384 27420 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:11:20.757977 27256 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.053s	user 0.016s	sys 0.001s
I20260709 13:11:20.650117 27353 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:20.715080 27379 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.895s	user 0.014s	sys 0.000s
W20260709 13:11:20.714346 27287 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.682s	user 0.008s	sys 0.004s
W20260709 13:11:20.714206 27224 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.002s
W20260709 13:11:20.714063 27411 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.387s	user 0.001s	sys 0.002s
I20260709 13:11:20.731880 27429 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:20.768597 27423 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:11:20.769479 27426 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 13:11:20.777474 27412 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:11:20.777266 27417 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.704276 27433 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.780449 27256 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:20.753942 27284 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.003s
I20260709 13:11:20.781976 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 647) took 1379 ms. Trace:
W20260709 13:11:20.774724 23385 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.173s (SENT)
W20260709 13:11:20.782943 27303 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.787s	user 0.005s	sys 0.010s
I20260709 13:11:20.783067 27395 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:20.817744 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.000s	sys 0.008s
I20260709 13:11:20.694413 27424 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:20.843276 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.000s	sys 0.008s
W20260709 13:11:20.844100 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 684932us
I20260709 13:11:20.846979 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:20.820231 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 334856us
I20260709 13:11:20.797291 27353 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.849856 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 830) took 1087 ms. Trace:
I20260709 13:11:20.842332 27284 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.850040 27353 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.850325 23415 rpcz_store.cc:276] 0709 13:11:19.762331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.762497 (+   166us) service_pool.cc:224] Handling call
0709 13:11:19.763006 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.763438 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:19.763475 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:19.764057 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.011042 (+246985us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:20.011092 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.011163 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.849559 (+838396us) raft_consensus.cc:1685] finished
0709 13:11:20.849596 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.849773 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1360,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":246531,"threads_started":1,"wal-append.queue_time_us":510716}
I20260709 13:11:20.851992 27429 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:20.852353 27353 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:20.823840 22634 rpcz_store.cc:276] 0709 13:11:19.403134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.405652 (+  2518us) service_pool.cc:224] Handling call
0709 13:11:19.406134 (+   482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.446604 (+ 40470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:19.446662 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:19.531635 (+ 84973us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.022277 (+490642us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:20.022318 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.022394 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.781733 (+759339us) raft_consensus.cc:1685] finished
0709 13:11:20.781759 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.781908 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50788,"raft.queue_time_us":32610,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":15751,"thread_start_us":441436,"threads_started":1,"wal-append.queue_time_us":606614}
I20260709 13:11:20.837831 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:20.853660 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:20.857983 22706 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:11:20.857924 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83243.9us
I20260709 13:11:20.808844 27287 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.865065 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:20.812196 27224 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.868364 22708 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:20.847848 27446 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:20.852679 27284 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:20.730311 27428 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.842670 27303 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:20.824026 27417 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.887126 23418 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1011) took 1284 ms. Trace:
I20260709 13:11:20.823531 27433 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.887568 23418 rpcz_store.cc:276] 0709 13:11:19.602698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.625704 (+ 23006us) service_pool.cc:224] Handling call
0709 13:11:19.674854 (+ 49150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.721259 (+ 46405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:19.721292 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:19.775239 (+ 53947us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.298111 (+522872us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:20.298175 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.298277 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:20.886841 (+588564us) raft_consensus.cc:1685] finished
0709 13:11:20.886865 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:20.887056 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2363,"raft.queue_time_us":46499,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10981,"thread_start_us":434703,"threads_started":1,"wal-append.queue_time_us":573521}
I20260709 13:11:20.889619 27433 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:20.891323 27256 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:20.897120 22629 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:20.900055 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:20.886114 27424 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.011618 23415 raft_consensus.cc:2393] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:20.905825 22781 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:20.907279 27303 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:20.908743 27438 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:11:20.909621 22634 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
W20260709 13:11:20.855458 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 332345us
I20260709 13:11:20.920883 27417 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:20.923496 27224 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:20.926855 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.004s	sys 0.008s
W20260709 13:11:21.014798 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.004s	sys 0.008s
W20260709 13:11:21.015575 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 880330us
I20260709 13:11:20.937862 27450 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:20.939530 27252 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.025620 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:20.941499 27428 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.036239 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:20.941248 27147 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:20.877722 27446 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.070565 22704 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:11:21.072661 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.073592 22704 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:20.947811 22632 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:20.962038 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:20.965152 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.079260 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:11:21.080890 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:20.966948 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:20.967311 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:20.967883 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:20.905031 27287 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.018455 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.020180 22710 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:21.021608 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:21.022699 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 146413us
I20260709 13:11:21.023666 22749 leader_election.cc:400] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
W20260709 13:11:21.024317 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.084717 22709 raft_consensus.cc:2368] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11.
W20260709 13:11:21.034536 27272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.015s
W20260709 13:11:21.111899 27272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.000s	sys 0.017s
I20260709 13:11:21.113368 22749 leader_election.cc:403] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:11:21.078353 22705 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:21.113389 22708 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:21.088085 22707 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:21.117614 22708 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:21.025424 27276 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:21.034097 27413 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:21.122987 22709 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:21.126255 27267 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:21.127573 27267 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:21.129798 27253 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:21.130626 27253 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.134464 27064 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.135408 27064 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.137979 27272 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.138723 27272 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.139947 27064 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.141279 27272 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.143067 27253 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.147255 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:21.152522 27406 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.913s	user 0.013s	sys 0.002s
I20260709 13:11:21.152729 22702 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:21.038483 27450 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.155153 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2868) took 2825 ms. Trace:
W20260709 13:11:21.155151 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.117s (SENT)
I20260709 13:11:21.163604 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:21.164249 22709 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:21.165865 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:21.162825 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:21.170603 23415 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:21.150442 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2826) took 1133 ms. Trace:
I20260709 13:11:21.173887 22710 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:21.087149 27147 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:21.175846 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:21.193252 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.194801 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:21.195266 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.906s (SENT) [suppressed 35 similar messages]
I20260709 13:11:21.178035 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:21.197898 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:21.162881 22701 rpcz_store.cc:276] 0709 13:11:18.329615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.126808 (+797193us) service_pool.cc:224] Handling call
0709 13:11:19.160978 (+ 34170us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:19.172323 (+ 11345us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:11:19.622802 (+450479us) log.cc:844] Serialized 14 byte log entry
0709 13:11:20.034431 (+411629us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:20.049663 (+ 15232us) log.cc:844] Serialized 26 byte log entry
0709 13:11:20.049818 (+   155us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:20.049864 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:20.049942 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:21.151624 (+1101682us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:21.154836 (+  3212us) raft_consensus.cc:1685] finished
0709 13:11:21.154949 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.155090 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12637,"prepare.queue_time_us":601398,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":45,"raft.queue_time_us":20644,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":22045,"spinlock_wait_cycles":90315008,"thread_start_us":745744,"threads_started":2,"wal-append.queue_time_us":498455}
I20260709 13:11:21.171768 22777 rpcz_store.cc:276] 0709 13:11:20.017183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:20.017311 (+   128us) service_pool.cc:224] Handling call
0709 13:11:20.017702 (+   391us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:11:20.045551 (+ 27849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:20.045588 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:11:20.102605 (+ 57017us) log.cc:844] Serialized 78 byte log entry
0709 13:11:20.331160 (+228555us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:20.561305 (+230145us) log.cc:844] Serialized 14 byte log entry
0709 13:11:20.603185 (+ 41880us) log.cc:844] Serialized 14 byte log entry
0709 13:11:21.146905 (+543720us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b180045c700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:21.149554 (+  2649us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b180045c700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:21.149851 (+   297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.149974 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:21.150059 (+    85us) raft_consensus.cc:1685] finished
0709 13:11:21.150080 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.150343 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75915,"prepare.queue_time_us":312711,"prepare.run_cpu_time_us":174,"prepare.run_wall_time_us":4939,"raft.queue_time_us":588370,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":24504,"spinlock_wait_cycles":13480704,"thread_start_us":388703,"threads_started":2,"wal-append.queue_time_us":348791}
W20260709 13:11:21.202792 22676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.045s	sys 0.037s
W20260709 13:11:21.203444 22676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.046s	sys 0.037s
W20260709 13:11:21.209117 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.211519 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:21.217504 22704 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:21.226827 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.216413 27433 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:21.231442 27433 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.213551 27256 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.235612 27256 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.219666 27284 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:21.239392 27284 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.249862 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.251729 22710 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:21.214768 27424 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.258024 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2827) took 1228 ms. Trace:
I20260709 13:11:21.257750 27424 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.261207 27424 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.263710 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
W20260709 13:11:21.266724 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 243211us
I20260709 13:11:21.267416 27276 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.267887 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:21.268015 27276 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.269192 27367 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:21.268433 27369 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:21.280781 27369 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.283195 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.276355 27442 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:21.273922 27266 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:21.258443 22776 rpcz_store.cc:276] 0709 13:11:20.029508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:20.030199 (+   691us) service_pool.cc:224] Handling call
0709 13:11:20.030702 (+   503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:20.031259 (+   557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:20.031301 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:20.057395 (+ 26094us) log.cc:844] Serialized 52 byte log entry
0709 13:11:20.404684 (+347289us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:21.014661 (+609977us) log.cc:844] Serialized 14 byte log entry
0709 13:11:21.110390 (+ 95729us) spinlock_profiling.cc:243] Waited 7.25 ms on lock 0x7b180045c880. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:21.257516 (+147126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.257605 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:21.257651 (+    46us) raft_consensus.cc:1685] finished
0709 13:11:21.257745 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:21.257929 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242484,"prepare.queue_time_us":210225,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":5295,"raft.queue_time_us":253799,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":50344,"spinlock_wait_cycles":16668032,"thread_start_us":751354,"threads_started":2,"wal-append.queue_time_us":419308}
I20260709 13:11:21.277892 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.278920 27446 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.279383 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.283375 27369 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:21.289822 27399 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.280s	user 0.008s	sys 0.007s
W20260709 13:11:21.277171 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 122064us
I20260709 13:11:21.288326 27276 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.294231 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.303611 22782 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:21.294888 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 100715us
I20260709 13:11:21.295763 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.305351 22776 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:21.296885 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.297860 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.307030 22778 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:21.299875 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.308219 22779 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:21.309896 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.311914 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.313318 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.313974 22702 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:21.247284 27433 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.322476 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.324647 22704 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:21.325834 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:21.323235 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54432.5us
I20260709 13:11:21.327208 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.328361 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:21.329586 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.264246 27256 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:21.160383 27267 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.336025 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:21.353361 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.354595 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.355327 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:21.359126 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.360145 22704 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:21.292995 27273 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:21.350076 22750 connection.cc:256] RPC call timeout handler was delayed by -1.4657583439999371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:11:21.306097 27446 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:21.363758 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.365101 22782 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:21.370163 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 59107.6us
W20260709 13:11:21.369244 22750 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.831s (SENT)
I20260709 13:11:21.315099 27284 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.371248 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.345199 22776 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:21.337177 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.378965 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.376677 22704 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:21.382516 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.383737 22779 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:21.384785 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.388494 22703 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:21.388892 22704 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:21.389401 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.346757 27369 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:11:21.392194 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:21.384994 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91515.9us
I20260709 13:11:21.394047 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.385164 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:21.394444 22780 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:11:21.395344 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.397992 22633 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:21.400563 22631 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:21.400717 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.403311 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.404706 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.405668 22626 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:21.378458 26906 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:21.413084 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.409555 22626 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:21.385927 27267 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:21.414578 22630 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:21.426205 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.430949 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:21.432111 22710 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:11:21.432988 22710 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:21.434816 22702 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:21.368733 27273 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.438256 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.400264 27442 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:21.442437 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:21.441808 22626 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:21.455355 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86150.9us
W20260709 13:11:21.458976 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.882s (SENT)
W20260709 13:11:21.459836 22750 connection.cc:256] RPC call timeout handler was delayed by -1.220958343999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1996s for a response
W20260709 13:11:21.460203 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1708119840000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2938s for a response
W20260709 13:11:21.460526 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1388455479999493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
W20260709 13:11:21.460829 22750 connection.cc:256] RPC call timeout handler was delayed by -1.07656938599996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.40599999999999997s for a response
W20260709 13:11:21.461123 22750 connection.cc:256] RPC call timeout handler was delayed by -1.0688583439999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21650000000000003s for a response
I20260709 13:11:21.407752 27278 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:21.464529 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 46527.2us
I20260709 13:11:21.463732 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.463691 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.425240 27303 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:11:21.485184 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.488134 26906 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:21.495697 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:21.495570 26906 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.496353 22632 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:21.496675 22632 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:21.497870 22632 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.499177 22632 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:21.388696 27446 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.429335 27450 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:21.534052 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 43561.9us
I20260709 13:11:21.542986 27434 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.543671 27434 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.545819 27273 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.546022 27434 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.546836 27435 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.532282 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.547274 22704 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:21.557076 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.557780 22704 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:21.559264 27278 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:21.547689 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:21.548645 22750 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.810s (SENT)
I20260709 13:11:21.561020 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.509590 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.561829 22710 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:21.559623 27440 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.562533 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.563032 27440 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.564075 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.565233 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:21.561378 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:21.567411 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.569167 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.569588 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.539968 22632 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:21.574465 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.575917 22784 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:21.579160 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.581561 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.583595 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.585044 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.586340 27289 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.587153 27289 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:21.590221 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:21.591252 22708 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:11:21.593212 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.590111 27289 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.595187 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.547437 27273 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.597448 27273 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.600003 27273 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.603458 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.604604 22778 raft_consensus.cc:2410] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:11:21.605553 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.501580 27276 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.492015 27369 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:21.608436 22776 raft_consensus.cc:2410] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:11:21.608729 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66122.3us
I20260709 13:11:21.610677 22780 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:21.612912 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.613498 22777 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:21.614984 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:21.621563 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:21.622601 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.623355 22710 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:21.624938 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:21.554476 27403 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:11:21.632476 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.627782 22778 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:21.627259 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.628746 22705 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:21.628850 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.637152 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:21.629537 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:21.652060 27455 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:11:21.641342 27434 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:21.658828 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.630963 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:21.644064 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 218241us
I20260709 13:11:21.661548 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:21.662408 22628 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:21.643923 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.664352 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:21.632871 27256 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.666261 22779 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:21.645862 22750 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:21.667811 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:21.700428 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54635us
I20260709 13:11:21.700882 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.702795 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.676754 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.677590 22784 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:21.705406 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.690884 22704 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:21.639683 27289 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:11:21.643739 27291 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:21.642825 27432 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.649801 27379 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:21.634029 27276 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.712597 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.713588 22780 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
I20260709 13:11:21.691568 27440 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:21.728998 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.730161 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:21.740676 22782 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:21.602319 27435 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.731655 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:21.713040 27442 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:21.718542 22779 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:21.690876 27256 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.741112 22780 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:21.817497 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:21.798122 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68072.6us
W20260709 13:11:21.798290 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 316390us
I20260709 13:11:21.733304 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.823505 22780 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:11:21.825652 22780 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:11:21.809120 22778 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
W20260709 13:11:21.829955 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94975.5us
I20260709 13:11:21.831043 22779 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:11:21.839018 22784 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:21.738801 27220 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:11:21.816376 26981 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:21.738111 27071 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 13:11:21.841514 27464 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:11:21.825338 27434 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.736392 27369 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.902875 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 652) took 1032 ms. Trace:
I20260709 13:11:21.908179 22634 rpcz_store.cc:276] 0709 13:11:20.870444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:20.876072 (+  5628us) service_pool.cc:224] Handling call
0709 13:11:20.876649 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.013335 (+136686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.902454 (+889119us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b540003b1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:21.902793 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75260160}
I20260709 13:11:21.849186 27483 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:11:21.942852 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 203412us
I20260709 13:11:21.932235 26906 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.900527 27269 raft_consensus.cc:2764] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:11:21.948593 27439 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:21.949283 27446 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:21.895818 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.913272 26981 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:21.913929 27289 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:21.911847 27484 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:21.903486 27434 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:21.935451 27483 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:22.020231 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 124843us
W20260709 13:11:22.021162 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 287929us
W20260709 13:11:22.025974 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:22.026225 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.017287 26981 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:21.942219 27452 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 13:11:22.108901 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.016178 27289 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:22.108434 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.052299 27442 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:22.142493 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:21.963738 26906 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.145493 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.149122 22627 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:11:22.151997 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:22.152757 22778 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:11:22.138929 26981 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:22.144236 27267 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.713s	user 0.013s	sys 0.002s
I20260709 13:11:21.719656 27432 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.124751 27364 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:22.144102 27442 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:22.158864 22675 connection.cc:256] RPC call timeout handler was delayed by -1.4261898060000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 13:11:22.014920 27439 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.143430 27367 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:22.191491 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.071316 27484 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:22.179039 22675 connection.cc:256] RPC call timeout handler was delayed by -1.4027898060001007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16040000000000001s for a response
I20260709 13:11:22.181532 22778 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:22.187322 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 365402us
I20260709 13:11:22.190966 22779 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:22.205303 22783 raft_consensus.cc:2368] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14.
W20260709 13:11:22.202387 22675 connection.cc:256] RPC call timeout handler was delayed by -1.4000898060000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
W20260709 13:11:22.212850 22675 connection.cc:256] RPC call timeout handler was delayed by -1.357789806000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
W20260709 13:11:22.213474 22675 connection.cc:256] RPC call timeout handler was delayed by -1.320889806000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 13:11:22.213773 22675 connection.cc:256] RPC call timeout handler was delayed by -1.2614898060000996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
I20260709 13:11:22.188022 27442 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:22.215168 27267 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.245366 27355 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:22.246026 27355 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:22.247380 27481 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.227s	user 0.000s	sys 0.000s
I20260709 13:11:22.248458 27432 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.249017 26906 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.244971 27267 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.187552 27276 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:22.257815 27480 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:11:22.274310 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:22.279758 27482 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:11:22.279947 27355 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:22.240967 27470 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.399s	user 0.001s	sys 0.001s
I20260709 13:11:22.269369 22675 leader_election.cc:400] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:22.200327 27435 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.296058 22778 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:11:22.199498 27496 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:22.303637 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:22.243041 27107 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:11:22.322369 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.190215 27364 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.207304 27367 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.324975 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.332647 27267 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.302661 27276 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:22.316161 27405 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:22.197767 27289 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.187700 27483 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.329084 27496 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:22.348836 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 125159us
I20260709 13:11:22.360857 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2847) took 1079 ms. Trace:
I20260709 13:11:22.334524 27500 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:22.349692 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 107803us
W20260709 13:11:22.351127 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 243237us
I20260709 13:11:22.340445 27107 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:22.350248 27266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.005s	sys 0.033s
I20260709 13:11:22.317198 27486 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:11:22.360956 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.361407 22781 rpcz_store.cc:276] 0709 13:11:21.281682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.281798 (+   116us) service_pool.cc:224] Handling call
0709 13:11:21.282106 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:21.316690 (+ 34584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:21.316726 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:21.317309 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:11:21.665563 (+348254us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:21.665639 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:21.665755 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:22.360599 (+694844us) raft_consensus.cc:1685] finished
0709 13:11:22.360623 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:22.360788 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35591,"raft.queue_time_us":40257,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":16910,"thread_start_us":288010,"threads_started":1,"wal-append.queue_time_us":477473}
I20260709 13:11:22.357115 27405 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:22.360435 27434 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:22.375391 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88854.7us
W20260709 13:11:22.383865 22628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.005s	sys 0.013s
W20260709 13:11:22.385776 22628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.006s	sys 0.013s
I20260709 13:11:22.363173 27496 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:22.388391 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:22.372270 27483 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:11:22.378261 27447 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.016s	sys 0.017s
I20260709 13:11:22.401810 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.403753 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.404155 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.374282 27452 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:22.378343 27266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.005s	sys 0.034s
I20260709 13:11:22.405220 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.375916 27505 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:22.361219 27276 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.410735 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:22.382637 27455 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.003s	sys 0.022s
I20260709 13:11:22.414532 22632 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:11:22.396503 27289 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:22.395715 27267 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:22.418820 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.430522 27439 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.431146 27484 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.438238 27107 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.431699 27498 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:22.432606 27499 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:22.379294 27504 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:11:22.443780 27276 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.444842 27276 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:22.401432 27435 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 13:11:22.433408 27364 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.434772 27405 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.449607 27405 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:22.435756 27367 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.376611 27500 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:22.436892 27494 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:11:22.456416 27500 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:22.408730 27447 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.016s	sys 0.017s
I20260709 13:11:22.461588 27297 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:11:22.471948 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68326.4us
I20260709 13:11:22.474936 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:22.417243 27455 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.004s	sys 0.022s
W20260709 13:11:22.482836 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:22.487892 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.419237 27501 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:22.497879 27505 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:22.506851 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 142331us
I20260709 13:11:22.405202 27355 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:11:22.510291 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.434057 27507 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 13:11:22.519068 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 148941us
I20260709 13:11:22.419359 27508 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:11:22.501816 27499 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:22.543773 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:22.544014 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.526572 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:22.523595 27498 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:22.523324 27439 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:22.415673 27452 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.503576 27501 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.545079 27267 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:11:22.559234 27439 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.567951 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2914) took 1171 ms. Trace:
I20260709 13:11:22.572810 22702 rpcz_store.cc:276] 0709 13:11:21.396392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.435494 (+ 39102us) service_pool.cc:224] Handling call
0709 13:11:21.435794 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:21.436204 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:21.436232 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:21.447232 (+ 11000us) log.cc:844] Serialized 26 byte log entry
0709 13:11:22.567200 (+1119968us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b180039b8c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1
0709 13:11:22.567414 (+   214us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:22.567456 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.567541 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:22.567604 (+    63us) raft_consensus.cc:1685] finished
0709 13:11:22.567617 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:22.567858 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":17930,"spinlock_wait_cycles":5280000,"thread_start_us":35084,"threads_started":1,"wal-append.queue_time_us":118355}
I20260709 13:11:22.548055 27505 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.570092 27491 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:22.600332 22708 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:11:22.600297 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49980.9us
I20260709 13:11:22.603334 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.596719 27501 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.607154 23420 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:22.540441 27510 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:11:22.609318 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.609792 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.610760 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.612098 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.610805 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.617964 22704 raft_consensus.cc:2368] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:22.618335 22703 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:22.613337 22708 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:22.618534 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.615336 22702 pending_rounds.cc:85] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:22.619513 22702 pending_rounds.cc:107] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:11:22.619611 22709 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:22.619843 22702 raft_consensus.cc:2889] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:22.618942 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.620800 22704 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:22.622395 22705 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:22.622942 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.623665 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:22.629307 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.630338 22705 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:22.631577 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.633023 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.636299 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.636495 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.637245 22701 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
W20260709 13:11:22.637521 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113856us
I20260709 13:11:22.639732 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.626711 27267 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:22.643257 22710 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:22.644562 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.645536 22705 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:22.647118 27267 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.649662 27267 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.651469 27439 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.626314 26981 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.653170 27495 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:22.653632 27495 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:22.626292 27442 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.655750 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2026) took 1084 ms. Trace:
I20260709 13:11:22.656193 22782 rpcz_store.cc:276] 0709 13:11:21.571576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.584732 (+ 13156us) service_pool.cc:224] Handling call
0709 13:11:22.655532 (+1070800us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54003c3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:22.655679 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356200704}
I20260709 13:11:22.657291 27442 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.657538 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.660179 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2905) took 1636 ms. Trace:
I20260709 13:11:22.660670 22707 rpcz_store.cc:276] 0709 13:11:21.024149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.118473 (+ 94324us) service_pool.cc:224] Handling call
0709 13:11:22.659429 (+1540956us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:22.659616 (+   187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:22.659820 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:22.659846 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:22.659904 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:22.659934 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.659993 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:22.660111 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123554304}
I20260709 13:11:22.662817 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:22.663522 22629 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:22.627097 27491 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:22.667766 27478 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:22.668354 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.668197 27491 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.669154 22701 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:22.652529 27452 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.670332 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.671152 23420 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:22.673641 22750 leader_election.cc:400] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:22.685895 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.686244 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:22.686906 22705 raft_consensus.cc:2368] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:22.685949 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:22.687742 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.678226 22629 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:22.688082 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:22.675365 22708 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:22.683688 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.684576 22709 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:22.684506 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.690873 22710 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:22.691771 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:22.689356 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.692471 22710 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:22.701606 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:22.704761 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:22.642236 27496 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.709432 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:22.714417 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:22.711905 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.708096 27369 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:22.715607 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.704615 27276 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:22.708004 27499 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:22.752769 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:22.708251 27506 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:22.705463 27442 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:22.712009 22704 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:22.709184 27367 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:22.713290 22703 raft_consensus.cc:2410] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:22.750762 22781 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:22.727486 22701 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:11:22.754334 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44996.4us
I20260709 13:11:22.711026 27267 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:22.716418 27462 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.228s	user 0.000s	sys 0.004s
I20260709 13:11:22.771354 27529 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.774395 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.727421 26981 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.750077 27369 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.786453 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:22.790210 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.752138 27276 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.795923 27276 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:22.773607 27390 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:22.763175 27500 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:22.779816 27224 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:22.763584 27355 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:22.810673 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.812177 27439 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:22.813069 27535 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:22.812822 27439 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.817610 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.803141 27369 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:22.819634 22782 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:11:22.818547 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.805821 27435 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:22.806524 27442 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:22.815598 27224 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:22.831419 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:22.810367 27073 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:22.827716 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.794277 27297 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:11:22.834473 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.779477 27499 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.833001 27535 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:22.838213 27535 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:22.838799 27535 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:22.839669 27499 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.839820 27535 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.808133 27529 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.844288 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.803844 27267 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.848474 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:22.826489 27500 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:22.855192 27529 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.855659 27500 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.834306 27435 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.834151 27224 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:22.859102 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.819868 27355 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:22.866770 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.848948 27499 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:22.869120 27529 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:22.893289 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.893774 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.894977 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.897080 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.897643 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:22.898434 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.898856 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.900167 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.901516 27533 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:22.902125 27533 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:22.902571 27533 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:22.895709 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.906921 22631 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 16: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:22.913095 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.887256 27499 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:22.910192 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.924321 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.881012 27507 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:22.926213 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59545us
I20260709 13:11:22.926739 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.927584 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.929023 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.887933 27529 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:22.914582 27525 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:11:22.930030 22631 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:22.935379 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:22.938117 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.938921 22629 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:22.939858 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.916190 27499 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:22.919152 27496 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.917986 27355 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.943077 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:22.945313 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.946985 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:22.947908 27538 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:22.949869 27538 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:22.937000 27529 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:22.959471 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:22.962664 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:22.885147 27442 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.956082 27405 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:22.957620 27289 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:22.935293 27507 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:22.989499 27317 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:22.991102 27435 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:22.991596 27267 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:22.992453 22632 raft_consensus.cc:2410] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:11:22.995148 27500 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:23.001937 27386 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:23.002965 27386 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:23.003572 27537 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.004024 27386 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.004252 27537 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.005726 27386 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:23.007606 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.008577 27289 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.009310 27289 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.011078 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.011999 27452 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:23.012326 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.012634 27452 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.013176 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2846) took 1733 ms. Trace:
I20260709 13:11:23.015010 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.015190 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.015972 23415 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:23.017014 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.018011 23412 raft_consensus.cc:2410] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:11:23.020542 27289 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:23.008805 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.024109 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.024910 23421 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:23.025156 27139 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.025835 27139 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.013720 22775 rpcz_store.cc:276] 0709 13:11:21.280170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.280848 (+   678us) service_pool.cc:224] Handling call
0709 13:11:21.281231 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.287490 (+  6259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:21.287525 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:21.287571 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:21.287603 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.012885 (+1725282us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:23.012953 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.013109 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13831936}
I20260709 13:11:23.035182 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.036638 23415 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:23.059212 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.060731 23412 raft_consensus.cc:2410] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:11:22.973014 27439 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.068461 23415 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:23.072203 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.076814 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.077593 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.078210 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.082020 23412 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
W20260709 13:11:23.088146 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86500.5us
W20260709 13:11:23.089982 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 154723us
I20260709 13:11:23.094429 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2032) took 1494 ms. Trace:
I20260709 13:11:23.094988 22784 rpcz_store.cc:276] 0709 13:11:21.599697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.730541 (+130844us) service_pool.cc:224] Handling call
0709 13:11:23.094165 (+1363624us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54003412d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.094349 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90454912}
I20260709 13:11:23.096761 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.097422 22778 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:11:23.097978 22778 pending_rounds.cc:85] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 4
I20260709 13:11:23.098306 22778 pending_rounds.cc:107] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:11:23.098657 22778 raft_consensus.cc:2889] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:23.078536 27452 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.103041 27369 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.105970 27369 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.106038 22782 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:11:23.090161 27537 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:23.110140 23385 connection.cc:256] RPC call timeout handler was delayed by -1.5644606060000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
W20260709 13:11:23.111657 23385 connection.cc:256] RPC call timeout handler was delayed by -1.5106114489999527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:11:23.113843 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.110077 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.116151 23414 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:11:23.112264 23385 connection.cc:256] RPC call timeout handler was delayed by -1.2937114489999431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19870000000000002s for a response
I20260709 13:11:23.095358 27276 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.146358 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.223441 27284 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:23.199918 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2021) took 1631 ms. Trace:
W20260709 13:11:23.225178 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 78975.1us
I20260709 13:11:23.212896 27529 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:23.228058 27475 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.495s	user 0.005s	sys 0.006s
I20260709 13:11:23.228262 27529 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.228332 27224 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.175000 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.229933 27440 raft_consensus.cc:2764] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:11:23.225534 22776 rpcz_store.cc:276] 0709 13:11:21.568607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.568914 (+   307us) service_pool.cc:224] Handling call
0709 13:11:23.199627 (+1630713us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b5400131d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.199829 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":88064384}
W20260709 13:11:23.231577 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 56553us
I20260709 13:11:23.232741 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.169518 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.179858 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.234243 23412 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:23.170637 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.238013 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.239682 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:23.203689 27476 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.388s	user 0.002s	sys 0.000s
W20260709 13:11:23.124078 27457 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.801s	user 0.002s	sys 0.003s
I20260709 13:11:23.131088 27442 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.245155 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.147118 27413 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.247655 27440 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:11:23.135596 23421 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:11:23.263388 23416 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:23.265102 23421 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:11:23.172865 27533 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.266309 27413 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.126888 27433 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:23.254760 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.268158 23412 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:23.247279 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.125226 27139 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.147472 27545 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:23.269528 27545 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:23.208609 27490 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.270187 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.270637 27490 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.260932 22782 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:23.152209 27367 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:23.271889 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.272989 27433 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:23.273259 27490 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.273651 22775 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:23.273411 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:23.131608 27267 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 13:11:23.210510 27256 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.254201 27529 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:23.287662 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.257249 27289 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:23.294502 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.203063 27452 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
W20260709 13:11:23.260881 27291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.467s	user 0.009s	sys 0.007s
I20260709 13:11:23.282910 27284 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:23.313934 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.281436 27533 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:23.313809 27291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.571s	user 0.009s	sys 0.007s
I20260709 13:11:23.300148 27224 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:23.324609 22750 leader_election.cc:400] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11.
I20260709 13:11:23.311404 27139 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:23.260881 27442 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.330219 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.317265 27433 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:23.335029 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.320572 27529 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:23.337698 27278 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:23.337806 27269 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:23.346841 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.347940 22702 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:23.339296 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.345237 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.220611 27495 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.327976 27291 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:23.310673 27556 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:11:23.356853 22627 raft_consensus.cc:606] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.368045 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.368770 22703 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:23.337239 27433 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.373023 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:23.370817 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70597us
I20260709 13:11:23.386991 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.388091 22701 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
W20260709 13:11:23.390113 27552 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:11:23.391290 27477 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.571s	user 0.000s	sys 0.003s
I20260709 13:11:23.397452 27266 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:23.398058 27266 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:23.399367 27291 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.387305 22673 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.410279 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.366247 27435 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:23.359571 27413 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.431102 22702 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:11:23.426074 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.371078 27269 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.385315 27433 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:23.435797 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.417124 27495 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:23.441291 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.420461 27529 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:23.444104 22702 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:23.420079 27435 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:23.448182 27558 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:11:23.442762 27269 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.453203 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:23.459595 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.465853 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.466701 22702 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:23.459491 27266 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.506747 27266 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:23.507506 27252 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.461158 27435 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:23.506604 27506 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.514884 27506 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.517800 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 42) took 1214 ms. Trace:
I20260709 13:11:23.460264 27529 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:23.518275 22627 rpcz_store.cc:276] 0709 13:11:22.303285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.303481 (+   196us) service_pool.cc:224] Handling call
0709 13:11:23.517617 (+1214136us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400349c54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.517718 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371253120}
W20260709 13:11:23.527302 23420 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1020) took 2550 ms (client timeout 1525 ms). Trace:
I20260709 13:11:23.466207 27267 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:11:23.527837 23420 rpcz_store.cc:269] 0709 13:11:20.969063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:20.969235 (+   172us) service_pool.cc:224] Handling call
0709 13:11:23.519128 (+2549893us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54004e9654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.519253 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3616826752}
I20260709 13:11:23.593472 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.594175 23420 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:11:23.595885 27508 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:23.596361 27508 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:23.528836 27452 consensus_queue.cc:1243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:11:23.529733 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.597335 27508 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.598022 22701 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:23.599112 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.599263 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.599829 22701 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:23.600769 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.530658 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.598402 27508 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:23.530745 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.603148 22707 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:23.604090 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.604621 22707 raft_consensus.cc:2376] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9. Re-sending same reply.
I20260709 13:11:23.605430 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.531584 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.606899 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.607559 22702 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:23.607623 27559 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:23.608174 27559 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:23.609189 27559 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:23.548623 22777 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2845) took 2269 ms (client timeout 2022 ms). Trace:
W20260709 13:11:23.611999 22777 rpcz_store.cc:269] 0709 13:11:21.278913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.279084 (+   171us) service_pool.cc:224] Handling call
0709 13:11:23.548367 (+2269283us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400132254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.548513 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469048576}
I20260709 13:11:23.599314 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2865) took 1431 ms. Trace:
I20260709 13:11:23.533397 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.614382 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:23.615700 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.616444 22710 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:23.617272 23420 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:23.534065 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.619184 22704 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:23.620370 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.621305 22704 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:23.534369 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.621991 22705 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:23.534734 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.539028 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2860) took 1387 ms. Trace:
I20260709 13:11:23.623162 22780 rpcz_store.cc:276] 0709 13:11:22.151188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.154036 (+  2848us) service_pool.cc:224] Handling call
0709 13:11:22.657640 (+503604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:22.658057 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:22.658088 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:22.658598 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:11:22.814007 (+155409us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:11:22.814053 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:22.814132 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:22.921167 (+107035us) raft_consensus.cc:1685] finished
0709 13:11:22.921200 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.538675 (+617475us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54003c3cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.538927 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":139,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":1157219712,"thread_start_us":85128,"threads_started":1,"wal-append.queue_time_us":171469}
I20260709 13:11:23.540733 27507 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.541003 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.541325 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:23.542429 27269 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.542714 27489 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:11:23.544353 27506 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:23.545645 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:23.548530 27256 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:23.559062 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.548951 27499 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.649399 23421 raft_consensus.cc:2376] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13. Re-sending same reply.
I20260709 13:11:23.560016 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:23.651136 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63857.8us
I20260709 13:11:23.560567 27447 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:23.561241 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.576699 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.577239 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:23.581197 27498 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:23.585713 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:23.588999 27535 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.591933 27442 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:23.626420 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85526.3us
I20260709 13:11:23.671223 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:23.626969 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 68931.3us
I20260709 13:11:23.613287 22778 rpcz_store.cc:276] 0709 13:11:22.168370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.350720 (+182350us) service_pool.cc:224] Handling call
0709 13:11:23.096962 (+746242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.099296 (+  2334us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.599103 (+499807us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54003412d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.599252 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689137536}
I20260709 13:11:23.671670 22710 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:23.630158 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.674243 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.675164 22710 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:23.676285 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.678162 22710 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:23.631806 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.679034 22704 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:23.643561 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.549218 27276 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.647488 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
W20260709 13:11:23.683683 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:23.652666 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:23.655033 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93954us
W20260709 13:11:23.629524 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94913.8us
I20260709 13:11:23.685338 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.686448 22709 raft_consensus.cc:2410] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:11:23.687026 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:23.688705 22629 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:11:23.546602 27284 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.697774 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.642067 27367 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:23.642671 27364 raft_consensus.cc:2764] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
W20260709 13:11:23.702394 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:23.707078 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.706990 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.652762 27559 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:23.651068 27220 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:23.662767 27498 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:23.716324 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.721933 27545 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:23.699189 27071 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:23.722952 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:23.701615 27290 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:23.724471 27290 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.701833 27267 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.727054 27267 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.689152 27452 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:23.729164 27267 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:23.730072 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.730557 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2868) took 1426 ms. Trace:
I20260709 13:11:23.730980 22775 rpcz_store.cc:276] 0709 13:11:22.304389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.059194 (+754805us) service_pool.cc:224] Handling call
0709 13:11:23.059701 (+   507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.274836 (+215135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.730291 (+455455us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003273d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:23.730491 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491460352}
I20260709 13:11:23.731848 27502 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:23.732299 27502 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.732709 27502 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:23.735994 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.716274 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:23.737258 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:23.737715 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:23.737931 27533 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:23.738407 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 718) took 1277 ms. Trace:
I20260709 13:11:23.667414 27269 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.657390 27447 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:23.739182 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:23.737658 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.708772 22632 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:23.736199 27290 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:23.747298 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:23.738842 22776 rpcz_store.cc:276] 0709 13:11:22.461263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.231175 (+769912us) service_pool.cc:224] Handling call
0709 13:11:23.231613 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.232347 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.232390 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:23.248142 (+ 15752us) log.cc:844] Serialized 26 byte log entry
0709 13:11:23.413666 (+165524us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:23.413735 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.413842 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:23.738165 (+324323us) raft_consensus.cc:1685] finished
0709 13:11:23.738187 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.738334 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":118938,"threads_started":1,"wal-append.queue_time_us":445239}
I20260709 13:11:23.682437 27439 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:11:23.745599 22783 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2023) took 2176 ms (client timeout 1657 ms). Trace:
I20260709 13:11:23.750824 22632 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:23.751698 22750 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:23.753401 27364 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:23.753798 27273 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:23.754069 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:23.750633 22783 rpcz_store.cc:269] 0709 13:11:21.569228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.569359 (+   131us) service_pool.cc:224] Handling call
0709 13:11:23.745295 (+2175936us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54001860d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.745505 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460197760}
I20260709 13:11:23.754241 27273 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:23.748559 27369 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:23.757314 27273 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:23.668411 27256 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.757236 27545 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:23.757736 27369 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:23.758343 27369 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:23.758589 27545 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:23.759900 27256 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:23.760571 22703 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 4.
I20260709 13:11:23.761380 27278 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:23.699091 26906 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:11:23.748559 27267 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:23.754678 27276 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:23.707453 27284 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.762909 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.799901 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2898) took 2779 ms. Trace:
I20260709 13:11:23.722888 27071 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:23.722807 27564 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:23.738415 27533 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.771821 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:23.788257 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.683435 27499 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.683763 27535 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.800344 22706 rpcz_store.cc:276] 0709 13:11:21.020428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:21.165702 (+145274us) service_pool.cc:224] Handling call
0709 13:11:21.166174 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:21.166578 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:11:21.334299 (+167721us) log.cc:844] Serialized 14 byte log entry
0709 13:11:23.399165 (+2064866us) spinlock_profiling.cc:243] Waited 8.26 ms on lock 0x7b1800443e00. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:11:23.399657 (+   492us) log.cc:844] Serialized 14 byte log entry
0709 13:11:23.797879 (+398222us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:23.797955 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:23.798027 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:23.798148 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:23.799815 (+  1667us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119117,"prepare.queue_time_us":262692,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":26039,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"spinlock_wait_cycles":18988928,"thread_start_us":600002,"threads_started":3,"wal-append.queue_time_us":212306,"wal-append.run_cpu_time_us":1489,"wal-append.run_wall_time_us":1043084}
I20260709 13:11:23.803202 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.768021 27561 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:11:23.805276 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.754245 27364 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:23.809263 22701 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:23.815220 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2470) took 1201 ms. Trace:
I20260709 13:11:23.815682 22708 rpcz_store.cc:276] 0709 13:11:22.613378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.631298 (+ 17920us) service_pool.cc:224] Handling call
0709 13:11:23.815009 (+1183711us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b5400315454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:23.815149 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98856704}
W20260709 13:11:23.817127 23414 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.005s	sys 0.008s
I20260709 13:11:23.817646 22708 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:11:23.819847 22709 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:23.809058 22705 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:23.812960 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.830936 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:23.794436 27278 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.747160 27220 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.767271 27447 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.873121 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:23.806548 27353 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:23.806932 27071 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:23.755312 27269 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:23.899691 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:23.809938 27533 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:23.799736 27529 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.814097 27561 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.903970 27447 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:23.904134 27269 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:23.905745 27452 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:23.905946 27284 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:23.908862 27555 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.551s	user 0.007s	sys 0.000s
I20260709 13:11:23.900678 22709 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:23.913172 27273 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:23.877179 27278 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:23.911209 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:23.928369 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.023705 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.027305 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.807152 27564 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.929386 27561 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:24.073568 27273 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:24.089344 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:24.090843 22628 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:24.092120 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.093348 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.093947 27512 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:24.097035 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.097769 22635 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:24.100091 22775 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:24.101912 22635 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:24.094457 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:23.880229 27435 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:23.990041 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:23.814108 27364 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:23.826313 27405 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:24.013113 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:23.925549 27526 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:24.033680 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:24.036547 22633 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:24.127522 22709 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:24.128269 22775 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:11:24.105147 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.129863 27364 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:24.130811 22629 raft_consensus.cc:2410] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:11:24.130407 27526 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.135097 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.141378 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.140851 22775 raft_consensus.cc:2376] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11. Re-sending same reply.
I20260709 13:11:24.064296 27564 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:24.121587 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.126173 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:24.040371 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:24.149012 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.104583 27395 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:23.814090 27498 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.154605 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 115506us
I20260709 13:11:24.156213 27491 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:24.157629 27422 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:24.163546 22709 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:24.173031 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.188752 22775 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:24.128654 27284 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:23.987939 23419 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:24.191259 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.196573 22707 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 13:11:24.198091 22707 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:24.167827 27569 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.107823 27512 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.210752 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.034s	sys 0.049s
I20260709 13:11:24.216590 22775 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:11:24.182116 27395 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.218808 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.223101 22777 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:24.220465 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.034s	sys 0.050s
I20260709 13:11:24.227123 22777 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:24.214761 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:24.185909 23412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.013s	sys 0.039s
W20260709 13:11:24.232530 23412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.013s	sys 0.040s
I20260709 13:11:24.193887 27498 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:24.184732 27491 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:24.219110 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 481879us
W20260709 13:11:24.235728 22702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.037s
W20260709 13:11:24.213605 27266 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.706s	user 0.012s	sys 0.000s
W20260709 13:11:24.256320 22702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.008s	sys 0.037s
I20260709 13:11:24.262423 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.262722 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.263566 22784 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:24.264725 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.266806 22701 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:24.262223 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.297216 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:24.297901 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.249764 27421 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.278957 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.283783 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:24.222361 27061 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 13:11:24.200997 27422 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.303941 27512 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.307538 27435 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.308414 27499 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:24.238041 27556 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:24.236238 27569 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.276787 27526 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:24.311295 27565 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:11:24.281385 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49973.5us
W20260709 13:11:24.281478 22781 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.015s	sys 0.021s
W20260709 13:11:24.342600 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.010s	sys 0.023s
W20260709 13:11:24.348800 22781 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.015s	sys 0.021s
W20260709 13:11:24.348917 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.011s	sys 0.024s
I20260709 13:11:24.350023 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 711) took 2083 ms. Trace:
I20260709 13:11:24.350713 22781 rpcz_store.cc:276] 0709 13:11:22.266475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.846361 (+579886us) service_pool.cc:224] Handling call
0709 13:11:22.865347 (+ 18986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:22.865591 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:22.876806 (+ 11215us) log.cc:844] Serialized 14 byte log entry
0709 13:11:23.403068 (+526262us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b1800028960. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:11:24.349484 (+946416us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.349598 (+   114us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:24.349669 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.349769 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.349943 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11046,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":1344,"spinlock_wait_cycles":6441856,"thread_start_us":946065,"threads_started":1}
I20260709 13:11:24.270141 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.209882 27284 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.314318 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.352559 22777 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:24.352818 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.276525 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.246178 27107 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:24.260083 27491 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:24.266784 27432 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.266480 26906 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.221388 27071 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.304014 27395 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:24.316200 27450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.011s	sys 0.038s
I20260709 13:11:24.260649 27535 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:24.328352 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.326486 27353 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:24.348886 27422 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:24.286283 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.358199 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:24.392230 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:24.378865 27450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.011s	sys 0.038s
I20260709 13:11:24.274071 27566 raft_consensus.cc:476] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Not starting pre-election -- already a leader
I20260709 13:11:24.286981 27220 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:24.334097 27405 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:24.355050 27107 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:24.360992 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:24.412648 22704 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:11:24.414724 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.415150 22704 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:24.417737 22704 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:24.393601 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:24.375910 27512 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:24.400058 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:24.389695 27353 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:24.401692 27369 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.001s
I20260709 13:11:24.391376 27395 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.324976 27421 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.421367 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.411566 27405 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:24.419678 27107 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:24.430928 22701 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:11:24.446058 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.023s	sys 0.098s
W20260709 13:11:24.446638 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.024s	sys 0.098s
W20260709 13:11:24.447052 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 697220us
I20260709 13:11:24.447810 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:24.349968 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 595959us
W20260709 13:11:24.448571 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 55069.3us
I20260709 13:11:24.448925 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:24.427448 22706 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:24.419628 27528 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:24.380081 27432 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.411566 27422 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:24.449090 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:24.449806 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.383167 26906 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:24.480780 22708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.005s	sys 0.025s
I20260709 13:11:24.489205 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:24.491632 27071 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.692647 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.493261 27573 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:24.678157 27267 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:24.722916 27573 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.724326 26906 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:24.725735 27573 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.494059 27266 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.494817 27422 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:24.727612 27422 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.498148 22775 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:24.499171 27432 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:24.505772 27502 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.773s	user 0.006s	sys 0.005s
I20260709 13:11:24.446058 27395 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.512615 27566 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:24.513144 27500 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:24.513579 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.737290 22632 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:24.515002 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:24.452860 27367 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.737s	user 0.013s	sys 0.000s
I20260709 13:11:24.519924 27569 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:24.520939 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:24.527769 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.529886 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.748072 22776 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:24.610914 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:24.431984 27512 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.555528 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.488768 27220 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:24.592506 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.670835 22701 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
W20260709 13:11:24.672639 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.760567 22701 pending_rounds.cc:85] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:11:24.760908 22701 pending_rounds.cc:107] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:11:24.761174 22701 raft_consensus.cc:2889] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:24.760102 23411 raft_consensus.cc:2376] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16. Re-sending same reply.
W20260709 13:11:24.673434 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 224418us
I20260709 13:11:24.566226 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.435204 27369 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.763414 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:24.765280 27369 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:24.767012 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:24.556071 22708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.025s
I20260709 13:11:24.768471 27499 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:24.768872 27499 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:24.769225 27499 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:24.769519 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.771042 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2933) took 1180 ms. Trace:
I20260709 13:11:24.771437 22708 rpcz_store.cc:276] 0709 13:11:23.590391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.828431 (+238040us) service_pool.cc:224] Handling call
0709 13:11:23.828753 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:23.829046 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:23.829346 (+   300us) log.cc:844] Serialized 14 byte log entry
0709 13:11:24.770701 (+941355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:24.770759 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:24.770803 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.770867 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:24.770982 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":20363,"raft.queue_time_us":134,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":938616,"threads_started":1}
I20260709 13:11:24.773135 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.529929 27284 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:24.677735 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.679306 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:24.695961 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 206841us
I20260709 13:11:24.779522 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.780400 22630 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:24.735437 27566 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:24.699769 27561 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.000s
I20260709 13:11:24.701298 27353 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:24.703925 23417 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46262 (request call id 744) took 1693 ms (client timeout 1827 ms). Trace:
W20260709 13:11:24.787209 23417 rpcz_store.cc:269] 0709 13:11:23.010551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.010726 (+   175us) service_pool.cc:224] Handling call
0709 13:11:24.703669 (+1692943us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400354b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:24.703831 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3400914304}
I20260709 13:11:24.704129 27421 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:24.789391 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.790385 27561 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.692101 27559 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
W20260709 13:11:24.716095 27364 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.005s
I20260709 13:11:24.796319 27500 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:24.809159 27364 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.810012 27500 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:24.811939 27364 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.716531 27564 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.718222 27435 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:24.815359 27435 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:24.816777 27435 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.720817 27533 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.003s
I20260709 13:11:24.819010 27533 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.721457 27447 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.714s	user 0.006s	sys 0.011s
W20260709 13:11:24.721894 22532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 145) took 29524 ms (client timeout 15000 ms). Trace:
I20260709 13:11:24.741425 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.766819 27569 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:24.531472 27528 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:24.855172 22710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.014s	sys 0.023s
W20260709 13:11:24.855628 22710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.014s	sys 0.023s
I20260709 13:11:24.857226 23420 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:24.857827 22776 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:24.777412 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.858978 22632 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:24.697299 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:24.691987 27278 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.590s	user 0.007s	sys 0.023s
I20260709 13:11:24.757123 27367 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:24.795223 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:24.732915 27266 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.799826 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.802536 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:24.921842 27266 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:24.796280 27284 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:24.820987 22627 raft_consensus.cc:606] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:24.788764 27566 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:24.735685 27502 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:24.941898 27491 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.559s	user 0.007s	sys 0.005s
W20260709 13:11:24.847873 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:24.976161 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:24.975674 22778 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:24.792276 27433 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 1.347s	user 0.011s	sys 0.005s
I20260709 13:11:24.776301 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:24.937422 27267 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.980907 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:24.981932 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 205711us
I20260709 13:11:24.867079 23414 raft_consensus.cc:1240] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:24.869498 22783 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:24.913160 22704 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:24.928790 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 133694us
I20260709 13:11:24.935213 22627 raft_consensus.cc:614] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:24.927193 22709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.018s	sys 0.022s
I20260709 13:11:24.849617 27545 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:24.945533 27284 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.947258 27447 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.059954 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2956) took 1054 ms. Trace:
W20260709 13:11:25.065567 22707 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.000s	sys 0.068s
W20260709 13:11:25.382818 22707 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.097s	user 0.000s	sys 0.069s
W20260709 13:11:25.383411 22777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.010s	sys 0.009s
W20260709 13:11:25.384004 22777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.009s
W20260709 13:11:24.820755 22532 rpcz_store.cc:269] 0709 13:10:55.197351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.197706 (+   355us) service_pool.cc:224] Handling call
0709 13:11:24.721755 (+29524049us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:20.632151 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:20.771107 (+138956us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:20.771160 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:11:20.771180 (+    20us) tablet.cc:662] Decoding operations
0709 13:11:20.772079 (+   899us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:20.772152 (+    73us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:20.772188 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:11:20.772888 (+   700us) tablet.cc:701] Row locks acquired
0709 13:11:20.772913 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:11:20.773116 (+   203us) write_op.cc:270] Start()
0709 13:11:20.773285 (+   169us) write_op.cc:276] Timestamp: P: 1783602680773051 usec, L: 0
0709 13:11:20.773312 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:11:20.779768 (+  6456us) log.cc:844] Serialized 2116 byte log entry
0709 13:11:21.024518 (+244750us) op_driver.cc:464] REPLICATION: finished
0709 13:11:21.191298 (+166780us) write_op.cc:301] APPLY: starting
0709 13:11:21.191438 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:21.390213 (+198775us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:21.399144 (+  8931us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:21.942132 (+542988us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:21.954107 (+ 11975us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 13:11:21.954163 (+    56us) write_op.cc:312] APPLY: finished
0709 13:11:21.954507 (+   344us) log.cc:844] Serialized 57 byte log entry
0709 13:11:21.954744 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:21.954877 (+   133us) write_op.cc:454] Released schema lock
0709 13:11:21.954927 (+    50us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:21.955367 (+   440us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:21.955395 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":233008,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":150430,"apply.run_cpu_time_us":16702,"apply.run_wall_time_us":764080,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25067,"num_ops":5,"prepare.queue_time_us":155031,"prepare.run_cpu_time_us":8853,"prepare.run_wall_time_us":263916,"raft.queue_time_us":80439,"raft.run_cpu_time_us":9496,"raft.run_wall_time_us":99674,"replication_time_us":251029,"thread_start_us":364636,"threads_started":4,"wal-append.queue_time_us":136371,"wal-append.run_cpu_time_us":41465,"wal-append.run_wall_time_us":2044569}]]}
I20260709 13:11:25.384639 27447 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.386068 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.320763 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:25.327857 22600 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.830s	user 0.010s	sys 0.001s
W20260709 13:11:25.328779 22706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.005s	sys 0.012s
W20260709 13:11:24.988667 22709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.019s	sys 0.022s
W20260709 13:11:25.329300 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.401304 22775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.001s	sys 0.013s
W20260709 13:11:24.992834 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 490061us
I20260709 13:11:25.437659 27071 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.074244 22783 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:11:25.337352 22779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.039s	user 0.005s	sys 0.078s
W20260709 13:11:25.439152 22779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.005s	sys 0.079s
I20260709 13:11:25.438825 27071 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.993871 22784 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 8.
I20260709 13:11:25.137118 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.443220 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:25.444862 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.445873 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:24.844782 23415 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:25.447052 23418 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
W20260709 13:11:25.340493 23416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.000s	sys 0.021s
W20260709 13:11:25.449606 23416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.000s	sys 0.021s
W20260709 13:11:25.020112 27107 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.006s
I20260709 13:11:24.995945 27224 consensus_queue.cc:1059] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:11:25.142594 27284 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.465880 27071 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.466992 27284 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.467973 27284 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.044445 27291 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:25.471088 27071 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.471413 27291 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.472258 27071 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:25.246449 23412 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.008s	sys 0.007s
W20260709 13:11:25.474694 23412 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.008s	sys 0.007s
I20260709 13:11:24.992882 27491 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.475373 27107 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.477150 27609 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:25.274873 27499 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.505s	user 0.014s	sys 0.002s
I20260709 13:11:25.479764 27224 raft_consensus.cc:3055] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:25.479563 27499 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.480365 27224 raft_consensus.cc:740] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.023937 27267 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.481812 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2864) took 3313 ms. Trace:
I20260709 13:11:24.927012 27278 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.016336 27353 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.194453 27364 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.017479 27395 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:25.204909 27405 consensus_meta.cc:287] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.751s	user 0.013s	sys 0.002s
I20260709 13:11:25.506290 22780 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:24.995049 27433 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.506227 27364 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.509362 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2882) took 1322 ms. Trace:
I20260709 13:11:25.509804 27433 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.479833 27569 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.511242 27569 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.511951 22626 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:11:25.511579 27405 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.513659 23421 raft_consensus.cc:1240] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:25.166038 27502 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.137315 27512 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.053205 27528 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.516937 27278 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.518795 27499 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.520823 27499 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:25.095852 27529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.017s	sys 0.055s
I20260709 13:11:25.017388 27533 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:25.530525 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 476496us
W20260709 13:11:25.531709 27529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.223s	user 0.018s	sys 0.056s
I20260709 13:11:25.496006 27291 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.024243 27545 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:24.994572 27556 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:25.534240 27556 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.534916 27556 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:25.534966 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:25.536613 27545 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.537547 27545 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:24.995299 27559 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.539542 27559 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.191560 27561 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:24.952661 27564 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.279954 27566 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.560361 22781 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
W20260709 13:11:25.169790 27580 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 13:11:25.171056 27581 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
I20260709 13:11:24.955047 27578 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:11:25.209894 27583 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.077s	user 0.001s	sys 0.001s
I20260709 13:11:25.562752 23414 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:25.188246 27582 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:11:25.482359 22779 rpcz_store.cc:276] 0709 13:11:22.168020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:22.286080 (+118060us) service_pool.cc:224] Handling call
0709 13:11:23.745441 (+1459361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.748943 (+  3502us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:24.231907 (+482964us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.440094 (+1208187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:25.441174 (+  1080us) log.cc:844] Serialized 26 byte log entry
0709 13:11:25.441517 (+   343us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:25.441563 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.441731 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.455054 (+ 13323us) raft_consensus.cc:1685] finished
0709 13:11:25.455084 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.481504 (+ 26420us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54001860d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:25.481691 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3008,"prepare.queue_time_us":669917,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":1599,"raft.queue_time_us":11808,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":352,"spinlock_wait_cycles":3355358464,"thread_start_us":1600010,"threads_started":2,"wal-append.queue_time_us":1005306}
W20260709 13:11:25.265360 27585 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:11:25.372275 22704 rpcz_store.cc:276] 0709 13:11:24.006033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.436440 (+430407us) service_pool.cc:224] Handling call
0709 13:11:24.437002 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.986438 (+549436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.055242 (+ 68804us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54000e6d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.059843 (+  4601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016780544}
I20260709 13:11:25.566740 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2093) took 1541 ms. Trace:
I20260709 13:11:25.567389 22631 rpcz_store.cc:276] 0709 13:11:24.025451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.029380 (+  3929us) service_pool.cc:224] Handling call
0709 13:11:24.029821 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:24.039905 (+ 10084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:24.039934 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:24.046786 (+  6852us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.419224 (+372438us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:24.419266 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.419339 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.565878 (+1146539us) raft_consensus.cc:1685] finished
0709 13:11:25.565941 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.566294 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":372065,"threads_started":1,"wal-append.queue_time_us":880288}
I20260709 13:11:25.568616 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2934) took 1977 ms. Trace:
W20260709 13:11:25.267397 27586 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:11:25.380566 22782 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:25.383029 27576 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:25.378706 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 251093us
W20260709 13:11:25.395337 22600 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.010s	sys 0.002s
W20260709 13:11:25.398761 22706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.006s	sys 0.012s
I20260709 13:11:25.400695 27447 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:25.370676 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.008s	sys 0.016s
I20260709 13:11:25.409412 23417 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:25.425395 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 723) took 1160 ms. Trace:
I20260709 13:11:25.412978 22526 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:25.429741 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 43) took 1889 ms. Trace:
I20260709 13:11:25.431919 27450 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:25.455492 22775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.001s	sys 0.014s
I20260709 13:11:25.550655 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:24.936964 27498 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.586669 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2941) took 1967 ms. Trace:
W20260709 13:11:25.500777 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60950.2us
I20260709 13:11:25.587188 27498 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:25.589360 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 1.09177e+06us
I20260709 13:11:25.502228 27353 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.503038 27491 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.591316 27353 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.130046 27452 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:25.595142 27452 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:25.597030 27452 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.515344 27405 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.598531 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2938) took 1935 ms. Trace:
I20260709 13:11:25.599582 27071 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.600190 27452 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:25.509891 22780 rpcz_store.cc:276] 0709 13:11:24.188943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.189122 (+   179us) service_pool.cc:224] Handling call
0709 13:11:24.189544 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.507123 (+1317579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.508999 (+  1876us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400325fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.509266 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3026676096}
I20260709 13:11:25.530705 27603 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.530504 27601 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.603503 27603 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.603808 27601 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.604506 27605 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.532498 27533 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:25.228257 27535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.004s
I20260709 13:11:25.579502 22627 rpcz_store.cc:276] 0709 13:11:23.540568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.541165 (+   597us) service_pool.cc:224] Handling call
0709 13:11:25.429038 (+1887873us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400349754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:25.429336 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2748084096}
I20260709 13:11:25.533192 22703 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:25.514407 27569 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.482163 27224 consensus_queue.cc:260] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.618263 27450 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.619939 27224 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:25.621668 27450 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.529569 27107 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.509900 27364 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.560218 27291 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:25.328584 22601 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.006s	sys 0.007s
I20260709 13:11:25.461313 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2051) took 1196 ms. Trace:
W20260709 13:11:25.581586 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.009s	sys 0.016s
I20260709 13:11:25.551841 27499 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:25.584594 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.585642 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:25.585953 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2893) took 1140 ms. Trace:
I20260709 13:11:25.597568 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2960) took 1474 ms. Trace:
I20260709 13:11:25.514140 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:25.594285 27353 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.562269 27566 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.591161 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2928) took 2002 ms. Trace:
I20260709 13:11:25.569221 22705 rpcz_store.cc:276] 0709 13:11:23.616627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.829046 (+212419us) service_pool.cc:224] Handling call
0709 13:11:23.829441 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.844135 (+ 14694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.844177 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:23.899754 (+ 55577us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.434346 (+534592us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:11:24.434423 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.434530 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.567956 (+1133426us) raft_consensus.cc:1685] finished
0709 13:11:25.568004 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.568355 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21008,"raft.queue_time_us":14300,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":24035,"thread_start_us":371604,"threads_started":1,"wal-append.queue_time_us":945024}
I20260709 13:11:25.571199 27576 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.603981 22526 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 6 to 9, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:25.599035 22709 rpcz_store.cc:276] 0709 13:11:23.618516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.190302 (+571786us) service_pool.cc:224] Handling call
0709 13:11:24.212437 (+ 22135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.212758 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:11:24.245605 (+ 32847us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.532209 (+1286604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.532377 (+   168us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:25.532489 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.533593 (+  1104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.553507 (+ 19914us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39747,"prepare.queue_time_us":83496,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":93798,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":114,"spinlock_wait_cycles":1709952,"thread_start_us":1210924,"threads_started":2,"wal-append.queue_time_us":957127}
W20260709 13:11:25.614084 22519 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 1f99ad34cc414233929e0a92f8c3d8db is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:25.622717 27533 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:25.682828 22519 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70460.7us
I20260709 13:11:25.602437 27559 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.550367 27395 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:11:25.623873 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2891) took 1245 ms. Trace:
I20260709 13:11:25.684546 22782 rpcz_store.cc:276] 0709 13:11:24.379026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.788005 (+408979us) service_pool.cc:224] Handling call
0709 13:11:24.788310 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.570192 (+781882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.623512 (+ 53320us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b5400341cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.623807 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297480576}
I20260709 13:11:25.685353 27395 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:25.685899 27395 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.625311 27267 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:25.614670 27535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.009s	sys 0.005s
I20260709 13:11:25.627533 27578 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.587229 22707 rpcz_store.cc:276] 0709 13:11:23.619446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.202236 (+582790us) service_pool.cc:224] Handling call
0709 13:11:24.202584 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.202961 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:25.383969 (+1181008us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.557904 (+173935us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.572844 (+ 14940us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:25.581248 (+  8404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.586340 (+  5092us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.586569 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89656,"prepare.queue_time_us":91794,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":112264,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":374,"thread_start_us":1208174,"threads_started":3,"wal-append.queue_time_us":154810}
I20260709 13:11:25.687757 27395 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.688688 27578 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.628755 27601 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.605252 27605 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.590413 27498 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.622238 27284 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.632314 27291 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.633848 22775 rpcz_store.cc:276] 0709 13:11:24.445805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.844927 (+399122us) service_pool.cc:224] Handling call
0709 13:11:24.845379 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.845688 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:11:24.863959 (+ 18271us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.446136 (+582177us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:25.585191 (+139055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.585305 (+   114us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:25.585485 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.585623 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.585869 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18203,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":62,"raft.queue_time_us":10049,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":182,"spinlock_wait_cycles":31060352,"thread_start_us":720599,"threads_started":1,"wal-append.queue_time_us":550978}
W20260709 13:11:25.635608 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 848327us
I20260709 13:11:25.693568 27353 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:25.694782 27278 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.695297 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:25.695788 27278 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:25.636483 22601 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.214s	user 0.006s	sys 0.007s
I20260709 13:11:25.633088 22784 rpcz_store.cc:276] 0709 13:11:24.264185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.264426 (+   241us) service_pool.cc:224] Handling call
0709 13:11:25.460699 (+1196273us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54003c3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:25.460992 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526805760}
W20260709 13:11:25.698072 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.27596e+06us
I20260709 13:11:25.637938 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 727) took 1316 ms. Trace:
W20260709 13:11:25.686832 22600 connection.cc:256] RPC call timeout handler was delayed by -1.037818199999947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 13:11:25.699337 22781 rpcz_store.cc:276] 0709 13:11:24.321314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.784375 (+463061us) service_pool.cc:224] Handling call
0709 13:11:24.784705 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.561201 (+776496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.637576 (+ 76375us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b54003264d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.637858 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782589696}
I20260709 13:11:25.638139 27499 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.700525 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:25.639406 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 2936) took 2022 ms. Trace:
W20260709 13:11:25.639814 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.703827 22703 rpcz_store.cc:276] 0709 13:11:23.617261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.106765 (+489504us) service_pool.cc:224] Handling call
0709 13:11:24.107082 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:25.622533 (+1515451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.639102 (+ 16569us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002c2ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.639341 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3278404480}
I20260709 13:11:25.705727 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:25.633908 22706 rpcz_store.cc:276] 0709 13:11:24.123999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.452026 (+328027us) service_pool.cc:224] Handling call
0709 13:11:24.452281 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.452499 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:24.503394 (+ 50895us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.597022 (+1093628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.597118 (+    96us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:25.597186 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.597282 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.597503 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50687,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":49,"raft.queue_time_us":7151,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":136,"thread_start_us":1077424,"threads_started":1,"wal-append.queue_time_us":757789}
I20260709 13:11:25.641881 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:25.641891 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:25.643013 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:25.644124 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:25.645272 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:25.649567 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 99434.2us
I20260709 13:11:25.647845 22710 rpcz_store.cc:276] 0709 13:11:23.588955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:23.820668 (+231713us) service_pool.cc:224] Handling call
0709 13:11:23.821050 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:23.971238 (+150188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:23.971296 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:24.042747 (+ 71451us) log.cc:844] Serialized 26 byte log entry
0709 13:11:24.856026 (+813279us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:24.856070 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:24.856145 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:25.571647 (+715502us) raft_consensus.cc:1685] finished
0709 13:11:25.571703 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.590937 (+ 19234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5921,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":33344,"thread_start_us":700537,"threads_started":1,"wal-append.queue_time_us":1069934}
I20260709 13:11:25.572628 22777 rpcz_store.cc:276] 0709 13:11:24.265352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.745818 (+480466us) service_pool.cc:224] Handling call
0709 13:11:24.746248 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.746536 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:11:24.778268 (+ 31732us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.424423 (+646155us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.424596 (+   173us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:25.424719 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.424904 (+   185us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.425222 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36746,"prepare.queue_time_us":32184,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":39923,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":364,"thread_start_us":605909,"threads_started":1,"wal-append.queue_time_us":657929}
I20260709 13:11:25.658558 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.659744 23411 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:25.660002 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:25.664404 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 78997.1us
I20260709 13:11:25.679793 27614 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:25.681620 27603 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.707284 22780 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:25.654310 27576 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:25.665377 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 75604.9us
W20260709 13:11:25.638733 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.720700 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:25.721634 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.722532 27266 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:25.723659 27061 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:25.724201 27061 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.724334 22779 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:25.726032 22776 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:25.726850 27512 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.727280 22778 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:25.727685 27512 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.716259 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:25.729658 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:25.730700 27071 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.730994 27491 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.731552 27071 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.731777 27491 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.735462 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.736306 27499 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:25.736788 27499 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.737179 27499 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:25.737754 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.739146 27566 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:25.739586 22706 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:25.724599 27061 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:25.734395 27071 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.743659 27512 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.735643 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:25.747745 27278 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.686403 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:25.749303 22782 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:25.751093 27447 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:25.751953 27556 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.753386 27590 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:25.753455 23414 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:25.753914 27590 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.755187 27590 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.755837 27353 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:25.756522 27353 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:25.756789 27590 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:25.757046 27353 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:25.757913 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:25.755518 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:25.759076 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:25.752274 27447 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:11:25.760756 27422 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:25.762816 27107 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:25.762863 27499 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.763473 27107 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.764896 27107 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.765931 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:25.766795 27107 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:25.717175 27614 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:25.767455 27614 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:25.770767 27395 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:25.771884 27291 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.772800 27499 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.734365 27491 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.773844 27512 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:25.776139 27491 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:25.776677 27491 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:25.776333 27512 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:25.676461 22526 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 10, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 10 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:25.700596 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.779580 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.778899 22526 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:25.780318 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:25.781266 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:25.782514 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.782398 22631 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
W20260709 13:11:25.783197 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.748479 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:25.786235 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:25.787420 27266 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:25.787961 27266 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.788473 27266 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:25.789582 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.790817 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:25.791651 23414 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:25.792040 27502 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:25.776386 27605 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.793056 27502 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:25.793578 27502 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:25.794044 27502 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:25.790047 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.795384 23419 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:25.796432 27107 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:25.797174 27107 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.799623 27533 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.748519 27578 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.777226 27491 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:25.807413 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.809257 27107 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.810685 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:25.813265 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.813985 22626 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:25.813982 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 758) took 1312 ms. Trace:
W20260709 13:11:25.835517 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50631.2us
I20260709 13:11:25.836211 27623 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:25.781198 22526 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:25.863219 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77127.4us
I20260709 13:11:25.864629 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:25.872315 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:25.875989 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:25.869313 27367 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:25.883380 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:25.824628 23412 rpcz_store.cc:276] 0709 13:11:24.501425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:24.501590 (+   165us) service_pool.cc:224] Handling call
0709 13:11:24.501948 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:24.713390 (+211442us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:25.475550 (+762160us) log.cc:844] Serialized 14 byte log entry
0709 13:11:25.572852 (+ 97302us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:25.607528 (+ 34676us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:25.649283 (+ 41755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:25.813500 (+164217us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54004f5bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:25.813569 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:25.813892 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":532542,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":1865,"raft.queue_time_us":29107,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":288,"spinlock_wait_cycles":485490432,"thread_start_us":840803,"threads_started":3,"wal-append.queue_time_us":222445}
W20260709 13:11:25.888202 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 122490us
I20260709 13:11:25.889213 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:25.894076 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:25.884279 27556 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:25.903455 22526 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 1 to 8, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:25.884570 27367 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:25.902143 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.915519 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:25.909550 22703 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:25.925578 27578 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.926931 27533 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.927001 27578 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.928709 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:25.928215 27533 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.929195 27612 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.929481 22703 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:25.929941 27612 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.931377 27533 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.877960 27061 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.934468 23411 raft_consensus.cc:2410] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:11:25.934719 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:25.941908 27061 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:25.942229 27422 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:25.942822 27422 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.944442 22526 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:25.937173 23414 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:25.912936 27367 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:25.962682 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54258.1us
I20260709 13:11:25.895179 27447 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:11:25.908000 27502 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:25.970361 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 101145us
W20260709 13:11:25.970438 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 154296us
I20260709 13:11:25.969373 27578 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:25.971177 27622 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:25.973678 27622 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.974838 27622 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.976063 27622 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:25.976866 27491 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.977807 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:25.908663 27224 consensus_queue.cc:1059] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:11:25.978930 22706 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:25.979574 27224 raft_consensus.cc:3055] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:25.961860 27533 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:25.979990 27224 raft_consensus.cc:740] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:25.980473 27533 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.981216 27224 consensus_queue.cc:260] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:25.979018 27628 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:25.966872 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:25.993772 22782 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:26.002518 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:26.006881 27621 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:26.007795 27621 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.008924 27502 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.012109 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.013008 22705 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:26.015430 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:26.015929 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.016182 22707 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:26.017997 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:25.997836 27605 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.018800 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:26.019009 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.019930 22631 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:26.018796 27512 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:26.020812 27512 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.021327 27512 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.001818 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.002684 27224 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:26.019397 27605 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.025190 27578 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:26.026072 27578 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.026692 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:26.028604 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:26.031059 27619 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:26.031673 27619 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:26.032196 27619 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:25.991380 27628 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.034399 27628 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:25.971611 22526 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.036671 27578 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.036424 22526 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:26.041301 27353 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.042796 27512 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.047194 27533 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.047827 27266 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:25.997824 27071 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:26.051613 27071 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.052066 27071 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:26.053809 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:26.055138 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 217925us
I20260709 13:11:26.051128 27612 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.049957 27452 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:26.042388 23414 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:26.064013 22707 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:26.067976 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.069300 23414 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:26.071565 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 71606us
I20260709 13:11:26.071125 23419 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
W20260709 13:11:26.070183 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 67771.5us
I20260709 13:11:26.079056 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:26.087636 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.088358 27512 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.076208 27291 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:26.095785 23421 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:26.056388 27621 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.091526 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.102082 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:26.101311 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:26.105372 23417 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.106563 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:26.066691 27614 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.110538 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.096227 27556 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:26.118526 27353 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.096912 27291 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:26.120927 27291 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:26.123730 27618 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:11:26.123535 27422 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.124878 27422 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:26.106150 27623 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:11:26.105275 27590 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:11:26.112376 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:26.130275 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 139128us
W20260709 13:11:26.132478 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.136483 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:26.131510 22709 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:26.129642 27266 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.141065 27559 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:26.142309 27559 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:26.142793 27559 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:26.143049 27610 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:26.143687 27610 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:26.143886 27559 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.145406 27559 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:26.145022 27610 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.143599 27367 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.132333 27556 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.147698 27556 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:26.127079 27435 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:26.038501 22526 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:26.150463 27435 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.130100 27614 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.150911 27435 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:26.151537 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:26.154904 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65955.5us
I20260709 13:11:26.162251 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.115159 27512 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.167230 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.122599 27291 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:26.117048 27491 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.171824 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.173148 27621 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:26.176286 23414 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:26.157635 27367 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:26.180806 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 68534.3us
I20260709 13:11:26.182813 27500 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.183569 27500 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.186223 27500 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.175884 27605 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.186966 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:26.189488 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 322858us
I20260709 13:11:26.178807 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:26.154914 27071 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.193207 22703 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:26.195537 27624 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:11:26.196184 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55977.7us
I20260709 13:11:26.178690 27452 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:11:26.189079 27491 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:26.212275 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.220630 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.202773 27500 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:26.179265 27512 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.203595 27624 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:26.206374 22675 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.598s (SENT) [suppressed 21 similar messages]
W20260709 13:11:26.239821 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.241080 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.241390 22627 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:11:26.241734 22627 pending_rounds.cc:85] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 10
W20260709 13:11:26.241933 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50603.5us
I20260709 13:11:26.242389 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.242873 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.246398 27512 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.217541 27367 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:26.248453 27367 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:26.240629 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.260207 23417 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:26.260772 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:26.262305 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86617.7us
I20260709 13:11:26.264804 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:26.268844 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.241972 22627 pending_rounds.cc:107] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 14.11
W20260709 13:11:26.269819 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.270316 22627 raft_consensus.cc:2889] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:26.268249 27491 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:26.267928 27500 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.275435 27500 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.280078 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.281080 22778 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:26.191872 22526 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.287067 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.287915 22783 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:26.288008 22526 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.288914 22782 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:26.300062 27502 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:26.300851 27502 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.300009 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.289942 22526 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.306963 22780 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:26.302007 27502 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.307775 22526 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:26.315129 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.317121 22782 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:26.325976 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.328907 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:26.329046 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.297279 27367 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.338060 27561 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:11:26.351763 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.352981 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:26.324481 27626 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.358225 22626 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:26.360191 22778 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:26.360718 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.363034 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.363813 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.364117 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.364790 22630 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:26.367007 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.367918 22782 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:26.368994 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.329872 27625 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:11:26.375121 27605 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:26.375329 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 132611us
I20260709 13:11:26.375955 27605 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.376595 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.376408 27491 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.377723 27561 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:26.334537 27266 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:26.379176 27266 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.363842 22634 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:26.387601 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.398257 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.402232 22634 raft_consensus.cc:2410] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:26.429064 27576 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:26.429672 27576 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:26.432134 27605 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.417969 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:26.441551 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57482.3us
I20260709 13:11:26.380127 27626 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.447438 22634 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:11:26.448897 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.449715 22780 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:26.397245 27452 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:26.455761 22783 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
W20260709 13:11:26.468551 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 131682us
I20260709 13:11:26.471377 27619 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.476107 27266 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.477710 27625 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:26.455972 27071 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.479461 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.480372 22780 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:26.480211 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.481588 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.481948 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:26.482352 22780 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:26.482676 22776 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:26.483008 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.363050 27624 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.484570 23419 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:26.484150 22783 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:26.504678 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.502015 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:26.381167 27628 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.515272 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.516562 22780 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:26.518905 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:26.521032 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 315384us
I20260709 13:11:26.521270 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:26.521274 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 64626.5us
I20260709 13:11:26.474465 27619 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.532256 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.525949 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.500844 27626 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.536863 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.541302 22782 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 4.
I20260709 13:11:26.546315 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.542636 22778 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:26.545766 22777 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:26.547452 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.559149 22779 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:26.531920 27452 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:26.553787 23417 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:26.510501 27071 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.452919 27576 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.565637 22780 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:26.570721 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.547549 27624 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:26.573297 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:26.576884 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.549578 27561 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:26.560483 27619 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:26.550396 27107 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:11:26.580204 27605 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:26.563469 27626 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:11:26.568647 27452 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.451491 22526 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 9, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:26.582554 27071 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.600472 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.602056 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.602603 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.602820 27621 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:26.602982 22783 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:26.599592 27071 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.604094 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.605556 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.606417 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.607021 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.607324 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.607758 22631 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:26.607962 22784 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:26.611946 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.617274 22784 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:26.617995 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.588443 27628 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:26.610015 27638 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:11:26.607798 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:26.641588 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56448.9us
I20260709 13:11:26.647086 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:26.649415 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.650578 23412 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:26.651369 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.651867 22631 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:26.661330 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:26.661695 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.592095 27626 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:26.592886 27452 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:26.664036 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:26.666438 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:26.663638 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 79478.4us
I20260709 13:11:26.668497 22782 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:26.669929 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:26.670603 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.679495 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:26.679740 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.683591 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:26.684867 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:26.685698 27502 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:26.686148 27502 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.686596 23417 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:11:26.686153 22749 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:26.687112 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.687711 27266 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:26.688098 27266 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:26.688421 27266 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:26.688452 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.689256 22635 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 6.
I20260709 13:11:26.690160 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.690342 27071 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:26.680420 27559 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.002s
I20260709 13:11:26.693270 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.686543 27502 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:26.694677 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.696081 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:26.697336 27071 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:26.686713 27590 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:11:26.699148 27061 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:26.699549 27061 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:26.699616 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 183857us
I20260709 13:11:26.700628 27061 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.701200 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:26.702109 27061 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:26.703462 27071 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:11:26.707145 27071 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:26.708496 27071 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:26.708971 27071 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.710035 27071 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.711292 27071 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.669369 27625 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:11:26.596766 22526 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.659540 27621 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:26.591405 27605 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:26.715485 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.619647 27435 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.665577 27626 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.654789 27619 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.717569 22526 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:26.739795 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:26.740091 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 207175us
I20260709 13:11:26.715813 27625 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:26.737226 27626 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:26.733700 27556 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.007s
I20260709 13:11:26.716856 27367 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:26.734961 27578 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:26.739712 27559 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:26.733976 27628 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:11:26.765121 27628 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:26.765654 27628 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.766788 27628 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.767555 27626 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:26.768172 27628 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.782725 22630 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:26.733003 27610 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:11:26.792379 27605 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.796895 27435 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.801748 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:26.802605 23412 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:26.803642 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.804440 22703 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:26.805845 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.806197 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:26.806694 22709 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:26.806934 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.806197 22710 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:26.807808 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.807085 22703 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:26.805878 22706 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:26.806558 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.810141 22704 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:26.810141 27367 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.813442 27367 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:26.814136 22526 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 2 to 10, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:26.816030 22526 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:26.818455 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.821157 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:26.824559 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.828112 22782 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:11:26.722132 27266 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:26.830525 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 160673us
I20260709 13:11:26.834915 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:26.839361 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:26.839439 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:26.840950 22627 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:26.836273 27512 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.843864 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:26.844219 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:26.844658 22710 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.845078 23419 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:26.845358 27619 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:26.847319 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:26.849632 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:26.853509 22710 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:26.751099 27556 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.854921 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.855717 22631 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:26.857241 27556 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.857439 23412 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:26.859126 23421 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 19: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:26.840267 22704 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:26.715188 27502 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:26.844115 27512 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:26.861389 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 168746us
I20260709 13:11:26.862495 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:26.863802 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.754851 27578 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.865664 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.836166 27278 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:26.860117 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:26.871156 22706 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:26.878230 27619 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:26.883404 22635 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:26.887283 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:26.863646 27625 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:26.866758 27559 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:26.882831 27528 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:26.893812 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.891283 27621 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.897995 27266 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:26.901002 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51561.2us
I20260709 13:11:26.904393 27561 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.905047 27561 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:26.869822 27578 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:26.899299 23419 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:26.868757 22630 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:26.924124 23416 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:26.912640 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:26.931608 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:26.941849 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55736.7us
I20260709 13:11:26.953545 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:26.954684 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:26.939427 27291 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:26.954581 27605 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:26.955693 27647 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:27.003700 27605 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:26.934237 27278 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:26.956859 22601 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:26.957589 22704 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:26.953788 27107 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:27.019029 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63864.8us
W20260709 13:11:27.020829 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 398167us
I20260709 13:11:26.934388 27556 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:11:27.027122 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:27.031704 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:27.032560 27605 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:26.932981 27559 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:11:26.817610 22526 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:27.039739 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.053839 22630 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:11:27.060511 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:27.060861 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 124446us
I20260709 13:11:27.061760 22630 pending_rounds.cc:85] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 7
I20260709 13:11:27.062141 22630 pending_rounds.cc:107] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:11:27.062510 22630 raft_consensus.cc:2889] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:27.065068 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:27.065744 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:27.066416 22778 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:27.064848 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:27.017508 27107 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:27.035684 27528 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:27.095232 22706 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:27.097471 22778 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 19.
W20260709 13:11:27.081458 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 247164us
I20260709 13:11:27.077612 27647 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:27.075711 27619 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:27.055033 27061 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:11:27.106993 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 75367.5us
I20260709 13:11:27.082827 27107 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.083364 27621 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:27.060935 27590 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:11:27.034698 27278 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.106988 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:27.119596 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.110810 27632 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:27.139257 27512 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:27.140820 27502 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:27.142239 27061 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:27.142578 23412 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:27.144114 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:27.145684 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:27.116015 27621 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:27.164825 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:27.170921 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 106393us
I20260709 13:11:27.145129 27395 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:27.163710 27626 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:27.187213 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 131339us
I20260709 13:11:27.179648 27395 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:27.205710 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 303843us
I20260709 13:11:27.171892 27621 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.172384 27452 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.205153 27624 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 13:11:27.189414 27605 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:11:27.261893 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.267366 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.283926 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.283558 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54059.2us
W20260709 13:11:27.287859 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 157227us
I20260709 13:11:27.164225 27512 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:27.295176 22627 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:27.295106 22526 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:27.323119 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.328382 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:27.332703 23411 raft_consensus.cc:1240] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:27.308467 27626 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:27.335001 27654 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:11:27.336653 27512 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:27.338997 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:27.338507 27395 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:27.344859 27664 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:11:27.363770 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46007.3us
I20260709 13:11:27.324020 27561 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:27.427762 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 308229us
W20260709 13:11:27.429103 27578 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.011s
I20260709 13:11:27.432128 22781 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:27.347262 27660 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:11:27.435822 22628 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:11:27.437866 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.464035 27491 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.464870 27452 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.464615 27491 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.465893 27512 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.466925 27628 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.467190 27491 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.467536 27628 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.434841 27626 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:27.463617 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:27.470510 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:27.473189 23421 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
W20260709 13:11:27.473591 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.477265 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45347.5us
I20260709 13:11:27.492552 22749 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.469708 27395 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:27.462467 27578 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:27.507208 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 74081.5us
I20260709 13:11:27.490375 27651 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:27.495878 27622 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:11:27.498394 27628 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:27.503037 27670 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:11:27.501215 27435 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:27.520362 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.531332 27452 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.534876 27452 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.537547 27452 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.484468 22526 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 8 to 12, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:27.542814 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.540697 22526 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 5 to 6, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:27.545185 22782 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:27.547191 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:27.549302 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:27.551131 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:27.553581 27578 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.555510 27625 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:11:27.555809 27657 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:27.556025 27673 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:27.515775 27435 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.541345 27561 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:27.548070 22781 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:27.557257 27625 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:27.558763 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:27.559367 27624 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:11:27.562182 27659 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:11:27.541648 27642 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:27.563310 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.566555 27642 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.569818 27642 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.557276 27435 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.574048 27634 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:27.565392 27624 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.574764 27634 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:27.575985 27624 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.576222 27634 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.577536 27624 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:27.582011 27665 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.197s	user 0.004s	sys 0.003s
I20260709 13:11:27.545751 27651 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.583472 27651 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.585960 23412 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:27.590161 23412 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
W20260709 13:11:27.590404 27661 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.306s	user 0.001s	sys 0.001s
I20260709 13:11:27.594591 27528 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:27.595381 27528 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.596750 27528 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.598006 27528 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.563678 27107 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.601506 22635 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:27.603438 27107 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:27.541358 27621 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.604604 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:27.611402 27566 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:27.614759 27578 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:11:27.615613 27422 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:27.616091 27422 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.616371 27628 consensus_queue.cc:1059] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:27.617475 27500 raft_consensus.cc:3055] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:27.617895 27500 raft_consensus.cc:740] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:27.618337 22526 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:27.618970 27500 consensus_queue.cc:260] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.619884 27674 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:27.620319 27674 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.586345 27651 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:27.621038 27576 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:27.624315 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.624193 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:27.629050 22747 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:27.631933 22781 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:27.633592 27669 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:27.635512 27669 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.636525 27669 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.638123 27500 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:27.629915 27676 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:11:27.640213 27657 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:27.640698 27657 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.641678 27657 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.643298 27657 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:11:27.631706 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88987.8us
I20260709 13:11:27.652894 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.620606 22526 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:27.662149 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 128947us
I20260709 13:11:27.647017 27634 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:27.672148 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 117170us
I20260709 13:11:27.647406 27651 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:11:27.672986 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.630604 27622 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:11:27.657574 27491 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:27.690056 27107 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:27.690761 27107 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.691772 27107 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.621304 27674 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:27.714588 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.688232 27491 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.672010 27435 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:27.734932 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:27.735406 22710 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:27.740428 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:27.736038 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:27.741707 22526 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:27.742744 22704 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.743185 27559 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:27.743638 27559 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:27.736722 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:27.744719 27559 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.744511 22526 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 8, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:27.745893 27559 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:11:27.746378 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.749809 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2119) took 1143 ms. Trace:
I20260709 13:11:27.750313 22629 rpcz_store.cc:276] 0709 13:11:26.605976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.611384 (+  5408us) service_pool.cc:224] Handling call
0709 13:11:27.749509 (+1138125us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b540006b3d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:27.749707 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070366976}
I20260709 13:11:27.753298 22704 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:27.756349 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:27.745258 22703 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.758586 27676 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:27.759042 27676 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:27.759112 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:27.759968 27676 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.761210 27676 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:27.761834 22629 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:27.764696 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:27.765704 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:27.766665 22630 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:27.767032 27621 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:27.769598 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.738586 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:27.770799 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.771158 27674 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:27.771452 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:27.773073 27648 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:27.772768 27422 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.773995 27647 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:27.773548 27648 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.775573 27648 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:27.776974 27632 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:11:27.777545 27632 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:27.778440 27578 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:27.778517 27632 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.783722 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:27.783414 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.732430 27626 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.797726 27291 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:27.810873 22703 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:27.805604 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:27.798866 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:27.861693 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.870740 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87081us
W20260709 13:11:27.847401 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:27.814329 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.740466 27491 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:27.800755 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:27.878909 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77660.2us
W20260709 13:11:27.884014 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83332.5us
I20260709 13:11:27.796989 27642 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:27.885818 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:27.932848 27422 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:27.866986 27500 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:11:27.810349 27625 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:27.882860 27605 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:27.900719 22635 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:27.905809 27435 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:27.877140 27651 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:27.912139 23414 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:27.915315 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:27.806205 27667 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.347s	user 0.003s	sys 0.000s
I20260709 13:11:27.809564 27291 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.885217 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:27.933143 22633 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:27.805733 27626 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:27.958200 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73099.8us
I20260709 13:11:27.962551 23421 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:27.799577 27662 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:27.935724 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64152.2us
W20260709 13:11:27.936944 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57162.4us
I20260709 13:11:27.955171 27291 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:27.971496 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.972038 27578 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:27.793790 27660 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.139s
I20260709 13:11:27.976540 27660 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:27.976935 27660 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:27.965075 23414 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46274 (request call id 880) took 1456 ms (client timeout 1539 ms). Trace:
I20260709 13:11:27.964483 22747 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.951972 27651 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.936319 27642 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.949045 27605 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:27.973559 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:27.974080 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.984972 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.938642 27625 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.953737 27561 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:27.977502 23414 rpcz_store.cc:269] 0709 13:11:26.500274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.500516 (+   242us) service_pool.cc:224] Handling call
0709 13:11:27.956921 (+1456405us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54005901d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:27.957053 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3196818304}
I20260709 13:11:27.954250 27628 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:27.978528 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.957759 27652 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:27.985222 22630 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:27.981087 27651 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:28.005153 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.995383 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:27.978777 27662 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:27.983987 27642 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:27.986089 27605 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:28.018359 23415 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:11:28.023624 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:28.016589 22749 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:28.039847 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44547.9us
I20260709 13:11:28.039953 27680 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:28.041028 27626 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:28.041203 27071 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:28.041517 27626 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:28.041798 27071 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:28.042008 27626 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:28.042758 27622 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.043018 27071 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.043311 27622 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.044665 27669 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:28.045218 27669 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.045717 27367 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.046231 27367 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.046978 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:28.047508 27669 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.047654 23414 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:28.048470 27367 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.049420 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:28.050309 27447 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:28.050711 27278 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.050706 27447 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.051275 27278 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.051915 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:28.052189 27622 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.052647 23416 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:28.052990 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:28.053658 27278 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:27.983722 27657 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:27.994554 27528 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:28.054591 27657 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:28.054804 27528 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:28.055178 27657 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:28.055359 27528 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:28.004047 27266 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:28.056620 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.057163 27266 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:28.057452 22708 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:28.041159 27680 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:28.058261 27266 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.058944 22747 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:27.989606 27625 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:28.059686 27680 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.053788 23414 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:27.990193 27578 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:28.063560 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:27.997874 27652 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:28.006367 27061 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:28.051977 27447 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:27.997541 27628 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:28.082659 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 96505us
I20260709 13:11:28.067298 27500 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:28.066102 27578 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:27.994410 27561 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.096957 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:28.097781 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:28.105409 22635 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:28.001753 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:28.108541 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 106893us
I20260709 13:11:28.109633 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:28.100708 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.017093 27566 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 13:11:28.126844 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.072431 27690 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:28.131347 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.131558 22633 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:11:28.083053 27061 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:28.133772 27691 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:28.099843 27624 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:28.101151 27500 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.127735 27422 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:11:28.154078 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 159375us
I20260709 13:11:28.103297 27578 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:28.157738 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:28.156250 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46786.2us
I20260709 13:11:28.166260 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:28.167621 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:28.175993 27657 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.177374 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:28.178160 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.179159 27701 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:28.179735 27701 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.180167 27701 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:28.182929 22778 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:11:28.183362 27657 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.185148 27576 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:11:28.189911 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:28.205055 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49208.2us
W20260709 13:11:28.207566 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49935.6us
W20260709 13:11:28.216540 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.130367 27647 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:28.232754 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.231050 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.235451 27696 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:28.236153 27696 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.236699 27696 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:28.142095 27690 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.239075 27690 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:28.242925 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:28.243909 22635 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:28.249703 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.250746 22782 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:28.252542 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 75241.2us
I20260709 13:11:28.192878 27500 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:28.254284 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:28.263446 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:28.268270 22784 raft_consensus.cc:2410] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:11:28.199981 27624 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.270476 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:28.247594 27682 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 13:11:28.276285 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:28.276289 22633 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 13:11:28.277467 22633 pending_rounds.cc:85] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 11
I20260709 13:11:28.277684 22633 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:11:28.277928 22633 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:28.272423 22776 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:11:28.278831 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.245705 27673 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:28.280483 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:28.243875 27702 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:28.293212 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.298856 22705 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:28.079628 27660 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.244012 27435 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:11:28.277001 27647 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:28.306042 22708 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:11:28.290014 27559 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.536s	user 0.009s	sys 0.003s
I20260709 13:11:28.312048 22776 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:28.308784 22703 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:28.254288 27652 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.314577 27652 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:28.315542 27491 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.278577 27624 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:28.321033 27634 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.000s
I20260709 13:11:28.321545 22705 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:28.263110 27422 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.289227 27576 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.323822 27576 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:28.333398 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 116752us
I20260709 13:11:28.326217 27691 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:28.301048 27702 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:28.345247 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64850.5us
I20260709 13:11:28.311632 27647 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.351590 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:28.309262 27673 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:28.365165 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.365382 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.326574 27561 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.368999 22783 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:28.326892 27622 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:28.369364 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:28.357679 27559 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:28.375247 27559 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.377672 27559 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.378782 27706 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:28.379524 27528 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.347901 27702 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.387071 23415 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:11:28.368292 27648 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.586s	user 0.008s	sys 0.005s
I20260709 13:11:28.361754 27647 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:28.400983 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.402653 22783 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:28.405181 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:28.387810 27691 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:28.407720 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:28.331808 27061 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.420660 27624 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:28.430917 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 90254.6us
I20260709 13:11:28.383270 27651 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.443894 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:28.411363 27698 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:28.404526 27622 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.450531 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:28.399125 27648 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.476485 22782 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 7. Current term is 9. Ops: []
I20260709 13:11:28.446660 27435 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.484510 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.450714 27628 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:28.483986 27707 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:11:28.496361 22704 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:28.474929 27697 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:28.493674 27694 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 13:11:28.494659 27703 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:11:28.507879 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.432001 27625 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:28.509860 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 158414us
I20260709 13:11:28.510965 22780 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:28.518308 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:28.527872 22635 raft_consensus.cc:606] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:28.528268 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.528494 22635 raft_consensus.cc:614] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:28.501427 27697 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:28.529279 22705 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:28.531126 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.532003 22705 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:28.488296 27435 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.534194 22708 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:11:28.559360 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.560271 22705 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:28.541972 22703 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:11:28.542147 27576 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.544178 27701 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.471966 27673 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.563216 22779 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
W20260709 13:11:28.534184 27291 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.565s	user 0.013s	sys 0.000s
I20260709 13:11:28.529253 27705 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.531206 27697 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:28.579236 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:28.538161 27707 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.611493 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.615026 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:28.616300 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.530064 27642 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:28.616950 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:28.587409 27605 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.002s
W20260709 13:11:28.662189 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:28.716133 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:28.719519 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.506s (SENT)
I20260709 13:11:28.720052 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:28.720326 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 93071.4us
I20260709 13:11:28.604259 27528 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.604432 27624 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:28.604530 27662 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.730116 27621 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:11:28.610070 27648 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.736393 23411 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
W20260709 13:11:28.738397 22750 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.571s (SENT)
I20260709 13:11:28.586823 27291 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.740338 22705 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:28.774034 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.609184 27590 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 13:11:28.723768 27107 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.799606 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:28.775293 27578 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.000s
I20260709 13:11:28.722415 27674 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:11:28.803953 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.735987 27716 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.617895 27657 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:11:28.715844 27605 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:28.805980 27696 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.569s	user 0.005s	sys 0.015s
W20260709 13:11:28.835271 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 96886.6us
W20260709 13:11:28.836323 22750 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.576s (SENT)
I20260709 13:11:28.836843 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:28.598246 27705 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.810989 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.825851 22635 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:28.842254 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.828625 27625 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.800174 27651 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:28.829125 27657 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:28.853209 27657 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:28.854269 27657 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.855890 27657 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:28.859565 27721 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:11:28.813673 27107 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.863072 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:28.863897 22628 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:28.865209 27107 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.829684 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.830729 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.867101 22782 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:28.867906 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.868112 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:28.868753 22782 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
W20260709 13:11:28.870852 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.870716 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.871559 22702 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:28.831104 27605 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.804874 27266 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:28.838732 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.875816 22703 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:28.812947 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:28.876806 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.716243 27707 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:28.849152 27626 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.807s	user 0.010s	sys 0.006s
I20260709 13:11:28.850101 27435 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:28.850296 27717 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
I20260709 13:11:28.852579 27701 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:28.881508 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:28.889539 22703 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:28.821866 27578 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.881520 27706 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:28.891628 27626 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.881819 27590 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.895341 27590 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.898730 27634 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:28.900511 27634 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.883101 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:28.842960 22705 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:28.885851 27447 consensus_meta.cc:287] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.786s	user 0.007s	sys 0.007s
I20260709 13:11:28.931113 22702 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:28.887091 22635 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:11:28.932616 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 198210us
I20260709 13:11:28.887734 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.933650 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.934525 22702 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:28.882184 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:28.934947 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:28.935909 22782 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:28.884255 27500 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.002s
W20260709 13:11:28.937634 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.938628 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.942886 22701 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:11:28.900748 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:28.946904 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:28.947973 22632 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:11:28.874348 27702 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.953400 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:28.955005 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118722us
I20260709 13:11:28.843132 27696 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:28.970963 22703 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:28.904567 27690 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.665s	user 0.011s	sys 0.003s
I20260709 13:11:28.931744 27625 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.931358 27435 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:28.978832 22628 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:28.951095 27706 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:28.982313 23412 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:28.919804 27632 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.984294 22702 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:11:28.985144 22705 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:28.968000 27561 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.000880 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:28.927554 27626 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:28.997278 22750 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.584s (SENT)
I20260709 13:11:29.004959 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:28.997881 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:28.965435 27605 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:28.829183 27716 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.968936 27624 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:28.977672 27590 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.946234 27634 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:28.983850 27642 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.037375 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.037572 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.038116 22701 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:28.991993 27625 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.018042 27632 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.038988 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.039693 22703 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:29.040704 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.040956 27632 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:28.988343 27690 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.041865 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.042449 27624 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.043321 27690 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.045819 27624 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.006899 22705 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.008030 27712 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:29.047087 27712 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.048537 27634 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:29.050160 27712 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.051782 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.052016 27716 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.053278 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:28.919148 27647 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.053999 23412 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:29.054072 22709 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:29.055565 27647 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.056452 22706 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:29.009156 27700 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.016305 27704 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:28.941184 27447 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.005509 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.024287 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:28.954317 27500 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.017113 27491 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.022156 27605 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.077243 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.077948 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.031585 27705 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:29.031854 27702 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:29.032620 27676 consensus_meta.cc:287] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.271s	user 0.021s	sys 0.000s
I20260709 13:11:29.033169 27578 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.034773 27422 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:28.999511 27706 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.036664 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.037231 27696 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.095778 22702 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:29.008562 27713 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.042174 22710 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:29.098835 22783 raft_consensus.cc:2368] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13.
I20260709 13:11:29.060241 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.061136 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.062980 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:29.066455 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.100255 22705 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.100306 22628 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:29.099588 23412 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:29.069869 23419 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:29.073167 22701 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:29.073701 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.102528 23417 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:11:29.034842 27652 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.720s	user 0.013s	sys 0.002s
I20260709 13:11:29.105816 27500 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.107236 27706 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.081750 27071 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.108471 27071 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.109925 27605 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.111279 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.111984 22705 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:29.113036 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.114144 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.073030 27700 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:29.116250 27725 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:11:29.116750 27697 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.118121 22708 raft_consensus.cc:2368] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:29.103672 27713 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.120436 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:29.120759 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.045s (SENT)
I20260709 13:11:29.121192 27713 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.121332 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.122327 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.123041 22701 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:29.127111 22705 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.040853 27625 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.068778 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:29.128469 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59808.9us
I20260709 13:11:29.074755 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.129585 22704 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.069871 23420 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:29.071208 27707 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.078716 22635 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:29.137651 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.086972 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.138742 22776 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:29.086402 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.089187 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.089776 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.064302 27367 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:29.093173 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.144686 23418 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
W20260709 13:11:29.094004 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 96771.3us
W20260709 13:11:29.145722 22750 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.817s (SENT)
I20260709 13:11:29.146149 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:29.094915 27734 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:11:29.072284 27712 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.150128 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.071359 27704 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.067328 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.103770 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.167295 23421 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:29.103935 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:29.130410 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 73888.4us
I20260709 13:11:29.078298 22634 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
W20260709 13:11:29.169658 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 65977.9us
I20260709 13:11:29.129402 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:29.131278 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.175163 22702 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:11:29.168848 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 65006.7us
I20260709 13:11:29.177569 22701 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:29.073215 27435 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:11:29.147073 22782 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:29.151095 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.136783 27634 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:29.152916 27422 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:29.153597 27491 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:29.154770 27737 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:11:29.174425 27691 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.008s	sys 0.012s
I20260709 13:11:29.164538 27071 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.120466 27697 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.156452 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.096645 22707 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:11:29.161332 22705 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:29.194456 27669 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:29.084996 27628 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:11:29.117398 27622 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.196626 27674 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.235426 27622 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:29.235491 27674 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.238047 27674 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.159691 27625 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:11:29.135407 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.179662 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.240974 22701 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.181348 22775 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:29.241020 23417 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:11:29.192575 27278 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:29.184304 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.147579 27578 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:29.151904 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:29.193202 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47523.4us
I20260709 13:11:29.244064 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.244745 22779 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:29.197679 27634 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:29.247291 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.193696 22706 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:29.195044 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.248832 22776 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:11:29.195650 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73072.6us
W20260709 13:11:29.231472 27691 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.008s	sys 0.013s
W20260709 13:11:29.198683 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.203766 27713 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:29.208389 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.209499 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.211337 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.255779 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.256479 23420 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.234992 27669 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.216104 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:29.258548 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 63617.9us
I20260709 13:11:29.202351 27512 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:29.256593 22709 raft_consensus.cc:2376] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9. Re-sending same reply.
I20260709 13:11:29.219045 27704 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.222209 27367 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.263540 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.264442 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.264657 22701 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:29.265172 22708 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.250659 27578 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.163372 22704 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:29.236002 27622 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.176427 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.242497 23412 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
W20260709 13:11:29.272444 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.679s (SENT)
W20260709 13:11:29.197679 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68374.8us
I20260709 13:11:29.252864 27634 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:29.159468 27676 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:29.155130 27528 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:29.256172 23421 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.233186 27632 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:11:29.258927 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:29.283689 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.866s (SENT)
I20260709 13:11:29.284514 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.259508 22776 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:11:29.287024 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.637s (SENT)
I20260709 13:11:29.287832 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.259423 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.256139 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:29.218931 22629 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:29.258998 23418 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:29.170843 27712 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.232419 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.260910 27669 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:29.263595 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.292579 22709 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.252866 27723 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:29.272362 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.233942 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.273658 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.254254 27691 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:29.299867 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:29.292824 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:29.301218 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.297401 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.295053 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:29.302563 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.302834 22704 raft_consensus.cc:2393] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.296761 22782 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:29.271637 27578 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:29.302078 22708 raft_consensus.cc:2376] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13. Re-sending same reply.
I20260709 13:11:29.298117 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.303567 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.310542 22701 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.311576 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.312269 27662 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:29.313517 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.314285 23418 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:29.315317 27691 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.318475 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.319396 22709 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.320462 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.321782 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.283313 27528 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.324318 27712 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.272179 27367 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.324684 27528 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.326036 27528 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:29.327251 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3007) took 1016 ms. Trace:
I20260709 13:11:29.328084 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.328846 22776 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:29.329762 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.331285 27680 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:29.331882 27680 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.332878 27680 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.334303 27680 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:29.268121 27512 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:29.309005 27707 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:29.335101 27707 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.294073 27676 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.335454 27512 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.336333 27676 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.338762 22784 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:29.339154 27561 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.339525 27621 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:29.340103 27621 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.341952 27698 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:29.342558 27698 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:29.343897 27736 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.328s	user 0.000s	sys 0.002s
I20260709 13:11:29.345144 27622 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:29.346416 27733 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
I20260709 13:11:29.346807 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.347479 22704 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.348212 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2919) took 1020 ms. Trace:
I20260709 13:11:29.329855 27435 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:29.291388 27632 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.306917 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.331476 27728 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:29.311753 27738 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:29.335493 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1066) took 1433 ms. Trace:
W20260709 13:11:29.305153 22750 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.934s (SENT)
I20260709 13:11:29.350385 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.355997 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2701) took 1261 ms. Trace:
I20260709 13:11:29.356102 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:29.366523 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59691.6us
I20260709 13:11:29.327751 22710 rpcz_store.cc:276] 0709 13:11:28.310887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.311049 (+   162us) service_pool.cc:224] Handling call
0709 13:11:28.356594 (+ 45545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:29.099036 (+742442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:29.326920 (+227884us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54002c38d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:29.327170 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576210816}
I20260709 13:11:29.361258 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.370645 22702 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:29.357786 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.367316 22629 rpcz_store.cc:276] 0709 13:11:28.094965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.095146 (+   181us) service_pool.cc:224] Handling call
0709 13:11:28.126898 (+ 31752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:29.292211 (+1165313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:29.355650 (+ 63439us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400071554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:29.355907 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2510929280}
I20260709 13:11:29.366895 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.368278 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.348749 22777 rpcz_store.cc:276] 0709 13:11:28.327653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.337844 (+ 10191us) service_pool.cc:224] Handling call
0709 13:11:28.361789 (+ 23945us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:28.729025 (+367236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:28.729053 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:28.774836 (+ 45783us) log.cc:844] Serialized 26 byte log entry
0709 13:11:28.949090 (+174254us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:28.949131 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:28.949206 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:29.342970 (+393764us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:29.347742 (+  4772us) raft_consensus.cc:1685] finished
0709 13:11:29.347774 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:29.348122 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":66939,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":30579,"spinlock_wait_cycles":3466112,"thread_start_us":365082,"threads_started":2,"wal-append.queue_time_us":207999}
I20260709 13:11:29.341493 27621 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.283113 27559 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.379143 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.368278 22708 raft_consensus.cc:2376] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14. Re-sending same reply.
I20260709 13:11:29.307143 22779 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:29.383409 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.383462 22704 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:29.367514 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.390901 22701 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:29.392685 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.370512 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.366319 23411 rpcz_store.cc:276] 0709 13:11:27.901988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:27.902725 (+   737us) service_pool.cc:224] Handling call
0709 13:11:27.903205 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:28.780630 (+877425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:29.335129 (+554499us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54005028d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:29.335403 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915659520}
I20260709 13:11:29.394519 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.322432 22709 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:11:29.296187 27723 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:29.303543 27566 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:29.304162 27624 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:29.443627 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.444626 22701 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10. Re-sending same reply.
I20260709 13:11:29.445894 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.423887 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.447235 22779 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:11:29.296445 27704 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:29.406677 22782 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:29.400717 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.442147 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:29.428092 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 190550us
I20260709 13:11:29.449136 22710 raft_consensus.cc:2376] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5. Re-sending same reply.
I20260709 13:11:29.358651 27739 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:29.431447 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.434027 23421 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:29.437731 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.452256 22708 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:29.335503 27707 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.454267 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.454689 22598 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:29.449174 22631 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:29.378044 27632 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:29.457783 22777 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:29.458204 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.460739 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.391726 27642 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:29.378530 27738 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:29.435724 27728 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:29.378810 27737 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.460886 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.359328 27447 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.484124 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.450564 27624 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.469529 27672 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:29.469812 27367 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:29.471943 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:29.502425 22634 raft_consensus.cc:606] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.502951 22634 raft_consensus.cc:614] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:29.436746 27621 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:29.456408 27739 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.424214 27625 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:11:29.467490 27738 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:29.437945 27561 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:29.435837 27705 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:29.456190 22701 raft_consensus.cc:2376] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12. Re-sending same reply.
I20260709 13:11:29.479765 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.484066 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.513761 22702 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:29.515496 22631 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.486385 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.516534 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:29.478844 22633 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:29.497391 27624 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:29.469552 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 175608us
W20260709 13:11:29.526911 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:29.468300 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.875s (SENT)
I20260709 13:11:29.496486 22710 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:29.498194 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.528236 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.500815 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.529464 22707 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:29.428217 27559 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.506244 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.485352 27674 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:29.511189 22628 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:29.512303 27642 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:29.442145 27712 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:29.496179 27672 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.521054 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.548563 22701 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:29.524282 22779 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:29.550678 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.501734 23420 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:29.540616 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:29.543468 22750 leader_election.cc:336] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.083s (SENT)
I20260709 13:11:29.551779 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.530078 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.548952 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2928) took 1218 ms. Trace:
I20260709 13:11:29.550028 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.513561 27561 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:29.530827 27291 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.552826 22775 rpcz_store.cc:276] 0709 13:11:28.330466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.384805 (+ 54339us) service_pool.cc:224] Handling call
0709 13:11:29.548752 (+1163947us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54003255d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:29.548882 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1607274752}
I20260709 13:11:29.530879 27691 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.492493 27737 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.552682 22702 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:29.552567 22747 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.563040 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.569193 22710 raft_consensus.cc:2368] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:29.533402 27749 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:29.553180 22701 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:29.549885 27672 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:29.545938 27712 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.566222 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.546773 27674 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:29.575129 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.575122 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.557008 27561 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.601152 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.577793 27757 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:29.566150 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:29.589577 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.605824 22702 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:29.592175 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.580356 22707 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:29.584757 27642 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:29.601933 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 133672us
I20260709 13:11:29.601954 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:29.606688 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.581704 27712 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:29.608624 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77877.5us
W20260709 13:11:29.614806 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.574532 27628 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:29.608417 22710 raft_consensus.cc:2393] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.611940 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.579167 27705 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.624119 22627 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:29.644779 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.645524 22702 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:29.647504 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.648507 22710 raft_consensus.cc:2368] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:29.647521 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.653908 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.654459 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.580794 27691 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.664880 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.665797 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:11:29.667336 27755 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:11:29.667585 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.643080 27628 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:29.675859 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.666431 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.679790 22702 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:29.682446 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:29.682964 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.683383 22710 raft_consensus.cc:2376] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
I20260709 13:11:29.687893 22775 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:29.688345 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.689497 22702 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.591478 27749 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.693555 22708 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 9: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.734563 27278 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:29.735893 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.737779 27749 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:29.694418 27748 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 13:11:29.699586 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.570603 27728 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.740748 27578 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.741437 22709 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:29.730072 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.742579 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.734977 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.701694 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.746528 22709 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:29.703706 27697 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:29.703531 27628 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:29.704852 27752 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.114s	user 0.003s	sys 0.000s
W20260709 13:11:29.705724 27754 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:11:29.706024 27742 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:29.706837 27737 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.766997 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.757226 27697 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.707525 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.767982 22706 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:29.768787 22707 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:11:29.769511 22706 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:29.770040 27697 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.687475 27747 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.770989 22600 leader_election.cc:400] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13.
I20260709 13:11:29.771627 22600 leader_election.cc:403] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:29.716837 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.777817 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.778770 22628 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:29.778891 22706 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:11:29.779631 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:29.756917 27753 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:29.715785 27691 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.718199 27676 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.720822 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.722509 27707 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.727159 27660 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.662317 27739 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.729853 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.732999 22631 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:29.733453 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.691267 27700 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.703392 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.747205 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:29.822650 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 75536.7us
I20260709 13:11:29.738397 22708 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:29.735419 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.742951 22705 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.748508 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:29.743007 22782 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:29.743324 22704 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:29.750533 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.709867 27723 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.692320 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.828627 22702 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10. Re-sending same reply.
I20260709 13:11:29.646976 27674 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.681030 27559 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.757776 22709 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:29.759325 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:29.760399 22675 connection.cc:256] RPC call timeout handler was delayed by -2.1376537480000479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:11:29.838433 22675 connection.cc:256] RPC call timeout handler was delayed by -2.1331537480000407s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
W20260709 13:11:29.838896 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0872537479999664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
W20260709 13:11:29.839268 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0611537480001516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
W20260709 13:11:29.839552 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0350537480001094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 13:11:29.839843 22675 connection.cc:256] RPC call timeout handler was delayed by -2.0116537480000716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:11:29.840112 22675 connection.cc:256] RPC call timeout handler was delayed by -1.95495374799998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
W20260709 13:11:29.840397 22675 connection.cc:256] RPC call timeout handler was delayed by -1.9126537480001389s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
W20260709 13:11:29.840700 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8721537480000734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 13:11:29.841020 22675 connection.cc:256] RPC call timeout handler was delayed by -1.8037537480001902s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
I20260709 13:11:29.781855 27753 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.765240 27742 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.846362 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.711943 27395 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:29.713274 27669 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.771054 27747 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.772631 27605 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.717923 27061 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:29.782024 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.759814 27698 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.783721 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.784621 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.785454 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.722150 27672 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.735467 27622 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:29.798450 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.802353 22628 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:29.844465 27662 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:29.801625 22784 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:11:29.862774 22784 pending_rounds.cc:85] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 6
I20260709 13:11:29.802150 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.863065 22784 pending_rounds.cc:107] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:11:29.863375 22784 raft_consensus.cc:2889] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:29.815788 27676 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.852073 27753 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:29.829767 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.868754 22702 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:29.870041 22702 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:29.870191 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:29.831193 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 97838.3us
I20260709 13:11:29.836334 22710 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.836778 22777 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 6: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:29.870914 23412 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:29.872475 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.756527 22633 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
W20260709 13:11:29.841356 22675 connection.cc:256] RPC call timeout handler was delayed by -1.6921537480000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:11:29.874116 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5787537480000537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22050000000000003s for a response
I20260709 13:11:29.764202 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:29.874522 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5778537480000523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 13:11:29.838547 22779 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:29.770108 22707 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:11:29.874908 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5670537480000348s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 13:11:29.875259 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5526537480000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22340000000000002s for a response
W20260709 13:11:29.875551 22675 connection.cc:256] RPC call timeout handler was delayed by -1.5157537480001793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2275s for a response
W20260709 13:11:29.875881 22675 connection.cc:256] RPC call timeout handler was delayed by -1.4086537480000061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23940000000000003s for a response
I20260709 13:11:29.744594 27634 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.750180 27712 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.809427 27512 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:29.856449 23416 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
W20260709 13:11:29.856700 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 72220.9us
I20260709 13:11:29.888049 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:29.853435 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:29.888666 22673 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:29.888991 22703 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:29.887934 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:29.857067 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 71771.7us
I20260709 13:11:29.890137 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.837483 27729 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:29.860915 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.893994 23420 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:29.894253 22710 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:29.799767 22780 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:29.899758 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.900256 27669 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.901118 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.858714 27605 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:29.903297 27422 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:29.903769 27422 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.904121 27422 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:29.904877 27605 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.905567 27528 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:29.833197 27737 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13.
I20260709 13:11:29.817837 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:29.851864 27723 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:29.850696 27395 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:29.906791 27766 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:29.866236 27672 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.866585 27676 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.867661 27707 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.868877 22701 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:29.763880 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.747727 27738 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:29.876209 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0765537480001512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2763s for a response
I20260709 13:11:29.880515 22707 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:29.881194 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.930055 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.907760 27512 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.931380 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.931656 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:29.859421 27700 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:29.932695 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.933413 27691 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.886549 27632 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.934388 27691 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.883646 22702 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:29.859035 27739 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:29.860411 27628 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:29.936546 27632 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.890264 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.937290 23421 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:29.891629 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.852723 27578 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.891983 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:29.938709 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:29.887444 27680 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.508s	user 0.013s	sys 0.000s
I20260709 13:11:29.939903 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.941375 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.862567 27662 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.942924 27662 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:29.948493 22626 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:29.942471 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.893100 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.902326 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.955754 27698 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.956156 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.924253 27766 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.956988 27766 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:29.957425 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:29.858990 27061 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.864616 22631 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:11:29.958572 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:29.959574 22780 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:29.960022 22777 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:29.924656 23420 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:29.925369 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:29.963274 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71392.2us
W20260709 13:11:29.921756 27767 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:11:29.926509 22703 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:29.926940 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:29.969779 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76748.4us
I20260709 13:11:29.970359 22775 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:29.932802 27700 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:29.933377 22782 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
W20260709 13:11:29.882261 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:29.883546 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.936210 27628 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.913170 23418 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:29.951865 27747 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.952174 27712 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:29.953145 27634 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:29.953785 27395 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.906755 27624 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:29.962020 27738 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:29.880216 27278 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:29.915012 27626 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:30.020195 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.964185 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:29.967073 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:30.025815 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:29.967638 27680 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:29.973413 23421 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:30.029035 23418 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 17: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:30.029141 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:30.026293 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59305.4us
I20260709 13:11:29.977391 23412 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:11:29.982136 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:29.980237 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.991735 22710 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:29.998775 27669 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:29.989364 27700 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:29.933277 27512 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.002871 22784 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
W20260709 13:11:30.039373 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59284.7us
I20260709 13:11:30.040231 22600 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:29.998886 27672 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:11:29.983340 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:29.962395 27561 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.075371 22783 raft_consensus.cc:2393] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.014751 27621 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:30.017060 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:30.080226 23420 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:30.023116 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:30.025141 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93572.1us
I20260709 13:11:30.082137 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:30.082345 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:30.031528 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67451.5us
I20260709 13:11:30.083722 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:30.032940 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:30.034322 22598 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:30.035362 22747 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:30.048069 22706 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:29.982824 27728 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:30.077982 22634 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.085196 22626 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.086014 22627 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:30.082907 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:30.091480 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 122605us
I20260709 13:11:30.038026 27726 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:11:30.096966 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 66030.9us
W20260709 13:11:30.098053 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 63783.2us
I20260709 13:11:30.075860 27764 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:30.097878 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:30.097443 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:30.037878 27729 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:30.103220 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:30.024854 27673 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:11:30.109874 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:30.035820 27528 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:11:30.111580 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:30.079568 27691 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.106189 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:30.107954 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:30.108385 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:30.108891 23416 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:11:30.113792 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.114460 22780 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:30.087337 27622 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:30.094985 27676 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:30.090255 27628 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.037410 27753 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.101042 27278 raft_consensus.cc:2804] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:30.098733 27578 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:30.074393 27447 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:11:30.101244 27697 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:30.101521 27704 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:30.100198 27726 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.038757 27674 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:11:30.099875 27071 raft_consensus.cc:2764] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:30.101676 27491 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:30.122732 27712 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:30.119302 27729 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:30.130676 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.115353 27500 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:30.112496 27764 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:30.144109 22781 raft_consensus.cc:2410] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:11:30.090194 27669 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:11:30.119154 27676 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.104000 27626 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:11:30.125690 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.134549 27622 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:30.188947 27771 raft_consensus.cc:2764] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:11:30.119421 27691 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.147454 27726 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:30.154811 27712 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.147106 27491 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.194073 22777 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:30.154307 27578 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:30.151168 27697 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:30.158335 27278 raft_consensus.cc:697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:30.156221 27673 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:11:30.145356 27422 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.248807 27422 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.184813 27676 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:30.178406 27705 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:30.217882 23416 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:11:30.159056 27061 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.243149 27278 consensus_queue.cc:237] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.249883 27624 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.159539 27704 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:30.280658 27704 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.222846 27578 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:30.265980 22780 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:30.223960 27491 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:30.266530 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 160463us
I20260709 13:11:30.274901 27561 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:30.201725 27712 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:30.289057 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 179085us
I20260709 13:11:30.315622 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:30.282831 27705 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:30.289149 27578 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:30.248838 27766 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.318747 27729 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.291752 27628 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:11:30.344590 27739 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:30.345130 27739 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:30.348042 27742 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.349500 27742 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:11:30.353569 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 270065us
I20260709 13:11:30.350426 27742 consensus_queue.cc:1243] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 13.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:11:30.353530 27753 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.331560 27622 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.346100 27739 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.368157 27739 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:30.370381 22776 raft_consensus.cc:2410] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:11:30.373651 27770 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.453s	user 0.007s	sys 0.000s
I20260709 13:11:30.370576 27700 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.375010 22631 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:30.380796 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 65319.6us
I20260709 13:11:30.326987 27697 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.387156 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:30.391987 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 284889us
W20260709 13:11:30.409963 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:30.441063 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43681.7us
I20260709 13:11:30.325623 27764 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.389662 27512 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:30.467574 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:30.395563 27674 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:11:30.355865 27662 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.455009 27780 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:30.463959 27766 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.445612 27528 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:11:30.466215 27691 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.465921 27491 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.484503 22780 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:30.486358 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.495473 22777 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
W20260709 13:11:30.449924 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63312.6us
I20260709 13:11:30.512523 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:30.490199 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 43918.8us
W20260709 13:11:30.490144 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 189420us
I20260709 13:11:30.515471 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.516213 22709 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:30.516269 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:30.514328 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.517508 22708 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:30.517436 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:30.519619 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.521521 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:30.522094 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:30.522169 22702 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:30.525977 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.527976 22705 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:30.529208 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.530787 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.531147 22705 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:11:30.526571 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.532166 22750 leader_election.cc:400] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:30.532670 22703 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:30.533403 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:30.468417 27700 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.539554 22707 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:30.442703 27422 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:11:30.543277 22704 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:11:30.458142 27712 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.546352 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:30.549017 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.487056 27780 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.473505 27680 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:30.551203 22703 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:30.553588 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.492549 27512 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:30.448415 27624 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:11:30.532052 27766 raft_consensus.cc:2764] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:11:30.563668 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:30.565212 22702 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 4. Current term is 5. Ops: []
I20260709 13:11:30.525406 27728 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:30.566286 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.570688 27491 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.575182 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.576574 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.577370 22710 raft_consensus.cc:2368] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 14. Current term is 15.
I20260709 13:11:30.577256 27726 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.578296 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.442996 27705 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.602134 27662 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.586732 27712 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.536659 27071 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:30.587750 27590 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:30.583647 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:11:30.605217 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:30.579698 27512 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.583057 27395 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:30.604929 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:30.657773 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53038.9us
I20260709 13:11:30.563951 27780 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:30.616837 22708 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:30.631686 22783 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:30.631820 22705 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:30.632618 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.637610 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:30.638053 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.638633 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:30.594267 27726 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:30.678285 23411 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:30.652626 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:30.679270 23412 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:30.565825 27278 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:30.656550 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:30.677395 22707 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 5.
I20260709 13:11:30.684901 23416 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:30.659245 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:30.686138 22627 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.662429 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.687161 22710 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:30.685914 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.658494 22601 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:30.677425 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.689265 22708 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.675203 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.692632 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.693509 22705 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:30.699795 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.702123 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.718405 22705 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:30.665968 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:30.720865 23421 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:30.683624 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:30.688423 22707 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:30.726890 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:30.645625 27590 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:30.734091 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.740021 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.650420 27779 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.727691 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.663507 27662 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:11:30.735894 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77492.9us
I20260709 13:11:30.684755 27669 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.758441 27779 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.725804 27491 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:30.651049 27691 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:30.740744 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.777877 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:30.779170 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.674896 27626 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:30.693679 27278 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:30.736133 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58126.1us
I20260709 13:11:30.792167 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:30.752051 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.755389 22707 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:30.750936 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:30.794888 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.796321 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.797261 22707 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:30.798727 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47815.8us
I20260709 13:11:30.800768 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:30.801695 22630 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.802525 22784 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 7: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:30.770910 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 285382us
I20260709 13:11:30.775759 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.741771 27739 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:30.787520 22626 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:30.798779 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:30.803370 22708 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:30.773811 27491 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:30.839282 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:30.740154 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.808128 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.742072 27738 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:11:30.771685 27662 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:30.804193 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:30.844852 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.845786 22710 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:30.815408 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:30.847035 22784 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:30.800448 27278 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:30.786320 27626 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:30.799973 27691 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:30.863745 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:30.864856 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.865811 22705 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:30.851554 22708 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:11:30.855531 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:30.806938 27739 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:30.810273 27700 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:11:30.858304 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 312051us
W20260709 13:11:30.843346 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44702.2us
I20260709 13:11:30.877942 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:30.842900 27491 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:30.884939 22750 connection.cc:256] RPC call timeout handler was delayed by -1.0492115819999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2575s for a response
I20260709 13:11:30.773095 27669 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:30.886229 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:30.891657 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:30.868252 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.895509 22708 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.814935 27624 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:11:30.899401 27561 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:30.866251 27626 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:30.912415 27786 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 13:11:30.896481 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.915115 22708 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:30.861610 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.897931 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.863279 27753 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:30.858186 27632 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:30.916419 22704 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:30.916421 22710 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:30.876868 27674 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:30.908075 22750 leader_election.cc:400] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:30.922749 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.924153 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.925588 22710 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:30.925830 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:30.927608 23416 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:30.904706 27071 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:30.935763 27779 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:30.888996 27729 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:30.936858 27729 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.939934 27729 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:30.942252 27753 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:30.945163 27753 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:30.891952 27578 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.000s
I20260709 13:11:30.876291 27739 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:30.964771 22778 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:11:30.947156 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2956) took 1145 ms. Trace:
I20260709 13:11:30.965965 22779 rpcz_store.cc:276] 0709 13:11:29.801851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.882861 (+ 81010us) service_pool.cc:224] Handling call
0709 13:11:29.883175 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.087874 (+204699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.087905 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:30.087950 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:30.087982 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.946879 (+858897us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54001860d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:30.946935 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:30.947085 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470219520}
I20260709 13:11:30.966902 22708 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 6.
I20260709 13:11:30.967650 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.968206 22708 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:30.828263 27590 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:30.882033 22626 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:11:30.964552 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2139) took 1326 ms. Trace:
I20260709 13:11:30.973938 22626 pending_rounds.cc:85] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 6
I20260709 13:11:30.974403 22626 pending_rounds.cc:107] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:11:30.974865 22626 raft_consensus.cc:2889] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:30.850374 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:30.946110 27662 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:30.970965 22781 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2910) took 2734 ms (client timeout 3235 ms). Trace:
I20260709 13:11:30.973166 22634 rpcz_store.cc:276] 0709 13:11:29.638341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.682703 (+ 44362us) service_pool.cc:224] Handling call
0709 13:11:30.964347 (+1281644us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54000b81d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:30.964479 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":836548864}
W20260709 13:11:30.978744 22781 rpcz_store.cc:269] 0709 13:11:28.230043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:28.230734 (+   691us) service_pool.cc:224] Handling call
0709 13:11:30.963998 (+2733264us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b5400131d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:30.964209 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4390620032}
W20260709 13:11:30.971652 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72907.6us
I20260709 13:11:30.971771 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.883288 27704 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 13:11:30.977216 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:30.967458 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:30.980508 22779 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:30.986640 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.980381 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:30.982160 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:30.990113 22710 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:30.991454 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:30.977602 27622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.005s	sys 0.006s
I20260709 13:11:31.002123 22708 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:11:31.004019 22704 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:31.005561 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.008975 22708 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:11:31.014446 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:30.968895 27578 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.001544 27680 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:11:31.004935 27622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.006s	sys 0.006s
I20260709 13:11:31.025892 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:31.051362 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:31.007926 27624 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:30.985494 27738 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:11:31.021950 27704 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:31.012288 27794 raft_consensus.cc:2764] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:11:31.055562 22709 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:31.049477 27676 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.766s	user 0.014s	sys 0.000s
I20260709 13:11:31.039383 27680 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:31.018070 27737 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:11:31.065840 22633 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:30.978871 27691 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.072216 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.073889 22780 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:31.037959 27780 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:31.076996 27773 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.977s	user 0.006s	sys 0.008s
I20260709 13:11:31.078015 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2941) took 1835 ms. Trace:
I20260709 13:11:31.080430 27753 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.083057 27669 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.064853 27676 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.122707 27578 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:31.124500 27676 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.063026 27704 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.125839 27704 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.086301 27447 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:31.129647 27447 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.086970 27625 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:11:31.130446 27071 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:31.130821 27625 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.131429 22779 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:31.131778 27625 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.132102 27278 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.133215 27625 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:31.133514 22781 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:11:31.056105 27624 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.087800 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2081) took 1273 ms. Trace:
I20260709 13:11:31.135061 22776 rpcz_store.cc:276] 0709 13:11:29.814412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.819846 (+  5434us) service_pool.cc:224] Handling call
0709 13:11:31.087540 (+1267694us) spinlock_profiling.cc:243] Waited 523 ms on lock 0x7b5400325ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:31.087720 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1202396544}
I20260709 13:11:31.136581 27624 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:31.089339 27491 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.137815 22629 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:31.090040 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:31.055527 27626 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:31.099483 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.063907 27680 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:31.115934 22778 rpcz_store.cc:276] 0709 13:11:29.242938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.243111 (+   173us) service_pool.cc:224] Handling call
0709 13:11:30.964213 (+1721102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.965266 (+  1053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.077777 (+112511us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b5400131d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:31.077945 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3957140352}
I20260709 13:11:31.137300 27447 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.068547 22705 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:31.140677 22776 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
W20260709 13:11:31.169198 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.178488 22784 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
W20260709 13:11:31.192941 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 210857us
I20260709 13:11:31.193876 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.194350 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:31.185132 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.197476 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:31.198220 22705 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:31.198700 22780 raft_consensus.cc:2410] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:11:31.165055 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2730) took 1054 ms. Trace:
I20260709 13:11:31.139806 22783 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:11:31.202056 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.109653 22709 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:31.205292 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.199165 22632 rpcz_store.cc:276] 0709 13:11:30.111089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.111278 (+   189us) service_pool.cc:224] Handling call
0709 13:11:31.164837 (+1053559us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b5400071a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:31.164975 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083846912}
W20260709 13:11:31.114190 27698 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.015s	sys 0.013s
W20260709 13:11:31.256935 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60505.1us
I20260709 13:11:31.258356 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:31.156986 27790 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 13:11:31.163554 27800 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:11:31.154292 27792 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:11:31.259398 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:31.259701 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54534.1us
I20260709 13:11:31.273478 22783 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:31.260170 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:31.261823 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61413.4us
W20260709 13:11:31.261224 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:31.155742 27785 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
I20260709 13:11:31.148368 27712 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.282456 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:31.283272 22778 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:31.267096 22704 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:11:31.268541 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.269306 22705 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:11:31.264285 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:31.140321 27626 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.273357 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.164139 27528 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:31.257393 27107 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:31.268299 27447 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:31.207819 27632 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:31.145673 27657 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:31.220261 27071 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:31.275723 27491 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.266090 27797 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:31.284909 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:31.302000 22704 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:31.302296 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.285321 22705 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:31.304244 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:31.304844 22705 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:31.286394 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:31.261003 27783 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:11:31.257392 27698 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.106s	user 0.016s	sys 0.013s
I20260709 13:11:31.291874 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.203171 27753 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.263080 27500 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:11:31.319939 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.301265 27674 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.322070 22705 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:31.263850 27739 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.336979 22704 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:31.326910 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:31.302219 27578 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:31.286978 27395 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:11:31.348289 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79876.2us
I20260709 13:11:31.349184 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.351022 22635 raft_consensus.cc:2368] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:11:31.288439 27712 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.342118 22705 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:31.302917 27704 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:11:31.353049 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:31.343794 27806 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.301246 27657 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.323273 27797 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:11:31.300980 27107 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.340428 27674 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.355721 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:31.347177 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:31.445669 22631 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:31.385345 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:31.374362 23416 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:11:31.480465 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 124864us
I20260709 13:11:31.387188 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:31.373569 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3024) took 2162 ms. Trace:
I20260709 13:11:31.483558 23412 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:31.483856 22706 rpcz_store.cc:276] 0709 13:11:29.211246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:29.782637 (+571391us) service_pool.cc:224] Handling call
0709 13:11:30.043601 (+260964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:30.088783 (+ 45182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:31.373283 (+1284500us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54004105d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:31.373488 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599357696}
I20260709 13:11:31.485204 22704 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:31.437168 22598 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.489977 22631 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:31.449563 27796 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.510s	user 0.002s	sys 0.000s
W20260709 13:11:31.382084 27801 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:11:31.426726 27798 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.517s	user 0.000s	sys 0.003s
W20260709 13:11:31.414687 27808 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:11:31.351684 27807 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:31.348120 27632 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:31.382117 27739 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:31.488170 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 230081us
I20260709 13:11:31.508069 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.359808 27491 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.157s
I20260709 13:11:31.503361 27764 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:31.416036 27651 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:31.498273 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61247.5us
I20260709 13:11:31.356990 27673 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:31.356710 27783 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.486249 27782 consensus_queue.cc:1059] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:11:31.457461 27806 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:31.457078 27780 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.416502 27712 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.594897 27823 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.594730 27632 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.512669 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.595597 27823 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.512790 22783 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:11:31.486683 27704 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:31.516342 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:31.348053 27278 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.517859 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:31.601414 22775 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:31.463449 27818 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:31.465401 27107 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:31.603382 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 254279us
I20260709 13:11:31.528053 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.530580 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.604629 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.414299 27816 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:31.605605 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 75122.1us
I20260709 13:11:31.605954 27816 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.606974 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.607192 27822 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.607530 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.608060 27822 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:31.608367 27816 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.608979 27816 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:31.609622 27816 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.609509 27822 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:11:31.358875 27669 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 13:11:31.610435 27822 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:31.611019 23416 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:11:31.611668 27626 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:31.530989 23412 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:31.612093 27626 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:31.545919 22598 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.612557 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:31.514091 27807 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:31.613479 27807 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:31.613366 23416 raft_consensus.cc:2368] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:11:31.613018 27626 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.616185 27626 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:31.547822 22706 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:31.622725 22673 leader_election.cc:400] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:11:31.623142 22673 leader_election.cc:403] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:11:31.622886 27691 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:11:31.364185 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 61987.4us
I20260709 13:11:31.586910 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:31.584657 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46270.9us
I20260709 13:11:31.585026 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:31.439085 27657 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.584183 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:31.627898 27657 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:31.628207 22710 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:31.628742 27590 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:31.629587 22780 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:11:31.630863 27621 raft_consensus.cc:3055] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:31.631289 27621 raft_consensus.cc:740] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.422184 27753 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.634002 27753 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.595405 27712 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:11:31.532338 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 212501us
I20260709 13:11:31.623847 27797 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:11:31.642235 23421 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:31.603653 27818 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:31.644270 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.659646 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.661799 23421 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:11:31.663658 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3041) took 1144 ms. Trace:
I20260709 13:11:31.664144 22709 rpcz_store.cc:276] 0709 13:11:30.519248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.519378 (+   130us) service_pool.cc:224] Handling call
0709 13:11:31.663427 (+1144049us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400087d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:31.663545 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1230994816}
W20260709 13:11:31.670185 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 124395us
I20260709 13:11:31.670898 22775 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:31.672205 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:31.670218 22782 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:31.670291 22710 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:31.531699 27676 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:11:31.529042 27809 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:11:31.576426 27651 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.703042 23412 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:31.639102 27817 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:31.640002 27673 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.634503 27753 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:31.721715 27561 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.029s	sys 0.029s
I20260709 13:11:31.668996 27818 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:11:31.734218 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:31.741082 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81513us
I20260709 13:11:31.746418 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.747836 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.748888 27691 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:11:31.737655 27625 consensus_meta.cc:287] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.604s	user 0.016s	sys 0.000s
I20260709 13:11:31.749903 27797 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:31.730288 27669 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:31.754212 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:31.749928 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:31.739893 27561 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.029s	sys 0.029s
W20260709 13:11:31.639907 27624 consensus_queue.cc:1175] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 13 status { last_received { term: 13 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 13 index: 9 } }
I20260709 13:11:31.600071 27806 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.760010 27672 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.737121 27739 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:31.761755 27625 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.745498 27823 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:31.768442 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 96313.8us
I20260709 13:11:31.766903 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3048) took 1239 ms. Trace:
I20260709 13:11:31.785432 27828 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:31.765552 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.731209 27657 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:11:31.735702 27422 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:11:31.792651 22701 rpcz_store.cc:276] 0709 13:11:30.527127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.527241 (+   114us) service_pool.cc:224] Handling call
0709 13:11:30.527580 (+   339us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:11:30.538160 (+ 10580us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.538196 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:11:30.547692 (+  9496us) log.cc:844] Serialized 78 byte log entry
0709 13:11:30.826854 (+279162us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:30.881602 (+ 54748us) log.cc:844] Serialized 14 byte log entry
0709 13:11:30.881850 (+   248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.881929 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:31.139765 (+257836us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b180036dc60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:31.448659 (+308894us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:31.766549 (+317890us) raft_consensus.cc:1685] finished
0709 13:11:31.766580 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:31.766832 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54439,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3526,"raft.queue_time_us":107005,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":23407,"spinlock_wait_cycles":52735616,"thread_start_us":183210,"threads_started":1,"wal-append.queue_time_us":373218}
I20260709 13:11:31.763752 27447 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:31.781392 27680 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.525s	user 0.011s	sys 0.002s
W20260709 13:11:31.795675 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48294.4us
I20260709 13:11:31.764966 27561 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:31.765086 27691 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:31.793879 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:31.792009 27625 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.781886 27794 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:31.809521 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:31.809862 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:31.811045 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:31.811532 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46058.1us
I20260709 13:11:31.844753 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.801191 27828 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:31.836887 23385 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.501s (SENT) [suppressed 45 similar messages]
I20260709 13:11:31.796789 27823 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.789382 27672 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:31.831166 27788 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.920s	user 0.000s	sys 0.007s
I20260709 13:11:31.831982 27778 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:11:31.854246 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46820.1us
I20260709 13:11:31.823199 27680 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.832795 27820 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:31.833652 27825 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:31.840523 22750 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.067s (SENT)
I20260709 13:11:31.855286 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.855517 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.800230 27739 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:31.802291 27674 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:31.862489 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 236168us
W20260709 13:11:31.873644 23385 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.100s (SENT)
W20260709 13:11:31.878409 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.122s (SENT)
I20260709 13:11:31.879128 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.879113 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:31.861493 27739 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.841372 27528 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.841699 27771 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:31.900053 22626 raft_consensus.cc:2368] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 8. Current term is 9.
I20260709 13:11:31.883401 23416 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:31.885462 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.875690 27825 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.902699 27697 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.902961 27742 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:31.903914 27825 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.904187 27742 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:31.904747 27742 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:31.863229 27823 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.923079 23420 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 926) took 1542 ms. Trace:
I20260709 13:11:31.923902 27823 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:31.924829 27826 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:11:31.923565 27674 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:31.926775 27830 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:11:31.879720 27778 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:31.882786 27704 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:31.879930 27071 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.001s
I20260709 13:11:31.864557 27797 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:11:31.933282 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:31.917908 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62725.2us
I20260709 13:11:31.923523 23420 rpcz_store.cc:276] 0709 13:11:30.381162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.381353 (+   191us) service_pool.cc:224] Handling call
0709 13:11:30.397881 (+ 16528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:30.398280 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:30.398313 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:30.447738 (+ 49425us) log.cc:844] Serialized 26 byte log entry
0709 13:11:30.732796 (+285058us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:30.732850 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:30.732936 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:31.693514 (+960578us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:31.922720 (+229206us) raft_consensus.cc:1685] finished
0709 13:11:31.922748 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:31.923008 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":574,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":4207,"spinlock_wait_cycles":17020288,"thread_start_us":191259,"threads_started":1,"wal-append.queue_time_us":372785}
I20260709 13:11:31.938292 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:31.909770 27819 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:11:31.797272 27673 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.900849 27691 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:31.952170 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:31.943845 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:31.953817 27566 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:31.842585 27447 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.957849 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:31.908808 27827 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:11:31.915947 27821 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:31.911113 27806 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.963733 22780 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:31.964304 22780 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:31.905846 27742 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.918757 27672 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.921619 27822 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:31.941066 27697 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:31.859215 27794 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.939533 27071 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:31.898980 27632 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:11:31.983916 27813 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.984556 27813 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.987392 27813 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.989223 27813 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.989801 27813 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:31.909646 27528 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.967653 22780 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.945547 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:31.978546 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:31.967025 27669 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:11:31.981913 27839 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.913110 27807 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.909868 27828 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:31.948385 27626 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.954560 27778 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:31.954850 27823 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:31.929433 27704 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:31.968237 27771 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:32.004609 22635 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
W20260709 13:11:31.992400 27662 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.000s	sys 0.019s
I20260709 13:11:31.967592 27820 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:32.011968 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:32.012912 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52658.1us
I20260709 13:11:32.009481 22778 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:32.020042 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.020834 22704 raft_consensus.cc:2368] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14.
I20260709 13:11:31.968745 27674 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:11:31.979079 27697 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:31.980145 27651 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:31.962728 27797 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:11:31.909153 27561 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.023455 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.007891 27821 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:32.006923 27753 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.024247 27697 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:32.025966 27625 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.026784 27831 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:11:31.974476 27841 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:11:32.028774 27837 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:11:31.976481 27691 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:11:32.065287 22710 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:32.030931 27807 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.031356 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.017359 27839 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:32.021581 27662 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.000s	sys 0.020s
I20260709 13:11:32.014196 27422 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:32.016831 27742 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:32.038178 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.022660 27823 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.038462 27818 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.039256 27845 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:32.021099 27626 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.041124 27813 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.091228 27422 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:32.092504 27422 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.095073 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.095875 22627 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:11:32.042363 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:32.044078 27447 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.045320 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:32.049295 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103859us
W20260709 13:11:32.049899 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 71843.5us
I20260709 13:11:32.048239 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:32.063539 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.067106 22707 raft_consensus.cc:3060] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:32.069538 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.111595 22701 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:32.105127 23415 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:32.112968 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.113358 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:32.116195 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:32.118466 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70324.4us
I20260709 13:11:32.119247 22778 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:32.119638 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.123680 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.124491 22627 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:32.129750 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.099418 27845 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:32.137501 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:32.139693 27621 consensus_queue.cc:260] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.141021 27621 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:32.143821 27823 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.145946 23418 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:32.127281 27807 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:32.147987 27807 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.148430 27807 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:32.154515 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:32.158316 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2966) took 1515 ms. Trace:
I20260709 13:11:32.161691 22779 rpcz_store.cc:276] 0709 13:11:30.643335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.987762 (+344427us) service_pool.cc:224] Handling call
0709 13:11:30.988088 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.131857 (+143769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.158020 (+1026163us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540015a9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:32.158239 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329058560}
I20260709 13:11:32.046465 27821 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:32.163404 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.163493 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:32.168326 22779 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:32.167114 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:32.171141 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.171640 22707 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:32.129642 27828 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:32.127928 27680 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.174907 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.131296 27809 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:32.126130 27628 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 13:11:32.202015 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.207710 22779 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:32.210027 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.188578 27700 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.211834 22783 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:32.215615 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.187642 27828 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:32.193007 27809 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:32.225051 27809 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:32.216213 27845 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.226212 27845 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:32.226718 27700 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.176702 27632 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:32.218331 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:32.221757 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:32.222020 27850 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:11:32.223539 27818 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.217509 27828 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:32.237504 22703 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:11:32.176401 27723 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:11:32.260738 22703 raft_consensus.cc:1275] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:32.262805 27626 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.263447 27626 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:32.263717 27605 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:32.263854 27626 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:32.264297 27605 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:32.262372 27855 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:11:32.223562 27821 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.237879 27632 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:32.265214 27605 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 10, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.235926 27680 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:32.275327 27680 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.279076 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.280336 22703 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:32.242357 27794 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.248915 27528 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.249042 27739 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:32.264997 27626 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.286145 22703 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:32.279174 27632 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:32.277494 27837 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:32.304628 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:32.308140 27857 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:11:32.308616 27700 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.296362 27739 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.327244 27626 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:32.329313 27839 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.343833 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.348059 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.349130 22704 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:11:32.353852 22633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2729) took 2245 ms (client timeout 2095 ms). Trace:
I20260709 13:11:32.336740 27605 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:11:32.354460 22633 rpcz_store.cc:269] 0709 13:11:30.107917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.108112 (+   195us) service_pool.cc:224] Handling call
0709 13:11:32.353572 (+2245460us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b540003b1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:32.353716 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188496768}
I20260709 13:11:32.367542 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.373183 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2967) took 1729 ms. Trace:
I20260709 13:11:32.373761 22781 rpcz_store.cc:276] 0709 13:11:30.643633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:30.989014 (+345381us) service_pool.cc:224] Handling call
0709 13:11:30.989366 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:31.133924 (+144558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:32.372815 (+1238891us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b5400324bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:32.373093 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330298752}
I20260709 13:11:32.379413 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.380333 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.381806 22703 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:32.322409 27837 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.395264 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.395488 27697 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.397087 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.398075 27691 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:32.398854 27691 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:32.399297 27697 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.356272 27739 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.400413 27691 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:32.401616 27853 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:32.403402 27632 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.378574 27807 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.405702 27632 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.389346 27061 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:32.410858 27061 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.409307 27680 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.428717 22782 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:32.433537 22675 leader_election.cc:400] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:11:32.435139 22675 leader_election.cc:403] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:32.390769 27820 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:32.447530 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.448289 22703 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:32.442553 27691 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:11:32.488888 22703 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:32.493219 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.499701 22704 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:32.474418 27820 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:32.442955 27697 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:11:32.514349 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.515728 22704 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:32.495309 27061 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.532423 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.542263 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.544512 22704 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:32.557521 27107 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:32.558107 27107 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:32.559369 27107 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.561059 27107 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:32.564842 27828 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.576467 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.577276 22704 raft_consensus.cc:2376] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10. Re-sending same reply.
I20260709 13:11:32.586001 22780 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:32.588840 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:32.589366 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:32.589737 23418 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:32.590054 22633 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:11:32.593401 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:32.604491 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94294.1us
I20260709 13:11:32.573671 27807 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.608721 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.611428 22627 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:11:32.609509 22704 raft_consensus.cc:2376] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12. Re-sending same reply.
I20260709 13:11:32.613050 22701 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:32.611876 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.614298 22707 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 7: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:32.616456 22780 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:11:32.616882 22780 pending_rounds.cc:85] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 6
I20260709 13:11:32.610507 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:32.556736 27820 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.617858 22780 pending_rounds.cc:107] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:11:32.619273 23418 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:11:32.621359 22780 raft_consensus.cc:2889] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:32.620419 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:32.621989 22631 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:32.622105 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.623220 22707 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:32.624202 27828 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.624539 27061 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:32.628309 27809 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:32.634191 27867 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:11:32.637573 27704 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:32.641423 27704 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:32.642482 27704 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.644177 27704 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:32.621064 27807 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:32.647946 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.649080 22701 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:32.655884 23411 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:32.622679 27821 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:11:32.659018 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.648679 27807 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:32.671461 22631 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:32.671790 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.663273 27691 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:11:32.673916 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.671952 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.678690 22704 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:11:32.681690 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:32.682425 22708 raft_consensus.cc:2376] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12. Re-sending same reply.
I20260709 13:11:32.687929 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:32.672350 27807 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:32.691152 22708 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:32.694677 23411 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:11:32.697721 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:32.694869 22749 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:32.703037 22633 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:32.703804 27707 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:32.705260 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.705232 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.707185 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.708449 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.709049 23385 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.709782 27764 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.710139 22780 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:32.713204 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.718863 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.726238 22631 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:32.725515 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.706116 27500 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:32.727349 22634 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:11:32.727890 22780 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:11:32.707054 27662 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:32.729223 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.730244 22627 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:32.724884 23385 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.737721 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:32.738323 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.716883 27821 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:32.739537 22634 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:11:32.739833 22627 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:32.706818 27805 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:11:32.753402 22780 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:32.739773 27811 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:32.740003 27818 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:11:32.761907 27811 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.759023 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:32.764734 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:32.765147 22783 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:32.765507 22780 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:32.761031 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:32.732736 27500 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:32.717164 27764 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.767480 27500 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:32.769068 27764 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.712215 27828 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:11:32.769757 22703 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:32.771306 27821 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:32.763348 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.751767 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.771363 27705 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:32.775511 22631 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:32.776597 23384 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:32.774466 27705 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.778211 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:32.737161 27632 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:32.737200 27766 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.782214 22631 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:32.782753 22634 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:32.759495 27626 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.787194 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:32.772482 27821 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.792268 27766 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.810842 27278 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:32.766557 27818 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:11:32.816114 22783 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:32.805807 27626 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.785120 27828 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:11:32.835243 27811 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.840049 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:32.781170 27662 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:32.806708 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:32.911034 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 46816.9us
W20260709 13:11:32.913887 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 137375us
I20260709 13:11:32.839977 27878 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:11:32.916224 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 109600us
I20260709 13:11:32.875713 27766 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.843020 27278 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:32.891647 27395 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.936604 27395 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.938834 27729 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:11:32.939402 27691 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:32.939972 27691 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:32.941059 27691 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.939394 27729 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:32.943612 27729 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:11:32.945724 27705 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.948472 27822 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:32.935429 27278 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.952173 27278 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:32.939523 27500 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:32.955183 27500 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.955343 27395 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:32.955906 22779 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:32.958046 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:32.959138 23418 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:32.942490 27691 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:32.961966 27660 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:32.962630 27660 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:32.963469 27704 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.912703 27879 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:11:32.964092 27704 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:32.967892 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.930827 27878 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:32.985822 27878 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:32.988015 27626 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.994379 27697 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:32.992367 27729 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.995327 27697 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:32.997221 27729 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:32.998109 27697 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:32.994485 27704 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:33.001911 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:32.957890 27837 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.009637 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 700) took 1200 ms. Trace:
I20260709 13:11:33.010922 22633 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:32.952056 27662 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:33.012338 23412 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 931) took 1371 ms. Trace:
I20260709 13:11:33.019536 23412 rpcz_store.cc:276] 0709 13:11:31.641146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.668359 (+ 27213us) service_pool.cc:224] Handling call
0709 13:11:31.668696 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:31.703713 (+ 35017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.012071 (+1308358us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b540020a154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:33.012278 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65146752}
W20260709 13:11:33.021536 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:33.013398 22709 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:11:33.015041 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.015560 22704 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:33.014191 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.024777 22707 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:11:33.016710 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.017649 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:33.027452 22705 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 5: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:33.010123 22635 rpcz_store.cc:276] 0709 13:11:31.809418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.809594 (+   176us) service_pool.cc:224] Handling call
0709 13:11:33.009417 (+1199823us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b540022e054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:33.009563 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341972864}
I20260709 13:11:33.018666 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.029184 22710 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:33.035323 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 929325us
I20260709 13:11:33.035856 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.036729 22702 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:33.044329 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.054318 22705 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:11:33.054165 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.007556 27660 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.030877 27500 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:11:33.060460 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:33.077029 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:33.052532 27697 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.080098 22702 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:11:33.077373 22705 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:33.095690 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.096624 22706 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:33.052198 27729 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:11:33.100900 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1091) took 1307 ms. Trace:
I20260709 13:11:33.101444 23416 rpcz_store.cc:276] 0709 13:11:31.793384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.793579 (+   195us) service_pool.cc:224] Handling call
0709 13:11:33.100669 (+1307090us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b54005f7654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:33.100827 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":99210112}
I20260709 13:11:33.103084 23415 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:33.045405 27662 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:33.056012 27705 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:33.106680 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:33.056589 27632 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:33.058353 27781 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:33.080651 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.109216 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.107398 22707 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:33.111572 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.114223 27662 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.114809 27624 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:33.115383 27624 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.118316 27764 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:33.119109 27764 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.123235 27764 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.129748 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.130689 22706 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:33.132640 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.135448 22626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2150) took 2145 ms (client timeout 1742 ms). Trace:
W20260709 13:11:33.135952 22626 rpcz_store.cc:269] 0709 13:11:30.989609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.051046 (+ 61437us) service_pool.cc:224] Handling call
0709 13:11:33.135148 (+2084102us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54003d2f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:33.135363 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1949541120}
I20260709 13:11:33.137240 27783 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:33.137745 27783 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:33.138192 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:33.138729 27783 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.166597 22631 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:33.140955 27809 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.107594 27697 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.167507 27764 raft_consensus.cc:2764] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:11:33.168288 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:33.166975 22627 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:11:33.170131 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:33.172212 22626 pending_rounds.cc:85] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 9
I20260709 13:11:33.172605 22626 pending_rounds.cc:107] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:11:33.172909 22626 raft_consensus.cc:2889] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:33.174219 27278 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:33.117762 27624 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.175297 27807 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.180274 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.183415 27691 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.184432 22777 raft_consensus.cc:1240] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:33.207744 22703 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:33.207777 27691 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.191016 27879 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:33.193045 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:33.203608 27846 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:33.211041 27846 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:33.210716 27879 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.211352 27624 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:33.211539 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:33.212069 27624 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:33.212141 27846 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.213305 27624 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.213729 27846 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:33.181151 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:33.222891 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 276847us
I20260709 13:11:33.223634 22706 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:11:33.223842 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:33.228835 22707 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:33.229283 22634 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:33.238137 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:33.243647 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:33.244280 22632 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:33.210815 27807 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.266297 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.268555 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 939) took 1104 ms. Trace:
I20260709 13:11:33.269479 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 2990) took 1260 ms. Trace:
I20260709 13:11:33.273257 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2987) took 1312 ms. Trace:
I20260709 13:11:33.223038 27783 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:33.282866 23415 rpcz_store.cc:276] 0709 13:11:32.164459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.164592 (+   133us) service_pool.cc:224] Handling call
0709 13:11:33.102860 (+938268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.103624 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.268244 (+164620us) spinlock_profiling.cc:243] Waited 938 ms on lock 0x7b54005f7650. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:33.268467 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2157043456}
I20260709 13:11:33.282203 22632 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:33.282732 22777 rpcz_store.cc:276] 0709 13:11:31.961345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.988656 (+ 27311us) service_pool.cc:224] Handling call
0709 13:11:31.989057 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:33.208006 (+1218949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.272888 (+ 64882us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400132254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:33.273166 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2748627840}
I20260709 13:11:33.282939 22778 rpcz_store.cc:276] 0709 13:11:32.009542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.036848 (+ 27306us) service_pool.cc:224] Handling call
0709 13:11:33.269260 (+1232412us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b540019b254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:33.269401 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27202432}
I20260709 13:11:33.214429 27764 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:33.288964 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:33.307335 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.307883 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:33.289793 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:33.308913 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.290503 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.310969 22631 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:33.311029 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.291399 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:33.311745 22779 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:33.311865 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.312429 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.313395 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.313966 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.314595 22777 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:33.316305 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:33.292142 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.297448 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 185955us
I20260709 13:11:33.263834 27500 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.279822 27782 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:11:33.297633 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.280906 27278 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:33.235955 27809 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.307802 22635 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:33.316406 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:33.304152 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:33.319406 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.263566 27071 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.293749 22634 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:33.293697 27705 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.327741 22777 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:33.340662 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.339387 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.307081 27764 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:33.367597 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:33.356369 27071 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.374259 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.415778 22777 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:33.366950 23417 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:11:33.417126 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.390894 22783 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:33.376547 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:33.359179 27705 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.433835 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:33.370297 27823 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:33.369977 27660 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:33.367193 27811 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:33.329255 23411 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:33.328171 27278 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:33.365715 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:33.455021 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 89383us
I20260709 13:11:33.360713 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 2983) took 1402 ms. Trace:
I20260709 13:11:33.371990 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:33.400851 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:33.321440 27794 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:33.365639 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:33.411087 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.326066 27395 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.319265 27500 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.314895 27707 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:11:33.479525 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.499846 22707 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:33.414337 27766 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.325431 27691 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.456514 22784 rpcz_store.cc:276] 0709 13:11:31.958093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:31.958265 (+   172us) service_pool.cc:224] Handling call
0709 13:11:31.958704 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:32.159001 (+200297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:32.159038 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.173245 (+ 14207us) log.cc:844] Serialized 26 byte log entry
0709 13:11:32.270815 (+ 97570us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:32.270872 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.360237 (+1089365us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400325ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:33.360304 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:33.360355 (+    51us) raft_consensus.cc:1685] finished
0709 13:11:33.360368 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:33.360637 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34062,"raft.queue_time_us":39332,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":11223,"spinlock_wait_cycles":381003648,"thread_start_us":97151,"threads_started":1,"wal-append.queue_time_us":101790,"wal-append.run_cpu_time_us":1814,"wal-append.run_wall_time_us":1001818}
I20260709 13:11:33.436441 27879 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.343585 27809 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.366648 27742 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:33.516813 22627 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:33.451323 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:33.453025 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:33.497733 22781 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:33.431159 27764 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:33.501376 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.495785 27605 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:11:33.515077 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.019s	sys 0.029s
I20260709 13:11:33.520756 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:33.532994 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.020s	sys 0.029s
W20260709 13:11:33.533512 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 618539us
W20260709 13:11:33.522470 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 71256.7us
I20260709 13:11:33.446213 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.457008 27697 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.539683 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:33.457826 27278 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:33.424218 22629 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 5.
I20260709 13:11:33.448634 27887 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.113s
W20260709 13:11:33.555081 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:33.502416 27395 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.569502 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.573122 22708 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:33.543418 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:33.537673 27878 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.551s	user 0.015s	sys 0.000s
I20260709 13:11:33.546108 27879 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:33.575820 27878 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.457420 27811 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.517930 27691 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:11:33.601408 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.602531 22708 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:33.513511 27766 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.532312 27742 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.602995 27821 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:33.617195 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.611382 27605 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:33.649703 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:33.638140 27821 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:33.644841 27605 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:33.656131 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 416712us
I20260709 13:11:33.645961 27697 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.663237 22708 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:33.618450 27672 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:11:33.677839 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.665287 27779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.008s	sys 0.009s
I20260709 13:11:33.656025 27782 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:33.656375 27821 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:33.687777 22708 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:11:33.681751 27888 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.691488 22701 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:11:33.700778 22707 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:33.696667 27888 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.671330 27878 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.667232 27500 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.712843 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.713596 22707 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
W20260709 13:11:33.688400 27779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.008s	sys 0.009s
I20260709 13:11:33.680684 27705 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:33.720157 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.724746 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.735899 22707 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:33.736208 22775 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
W20260709 13:11:33.721196 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 265199us
W20260709 13:11:33.710775 27820 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.009s	sys 0.007s
W20260709 13:11:33.740247 27820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.009s	sys 0.007s
I20260709 13:11:33.693255 27782 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:33.674423 27697 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:33.746493 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 427172us
I20260709 13:11:33.752144 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.753280 22775 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:33.759001 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:33.760288 22784 raft_consensus.cc:2410] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:11:33.766816 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:33.732899 27672 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.799472 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:33.717070 27809 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.805712 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.809204 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.747880 27782 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:33.715359 27605 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.811545 27071 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.805020 27395 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.827044 27820 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:33.802537 27705 consensus_queue.cc:1048] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:11:33.829191 27705 consensus_queue.cc:1243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:11:33.801204 27794 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:33.831218 27794 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.811156 27672 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.833658 27794 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:33.812649 27707 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:33.834573 27766 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.810348 27805 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:33.835371 27805 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:33.835472 27766 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:33.835865 27805 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:33.835906 27766 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:33.836946 27766 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.838135 27672 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:33.838382 27605 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:33.809923 27728 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:33.840354 27728 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.842891 27697 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:33.801528 27811 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:33.855418 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46307.5us
I20260709 13:11:33.860373 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.812439 27823 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:33.870719 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.875811 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:33.871682 27890 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:11:33.863341 27899 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.885557 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.861485 27781 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:33.912896 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.879315 27728 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:33.881462 27811 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.882299 27697 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.890452 27822 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:33.934235 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.937194 22703 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:33.931846 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:33.939353 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.942907 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:33.948623 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:33.910915 27781 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:33.956936 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:33.961365 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:33.929082 27660 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:33.890246 27729 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:11:33.930352 27841 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.932423 27707 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:33.941756 27728 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.960258 27781 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:33.967042 27107 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:33.982728 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 243193us
W20260709 13:11:33.985601 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 319317us
I20260709 13:11:33.986268 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:33.988559 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:33.989737 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:33.991897 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:33.959882 27395 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:11:33.937614 27811 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.976718 27910 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:33.918567 27899 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.943912 27697 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:33.967289 27278 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:33.994069 27107 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:33.981772 27728 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:34.034330 27908 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:11:34.042181 22778 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:34.042505 23419 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.013s	sys 0.007s
I20260709 13:11:34.021242 27672 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:34.026338 27107 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:34.048290 23419 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.013s	sys 0.007s
I20260709 13:11:34.020862 27916 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:34.040009 27898 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:34.039650 27624 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:33.977294 27841 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.080870 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.067587 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:34.048390 27897 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:11:34.063902 27916 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:34.079455 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 87613.4us
I20260709 13:11:34.090420 22629 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
W20260709 13:11:34.104938 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 115327us
I20260709 13:11:34.069242 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.124861 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:34.128106 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 140076us
I20260709 13:11:34.018730 27910 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.134913 22630 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:34.080371 27624 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.059383 27914 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:34.044174 27917 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:11:34.150429 27915 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.151291 27915 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:34.133658 27783 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.826s	user 0.012s	sys 0.003s
I20260709 13:11:34.107301 27898 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:34.162531 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.124735 27916 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:34.135932 27823 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:34.141232 27894 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:11:34.192664 27672 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.154570 27915 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.138832 27728 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.148926 27624 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:34.157121 27901 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:11:34.213742 22703 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:34.148032 27914 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:34.243081 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127685us
I20260709 13:11:34.246308 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:34.217732 27896 log.cc:927] Time spent T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
W20260709 13:11:34.230877 27900 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.294s	user 0.002s	sys 0.000s
W20260709 13:11:34.255673 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 93637.9us
W20260709 13:11:34.253072 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142381us
I20260709 13:11:34.215003 27707 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.259337 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.265733 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:34.280233 27764 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.737s	user 0.017s	sys 0.000s
I20260709 13:11:34.280941 27764 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.285998 27764 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:34.266942 27907 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.250s	user 0.000s	sys 0.002s
I20260709 13:11:34.245260 27914 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:34.223943 27823 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.233206 27672 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.274664 27811 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.270813 27931 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:34.318509 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69274.4us
I20260709 13:11:34.274288 27707 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.381947 27707 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.314047 27278 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.308495 27697 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.351816 27811 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:34.384334 27936 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:34.401290 27936 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.401741 27936 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:34.403925 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:34.355800 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 3000) took 1047 ms. Trace:
I20260709 13:11:34.359561 27766 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:34.386909 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:34.353672 27823 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.363338 27899 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.383864 27841 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.413015 27811 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:34.413689 27811 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.371017 27742 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.316036 27931 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.410871 27707 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:34.415908 27931 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.416256 27707 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.416822 27707 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.379094 27943 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.379803 27937 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:34.362730 27918 log.cc:927] Time spent T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.304s	user 0.001s	sys 0.001s
W20260709 13:11:34.385008 27934 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:11:34.385455 27913 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
W20260709 13:11:34.385810 27919 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.230s	user 0.001s	sys 0.001s
I20260709 13:11:34.382854 27697 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.420258 27660 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.420629 27697 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.420953 27660 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.323655 27783 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.421964 27783 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.390292 27915 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:11:34.387655 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.423568 22629 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:34.424501 27828 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:34.425211 27881 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.335647 27929 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:34.426218 27929 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:34.426666 27929 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:34.399564 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 950) took 1260 ms. Trace:
I20260709 13:11:34.427623 23416 rpcz_store.cc:276] 0709 13:11:33.139475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.171419 (+ 31944us) service_pool.cc:224] Handling call
0709 13:11:33.171756 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:33.172165 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.172199 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:33.172840 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 13:11:33.618432 (+445592us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:33.618481 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.618562 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.395250 (+776688us) spinlock_profiling.cc:243] Waited 54.3 ms on lock 0x7b180048cbe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:34.399146 (+  3896us) raft_consensus.cc:1685] finished
0709 13:11:34.399182 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.399479 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1108,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":124943616,"thread_start_us":395088,"threads_started":1,"wal-append.queue_time_us":690293}
I20260709 13:11:34.428601 27910 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.410593 22778 rpcz_store.cc:276] 0709 13:11:33.308665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.313635 (+  4970us) service_pool.cc:224] Handling call
0709 13:11:34.355556 (+1041921us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b540033d454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:34.355719 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1495820672}
I20260709 13:11:34.400384 27902 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.430691 27902 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.431015 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.431700 23418 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:11:34.431947 27626 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:34.432507 27626 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:34.432948 27822 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.433468 27822 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.399259 27779 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:11:34.425922 27881 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.425050 27828 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.436151 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3081) took 1388 ms. Trace:
I20260709 13:11:34.397691 27888 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.434340 27729 consensus_queue.cc:1059] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
W20260709 13:11:34.388037 27920 log.cc:927] Time spent T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.280s	user 0.000s	sys 0.001s
W20260709 13:11:34.388675 27933 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:11:34.435034 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.290395 27898 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.436677 22710 rpcz_store.cc:276] 0709 13:11:33.047426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.047876 (+   450us) service_pool.cc:224] Handling call
0709 13:11:33.053066 (+  5190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:33.170183 (+117117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:33.170215 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:33.170800 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:11:33.616605 (+445805us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:33.616668 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:34.435548 (+818880us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54004100d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:34.435642 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.435713 (+    71us) raft_consensus.cc:1685] finished
0709 13:11:34.435734 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.436032 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1273,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":95,"spinlock_wait_cycles":268446080,"thread_start_us":444702,"threads_started":1,"wal-append.queue_time_us":644980}
I20260709 13:11:34.387229 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.490152 22634 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:34.388012 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:34.406051 27846 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.170s	user 0.011s	sys 0.006s
I20260709 13:11:34.425098 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.418256 27943 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.438834 27278 raft_consensus.cc:3055] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:11:34.404781 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:34.438490 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:34.441116 27886 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.082s	user 0.000s	sys 0.005s
W20260709 13:11:34.421330 27932 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:11:34.441195 27837 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:34.442329 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.442696 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 3003) took 1133 ms. Trace:
I20260709 13:11:34.102458 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:34.395704 27818 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.446269 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.447911 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:34.446485 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.453526 27632 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:34.382220 27605 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 13:11:34.492484 22626 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:34.433626 27626 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.494726 27278 raft_consensus.cc:740] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:34.496533 27910 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.527523 27278 consensus_queue.cc:260] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.528457 27910 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.529340 27278 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.497721 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 2106) took 1734 ms. Trace:
I20260709 13:11:34.509681 22705 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:34.531246 22776 rpcz_store.cc:276] 0709 13:11:32.763658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:32.767302 (+  3644us) service_pool.cc:224] Handling call
0709 13:11:32.767622 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:32.777601 (+  9979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:32.777623 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:32.787175 (+  9552us) log.cc:844] Serialized 26 byte log entry
0709 13:11:33.037496 (+250321us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:33.037630 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:33.037732 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:34.440802 (+1403070us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b24003d4f78. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:11:34.497369 (+ 56567us) raft_consensus.cc:1685] finished
0709 13:11:34.497397 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:34.497654 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9533,"raft.queue_time_us":11577,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":1842,"spinlock_wait_cycles":16911232,"thread_start_us":146181,"threads_started":1,"wal-append.queue_time_us":465410}
I20260709 13:11:34.533274 27783 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.506378 27881 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.521864 27943 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.496410 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:34.537177 27779 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:11:34.538259 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:34.538450 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 180068us
I20260709 13:11:34.539788 27943 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:34.540306 27943 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.497292 27820 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.540910 27943 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:34.541281 27942 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:11:34.499156 22781 rpcz_store.cc:276] 0709 13:11:33.309589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:33.312169 (+  2580us) service_pool.cc:224] Handling call
0709 13:11:34.442507 (+1130338us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54001883d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:34.442636 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424347520}
I20260709 13:11:34.500610 27632 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.543563 27811 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.544173 27811 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:34.544872 27660 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.544948 27632 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.500820 27878 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:34.545845 27822 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.546154 27878 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:34.547313 27878 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:34.549974 27782 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.002s
I20260709 13:11:34.550406 27820 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.550675 27782 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.552693 27820 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:34.553153 27805 consensus_meta.cc:287] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.002s
I20260709 13:11:34.499552 27818 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.553756 27805 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.512495 27902 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.555075 23412 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:34.498757 27697 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:34.500321 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53933.9us
I20260709 13:11:34.558468 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:34.559175 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:34.559777 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.560134 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:34.561235 27816 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:34.564114 27881 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:34.564575 27881 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:34.564949 27881 raft_consensus.cc:3060] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.567260 22633 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
W20260709 13:11:34.493497 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 74653.3us
I20260709 13:11:34.571532 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:34.572522 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 315828us
I20260709 13:11:34.573304 22749 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:34.572314 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:34.575937 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:34.582966 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.584084 23419 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:34.577813 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.585306 23421 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:34.589610 23421 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:34.596316 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.597155 23420 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:34.594406 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.598795 23416 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:11:34.594365 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.522398 27672 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.594581 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:34.606943 27107 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 13:11:34.556931 27805 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.614300 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:34.574158 27779 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:34.609431 27626 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:34.583318 27888 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:34.595330 27846 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:11:34.609769 27936 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.594048 27941 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.574493 27697 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.514220 27828 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.586004 27917 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:34.608490 27931 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.577147 27942 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:34.646832 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 381197us
I20260709 13:11:34.648058 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.498350 27837 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:34.616083 27728 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:34.634654 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58449.9us
I20260709 13:11:34.616916 27779 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.618924 27818 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.630128 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:34.639382 27821 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.005s
I20260709 13:11:34.634377 27823 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:34.616374 27107 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.617250 27624 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.631649 27888 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:34.659456 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:34.642205 27820 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:11:34.641217 27917 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.639736 27697 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.643635 27910 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.671902 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.673616 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.770318 22629 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:11:34.675710 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.771494 23418 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:34.647313 27942 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:34.735569 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:34.776213 22784 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:34.738756 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.740334 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.658972 27728 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.736931 27782 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:34.660431 22708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.000s	sys 0.015s
I20260709 13:11:34.681916 22775 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
W20260709 13:11:34.788812 22708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.000s	sys 0.016s
I20260709 13:11:34.788928 22775 pending_rounds.cc:85] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 6
I20260709 13:11:34.789381 22775 pending_rounds.cc:107] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:11:34.789685 22775 raft_consensus.cc:2889] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:34.658860 27816 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:34.657126 27807 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:34.675307 27805 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.780570 27818 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:34.794795 22631 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:34.763965 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:34.668365 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.795217 27818 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.760491 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:34.794941 27805 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.797964 27818 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:34.800429 27805 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.801002 27805 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.801482 27805 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.651703 27929 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.667203 27779 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.678982 27823 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:34.772068 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 100244us
I20260709 13:11:34.734643 22783 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:34.773716 22632 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:11:34.670866 27903 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.634851 27941 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.777307 22628 raft_consensus.cc:2368] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12.
I20260709 13:11:34.779105 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:34.845754 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 66793.6us
I20260709 13:11:34.783852 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:34.734308 27660 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:34.742805 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83480.6us
I20260709 13:11:34.848676 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:34.733844 27783 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.680632 27822 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:34.830307 27916 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.000s
W20260709 13:11:34.767059 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 205811us
I20260709 13:11:34.824311 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:34.861009 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.767939 27917 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:34.768569 27902 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:34.835189 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.679227 27422 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
W20260709 13:11:34.841157 22674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.019s	sys 0.039s
W20260709 13:11:34.873998 22674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.019s	sys 0.039s
I20260709 13:11:34.875730 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.764880 27944 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:34.827517 27947 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:34.792791 27728 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.681475 27821 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.862528 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.891384 22631 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:34.802543 27707 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.865788 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.804622 27805 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:34.902150 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:34.866947 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:34.866981 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.841176 27823 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.878134 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:34.784910 27942 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:34.881590 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:34.883591 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:34.886080 22708 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:34.890384 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:34.915849 22704 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:34.916375 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:34.891579 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:34.918020 22708 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:34.918329 22710 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:34.864877 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:34.894896 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 155060us
I20260709 13:11:34.898556 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:34.856498 27898 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:34.923592 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:34.795392 27816 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.910781 22705 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:34.855412 27822 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.922010 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:34.939029 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:34.903909 22629 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:34.934958 23411 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:34.772809 27837 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.874095 27910 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:34.884106 27915 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:34.896199 27766 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.897994 27948 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:34.864529 27783 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.883713 27944 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.850274 27943 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:34.965674 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 98802.6us
I20260709 13:11:34.943856 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:34.966863 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:34.966928 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:34.890544 27811 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.974921 27811 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.977430 27811 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.851147 27903 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.937850 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:34.954093 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:34.982537 22778 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
W20260709 13:11:34.953928 27781 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.954s	user 0.014s	sys 0.005s
I20260709 13:11:34.863534 27660 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.884485 27902 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:34.960225 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61777.6us
I20260709 13:11:34.921962 27898 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:34.973889 27632 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:34.942466 27822 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:34.978551 27807 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:34.936789 27947 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:34.941097 27846 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:34.870421 27916 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:34.959528 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78052.4us
W20260709 13:11:34.990115 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52398.8us
I20260709 13:11:34.951294 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:35.026228 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:34.999221 22709 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:11:34.962602 27936 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:34.978823 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 198827us
I20260709 13:11:35.027971 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.028070 22747 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:35.029070 22775 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:35.031884 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:34.998482 27898 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:34.997664 27931 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.018280 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.048336 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:35.048907 22779 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:35.049547 27941 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.899096 27888 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.023048 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.050930 22702 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:35.052186 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.053161 22702 raft_consensus.cc:2393] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.054097 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:34.978269 27764 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:35.055233 27764 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.055828 27764 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:35.038539 22775 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:11:35.007421 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:34.998677 27820 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:34.960430 27766 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.058216 22701 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:35.059846 27766 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:34.965495 27944 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:35.062078 27917 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.998973 27958 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:35.067766 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:35.068017 22779 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:35.069427 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.070216 22778 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:35.072638 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.067708 22702 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:35.079754 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.080657 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:35.082787 22702 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 13:11:35.083180 22780 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:35.074368 22775 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:35.093959 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.101588 22708 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:35.034827 27107 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.094944 22775 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:11:34.969733 27948 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:34.999493 27781 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:34.991724 27957 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 13:11:35.046775 27624 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.131074 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:35.109854 27914 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.782s	user 0.013s	sys 0.000s
I20260709 13:11:35.008495 27632 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.148604 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:35.146961 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:35.167019 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 140832us
I20260709 13:11:35.158560 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:35.111230 27958 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:35.183863 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 232668us
I20260709 13:11:35.183918 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.187863 22702 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:11:35.190441 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 164858us
I20260709 13:11:35.194888 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:35.139629 27940 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:35.197521 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 169550us
I20260709 13:11:35.197433 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:35.202132 22749 leader_election.cc:400] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12.
I20260709 13:11:35.209888 22749 leader_election.cc:403] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:11:35.143985 27820 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.155995 27945 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.213270 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:35.142228 27929 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.213786 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.214855 22628 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:35.173980 27963 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.140460 27697 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.225099 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.216427 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:35.171269 27950 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.150940 27914 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.183398 27960 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:11:35.231575 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:35.235569 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:35.195900 27742 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.162592 27878 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:11:35.197827 27821 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.173745 27766 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:35.196434 27962 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:35.211263 27949 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:35.250140 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.250579 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.394773 22703 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:35.396355 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:35.397125 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 161701us
I20260709 13:11:35.397994 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.268419 22634 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:35.269706 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.269765 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.270399 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.266331 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.407385 22632 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 16: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.255211 22601 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:35.408036 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 152953us
I20260709 13:11:35.225899 27945 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.277920 27742 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:35.423898 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:35.276002 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.222911 27930 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:35.201825 27624 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:11:35.226325 27107 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:11:35.329746 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.439687 22709 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:35.274811 27954 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:35.272666 27915 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:35.163002 27846 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.425494 27957 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.290133 27943 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.448824 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.255926 27960 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:35.435895 27278 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.906s	user 0.014s	sys 0.001s
W20260709 13:11:35.357092 27881 consensus_meta.cc:287] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.792s	user 0.006s	sys 0.006s
I20260709 13:11:35.223344 27820 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.252740 27961 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:11:35.476066 27961 consensus_queue.cc:1243] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:11:35.398218 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:35.267439 22626 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:11:35.482513 27971 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.147s	user 0.002s	sys 0.000s
W20260709 13:11:35.484385 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86293.9us
I20260709 13:11:35.492168 27955 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:35.492684 27955 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.493039 27955 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:35.496731 27837 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:35.499827 27823 consensus_meta.cc:287] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.000s
I20260709 13:11:35.500452 27823 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.502059 22782 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:35.503016 27948 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.503859 27632 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.505554 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.505888 27914 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.506042 27987 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:35.435640 27917 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.506549 27987 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:35.331984 27821 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:35.509023 27821 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.509491 27821 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:35.509405 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.425357 27929 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:11:35.511089 27929 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:35.511487 27929 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:35.512374 27929 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.513691 27929 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:35.514508 27697 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.218575 27940 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.515138 27697 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.515558 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.516286 22775 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:35.215380 27964 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:35.406093 22631 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:35.517177 27940 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.517387 27697 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.418264 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.529271 22634 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:11:35.517235 27964 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.400151 27949 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.422178 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.273970 22629 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:35.424643 22627 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 5.
W20260709 13:11:35.486043 27952 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.476s	user 0.002s	sys 0.000s
I20260709 13:11:35.393810 27811 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:35.358922 27691 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.428087 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:35.531162 27626 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.910s	user 0.005s	sys 0.009s
I20260709 13:11:35.399200 27903 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.323999 27977 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 13:11:35.249251 27963 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.444399 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.563900 22709 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:35.443190 27957 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.446926 27781 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.425590 27916 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.352782 27979 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:35.357892 27766 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:35.461683 27965 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:11:35.254866 27950 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.427102 27930 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.419168 27941 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:35.426291 27660 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.455839 27960 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:35.472276 27881 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.475420 27820 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.519294 27809 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:35.515166 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.525537 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.194801 27958 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.445421 27915 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.532626 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:35.537853 27818 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.011s
I20260709 13:11:35.391448 27902 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.552709 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.558068 27691 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.543465 27964 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:35.507594 27987 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.639740 27987 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:11:35.641364 27959 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.492s	user 0.001s	sys 0.001s
I20260709 13:11:35.624580 27915 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:35.648532 27967 log.cc:927] Time spent T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.476s	user 0.002s	sys 0.000s
I20260709 13:11:35.594883 27941 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.655272 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.658941 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:35.588447 27728 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.672s	user 0.011s	sys 0.002s
W20260709 13:11:35.578500 27779 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.002s
I20260709 13:11:35.557760 27811 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.468104 27962 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.589574 27930 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:35.546932 27949 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:35.626929 27818 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.003s	sys 0.011s
I20260709 13:11:35.632465 27950 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.634338 27881 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.635852 27766 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.635934 27958 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.671011 27821 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.663573 27779 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.673638 27779 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.675585 27809 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:35.676191 27809 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:35.676576 27962 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.665598 27728 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.678017 27955 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.678634 27707 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.682883 27728 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.636462 27742 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.683005 27986 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:35.687065 27986 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:35.688097 27986 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.689437 27986 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:35.689592 27823 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.691304 27957 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.692502 27811 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.626915 27945 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.684170 27781 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:35.695250 27781 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:35.696836 27898 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.625s	user 0.007s	sys 0.003s
I20260709 13:11:35.696503 27781 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.697327 27898 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:35.699441 27764 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.643s	user 0.016s	sys 0.000s
I20260709 13:11:35.699678 27963 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:35.699913 27764 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.701014 27964 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.707288 27941 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.708840 27964 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.698300 27781 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:35.710785 27822 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.003s
I20260709 13:11:35.711328 27822 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.713492 27822 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:35.665112 27998 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:11:35.703975 27942 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.756s	user 0.005s	sys 0.008s
I20260709 13:11:35.715758 27942 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:35.704699 27944 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.644s	user 0.015s	sys 0.000s
I20260709 13:11:35.718036 27944 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.722966 27930 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.725586 27930 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.721319 27944 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.676548 27821 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:35.731101 27984 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.379s	user 0.000s	sys 0.003s
I20260709 13:11:35.732775 27949 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.734871 27949 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.734870 27915 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.736001 27898 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:35.737186 27915 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.739308 27955 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.739471 27979 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12.
I20260709 13:11:35.740090 27960 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.565091 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.787051 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.789098 22630 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:35.485605 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.791270 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.793289 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.518069 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.838740 22784 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:35.840023 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.840583 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:35.841572 22631 raft_consensus.cc:2410] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:11:35.841393 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.428946 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.793444 22626 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:11:35.550880 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.549697 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.844645 22629 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.844802 22627 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.795254 22628 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:35.796792 22633 raft_consensus.cc:2368] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13.
I20260709 13:11:35.530545 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.847452 22632 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:11:35.797938 22635 raft_consensus.cc:2368] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
W20260709 13:11:35.580090 22706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.000s	sys 0.028s
I20260709 13:11:35.848306 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:35.848847 22706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.000s	sys 0.028s
I20260709 13:11:35.466379 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.851051 22708 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:35.851336 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:35.408059 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.853137 22703 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 7.
I20260709 13:11:35.466410 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.827378 22710 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:35.854598 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.855792 22705 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:35.856235 22634 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.854650 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.857689 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:35.546691 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 309368us
I20260709 13:11:35.858588 22703 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:35.859390 22705 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:35.860908 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:35.834295 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.598623 22776 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:35.863565 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.865538 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.594738 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.867599 22777 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:11:35.869053 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:35.601475 22778 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:35.836011 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.871361 22633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59768 (request call id 714) took 1010 ms. Trace:
I20260709 13:11:35.872169 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.874395 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.875053 22775 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:35.876030 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.876925 22701 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:11:35.876953 22775 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:11:35.836890 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.879372 22783 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:35.837127 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 782) took 1161 ms. Trace:
I20260709 13:11:35.880007 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.836016 22780 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:35.881342 22702 raft_consensus.cc:2410] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 16 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:11:35.880641 22781 rpcz_store.cc:276] 0709 13:11:34.675798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.675957 (+   159us) service_pool.cc:224] Handling call
0709 13:11:34.676340 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:34.676729 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.676755 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:34.765080 (+ 88325us) log.cc:844] Serialized 52 byte log entry
0709 13:11:35.085624 (+320544us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:35.085671 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.085741 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:35.836322 (+750581us) raft_consensus.cc:1685] finished
0709 13:11:35.836361 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:35.836815 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2509,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6698,"thread_start_us":244112,"threads_started":1,"wal-append.queue_time_us":408837}
I20260709 13:11:35.884967 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.851547 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.580894 22709 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:35.871929 22633 rpcz_store.cc:276] 0709 13:11:34.860459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.860628 (+   169us) service_pool.cc:224] Handling call
0709 13:11:35.871108 (+1010480us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b540003b6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:35.871281 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148036992}
I20260709 13:11:35.885965 22706 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:35.887125 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:35.887295 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.848002 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.888340 22709 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:35.890661 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.891608 22702 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:35.866600 22776 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:35.897291 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:35.861135 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.899096 22705 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:11:35.901055 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:35.901849 22775 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
I20260709 13:11:35.842546 22707 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:35.905136 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:35.928104 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:35.930145 22707 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:35.928192 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.935838 22630 raft_consensus.cc:2393] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.936192 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.936964 22632 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:35.940905 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:35.941041 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.941903 23384 leader_election.cc:400] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13.
I20260709 13:11:35.942363 23384 leader_election.cc:403] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:35.943167 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:35.944698 22631 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.907023 22525 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 15, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:35.948102 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.950326 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.953111 23411 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:35.955766 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.956514 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:35.957643 23419 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:35.957989 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.633158 23416 raft_consensus.cc:1240] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:35.960964 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 962) took 1311 ms. Trace:
I20260709 13:11:35.961421 23416 rpcz_store.cc:276] 0709 13:11:34.649070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.649507 (+   437us) service_pool.cc:224] Handling call
0709 13:11:34.663302 (+ 13795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:35.959565 (+1296263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.960432 (+   867us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b5400503554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:35.960719 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2229863296}
I20260709 13:11:35.963107 27107 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.963512 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.663829 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.965910 23420 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:35.964182 27107 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.949282 22525 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 13, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 13 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:35.969535 27422 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:35.993539 27422 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:35.972128 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:35.995980 22628 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:11:35.955766 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:35.998152 23417 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:11:35.979023 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:35.990514 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.003499 22634 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:35.702911 27764 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:35.983593 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.167038 22708 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:35.989629 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:36.170394 22701 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:35.970667 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:36.008879 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.217181 27107 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.220253 27422 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.004014 22525 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 16. New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:35.992048 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:36.008298 27782 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:36.223176 27782 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:35.966526 22526 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 238) took 18539 ms (client timeout 15000 ms). Trace:
I20260709 13:11:35.967911 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.223661 27782 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:36.224073 22710 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:36.224404 22633 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:36.051939 27624 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:36.025031 27783 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.007486 27779 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:36.034466 27697 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.227170 27697 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.229770 27697 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.230876 27910 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:36.231438 27910 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:36.223201 22525 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:36.054796 27672 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:36.050596 27837 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.193s
I20260709 13:11:36.027362 27818 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.216s
I20260709 13:11:36.040328 27691 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.246320 27960 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.248623 27942 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.249604 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:36.249897 27624 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.250903 27779 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.252389 27779 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.253629 27987 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:36.254038 27972 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.720s	user 0.000s	sys 0.005s
I20260709 13:11:36.254969 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.255869 27624 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.257665 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3124) took 1364 ms. Trace:
I20260709 13:11:36.259374 27964 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.258122 22704 rpcz_store.cc:276] 0709 13:11:34.892892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:34.917050 (+ 24158us) service_pool.cc:224] Handling call
0709 13:11:34.917611 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:34.965240 (+ 47629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:34.965285 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:34.981832 (+ 16547us) log.cc:844] Serialized 26 byte log entry
0709 13:11:35.147560 (+165728us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:35.147637 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:35.147737 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.252827 (+1105090us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:36.257063 (+  4236us) raft_consensus.cc:1685] finished
0709 13:11:36.257090 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.257442 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57819,"raft.queue_time_us":48080,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":19537,"spinlock_wait_cycles":15374720,"thread_start_us":133067,"threads_started":1,"wal-append.queue_time_us":522743}
I20260709 13:11:36.260237 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2634) took 1054 ms. Trace:
I20260709 13:11:36.260099 27964 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.260666 22710 rpcz_store.cc:276] 0709 13:11:35.205988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.244645 (+ 38657us) service_pool.cc:224] Handling call
0709 13:11:36.259937 (+1015292us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54000a78d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:36.260149 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":994998016}
I20260709 13:11:36.261511 22704 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:36.254745 27729 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
W20260709 13:11:36.146967 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 192346us
W20260709 13:11:36.255911 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83557.7us
I20260709 13:11:36.265791 22630 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:36.262398 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2618) took 1116 ms. Trace:
I20260709 13:11:36.267328 22704 rpcz_store.cc:276] 0709 13:11:35.145597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.260658 (+1115061us) service_pool.cc:224] Handling call
0709 13:11:36.261109 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.262037 (+   928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.262287 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.273363 22702 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:36.244297 27992 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:36.287686 27992 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:36.289379 27992 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.013844 27632 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.277s
I20260709 13:11:36.291083 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:36.292110 23417 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:36.051062 27878 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:36.293161 22708 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:36.223778 22526 rpcz_store.cc:269] 0709 13:11:17.426669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:17.430833 (+  4164us) service_pool.cc:224] Handling call
0709 13:11:18.207303 (+776470us) spinlock_profiling.cc:243] Waited 75.8 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:11:35.966107 (+17758804us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:29.009915 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:29.010698 (+   783us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:29.010766 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:11:29.010799 (+    33us) tablet.cc:662] Decoding operations
0709 13:11:29.012770 (+  1971us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:29.012870 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:29.012897 (+    27us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:11:29.228198 (+215301us) tablet.cc:701] Row locks acquired
0709 13:11:29.228263 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:11:29.228508 (+   245us) write_op.cc:270] Start()
0709 13:11:29.228705 (+   197us) write_op.cc:276] Timestamp: P: 1783602689228410 usec, L: 0
0709 13:11:29.228748 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:11:29.354997 (+126249us) log.cc:844] Serialized 10509 byte log entry
0709 13:11:29.966086 (+611089us) op_driver.cc:464] REPLICATION: finished
0709 13:11:30.121292 (+155206us) write_op.cc:301] APPLY: starting
0709 13:11:30.129927 (+  8635us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:30.569567 (+439640us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:30.569696 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:31.036072 (+466376us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b18003fb7a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806c1c0ec 00007f1806c1eaf7 00007f1806c1ffb0 00007f180a9341bb 00007f180a97b5d4 00007f180a8278da 00007f180a82a3cb 00007f180a829fd3 00007f180a928e34
0709 13:11:31.455517 (+419445us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:31.456194 (+   677us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=0,mrs_lookups=0
0709 13:11:31.456294 (+   100us) write_op.cc:312] APPLY: finished
0709 13:11:31.464561 (+  8267us) log.cc:844] Serialized 236 byte log entry
0709 13:11:31.600400 (+135839us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:31.697945 (+ 97545us) write_op.cc:454] Released schema lock
0709 13:11:31.731684 (+ 33739us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:31.789547 (+ 57863us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:31.795647 (+  6100us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":131784,"reported_tablets":51,"spinlock_wait_cycles":174387968,"child_traces":[["op",{"apply.queue_time_us":145286,"apply.run_cpu_time_us":62592,"apply.run_wall_time_us":1643781,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117203,"num_ops":27,"prepare.queue_time_us":468431,"prepare.run_cpu_time_us":24729,"prepare.run_wall_time_us":514993,"raft.queue_time_us":153592,"raft.run_cpu_time_us":13133,"raft.run_wall_time_us":142279,"replication_time_us":737123,"spinlock_wait_cycles":15895936,"thread_start_us":638548,"threads_started":5,"wal-append.queue_time_us":420502,"wal-append.run_cpu_time_us":58808,"wal-append.run_wall_time_us":2642846}]]}
I20260709 13:11:36.294127 22704 pending_rounds.cc:85] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:36.294660 22704 pending_rounds.cc:107] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:11:36.295002 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.296550 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.297585 27936 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:36.057360 27898 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:36.298511 22782 raft_consensus.cc:2468] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:36.065230 27766 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:36.295032 22704 raft_consensus.cc:2889] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:36.299731 27766 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.242560 22525 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 11, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:36.034574 27742 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.303200 27742 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:35.972748 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:36.297477 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2621) took 1151 ms. Trace:
I20260709 13:11:36.305545 22710 rpcz_store.cc:276] 0709 13:11:35.146297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.262010 (+1115713us) service_pool.cc:224] Handling call
0709 13:11:36.262449 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.297070 (+ 34621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.297099 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.297145 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:36.297175 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.297240 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.297374 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.024377 27278 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:36.254508 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:36.315330 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 374662us
I20260709 13:11:36.312765 27278 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:36.254601 27987 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.027959 27816 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:36.319021 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:36.318065 27278 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:36.327157 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:36.327762 27764 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.329392 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:36.301611 22525 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 12 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.302600 27766 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.338766 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:36.324854 22601 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.351444 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:36.343430 27992 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:36.313807 27936 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.352870 22705 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:11:36.353231 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 99081.3us
I20260709 13:11:36.323987 27962 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:36.360972 22632 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:36.361558 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:36.323648 27632 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.376770 22709 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:36.343573 27742 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.379551 22633 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:36.352221 27964 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:36.334280 27782 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.379468 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:36.398837 22703 consensus_meta.cc:287] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.539s	user 0.018s	sys 0.006s
I20260709 13:11:36.437400 22703 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:36.439239 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3136) took 1008 ms. Trace:
W20260709 13:11:36.400629 22784 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.010s	sys 0.013s
I20260709 13:11:36.406493 22630 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:36.408442 27979 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:36.409333 22525 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:36.398485 27805 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.146s	user 0.038s	sys 0.035s
W20260709 13:11:36.450167 27805 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.038s	sys 0.036s
W20260709 13:11:36.407696 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52645.1us
W20260709 13:11:36.421361 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 102516us
I20260709 13:11:36.421989 27962 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:36.429095 22705 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:11:36.440313 27422 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:36.440960 27632 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:36.446875 22784 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.010s	sys 0.013s
I20260709 13:11:36.425622 27764 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.458557 28002 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:36.479516 27764 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.481225 27962 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.484891 27782 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.486379 27936 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.492307 27903 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:36.493002 27903 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.493580 27903 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:36.494136 27818 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:36.494953 27818 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:36.439597 22707 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
W20260709 13:11:36.400265 22706 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.504s	user 0.017s	sys 0.001s
I20260709 13:11:36.427055 22710 raft_consensus.cc:2393] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:36.445155 22703 rpcz_store.cc:276] 0709 13:11:35.430341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.854217 (+423876us) service_pool.cc:224] Handling call
0709 13:11:36.439153 (+584936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:11:36.498430 22706 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:36.476611 27807 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.050s	sys 0.066s
W20260709 13:11:36.442938 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 132424us
I20260709 13:11:36.479905 28002 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.458866 22525 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 13, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 13 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:36.470842 27979 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:36.456249 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:36.517603 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47418 (request call id 2630) took 1362 ms. Trace:
I20260709 13:11:36.518160 22701 rpcz_store.cc:276] 0709 13:11:35.154811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.213183 (+1058372us) service_pool.cc:224] Handling call
0709 13:11:36.517497 (+304314us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:36.520087 27807 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.449s	user 0.051s	sys 0.066s
I20260709 13:11:36.526360 27807 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.526907 27973 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.527688 27973 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.527912 27807 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.530974 27956 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:36.531843 27956 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.532454 22525 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 12, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 12 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:36.540417 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 91160us
I20260709 13:11:36.543745 27956 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:36.516656 27811 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.508091 27632 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.560932 27979 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:36.557879 22778 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.010s	sys 0.013s
W20260709 13:11:36.562958 22778 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.010s	sys 0.014s
I20260709 13:11:36.565095 27973 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.566655 27987 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:36.566848 27811 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.539391 22525 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 7 to 9, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:36.572317 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 217493us
I20260709 13:11:36.581157 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:36.591909 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:36.588716 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.600193 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2631) took 1420 ms. Trace:
I20260709 13:11:36.572010 27962 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:36.612099 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:36.600900 22702 rpcz_store.cc:276] 0709 13:11:35.171131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.407233 (+1236102us) service_pool.cc:224] Handling call
0709 13:11:36.452802 (+ 45569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.584497 (+131695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.584539 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:36.591293 (+  6754us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:36.591338 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.591403 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.591530 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:11:36.626827 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 296730us
I20260709 13:11:36.629837 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.632901 27742 raft_consensus.cc:2764] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:11:36.636364 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.657383 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:36.641805 22748 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.012s	sys 0.013s
I20260709 13:11:36.658507 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:36.666146 22748 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.013s	sys 0.013s
W20260709 13:11:36.641729 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.908s	user 0.006s	sys 0.020s
I20260709 13:11:36.674728 28008 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:36.675515 27954 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.676723 27954 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.677320 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.678450 27742 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:36.679003 27742 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.679462 27742 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:36.674196 27903 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.682020 27903 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.683794 27766 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.684602 27766 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:36.686486 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 101231us
I20260709 13:11:36.687194 27766 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:36.687899 27781 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.957s	user 0.016s	sys 0.014s
I20260709 13:11:36.679394 27954 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.688694 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:36.641511 22779 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.009s	sys 0.016s
I20260709 13:11:36.689647 28008 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:36.690088 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:36.689649 22779 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.009s	sys 0.020s
I20260709 13:11:36.691053 22630 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:11:36.695331 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.175s	user 0.008s	sys 0.021s
W20260709 13:11:36.701408 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81174.6us
I20260709 13:11:36.709501 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.724254 27811 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.730875 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:36.734014 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.735633 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.735028 27818 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.736837 27946 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:36.738161 27946 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:36.738745 27946 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.739905 27946 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.743024 27970 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:36.699363 27878 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:36.571368 28002 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.710120 28008 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:11:36.752823 27672 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:36.714402 27422 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:36.712954 27898 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:36.777114 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 58829.6us
I20260709 13:11:36.776011 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:36.634474 27278 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.743731 27970 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:36.823479 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:36.745015 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 1.22532e+06us
I20260709 13:11:36.767199 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.767262 27672 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.807421 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.822947 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2623) took 1670 ms. Trace:
I20260709 13:11:36.821206 27949 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 13:11:36.823818 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 104917us
I20260709 13:11:36.827461 22633 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:36.818928 27898 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.833763 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.834700 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:36.838095 27987 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:36.838917 27987 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:36.839026 28007 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:11:36.839792 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 75189.6us
W20260709 13:11:36.840332 27914 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.040s	sys 0.024s
W20260709 13:11:36.840715 27914 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.040s	sys 0.024s
I20260709 13:11:36.840085 27987 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:36.841636 27809 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.125s	user 0.023s	sys 0.008s
I20260709 13:11:36.842110 27809 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.842809 27916 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:36.842944 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 794) took 1008 ms. Trace:
I20260709 13:11:36.843432 27916 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.844624 27929 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.844506 27916 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:36.845440 27809 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.845219 27929 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.846539 22626 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:36.850723 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:36.851325 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.851231 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:36.851850 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:36.852051 22632 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:36.840719 22601 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.854009 27978 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:36.854281 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.854702 27978 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.855333 23412 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:36.855966 27978 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.857697 27970 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.857897 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:36.860944 27278 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:36.805671 27878 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:36.871966 22627 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:36.823426 27422 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.872627 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:36.843447 22784 rpcz_store.cc:276] 0709 13:11:35.834290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.876176 (+ 41886us) service_pool.cc:224] Handling call
0709 13:11:35.876683 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:35.877394 (+   711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.877428 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:35.878130 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:11:36.472905 (+594775us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:36.472965 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.473065 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.842687 (+369622us) raft_consensus.cc:1685] finished
0709 13:11:36.842708 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.842875 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":171776,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":156,"thread_start_us":594222,"threads_started":1,"wal-append.queue_time_us":816207}
I20260709 13:11:36.798188 27973 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:11:36.827984 22704 rpcz_store.cc:276] 0709 13:11:35.152613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.293213 (+1140600us) service_pool.cc:224] Handling call
0709 13:11:36.293788 (+   575us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:11:36.301075 (+  7287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.301110 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:11:36.301945 (+   835us) log.cc:844] Serialized 78 byte log entry
0709 13:11:36.435513 (+133568us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:36.457788 (+ 22275us) log.cc:844] Serialized 14 byte log entry
0709 13:11:36.505880 (+ 48092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.506083 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.822666 (+316583us) raft_consensus.cc:1685] finished
0709 13:11:36.822688 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.822880 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8061,"prepare.queue_time_us":21799,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":62,"raft.queue_time_us":12037,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":23707,"thread_start_us":132982,"threads_started":1,"wal-append.queue_time_us":516443}
I20260709 13:11:36.890595 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3025) took 1819 ms. Trace:
I20260709 13:11:36.891299 22779 rpcz_store.cc:276] 0709 13:11:35.071557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.071768 (+   211us) service_pool.cc:224] Handling call
0709 13:11:35.072167 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:35.402507 (+330340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:35.402550 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:35.403133 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:11:36.691379 (+1288246us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:36.691525 (+   146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.691687 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:36.890298 (+198611us) raft_consensus.cc:1685] finished
0709 13:11:36.890323 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:36.890523 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90836,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":15248,"thread_start_us":1617577,"threads_started":2,"wal-append.queue_time_us":1430713}
I20260709 13:11:36.892853 27672 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:36.894230 27672 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:36.836393 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:36.898983 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:36.901146 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:36.906899 27878 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:36.917106 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 85767.5us
I20260709 13:11:36.917972 22707 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:36.871061 22600 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:39075 timed out after 1.643s (SENT) [suppressed 5 similar messages]
I20260709 13:11:36.912822 22630 raft_consensus.cc:606] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:36.852764 27929 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:36.881299 27964 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.935479 22702 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:36.933068 22701 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:36.934208 23412 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:36.922884 27973 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:36.948616 22706 raft_consensus.cc:2410] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:11:36.932312 27624 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:36.952065 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:36.966120 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:36.962730 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:36.969878 27624 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:36.970238 22778 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:11:36.977872 22784 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:36.978886 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:36.954151 27278 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:36.944173 27964 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:36.991319 27910 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.759s	user 0.014s	sys 0.004s
W20260709 13:11:36.995976 27992 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.570s	user 0.014s	sys 0.000s
I20260709 13:11:36.958443 27973 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.020644 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.023689 22782 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:37.017206 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.026742 22778 raft_consensus.cc:2410] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:11:37.019107 27898 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:37.067006 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.071406 27930 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:37.075130 23412 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:37.079396 27878 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:37.080840 28011 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:11:37.082425 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.083254 22627 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:37.088017 27964 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.089206 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:37.089107 27624 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.090943 27878 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:37.091589 27878 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:37.092140 27878 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:37.018501 27278 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.108744 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.106354 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:37.020959 27910 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.111158 22631 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:37.118894 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.109858 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.124286 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:37.120260 23412 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:37.130180 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.131094 22782 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:37.131438 22706 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:37.141871 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:37.023443 27422 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.143595 22703 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:37.147235 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:37.150907 22778 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.153807 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:37.131182 27881 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.016s	sys 0.032s
I20260709 13:11:37.166808 22782 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:37.174067 22707 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:37.152508 27964 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:37.165966 27881 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.016s	sys 0.032s
I20260709 13:11:37.179785 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.180563 22782 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:37.078840 28008 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.097s
W20260709 13:11:37.179446 27962 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.534s	user 0.014s	sys 0.002s
W20260709 13:11:37.182444 28014 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.123s	user 0.002s	sys 0.003s
W20260709 13:11:37.184899 28017 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.094s	user 0.000s	sys 0.005s
I20260709 13:11:37.177417 28003 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:37.197608 22701 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:37.201649 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.202324 22782 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:37.199455 27807 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.020s	sys 0.019s
I20260709 13:11:37.199898 28003 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.184393 27964 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:37.212586 27807 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.020s	sys 0.020s
I20260709 13:11:37.224238 28003 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:37.248788 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:37.259956 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3152) took 1184 ms. Trace:
I20260709 13:11:37.260473 22703 rpcz_store.cc:276] 0709 13:11:36.075702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.499005 (+423303us) service_pool.cc:224] Handling call
0709 13:11:36.507292 (+  8287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:37.146441 (+639149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.259690 (+113249us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b540010d954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:37.259889 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462665984}
I20260709 13:11:37.265143 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:37.267153 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3147) took 1444 ms. Trace:
I20260709 13:11:37.275902 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:37.281975 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.285347 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.254004 27973 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:37.290091 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.296974 22635 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:37.298527 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.300117 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.295605 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.278652 27898 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:11:37.311151 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.325984 22778 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:11:37.292294 27881 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:11:37.332516 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 170489us
I20260709 13:11:37.338660 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:37.305099 27816 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.009s
I20260709 13:11:37.337738 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:37.345897 22634 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:37.348284 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.293124 27782 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:11:37.321540 27898 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.350627 22633 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:11:37.337157 22750 connection.cc:256] RPC call timeout handler was delayed by -1.278887111000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
I20260709 13:11:37.354069 22782 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:37.332510 27278 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:37.357204 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:37.358268 28021 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:11:37.362775 27937 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:11:37.363137 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.364001 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:37.364291 28019 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:11:37.365578 28024 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:11:37.366511 27962 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:11:37.371454 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 116960us
I20260709 13:11:37.348284 28008 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:37.352496 27816 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13.
I20260709 13:11:37.388516 27811 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:37.398439 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.399595 22776 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 13:11:37.400362 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.401185 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.450968 22780 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:37.401913 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:37.403705 28020 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:11:37.410167 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3140) took 1875 ms. Trace:
I20260709 13:11:37.412685 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.413527 22629 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:37.405783 27964 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:37.444804 22779 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:37.446683 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.452368 22709 rpcz_store.cc:276] 0709 13:11:35.534773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.427817 (+893044us) service_pool.cc:224] Handling call
0709 13:11:36.510286 (+ 82469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:36.555253 (+ 44967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.555316 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:36.619986 (+ 64670us) log.cc:844] Serialized 26 byte log entry
0709 13:11:36.922687 (+302701us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:36.922744 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.924696 (+  1952us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:37.409860 (+485164us) raft_consensus.cc:1685] finished
0709 13:11:37.409889 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:37.410088 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":214,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":16790,"thread_start_us":239614,"threads_started":1,"wal-append.queue_time_us":518684}
I20260709 13:11:37.457888 22781 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.419245 28008 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.451017 23416 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:37.452612 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77095.1us
I20260709 13:11:37.456104 22776 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 6
W20260709 13:11:37.464481 22750 connection.cc:256] RPC call timeout handler was delayed by -1.1618871110001692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20110000000000003s for a response
I20260709 13:11:37.353716 27898 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:37.454810 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3137) took 1930 ms. Trace:
I20260709 13:11:37.465009 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.381691 27962 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:37.442512 28015 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:11:37.465376 22708 rpcz_store.cc:276] 0709 13:11:35.524708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.414915 (+890207us) service_pool.cc:224] Handling call
0709 13:11:36.415805 (+   890us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:36.521196 (+105391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:36.521291 (+    95us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:36.592769 (+ 71478us) log.cc:844] Serialized 52 byte log entry
0709 13:11:36.980572 (+387803us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:36.982552 (+  1980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:36.982722 (+   170us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:37.454480 (+471758us) raft_consensus.cc:1685] finished
0709 13:11:37.454504 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:37.454728 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8574,"raft.queue_time_us":67164,"raft.run_cpu_time_us":409,"raft.run_wall_time_us":18295,"thread_start_us":270335,"threads_started":1,"wal-append.queue_time_us":578480}
I20260709 13:11:37.464466 22776 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:11:37.476492 22776 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:37.477120 22779 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:37.467492 22627 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:37.491729 22777 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:37.449995 27942 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:37.483168 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.497143 22782 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:11:37.497237 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.504428 22781 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:37.517371 22783 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.493093 27805 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.524269 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:37.513624 27742 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.833s	user 0.008s	sys 0.008s
I20260709 13:11:37.457284 27811 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.532974 22775 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:37.543020 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.543218 23384 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:37.495447 27816 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:11:37.561414 22783 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.551330 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.552722 27947 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:11:37.553524 28023 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:11:37.555267 28008 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:37.556174 27962 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:37.544875 22750 connection.cc:256] RPC call timeout handler was delayed by -1.0101848940000764s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 13:11:37.538028 27805 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.584363 27878 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.594467 27878 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:37.507366 27942 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.575021 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.529187 27979 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:37.615223 22779 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:37.567066 27947 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:37.623646 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.623904 22776 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:37.267647 22705 rpcz_store.cc:276] 0709 13:11:35.837064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.461774 (+624710us) service_pool.cc:224] Handling call
0709 13:11:36.462084 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:36.462346 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:11:36.945697 (+483351us) log.cc:844] Serialized 14 byte log entry
0709 13:11:37.266738 (+321041us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:37.266804 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:11:37.266862 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:37.266935 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:37.267078 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11576,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":260,"thread_start_us":763828,"threads_started":2}
I20260709 13:11:37.632632 22782 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:11:37.636520 22600 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.013s	sys 0.009s
W20260709 13:11:37.637030 22600 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.013s	sys 0.009s
W20260709 13:11:37.637348 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 771888us
I20260709 13:11:37.636397 27910 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:37.641156 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 104931us
I20260709 13:11:37.646136 22777 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:37.535472 27881 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:11:37.538695 27742 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.639751 27816 raft_consensus.cc:2804] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:37.658866 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:37.659144 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:37.662007 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 133161us
I20260709 13:11:37.651721 28043 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:37.656692 27962 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:37.624199 27979 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.675418 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.682325 27805 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:37.689366 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.665585 27916 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.687927 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:37.669658 28034 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:11:37.694087 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.665745 27816 raft_consensus.cc:697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:37.701676 27805 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:37.676980 27973 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:37.706231 23421 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:37.702445 28043 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.711350 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:37.713043 28043 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:37.714581 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 177483us
I20260709 13:11:37.714829 27881 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:37.715637 27881 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:37.716168 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:37.718884 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:37.719560 22632 raft_consensus.cc:606] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.720006 22632 raft_consensus.cc:614] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:37.720742 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.722011 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.725625 27809 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:37.726379 27946 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:37.726274 27809 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.727061 27946 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.728703 27809 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.729375 27946 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.733760 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:37.734194 22780 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:37.736989 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:37.737164 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:37.738638 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63364.2us
I20260709 13:11:37.739518 28045 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:37.739800 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.740092 28045 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:37.740530 28045 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:37.740747 27987 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:37.741595 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.742854 28042 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:37.743232 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.743891 27816 consensus_queue.cc:237] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.744568 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.745353 22778 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 16: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.746217 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.747460 28029 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:37.748255 28029 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:37.748843 27781 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.748725 28029 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.749451 27781 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.752519 27781 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.753222 27942 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.702333 27916 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:37.755396 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.756194 22777 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:37.756925 27979 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:37.757310 23416 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1123) took 1491 ms (client timeout 1886 ms). Trace:
W20260709 13:11:37.757735 23416 rpcz_store.cc:269] 0709 13:11:36.265624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:36.296233 (+ 30609us) service_pool.cc:224] Handling call
0709 13:11:37.757121 (+1460888us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54005e2254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:37.757224 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1677471616}
I20260709 13:11:37.691149 27947 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:37.760252 28036 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.203s	user 0.004s	sys 0.001s
I20260709 13:11:37.740540 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:37.760872 22783 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:11:37.739042 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.765511 22780 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:37.765940 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:37.762905 22778 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:37.778652 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:37.784686 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:37.756337 27916 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:37.793496 22702 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:37.794739 22781 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:37.793498 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:37.736497 27973 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:37.668190 28003 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.805274 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.813686 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:37.808024 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.779268 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.747799 27742 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.745368 27811 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.807052 22706 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:11:37.780855 28008 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:37.818754 22709 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:37.815770 22777 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:37.823455 22633 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:11:37.831535 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:37.827841 22705 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:37.837450 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:37.839313 22779 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.804085 28043 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:11:37.848584 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70255.5us
I20260709 13:11:37.845760 28043 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:37.822448 28029 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.852548 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:37.855800 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:37.856765 28003 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.858521 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.863849 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:37.867239 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:37.849227 27881 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:37.869298 22633 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:11:37.869721 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.872104 22632 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:37.875381 22630 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:11:37.878948 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.881313 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:37.854537 27742 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:37.868126 28043 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:37.907928 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:37.907974 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.910290 22782 raft_consensus.cc:2410] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:11:37.912158 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:37.912868 22781 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
W20260709 13:11:37.912201 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53795.6us
I20260709 13:11:37.907184 22704 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 5.
I20260709 13:11:37.930802 28002 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:37.931540 28002 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.932606 27624 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.933157 27624 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:37.933990 28002 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.935446 27624 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:37.935995 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:37.936985 22782 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 8: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.906507 27979 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:37.939023 22777 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:11:37.936152 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53203.1us
I20260709 13:11:37.944406 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:37.938447 22782 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:37.905156 27742 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:37.948969 22781 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:37.930297 27809 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:37.909487 27910 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:11:37.952569 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 79341.6us
I20260709 13:11:37.950277 27742 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:37.965008 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.966271 22782 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:37.968878 22630 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
W20260709 13:11:37.975538 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123394us
I20260709 13:11:37.976238 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:37.985505 22781 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:37.982340 22635 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:37.955341 27910 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:37.958477 27818 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:37.991273 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:38.002053 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:38.003912 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.004916 22782 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:37.963848 27809 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:38.013254 28061 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:38.014032 28061 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.017155 28061 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:37.963066 27807 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:11:37.991899 27910 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:38.029573 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.004438 27979 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:11:38.058900 28061 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:37.976786 27916 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 13:11:38.061466 23421 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1117) took 2138 ms (client timeout 2781 ms). Trace:
I20260709 13:11:37.992595 27964 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:11:38.004029 27764 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:11:38.082517 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:37.998462 27962 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:11:38.092633 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.097137 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.062353 23417 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:11:38.084596 23421 rpcz_store.cc:269] 0709 13:11:35.922692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:35.967316 (+ 44624us) service_pool.cc:224] Handling call
0709 13:11:38.061231 (+2093915us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54004f5bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:38.061394 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2868759296}
I20260709 13:11:38.086305 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.090411 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:38.046728 27898 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.003s
I20260709 13:11:38.092718 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.096254 22780 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:38.056604 27805 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:38.106226 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.106443 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.059737 28061 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.108413 22778 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:38.048988 27930 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 13:11:38.108466 27805 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:38.117045 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 149171us
I20260709 13:11:38.118403 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:38.119220 23414 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:38.119764 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.122164 27805 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:38.119700 27954 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.108115 27898 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:38.144205 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45028.6us
I20260709 13:11:38.061934 27809 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.141505 27811 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:38.166096 22626 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:38.170493 28057 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 13:11:38.171377 28066 raft_consensus.cc:993] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:38.172055 28066 raft_consensus.cc:1081] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:38.175886 27910 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.178822 27910 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:38.190667 28062 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:38.192806 28061 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:38.194170 27898 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.146835 27954 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.197225 27954 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:38.197850 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61568.5us
I20260709 13:11:38.198803 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:38.231654 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81375.5us
W20260709 13:11:38.251745 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 239235us
I20260709 13:11:38.201999 27811 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:38.255571 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:38.271525 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.272500 27910 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:38.297993 23414 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:38.315474 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 472857us
I20260709 13:11:38.316308 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.265467 27811 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:38.316823 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:38.319164 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86380us
I20260709 13:11:38.319466 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:38.325404 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:38.327086 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.327991 22780 raft_consensus.cc:2410] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:11:38.330475 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:38.331192 22775 raft_consensus.cc:2376] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10. Re-sending same reply.
I20260709 13:11:38.333019 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:38.333678 22775 raft_consensus.cc:2376] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12. Re-sending same reply.
I20260709 13:11:38.340111 22708 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:38.340327 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:38.341523 22778 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 19: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:38.344980 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:38.326122 22784 raft_consensus.cc:2376] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13. Re-sending same reply.
I20260709 13:11:38.350335 22626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:38.353345 22626 raft_consensus.cc:606] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:38.353736 22626 raft_consensus.cc:614] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:38.354908 28042 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.005s
I20260709 13:11:38.355618 28042 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:38.359812 22776 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:11:38.361864 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.335150 27764 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:38.342896 28073 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:11:38.378600 28073 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:38.379151 28073 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:38.379325 22701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:38.380111 22701 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:38.380511 22701 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:38.380525 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:38.365792 27807 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.346647 27962 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:38.374717 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.375783 28068 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:38.383426 22702 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:38.388242 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 189598us
I20260709 13:11:38.401232 22709 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:11:38.416960 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:38.418737 22776 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:38.431510 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56930.9us
I20260709 13:11:38.432773 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:38.417842 28045 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.677s	user 0.016s	sys 0.001s
I20260709 13:11:38.396854 27962 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:38.456279 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:38.395195 27807 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.437831 28003 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.439785 28068 raft_consensus.cc:2764] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:11:38.409228 27898 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:11:38.471540 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74790.9us
W20260709 13:11:38.522373 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89725.8us
W20260709 13:11:38.520464 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.469599 27910 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:38.441741 27949 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:11:38.466759 27954 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:38.466373 22701 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.655606 22709 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:38.463056 28054 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:11:38.676443 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42226 (request call id 799) took 1194 ms. Trace:
W20260709 13:11:38.591719 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62549.4us
I20260709 13:11:38.677102 22779 rpcz_store.cc:276] 0709 13:11:37.481649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.721726 (+240077us) service_pool.cc:224] Handling call
0709 13:11:38.676184 (+954458us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54003278d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:38.676370 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":254887680}
I20260709 13:11:38.677789 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.565372 27805 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.555557 28077 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:38.455111 28045 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.556387 28050 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:38.530376 28017 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.072s	user 0.000s	sys 0.011s
I20260709 13:11:38.520850 28003 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:38.677608 28034 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:11:38.587694 22702 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:38.638055 27942 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:11:38.643404 28078 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:11:38.649452 28043 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.722s	user 0.009s	sys 0.005s
I20260709 13:11:38.592504 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:38.690766 22701 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.701936 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.704630 28043 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.709342 28003 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:38.710826 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 130698us
I20260709 13:11:38.698989 27942 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:38.716207 27979 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:38.715945 27942 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.716857 27979 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:38.717600 27942 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:38.635403 27962 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.720369 28042 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:38.720885 28042 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:38.721450 28045 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:38.722239 28045 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:38.703301 28050 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.721868 28042 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.699561 27910 raft_consensus.cc:2764] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:11:38.728592 28042 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:38.689670 27954 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:38.726372 27781 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:38.729861 27781 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:38.730429 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1133) took 1358 ms. Trace:
I20260709 13:11:38.730060 27954 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.730973 27781 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.730928 23417 rpcz_store.cc:276] 0709 13:11:37.372493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.372655 (+   162us) service_pool.cc:224] Handling call
0709 13:11:38.061939 (+689284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:38.098083 (+ 36144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:38.730178 (+632095us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54004f5bd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:38.730354 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565521152}
I20260709 13:11:38.735494 27807 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.736300 27807 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:11:38.745692 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:38.755228 22776 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
W20260709 13:11:38.770023 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:38.753923 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:38.774482 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:38.776538 22750 connection.cc:256] RPC call timeout handler was delayed by -1.764511875999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
I20260709 13:11:38.785470 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:38.781292 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.789769 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:38.745872 28045 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:38.756042 28043 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:38.784948 28083 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.801873 28045 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:38.860152 27811 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
I20260709 13:11:38.860735 27811 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.867617 27946 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:38.846721 28066 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.869274 28066 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:38.879073 28066 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.888710 27811 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.889034 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:38.889766 22709 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:38.890002 27811 raft_consensus.cc:2764] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:11:38.891980 27811 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:38.892808 27811 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.896724 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:38.910504 22708 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:38.890760 27742 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.006s
I20260709 13:11:38.909169 27970 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:38.933780 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 229993us
I20260709 13:11:38.937062 22709 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:11:38.824620 28083 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:38.944953 22778 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:11:38.935772 22776 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:11:38.961472 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:38.938019 22525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.384s	user 0.012s	sys 0.013s
I20260709 13:11:38.963732 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:38.966140 22783 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:11:38.937592 27987 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:38.969342 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 263101us
I20260709 13:11:38.972795 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.947018 28088 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:38.969051 22525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.415s	user 0.012s	sys 0.013s
I20260709 13:11:38.993181 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.934216 27742 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:38.994370 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 173946us
I20260709 13:11:39.004498 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:39.005937 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.006345 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.006920 23421 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:39.010557 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.983017 28054 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.013377 28095 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:39.015952 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.028801 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:38.995095 28094 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:38.992198 27811 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:38.948268 27970 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.034754 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.039064 22703 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:39.009514 28043 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:39.051818 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:38.983750 27987 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.038877 27898 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:39.022542 28050 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.066900 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.064337 22705 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:11:39.074682 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 196552us
W20260709 13:11:39.062763 28073 consensus_meta.cc:287] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.683s	user 0.009s	sys 0.006s
I20260709 13:11:39.079054 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.080420 22707 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:39.080947 28050 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:11:39.063799 27916 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:39.082551 27742 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:38.995958 28088 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.050697 28096 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:39.050369 28094 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:39.075749 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.585s (SENT)
I20260709 13:11:39.044303 27782 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:11:39.065430 27811 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:39.104039 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.104367 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.071116 28042 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:11:39.080969 28093 log.cc:927] Time spent T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:11:39.124366 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 164402us
I20260709 13:11:39.129130 22702 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.143404 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.031994 28054 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.147770 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.113652 28095 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:39.161043 22707 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:39.162716 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.163642 22702 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:39.130280 27811 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.168385 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:39.243613 27764 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:11:39.231940 22707 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:11:39.212360 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 136663us
I20260709 13:11:39.192855 27811 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:39.267607 23419 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:11:39.257771 23421 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:11:39.213039 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.252522 28045 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.127912 28050 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
W20260709 13:11:39.251468 28102 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:11:39.282832 28077 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.265589 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.246745 28088 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:39.205911 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.259464 28094 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:39.378153 28103 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.116s	user 0.000s	sys 0.003s
I20260709 13:11:39.272187 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.400096 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3205) took 1015 ms. Trace:
I20260709 13:11:39.400700 22706 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.401049 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.402040 22707 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.211658 28114 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:39.211094 28111 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:39.402822 28111 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.099481 28073 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.402506 28114 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:39.403968 28114 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:39.269381 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.405771 28073 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.405907 23416 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:39.128332 27973 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:39.254592 28101 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:11:39.408478 28003 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.414845 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:39.289855 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 187219us
I20260709 13:11:39.415370 28003 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.416586 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.417501 28050 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:39.144732 28096 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:11:39.418159 28003 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.418741 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3191) took 1092 ms. Trace:
I20260709 13:11:39.267860 28083 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.261091 27970 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.419135 22703 rpcz_store.cc:276] 0709 13:11:38.326735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.328317 (+  1582us) service_pool.cc:224] Handling call
0709 13:11:38.348920 (+ 20603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.042872 (+693952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.418454 (+375582us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b54000a82d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:39.418675 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586389760}
I20260709 13:11:39.255364 28107 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:11:39.420691 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.421600 22703 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:11:39.423243 28042 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:39.342234 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.423858 28042 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:39.240953 22709 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:11:39.348047 28109 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.107s	user 0.003s	sys 0.000s
I20260709 13:11:39.343320 27947 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:39.351142 28104 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.098s	user 0.003s	sys 0.000s
I20260709 13:11:39.402277 28088 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
W20260709 13:11:39.336989 27942 consensus_meta.cc:287] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.006s
I20260709 13:11:39.372573 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:39.429072 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 56582.1us
I20260709 13:11:39.430198 28124 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:39.430720 28124 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.431241 28124 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:39.253890 28087 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.432148 28087 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.260092 27987 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.433336 27805 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.240589 27781 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:39.434839 28120 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.179s	user 0.002s	sys 0.000s
I20260709 13:11:39.434684 27781 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.436692 22634 raft_consensus.cc:1275] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:11:39.438225 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:39.438320 28115 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.259227 28054 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.439594 27878 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.440176 27878 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.441174 28087 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.439143 27781 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.442963 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.443372 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.444483 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.335147 28043 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:39.401872 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.445278 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.445871 22706 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:39.445996 23414 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:39.448566 27878 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.403225 28111 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:39.439191 28115 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.460654 28115 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:39.255224 28100 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:11:39.215583 28105 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:11:39.400566 22710 rpcz_store.cc:276] 0709 13:11:38.385051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.683608 (+298557us) service_pool.cc:224] Handling call
0709 13:11:38.699210 (+ 15602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:38.699521 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:38.699557 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:38.743621 (+ 44064us) log.cc:844] Serialized 26 byte log entry
0709 13:11:38.946665 (+203044us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:38.946733 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:38.954816 (+  8083us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:39.399832 (+445016us) raft_consensus.cc:1685] finished
0709 13:11:39.399863 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:39.400030 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":167297,"threads_started":1,"wal-append.queue_time_us":344785}
I20260709 13:11:39.408795 28077 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:39.480674 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:39.414084 27973 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:39.415886 22704 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:39.423408 28045 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.422518 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:39.438567 28108 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
W20260709 13:11:39.403535 27979 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.000s
I20260709 13:11:39.231680 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.492901 22629 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:39.210268 28056 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.004s	sys 0.016s
I20260709 13:11:39.445331 22701 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.461983 22626 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:11:39.467221 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:39.477377 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 309132us
W20260709 13:11:39.478650 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:39.467507 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51020us
I20260709 13:11:39.499687 22708 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:39.500213 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.485513 22703 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:39.500880 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.485879 22706 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:39.461328 28066 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:39.491211 22634 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:39.223361 22705 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:39.493804 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.510654 22709 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15.
W20260709 13:11:39.504866 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 60688.9us
I20260709 13:11:39.511652 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.512457 22706 raft_consensus.cc:2376] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14. Re-sending same reply.
I20260709 13:11:39.513360 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.514665 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.516538 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:39.495950 28056 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.408s	user 0.004s	sys 0.028s
I20260709 13:11:39.519109 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.492435 28083 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.521394 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.528478 22704 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:39.522739 22630 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:39.524843 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.520458 22703 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.536615 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.537544 22706 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:39.526926 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.539609 22701 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.540918 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.484472 28096 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:39.510928 27781 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.507995 28066 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:39.547206 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.550525 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.556753 22710 raft_consensus.cc:2393] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.558146 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.558357 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.550606 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.559166 22701 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.554651 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.511366 28107 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:11:39.535035 27422 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:39.570374 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.534548 28050 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.518016 28087 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.501751 27979 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.563280 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.566485 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:39.577818 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 177467us
I20260709 13:11:39.577766 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.475337 28043 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.579753 22702 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:39.526471 27881 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:11:39.548758 28096 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:39.544982 27946 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:11:39.576027 28111 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.571866 28124 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.535741 28054 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:11:39.589538 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.608196 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.590444 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.590899 22776 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:39.541291 28083 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.584348 28114 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.580865 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.547300 27807 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:11:39.628295 28114 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.628202 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.629124 28124 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.630077 22708 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 12: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.607475 27937 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:39.630983 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:39.631973 23385 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.866s (SENT)
I20260709 13:11:39.632553 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:39.633950 22630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 2814) took 1655 ms (client timeout 1594 ms). Trace:
I20260709 13:11:39.635039 22707 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:39.636124 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.637023 22710 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:39.639138 28002 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:39.639631 28002 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:39.640506 28002 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.607136 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.641707 22633 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:39.554060 27764 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:11:39.645488 28066 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.647238 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.648659 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:39.649420 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71766.7us
I20260709 13:11:39.651235 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.614114 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.652825 27782 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:39.654083 28029 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:39.656026 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:39.606510 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.660245 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:39.660467 23414 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:39.660980 23420 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:39.553766 27781 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.615317 22629 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:39.590543 27979 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.623924 28008 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:11:39.624783 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116368us
I20260709 13:11:39.632108 22702 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:39.639213 28063 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:39.649591 28111 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.614820 22626 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:39.603566 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.675208 22701 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:11:39.607872 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.675843 22632 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:39.608008 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:39.677604 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69715.9us
I20260709 13:11:39.668751 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.679162 22703 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:39.603226 28096 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:39.682301 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:39.634510 22630 rpcz_store.cc:269] 0709 13:11:37.978147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:37.978328 (+   181us) service_pool.cc:224] Handling call
0709 13:11:39.633664 (+1655336us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540022db54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:39.633809 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3519170816}
I20260709 13:11:39.685441 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.686432 22778 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:39.687227 22784 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:11:39.603266 22706 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:39.690248 22702 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
W20260709 13:11:39.696362 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53844.2us
I20260709 13:11:39.696628 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:39.706050 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:39.708971 23418 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:11:39.709446 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.711015 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:39.712440 22710 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:39.714708 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:39.723765 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:39.724550 22775 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:39.725678 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.726390 22776 raft_consensus.cc:2376] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12. Re-sending same reply.
I20260709 13:11:39.730690 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.732017 22784 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:39.733212 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.733863 22784 raft_consensus.cc:2376] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10. Re-sending same reply.
I20260709 13:11:39.683696 28042 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.736788 28042 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.717919 28119 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:11:39.717324 27964 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:11:39.729597 22782 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:39.710175 28118 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:11:39.734534 28077 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.734935 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.747771 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:39.747802 22784 raft_consensus.cc:2376] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11. Re-sending same reply.
I20260709 13:11:39.749832 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.751456 28083 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.752360 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:39.755759 28066 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:39.756060 28077 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.751554 22702 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
I20260709 13:11:39.757794 22702 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:11:39.760006 28094 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:39.761857 27937 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:39.760051 27781 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.764611 28096 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.766894 28124 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:39.767799 28124 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:39.768188 22703 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 6.
I20260709 13:11:39.769593 28124 raft_consensus.cc:2764] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:11:39.769989 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.771579 28118 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:11:39.772148 28118 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:39.773620 28118 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.672099 27811 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.781826 27811 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:39.695151 28097 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:11:39.581769 28087 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.784360 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:39.785058 28087 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:39.785295 28002 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:39.681036 27987 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:39.786331 22709 raft_consensus.cc:2368] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12.
I20260709 13:11:39.579624 28050 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.760272 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:39.789084 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.799654 22630 raft_consensus.cc:606] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:39.800292 22777 raft_consensus.cc:2410] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:11:39.807492 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.828572 22776 raft_consensus.cc:2393] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:39.822438 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 303348us
I20260709 13:11:39.797538 23420 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:39.804183 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:39.831027 22775 raft_consensus.cc:2376] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14. Re-sending same reply.
I20260709 13:11:39.832845 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:39.843925 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:39.846145 22704 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:39.849524 22631 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:39.852293 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.859637 22776 raft_consensus.cc:1240] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:39.864090 22710 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:11:39.867975 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 118239us
I20260709 13:11:39.791531 28096 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:39.884320 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:39.858946 27811 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:39.843886 27962 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:39.847820 28008 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.844180 27946 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.844861 27937 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:39.859170 28042 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:39.887254 27811 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.893570 27962 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:39.908579 28077 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:39.907737 27781 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:11:39.945930 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:39.946448 22632 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:39.929421 28042 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:39.950136 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:39.937667 28115 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:39.981568 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:39.913002 27937 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.909042 28008 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:39.911646 27946 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:39.928779 28058 consensus_queue.cc:1048] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:11:39.948554 27962 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:39.947721 28077 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.019585 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:40.063782 22706 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:40.064647 22630 raft_consensus.cc:606] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:40.065325 22630 raft_consensus.cc:614] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:40.065510 22706 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:40.059437 22709 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:39.953886 27781 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:39.979907 27973 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:39.986855 27672 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:11:40.072135 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 348481us
W20260709 13:11:40.075836 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 125618us
I20260709 13:11:40.080888 22710 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:39.992192 28029 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:40.033550 28045 raft_consensus.cc:2764] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:11:40.083971 22626 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:11:40.083585 23421 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:11:40.086786 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:39.984347 28063 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 13:11:40.093343 27811 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:40.080272 27973 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.098138 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:40.072224 28077 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:40.152590 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 72684.6us
I20260709 13:11:40.175204 22710 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:11:40.100553 28130 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
W20260709 13:11:40.176276 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 78281.6us
I20260709 13:11:40.106166 28029 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:40.186285 27782 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.533s	user 0.018s	sys 0.000s
I20260709 13:11:40.178332 28118 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:40.008301 28115 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.210068 27782 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:40.209671 28045 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:40.214789 28045 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:40.215490 28131 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.305s	user 0.000s	sys 0.000s
I20260709 13:11:40.217340 28045 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:40.238936 22779 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:40.244408 22704 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:11:40.286370 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81429.5us
I20260709 13:11:40.288110 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:40.289897 22710 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:40.295727 22779 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:40.300079 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:40.309791 22710 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:40.319684 28073 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:40.171020 28088 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 13:11:40.327855 28029 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:40.173861 28042 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.359202 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:40.321748 27946 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:40.388440 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 180006us
I20260709 13:11:40.209878 28118 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:40.384219 22780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.050s
W20260709 13:11:40.396711 22780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.000s	sys 0.051s
W20260709 13:11:40.383219 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83289.6us
W20260709 13:11:40.392854 22781 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.030s	sys 0.022s
W20260709 13:11:40.436028 22781 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.030s	sys 0.022s
W20260709 13:11:40.436196 22702 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.045s
W20260709 13:11:40.436739 22702 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.008s	sys 0.045s
W20260709 13:11:40.436825 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 148307us
W20260709 13:11:40.442668 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:40.394232 28136 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 13:11:40.461467 22701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.019s	sys 0.038s
W20260709 13:11:40.479539 22701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.019s	sys 0.038s
I20260709 13:11:40.481321 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2662) took 1067 ms. Trace:
W20260709 13:11:40.494566 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 96338.6us
I20260709 13:11:40.178537 27781 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:40.418298 28118 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:40.401088 27624 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:40.451326 28073 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:40.536881 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 107820us
I20260709 13:11:40.502285 28088 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:40.530391 27624 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:40.560992 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:40.577179 28050 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:40.507390 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:40.458199 27937 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:40.544708 27781 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:11:40.609476 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2824) took 1033 ms. Trace:
I20260709 13:11:40.543246 28088 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:40.628894 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 75515.8us
I20260709 13:11:40.560444 27624 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:40.617609 22634 rpcz_store.cc:276] 0709 13:11:39.576485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.576756 (+   271us) service_pool.cc:224] Handling call
0709 13:11:39.586709 (+  9953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.636902 (+ 50193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.636937 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.637484 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:11:39.984380 (+346896us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:39.984437 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:39.985500 (+  1063us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.609042 (+623542us) raft_consensus.cc:1685] finished
0709 13:11:40.609081 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.609357 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49654,"raft.queue_time_us":51036,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":109,"thread_start_us":344520,"threads_started":1,"wal-append.queue_time_us":663726}
W20260709 13:11:40.632841 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:40.617141 28069 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:40.623466 28124 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:40.658466 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:40.689093 28134 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:11:40.640169 28073 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:40.659161 28069 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.703049 28124 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:40.729960 22632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.045s	sys 0.067s
W20260709 13:11:40.731923 22632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.046s	sys 0.067s
W20260709 13:11:40.741811 22708 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.003s
W20260709 13:11:40.742466 22708 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.011s	sys 0.003s
W20260709 13:11:40.743041 22776 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.005s	sys 0.008s
W20260709 13:11:40.755743 22776 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.005s	sys 0.008s
W20260709 13:11:40.744136 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.007s
W20260709 13:11:40.756791 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.007s	sys 0.007s
W20260709 13:11:40.744390 22777 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.007s	sys 0.005s
W20260709 13:11:40.758409 22777 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.005s
W20260709 13:11:40.742790 23385 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.837s	user 0.004s	sys 0.008s
W20260709 13:11:40.764964 23385 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.005s	sys 0.009s
W20260709 13:11:40.765591 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 864843us
W20260709 13:11:40.743687 22778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.005s	sys 0.008s
W20260709 13:11:40.768085 22778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.978s	user 0.005s	sys 0.008s
W20260709 13:11:40.744771 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.006s	sys 0.007s
W20260709 13:11:40.769212 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.006s	sys 0.007s
W20260709 13:11:40.769646 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 1.06374e+06us
I20260709 13:11:40.757803 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 759) took 1525 ms. Trace:
I20260709 13:11:40.739187 28008 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:40.802170 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 294901us
I20260709 13:11:40.782171 22707 rpcz_store.cc:276] 0709 13:11:39.231933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.636936 (+405003us) service_pool.cc:224] Handling call
0709 13:11:39.637432 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.637815 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:11:39.955409 (+317594us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.757318 (+801909us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:40.757388 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:40.757447 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.757538 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.757725 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":584973,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"thread_start_us":963283,"threads_started":2}
I20260709 13:11:40.811206 28008 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:40.850118 28008 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:40.815047 22749 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:40.816068 28003 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:40.818157 28155 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:11:40.819208 28142 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:11:40.820434 28150 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:11:40.822780 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3217) took 1278 ms. Trace:
I20260709 13:11:40.829716 28058 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:11:40.832664 28077 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.638s	user 0.008s	sys 0.004s
I20260709 13:11:40.786054 28087 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:40.716431 28088 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.780825 27811 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:40.838665 28050 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:40.829763 28069 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:40.848696 22780 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3083) took 1161 ms. Trace:
I20260709 13:11:40.851722 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:40.909514 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:40.910051 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58424.7us
I20260709 13:11:40.736105 28124 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:40.854293 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3222) took 1273 ms. Trace:
I20260709 13:11:40.913765 22703 rpcz_store.cc:276] 0709 13:11:39.580713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.871692 (+290979us) service_pool.cc:224] Handling call
0709 13:11:39.871967 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.947088 (+ 75121us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:11:40.374789 (+427701us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.774411 (+399622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:40.774482 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:40.774555 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.853918 (+ 79363us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b54003d09d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:40.853996 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.854206 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":276157,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"spinlock_wait_cycles":172063616,"thread_start_us":813202,"threads_started":2}
I20260709 13:11:40.862035 22708 rpcz_store.cc:276] 0709 13:11:39.544101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.778358 (+234257us) service_pool.cc:224] Handling call
0709 13:11:40.222471 (+444113us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:40.222733 (+   262us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.223165 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.223198 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.223768 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.754501 (+530733us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:40.754576 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.754742 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.822381 (+ 67639us) raft_consensus.cc:1685] finished
0709 13:11:40.822406 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.822699 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1061,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":79994112,"thread_start_us":530172,"threads_started":1,"wal-append.queue_time_us":596423}
I20260709 13:11:40.855538 28029 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:11:40.919977 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.507s (SENT)
I20260709 13:11:40.909497 22780 rpcz_store.cc:276] 0709 13:11:39.687051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.712822 (+ 25771us) service_pool.cc:224] Handling call
0709 13:11:39.713261 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.788064 (+ 74803us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:40.397354 (+609290us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.773764 (+376410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:40.773862 (+    98us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:40.773940 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.848417 (+ 74477us) spinlock_profiling.cc:243] Waited 74.5 ms on lock 0x7b5400326ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:40.848472 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.848620 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27176,"prepare.queue_time_us":870151,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":6425,"raft.queue_time_us":6175,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":36363,"spinlock_wait_cycles":171302272,"thread_start_us":984575,"threads_started":2}
I20260709 13:11:40.922061 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:40.927392 22705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.012s	sys 0.015s
W20260709 13:11:40.934589 22705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.012s	sys 0.016s
I20260709 13:11:40.855144 28003 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:40.876248 28058 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:40.887589 28087 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:40.860589 28063 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:11:40.928264 28029 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:41.003896 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.227s (SENT)
I20260709 13:11:41.004678 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:40.952617 28115 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:40.956384 28156 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:11:40.977370 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3099) took 1257 ms. Trace:
W20260709 13:11:40.931995 28002 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.145s	user 0.018s	sys 0.000s
I20260709 13:11:40.948153 22631 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:41.001153 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3213) took 1458 ms. Trace:
I20260709 13:11:41.015187 22702 rpcz_store.cc:276] 0709 13:11:39.543024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.758584 (+215560us) service_pool.cc:224] Handling call
0709 13:11:39.758918 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.828910 (+ 69992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.828947 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.869412 (+ 40465us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.437217 (+567805us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:40.437269 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.437366 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.000820 (+563454us) raft_consensus.cc:1685] finished
0709 13:11:41.000856 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.001072 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46343,"raft.queue_time_us":70202,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":15184,"thread_start_us":505203,"threads_started":1,"wal-append.queue_time_us":707896}
I20260709 13:11:40.934911 28144 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:40.892170 28077 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.019753 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3231) took 1389 ms. Trace:
I20260709 13:11:41.012601 22777 rpcz_store.cc:276] 0709 13:11:39.719959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.851833 (+131874us) service_pool.cc:224] Handling call
0709 13:11:39.893981 (+ 42148us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.949477 (+ 55496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.949516 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.994669 (+ 45153us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.758976 (+764307us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:40.810335 (+ 51359us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.852029 (+ 41694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.852137 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:40.976851 (+124714us) raft_consensus.cc:1685] finished
0709 13:11:40.976887 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.977248 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93331,"prepare.queue_time_us":87655,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":103528,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":51758,"thread_start_us":659866,"threads_started":1,"wal-append.queue_time_us":837956}
W20260709 13:11:41.035815 22775 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.000s	sys 0.028s
W20260709 13:11:41.050769 22775 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.000s	sys 0.029s
I20260709 13:11:41.011787 28029 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.024421 28144 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:41.075250 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 160419us
I20260709 13:11:40.942920 28056 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.089s
W20260709 13:11:41.063525 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 419245us
W20260709 13:11:41.115411 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:41.079730 22750 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.012s	sys 0.017s
I20260709 13:11:41.031890 28063 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:41.079186 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3098) took 1359 ms. Trace:
W20260709 13:11:41.077350 22749 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Timed out: RequestConsensusVote RPC to 127.21.5.129:37283 timed out after 1.594s (SENT)
I20260709 13:11:41.028616 22706 rpcz_store.cc:276] 0709 13:11:39.630360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.068999 (+438639us) service_pool.cc:224] Handling call
0709 13:11:40.069333 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:40.069599 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:40.544299 (+474700us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.019307 (+475008us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:41.019381 (+    74us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:41.019439 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.019521 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.019679 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153859,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"thread_start_us":948669,"threads_started":2}
I20260709 13:11:41.028981 28115 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:11:41.123881 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:41.130363 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53059.9us
I20260709 13:11:41.124485 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:41.065675 28139 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.000s	sys 0.040s
W20260709 13:11:41.116251 27962 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.039s	user 0.014s	sys 0.002s
W20260709 13:11:41.121441 22750 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.012s	sys 0.017s
W20260709 13:11:41.122290 27970 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.006s	sys 0.021s
W20260709 13:11:41.144085 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 578958us
W20260709 13:11:41.147243 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.130021 22775 rpcz_store.cc:276] 0709 13:11:39.719675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.831753 (+112078us) service_pool.cc:224] Handling call
0709 13:11:39.848136 (+ 16383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:39.940693 (+ 92557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.940747 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:39.945805 (+  5058us) log.cc:844] Serialized 52 byte log entry
0709 13:11:40.451569 (+505764us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:41.078019 (+626450us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.078689 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.078786 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.078856 (+    70us) raft_consensus.cc:1685] finished
0709 13:11:41.078873 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.079083 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123313,"prepare.queue_time_us":354018,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":94873,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":37470,"thread_start_us":991356,"threads_started":2,"wal-append.queue_time_us":702628}
I20260709 13:11:41.122892 28144 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:41.117722 27937 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:41.143411 28139 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.000s	sys 0.042s
I20260709 13:11:41.160281 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:41.146414 27970 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.006s	sys 0.021s
I20260709 13:11:41.136229 28115 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:41.163537 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.593s (SENT)
W20260709 13:11:41.170113 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 166279us
I20260709 13:11:41.162495 28145 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:41.198349 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:41.176748 27973 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.177917 28045 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:41.179633 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:40.481840 22701 rpcz_store.cc:276] 0709 13:11:39.414060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.754360 (+340300us) service_pool.cc:224] Handling call
0709 13:11:39.754703 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:39.754922 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:39.950787 (+195865us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.480553 (+529766us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.480786 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:40.480844 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:40.480931 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.481053 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:40.481243 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":470576,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"thread_start_us":722840,"threads_started":2}
I20260709 13:11:41.185344 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.205627 22706 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:41.185938 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.186501 22707 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:11:41.188043 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.207132 22710 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:41.193636 28115 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.195034 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:41.207820 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 353521us
W20260709 13:11:41.197383 28171 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:11:41.156942 28094 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:11:41.173946 28153 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.463s	user 0.000s	sys 0.003s
W20260709 13:11:41.202019 28146 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.267s	user 0.002s	sys 0.000s
I20260709 13:11:41.148432 27962 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.169243 27937 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.117123 28087 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.159998 27811 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.082722 28058 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.215893 22709 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
W20260709 13:11:41.230525 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 50987.4us
I20260709 13:11:41.233036 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.233822 22631 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 19.
I20260709 13:11:41.231675 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3218) took 1684 ms. Trace:
I20260709 13:11:41.236558 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.207986 28096 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:41.210752 28085 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:41.235867 22705 rpcz_store.cc:276] 0709 13:11:39.547041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.800875 (+253834us) service_pool.cc:224] Handling call
0709 13:11:40.224730 (+423855us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:40.224959 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:40.225244 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:11:40.225570 (+   326us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.230874 (+1005304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:41.230952 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:41.231088 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.231363 (+   275us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.231582 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14196,"prepare.queue_time_us":820,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":127997,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":3620,"spinlock_wait_cycles":59863296,"thread_start_us":1004792,"threads_started":2,"wal-append.queue_time_us":908892}
I20260709 13:11:41.235968 28069 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:41.228355 28042 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:41.241935 28124 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.244565 28094 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 13:11:41.248915 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85423.9us
I20260709 13:11:41.215898 27946 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.253643 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:41.255918 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.264842 22777 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 19.
I20260709 13:11:41.208396 28145 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.257891 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59422 (request call id 51) took 1306 ms. Trace:
I20260709 13:11:41.268158 22630 rpcz_store.cc:276] 0709 13:11:39.945173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.945779 (+   606us) service_pool.cc:224] Handling call
0709 13:11:41.251767 (+1305988us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540022ea54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:41.251913 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257864448}
I20260709 13:11:41.232872 27942 consensus_queue.cc:1059] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 13:11:41.262463 22750 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.764s (SENT)
I20260709 13:11:41.252858 28115 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:41.250239 28087 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:41.234210 27811 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.252647 28003 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.281347 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 1006) took 1616 ms. Trace:
I20260709 13:11:41.281864 23416 rpcz_store.cc:276] 0709 13:11:39.665058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.672559 (+  7501us) service_pool.cc:224] Handling call
0709 13:11:39.672983 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.712085 (+ 39102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.712139 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.728699 (+ 16560us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.077301 (+348602us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:40.077363 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.077443 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.281069 (+1203626us) raft_consensus.cc:1685] finished
0709 13:11:41.281095 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.281264 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31121,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":71384,"thread_start_us":234157,"threads_started":1,"wal-append.queue_time_us":1044834}
I20260709 13:11:41.242278 28063 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
W20260709 13:11:41.284888 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 200806us
W20260709 13:11:41.301908 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.282510 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3096) took 1563 ms. Trace:
I20260709 13:11:41.255156 28085 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:41.333287 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 81802.9us
I20260709 13:11:41.325824 22783 rpcz_store.cc:276] 0709 13:11:39.718931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.753548 (+ 34617us) service_pool.cc:224] Handling call
0709 13:11:39.753902 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:39.893202 (+139300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.893254 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:39.911378 (+ 18124us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.384841 (+473463us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:40.384933 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.385016 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.173324 (+788308us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:41.276255 (+102931us) raft_consensus.cc:1685] finished
0709 13:11:41.276279 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.282424 (+  6145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125318,"raft.queue_time_us":153576,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":18494,"spinlock_wait_cycles":6308096,"thread_start_us":435199,"threads_started":1,"wal-append.queue_time_us":702955}
I20260709 13:11:41.256762 28096 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:41.285125 28178 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.070s	user 0.001s	sys 0.001s
W20260709 13:11:41.340065 22750 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.566s (SENT)
I20260709 13:11:41.342473 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:41.345284 22702 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:11:41.353048 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.356325 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.357445 22630 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:41.272298 27946 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.368221 22629 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:11:41.367079 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:41.374366 22750 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.585s (SENT)
W20260709 13:11:41.375135 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 143146us
I20260709 13:11:41.337177 27764 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:41.371366 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.376139 22675 leader_election.cc:400] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:41.385648 22675 leader_election.cc:403] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:41.327364 28185 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:11:41.379150 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.381031 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.381713 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.341974 27942 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:41.375502 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.613s (SENT)
I20260709 13:11:41.365962 28002 raft_consensus.cc:3055] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:41.385810 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:41.395123 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:41.387068 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.844s (SENT)
I20260709 13:11:41.401417 22627 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:41.405920 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:41.407475 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 185306us
I20260709 13:11:41.375941 28129 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:41.382521 28165 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:11:41.299557 28144 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.415205 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.377403 28045 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:41.408072 28168 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
W20260709 13:11:41.444506 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 74162.4us
I20260709 13:11:41.436079 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.394232 28185 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.436986 27973 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:41.463825 22635 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:41.401955 28002 raft_consensus.cc:740] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:41.469815 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.410701 28042 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:11:41.470425 28069 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:41.476876 28069 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.477339 28069 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:41.469691 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:41.468182 28178 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:11:41.468756 28172 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 13:11:41.448478 28188 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.399448 27764 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:41.493036 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.116s (SENT)
I20260709 13:11:41.409584 27942 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.495688 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:41.498658 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 112925us
W20260709 13:11:41.499274 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52543.1us
I20260709 13:11:41.470786 28185 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:11:41.494264 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118788us
I20260709 13:11:41.506886 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3102) took 1354 ms. Trace:
I20260709 13:11:41.487164 27962 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.488857 28129 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:41.511334 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.467666 27422 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:11:41.502388 28179 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:41.466943 28194 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:11:41.519659 22628 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:41.520725 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 105934us
I20260709 13:11:41.509348 22779 rpcz_store.cc:276] 0709 13:11:40.152495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.298519 (+146024us) service_pool.cc:224] Handling call
0709 13:11:40.395887 (+ 97368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.545782 (+149895us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.551094 (+  5312us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.624686 (+ 73592us) log.cc:844] Serialized 26 byte log entry
0709 13:11:41.000042 (+375356us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:41.026362 (+ 26320us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.026717 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.026811 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.506580 (+479769us) raft_consensus.cc:1685] finished
0709 13:11:41.506633 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.506811 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175973,"prepare.queue_time_us":25953,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":157894,"raft.run_cpu_time_us":427,"raft.run_wall_time_us":67421,"thread_start_us":339419,"threads_started":1,"wal-append.queue_time_us":507381}
I20260709 13:11:41.520409 28042 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:11:41.525739 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.666s (SENT)
I20260709 13:11:41.540066 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:41.525671 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.527163 27942 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.527637 28129 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:41.509620 28188 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.528860 27811 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.532218 28085 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.507193 27937 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.533406 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 748) took 1438 ms. Trace:
W20260709 13:11:41.513669 27624 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.873s	user 0.011s	sys 0.007s
I20260709 13:11:41.538409 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:41.550666 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.777s (SENT)
I20260709 13:11:41.520922 28124 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:41.554785 22632 rpcz_store.cc:276] 0709 13:11:40.095291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.095477 (+   186us) service_pool.cc:224] Handling call
0709 13:11:40.096046 (+   569us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:40.096491 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.096526 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:40.214890 (+118364us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b2c00000c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:40.215704 (+   814us) log.cc:844] Serialized 52 byte log entry
0709 13:11:40.734092 (+518388us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:41.134997 (+400905us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.142963 (+  7966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.143107 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.532957 (+389850us) raft_consensus.cc:1685] finished
0709 13:11:41.533086 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.533298 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":514343,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":14495,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":10830,"spinlock_wait_cycles":4883200,"thread_start_us":864998,"threads_started":2,"wal-append.queue_time_us":894307}
I20260709 13:11:41.563519 27422 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:41.564155 27422 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:41.565274 27422 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.567152 28194 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:41.566795 27624 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.567420 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:41.568068 28186 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:11:41.567638 28194 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:41.569882 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3084) took 1870 ms. Trace:
W20260709 13:11:41.574536 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.811s (SENT)
I20260709 13:11:41.574925 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:41.567322 28195 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:11:41.575946 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68790.2us
W20260709 13:11:41.563539 28183 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.183s	user 0.000s	sys 0.000s
W20260709 13:11:41.586215 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:41.630095 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:41.590804 23385 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.188s (SENT)
I20260709 13:11:41.570338 22778 rpcz_store.cc:276] 0709 13:11:39.699701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.699957 (+   256us) service_pool.cc:224] Handling call
0709 13:11:39.700440 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.768693 (+1068253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.768745 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.791570 (+ 22825us) log.cc:844] Serialized 26 byte log entry
0709 13:11:41.221086 (+429516us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:41.251648 (+ 30562us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.251926 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.252030 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.569576 (+317546us) raft_consensus.cc:1685] finished
0709 13:11:41.569605 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.569800 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118354,"prepare.queue_time_us":31300,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2948,"raft.queue_time_us":113089,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":17679,"spinlock_wait_cycles":8283392,"thread_start_us":1310704,"threads_started":2,"wal-append.queue_time_us":475785}
I20260709 13:11:41.631146 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:41.555693 28145 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:41.640038 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 167154us
W20260709 13:11:41.634244 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 59424.2us
W20260709 13:11:41.576687 28182 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:11:41.514251 28002 consensus_queue.cc:260] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.650199 28002 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:41.573743 27624 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.578804 28085 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:11:41.652310 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.646152 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3086) took 1944 ms. Trace:
I20260709 13:11:41.653158 22781 rpcz_store.cc:276] 0709 13:11:39.701573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.701787 (+   214us) service_pool.cc:224] Handling call
0709 13:11:39.751017 (+ 49230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:40.351866 (+600849us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b480012c258. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:11:40.436568 (+ 84702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.436614 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:40.437312 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 13:11:40.893230 (+455918us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:41.050932 (+157702us) log.cc:844] Serialized 14 byte log entry
0709 13:11:41.051377 (+   445us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.051612 (+   235us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.645724 (+594112us) raft_consensus.cc:1685] finished
0709 13:11:41.645861 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:41.646038 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145320,"prepare.queue_time_us":132581,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":169373,"raft.run_cpu_time_us":3991,"raft.run_wall_time_us":456547,"spinlock_wait_cycles":30043392,"thread_start_us":916240,"threads_started":2,"wal-append.queue_time_us":663304}
I20260709 13:11:41.582943 28042 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:11:41.648844 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68611.1us
I20260709 13:11:41.553591 28129 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.636736 27422 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:41.579080 28199 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:41.662185 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.638784 27878 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:41.636620 27811 raft_consensus.cc:2764] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:11:41.636304 28096 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:41.661293 28192 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:11:41.632967 27942 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:11:41.675537 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:41.657686 28042 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:41.690865 28042 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:41.691968 28173 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.483s	user 0.008s	sys 0.003s
W20260709 13:11:41.674405 28050 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.751s	user 0.015s	sys 0.000s
I20260709 13:11:41.701443 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.670885 28124 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:41.702142 27811 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:41.703819 27764 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.725906 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:41.726042 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54133.7us
I20260709 13:11:41.671124 27946 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.708956 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:41.732632 23417 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:41.695969 28050 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:41.739712 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.740281 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.740509 22781 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:41.745858 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.727453 28179 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:41.746552 22777 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:41.747385 27970 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:41.747978 27970 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.750069 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.725363 28085 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:41.755160 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.756193 22783 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:41.755787 28085 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.757647 22783 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 14. Ops: []
I20260709 13:11:41.759276 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.759887 28129 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:41.764102 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.765615 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.767288 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:11:41.768489 28163 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.733s	user 0.005s	sys 0.000s
I20260709 13:11:41.773765 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.775494 28008 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:41.775754 28208 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.776381 28208 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.779119 28208 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.780128 28073 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:41.780650 28073 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.782902 28073 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.783420 28073 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:41.750209 27970 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.783979 28073 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.784509 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:41.785252 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.786826 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:41.787062 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.787295 28073 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:41.787781 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.423s (SENT)
I20260709 13:11:41.788141 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.788283 23385 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.788316 28073 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:41.789402 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.789294 28073 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.791304 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2668) took 2150 ms. Trace:
I20260709 13:11:41.791750 22709 rpcz_store.cc:276] 0709 13:11:39.640485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.073682 (+433197us) service_pool.cc:224] Handling call
0709 13:11:40.074126 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:41.229272 (+1155146us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:41.791041 (+561769us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000a78d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:41.791240 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625359232}
I20260709 13:11:41.793762 28073 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:41.795439 28077 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:41.796387 28184 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:11:41.798887 28058 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:41.665679 28194 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:41.804212 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.804893 22634 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:41.805234 28194 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:41.806916 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.807554 22775 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:41.798915 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.811933 22776 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:41.812834 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.820941 22781 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:11:41.823863 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 117028us
I20260709 13:11:41.825664 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.822250 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.827718 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.828383 22781 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:41.670958 28203 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 13:11:41.825918 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.829505 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.829797 22630 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:11:41.806299 22777 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:41.834578 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:41.829792 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:41.840188 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 113344us
I20260709 13:11:41.840477 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.841483 22635 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:41.730475 28144 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:41.845438 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.847963 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:41.851998 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.853127 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.857646 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.857856 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:41.866179 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.867435 22626 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:41.867661 22781 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:41.872886 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.869035 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.875371 22781 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:41.875577 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:41.876276 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:41.875556 22775 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:41.857501 28197 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:41.877873 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.860230 28094 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:41.873994 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:41.888022 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 68212.5us
I20260709 13:11:41.894164 23417 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
W20260709 13:11:41.894447 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 324572us
I20260709 13:11:41.867898 22777 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:41.898765 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:41.868651 28050 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:41.898595 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:41.820830 28198 consensus_queue.cc:1175] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 9 status { last_received { term: 9 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 9 index: 7 } }
W20260709 13:11:41.902894 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 43740.1us
I20260709 13:11:41.904048 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:41.927949 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.929283 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.929667 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.930274 22777 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:41.931314 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.933580 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.934573 22702 raft_consensus.cc:2468] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:41.893187 28094 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:41.902747 28029 consensus_meta.cc:287] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.784s	user 0.000s	sys 0.015s
I20260709 13:11:41.940665 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.942775 22706 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:41.943001 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:41.946313 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 47701.2us
I20260709 13:11:41.959308 22781 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:41.955183 22776 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:41.900005 27946 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:41.931531 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.961767 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:41.962440 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.963840 27946 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:41.964298 27946 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:41.966813 28159 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:41.967073 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.967584 28159 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:41.968617 28159 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.969149 28029 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:41.955833 28188 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:41.971959 28077 raft_consensus.cc:2764] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:11:41.972745 28063 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:41.975802 28087 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.693s	user 0.014s	sys 0.000s
I20260709 13:11:41.981025 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.902156 28050 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:41.983832 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.984527 22627 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:41.985749 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:41.961850 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:41.987113 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:41.988090 22629 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:41.921617 28197 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:41.989267 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:41.990237 22775 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:41.990959 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:41.992502 22781 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:11:41.992419 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.993795 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:41.987991 22709 raft_consensus.cc:2376] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16. Re-sending same reply.
I20260709 13:11:41.995122 28201 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:41.995922 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:41.998785 22706 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:42.003348 22702 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.004156 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:41.998870 22633 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:42.006363 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.011057 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:42.010097 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 146317us
I20260709 13:11:41.958771 28094 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:41.950846 28205 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:42.012179 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.013840 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:42.014971 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.015247 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.015559 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.026643 22701 raft_consensus.cc:2376] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14. Re-sending same reply.
I20260709 13:11:42.026803 22710 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:42.027683 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.027798 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.028318 22701 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:42.028596 22710 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.029117 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.016319 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.026664 22703 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:42.029990 22708 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:42.015677 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.019318 22706 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:11:42.022254 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.023152 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:42.032285 22781 raft_consensus.cc:2368] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
W20260709 13:11:42.032524 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70769.1us
I20260709 13:11:42.033684 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.034500 22709 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15.
I20260709 13:11:42.024648 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:42.035859 22776 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:42.036026 22775 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:42.030869 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.038043 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.038671 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.039104 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:42.002964 28118 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.467s	user 0.020s	sys 0.002s
I20260709 13:11:42.039769 22710 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:42.039130 22703 raft_consensus.cc:2368] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:11:42.040745 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.040009 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.041656 22710 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.042026 22707 raft_consensus.cc:2410] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:11:42.040957 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.043478 22703 raft_consensus.cc:2393] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:42.017102 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.035308 22705 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:11:42.045385 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.029280 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.047197 22701 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:42.036978 22708 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:11:42.049606 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.049088 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.052511 22701 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:42.060892 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.061925 22710 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:11:42.062884 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.067862 22634 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:42.069440 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.069670 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.071419 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.073050 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:42.074752 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.075291 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.075888 22710 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:41.958266 28069 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.077746 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.082600 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.084856 22708 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.083928 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:42.089658 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65372.9us
I20260709 13:11:42.094251 22704 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:42.095203 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.081696 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:42.098875 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 158091us
I20260709 13:11:42.098189 22702 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.100368 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:42.106011 22710 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 12: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:42.106666 22776 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 16: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:42.072052 28056 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:42.108177 28209 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:42.120705 28209 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:42.121410 28148 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.041s	user 0.000s	sys 0.005s
W20260709 13:11:42.123535 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89708.8us
I20260709 13:11:42.124828 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.125686 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.125860 28073 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:42.126509 22632 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
W20260709 13:11:42.127286 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 138101us
I20260709 13:11:42.127633 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.128510 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:42.128957 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.129593 22632 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:42.129977 28208 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:42.118943 28056 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:42.132191 28056 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.126027 27937 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:42.130385 28208 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.078569 28124 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:42.090376 27811 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:42.136763 22627 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:42.139027 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.145197 28204 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:11:42.163360 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.164551 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.165714 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:42.151081 28208 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:42.091580 28199 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:42.167089 28199 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:42.145565 27937 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:42.167902 27937 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:42.168110 28199 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.170385 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.150233 28056 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:42.175063 27878 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:42.150534 22778 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:42.177327 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3094) took 2462 ms. Trace:
I20260709 13:11:42.164566 28207 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:42.180193 28207 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:42.180626 28207 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:42.182296 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:42.160434 28163 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:11:42.195849 28189 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:42.196727 28189 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.082505 28045 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:11:42.197202 28189 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.198388 28045 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:42.198815 28045 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:42.199841 28189 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:42.199649 28045 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.177738 22782 rpcz_store.cc:276] 0709 13:11:39.715105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.739435 (+ 24330us) service_pool.cc:224] Handling call
0709 13:11:39.739803 (+   368us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:39.877970 (+138167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:39.878024 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:39.882949 (+  4925us) log.cc:844] Serialized 52 byte log entry
0709 13:11:40.164397 (+281448us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:40.723811 (+559414us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.755115 (+ 31304us) log.cc:844] Serialized 14 byte log entry
0709 13:11:40.756115 (+  1000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:40.756227 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:41.793935 (+1037708us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b180035f560. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:42.120638 (+326703us) spinlock_profiling.cc:243] Waited 26.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:42.177036 (+ 56398us) raft_consensus.cc:1685] finished
0709 13:11:42.177061 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:42.177234 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":292777,"prepare.queue_time_us":695987,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":7357,"raft.queue_time_us":167175,"raft.run_cpu_time_us":617,"raft.run_wall_time_us":43751,"spinlock_wait_cycles":107271936,"thread_start_us":761394,"threads_started":2,"wal-append.queue_time_us":1048003}
W20260709 13:11:42.154917 28115 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.876s	user 0.015s	sys 0.000s
I20260709 13:11:42.080686 28118 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.205356 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:42.206255 22779 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:42.147533 22701 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:42.147862 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.085178 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.121862 28209 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.098431 28050 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.202046 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:42.331498 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:42.211676 28213 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:11:42.234022 28069 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:42.223426 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.333066 28045 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 14
W20260709 13:11:42.240638 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 80427.6us
I20260709 13:11:42.333675 28088 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:42.228350 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:42.195209 28185 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.002s
W20260709 13:11:42.288450 28002 consensus_meta.cc:287] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.638s	user 0.013s	sys 0.000s
I20260709 13:11:42.274971 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:42.288033 28042 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.596s	user 0.014s	sys 0.001s
I20260709 13:11:42.336027 28185 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.212113 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3095) took 2496 ms. Trace:
I20260709 13:11:42.202250 28179 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:42.226949 28118 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:42.333021 22703 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:42.241657 27946 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:42.385941 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 278481us
I20260709 13:11:42.288992 22775 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:11:42.399046 22627 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 14. Ops: [12.8-12.8]
I20260709 13:11:42.400151 28050 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:42.401178 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:42.401593 22635 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.002s
I20260709 13:11:42.526665 22635 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:42.505859 22784 rpcz_store.cc:276] 0709 13:11:39.715478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:39.750467 (+ 34989us) service_pool.cc:224] Handling call
0709 13:11:39.750958 (+   491us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:40.188332 (+437374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:40.188389 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:40.189371 (+   982us) log.cc:844] Serialized 52 byte log entry
0709 13:11:40.566319 (+376948us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b480012c258. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:11:40.664361 (+ 98042us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:41.767764 (+1103403us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:42.058988 (+291224us) spinlock_profiling.cc:243] Waited 5.8 ms on lock 0x7b18004b4e40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:11:42.059191 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 13:11:42.064620 (+  5429us) log.cc:844] Serialized 14 byte log entry
0709 13:11:42.196846 (+132226us) spinlock_profiling.cc:243] Waited 7.17 ms on lock 0x7b18004b4e40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:42.211382 (+ 14536us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18004b4e40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1806a93d70 00007f1806a87293 00007f1806a7ccea 00007f1806a7d23a 00007f1806ee7a2a 00007f1806ef70bf 00007f1806ef7038
0709 13:11:42.211715 (+   333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:42.211808 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:42.211856 (+    48us) raft_consensus.cc:1685] finished
0709 13:11:42.211869 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:42.212045 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194018,"prepare.queue_time_us":173278,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":125,"raft.queue_time_us":159966,"raft.run_cpu_time_us":2183,"raft.run_wall_time_us":659083,"spinlock_wait_cycles":107348608,"thread_start_us":1154198,"threads_started":3,"wal-append.queue_time_us":752278}
I20260709 13:11:42.402027 28002 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:42.292698 28193 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:42.402822 22776 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:42.410161 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:42.532227 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.536708 22781 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:42.418802 28029 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:42.488806 28207 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:42.492913 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 345165us
I20260709 13:11:42.406258 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.494249 22633 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
W20260709 13:11:42.544870 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 138716us
I20260709 13:11:42.545639 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:42.500854 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.546746 22783 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:42.501976 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.547514 22709 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:42.499817 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:42.499377 22626 consensus_meta.cc:287] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.629s	user 0.008s	sys 0.008s
I20260709 13:11:42.549782 22626 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:42.505676 22705 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
W20260709 13:11:42.505578 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 421809us
W20260709 13:11:42.505865 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 121249us
I20260709 13:11:42.506557 22630 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:42.502214 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.552999 22704 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:42.506402 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.553339 22634 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:42.502744 22710 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:11:42.521971 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.554126 22701 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:42.506139 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.890s (SENT)
I20260709 13:11:42.554699 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:42.524391 22775 pending_rounds.cc:85] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 3
I20260709 13:11:42.520223 22708 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:42.555454 22775 pending_rounds.cc:107] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:11:42.555816 22775 raft_consensus.cc:2889] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:42.502902 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:42.537813 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:42.556639 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.557009 22784 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:42.557399 22635 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.557967 22701 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:11:42.558712 22629 raft_consensus.cc:2368] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15.
I20260709 13:11:42.552594 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.551570 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.509778 22632 raft_consensus.cc:2410] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:11:42.556918 22702 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:42.529906 22780 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:42.560107 22707 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.546007 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.563046 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:42.563755 23417 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:42.495136 28058 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:42.568005 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.571663 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:42.572336 22703 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:42.501559 28185 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:42.581198 22626 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
W20260709 13:11:42.538509 28094 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.007s
I20260709 13:11:42.575876 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.573711 28203 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:42.562973 28201 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:11:42.581010 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74919.8us
W20260709 13:11:42.593786 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.154s (SENT) [suppressed 54 similar messages]
I20260709 13:11:42.563177 28077 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:42.503644 27781 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:11:42.533869 28223 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.499717 28042 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:42.600033 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:42.605887 22635 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:42.543220 28193 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:42.498353 27937 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:42.613118 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45231.9us
I20260709 13:11:42.588471 22626 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:11:42.617228 22707 raft_consensus.cc:1240] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:11:42.597092 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.535939 28208 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.623211 22704 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:11:42.566547 28066 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:42.574438 28147 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.640926 23417 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
I20260709 13:11:42.582279 27973 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.659924 28201 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:42.660105 27781 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:11:42.796588 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63468.8us
W20260709 13:11:42.673348 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59185.7us
W20260709 13:11:42.673954 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57445us
I20260709 13:11:42.673475 28203 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:11:42.613242 28219 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.677649 28145 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:42.611671 28094 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.588392 28187 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:42.679441 28223 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:42.681802 28226 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:11:42.578333 28194 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:42.683156 28179 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:42.684438 28207 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:42.610118 28189 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:11:42.695784 28222 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:42.696362 28058 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:42.709981 28042 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:42.710453 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:42.710750 22706 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:42.710951 28117 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:42.746554 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 149581us
I20260709 13:11:42.877750 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.878997 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:42.759172 22747 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:42.802830 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 77329.8us
I20260709 13:11:42.791770 27973 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.621048 22709 raft_consensus.cc:2410] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:11:42.819948 22784 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:42.791064 28124 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:11:42.845749 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46763.6us
I20260709 13:11:42.794637 27781 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:42.870604 22628 raft_consensus.cc:2368] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:11:42.804338 22703 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:42.880816 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:42.965104 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 84392.3us
I20260709 13:11:42.966223 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.845887 28187 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:42.832989 28145 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.851440 28222 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:42.971350 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:42.883323 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 124324us
I20260709 13:11:42.975185 28216 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:42.982074 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:42.974550 28066 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:42.967729 28045 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.005s
W20260709 13:11:42.900321 23385 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.866s (SENT)
I20260709 13:11:42.987769 23417 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:11:43.008947 23385 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:42.945407 28042 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:11:42.974776 28169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.028s	sys 0.035s
I20260709 13:11:42.873873 27970 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 13:11:42.903941 28189 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:42.989832 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:42.990785 28190 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:11:43.010527 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 110264us
I20260709 13:11:43.029688 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:42.936911 28077 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:11:43.016865 22777 raft_consensus.cc:2410] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
W20260709 13:11:42.999147 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 405359us
I20260709 13:11:42.991148 28207 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:43.035342 22750 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.168s (SENT)
I20260709 13:11:43.035777 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:42.975036 28224 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:42.993444 27946 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.038033 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:42.989784 28124 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:42.841907 28205 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.178s
I20260709 13:11:42.974910 28209 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:11:43.020906 28211 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:11:43.034364 28169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.030s	sys 0.037s
I20260709 13:11:43.035786 28042 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:42.766413 28147 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.833447 28193 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.826890 28219 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.879727 28179 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:42.984805 28201 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.048803 28189 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:43.052513 28069 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:43.095082 27937 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:43.095898 28207 consensus_queue.cc:1059] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 13:11:43.098698 28050 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.564s	user 0.012s	sys 0.001s
I20260709 13:11:43.098949 28124 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.099875 28223 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:43.100054 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:43.100742 22709 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:43.066582 28087 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:11:43.067221 28232 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:11:43.014868 28056 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:11:43.038755 28066 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:43.062206 22783 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.515s	user 0.014s	sys 0.000s
I20260709 13:11:43.104481 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:43.105142 22783 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:11:43.054558 28224 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:42.997255 28216 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.093540 28189 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.107926 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:43.108880 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:43.115509 23416 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:11:43.105027 28208 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:43.128598 22628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2209) took 1249 ms (client timeout 1612 ms). Trace:
I20260709 13:11:43.168972 22777 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2156) took 1314 ms. Trace:
I20260709 13:11:43.169600 22777 rpcz_store.cc:276] 0709 13:11:41.822356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.931263 (+108907us) service_pool.cc:224] Handling call
0709 13:11:43.135921 (+1204658us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003408d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:43.136202 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425193216}
W20260709 13:11:43.166895 22628 rpcz_store.cc:269] 0709 13:11:41.879474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:41.954380 (+ 74906us) service_pool.cc:224] Handling call
0709 13:11:43.128271 (+1173891us) spinlock_profiling.cc:243] Waited 902 ms on lock 0x7b54000b86d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:43.128494 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2075484416}
I20260709 13:11:43.165001 22782 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:11:43.173254 22782 pending_rounds.cc:85] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 9
I20260709 13:11:43.173586 22782 pending_rounds.cc:107] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:11:43.173866 22782 raft_consensus.cc:2889] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:43.064901 27781 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.078364 28187 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:43.181265 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 302352us
I20260709 13:11:43.106050 27978 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:11:43.121074 28223 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.121635 28094 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.064584 28058 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:43.205430 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:43.212386 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.215334 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:43.133591 22709 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:11:43.216890 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:43.219723 22784 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:43.227846 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:43.228894 22781 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:43.233659 22783 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:11:43.234033 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 169896us
W20260709 13:11:43.238332 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:43.238574 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:43.245111 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:43.245380 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:43.246726 22778 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:43.246795 22783 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:43.191808 27973 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:43.168826 28205 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:11:43.239355 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:43.262953 22675 leader_election.cc:400] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:11:43.263461 22675 leader_election.cc:403] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 13:11:43.273530 23385 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.719s (SENT)
I20260709 13:11:43.237550 27781 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:43.276599 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.278857 22784 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:43.243516 28045 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:11:43.353166 22781 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:43.322425 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:43.204502 28077 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:43.271963 28225 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:43.358685 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:43.189193 28042 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.213601 28223 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.370545 23421 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:43.377108 22631 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
W20260709 13:11:43.401847 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 165553us
W20260709 13:11:43.405615 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 167153us
I20260709 13:11:43.341796 28242 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:43.356452 28210 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:43.368666 28045 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:43.374851 27937 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:43.411943 22783 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:43.450959 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:43.452026 23419 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:11:43.441936 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:43.453896 22628 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:43.420769 28045 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.420701 28185 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.450944 28219 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:43.473222 28045 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:43.441449 28199 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:11:43.455462 28225 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:43.479197 28069 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:43.492676 28241 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:11:43.453004 28248 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
W20260709 13:11:43.519706 28234 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.881s	user 0.007s	sys 0.003s
I20260709 13:11:43.522197 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:43.523301 23414 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
W20260709 13:11:43.523752 28243 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
W20260709 13:11:43.526094 28244 log.cc:927] Time spent T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:11:43.527699 22750 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.192s (SENT)
I20260709 13:11:43.529104 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.529402 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:43.512918 28069 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.503754 28216 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.530261 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3126) took 1363 ms. Trace:
I20260709 13:11:43.532945 22628 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:11:43.548642 22632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 756) took 1145 ms. Trace:
I20260709 13:11:43.481803 28185 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:43.551239 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 228958us
I20260709 13:11:43.549901 28219 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.561398 28267 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:43.564394 28263 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.565137 28263 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.568032 28263 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.545434 22776 rpcz_store.cc:276] 0709 13:11:42.166412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.627356 (+460944us) service_pool.cc:224] Handling call
0709 13:11:42.762446 (+135090us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:42.791080 (+ 28634us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.791146 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:42.822493 (+ 31347us) log.cc:844] Serialized 52 byte log entry
0709 13:11:43.134243 (+311750us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:43.428061 (+293818us) log.cc:844] Serialized 14 byte log entry
0709 13:11:43.529888 (+101827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.529977 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:43.530025 (+    48us) raft_consensus.cc:1685] finished
0709 13:11:43.530038 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.530197 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46062,"prepare.queue_time_us":404530,"raft.queue_time_us":78748,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":36286,"thread_start_us":555748,"threads_started":3,"wal-append.queue_time_us":364373}
I20260709 13:11:43.561429 28219 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:43.561436 28250 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:11:43.569664 28147 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:43.570150 28262 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:43.537523 27978 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:43.537390 28224 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:43.545292 28194 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.701s	user 0.013s	sys 0.002s
I20260709 13:11:43.570392 23419 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:43.522599 28193 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:43.593152 28252 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 13:11:43.576023 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48370.1us
I20260709 13:11:43.573877 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:43.596822 22600 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:43.549319 22632 rpcz_store.cc:276] 0709 13:11:42.403531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.574343 (+170812us) service_pool.cc:224] Handling call
0709 13:11:42.696512 (+122169us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:42.696745 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.697027 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:11:42.992391 (+295364us) log.cc:844] Serialized 14 byte log entry
0709 13:11:43.548035 (+555644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:43.548112 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:11:43.548186 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.548298 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.548553 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27961,"prepare.queue_time_us":403425,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":126068,"spinlock_wait_cycles":31502464,"thread_start_us":822110,"threads_started":3,"wal-append.queue_time_us":364282}
I20260709 13:11:43.570942 28225 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.551352 28119 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:43.579584 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:43.591218 28259 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:43.591806 28194 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:43.601004 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.072s (SENT)
I20260709 13:11:43.617748 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.618438 22778 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:43.500224 28077 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:43.617127 22750 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.564s (SENT)
I20260709 13:11:43.582527 28262 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:43.624119 22750 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.594s (SENT)
I20260709 13:11:43.594475 27978 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.635106 28194 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:43.637210 28219 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.638867 28185 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:43.622737 28259 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.644609 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.652724 22630 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:43.674937 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 95446.6us
W20260709 13:11:43.669996 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 221035us
W20260709 13:11:43.672670 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 143357us
I20260709 13:11:43.615134 28118 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:11:43.677095 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53004.9us
I20260709 13:11:43.676111 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:43.678941 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:43.623715 28211 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:43.739480 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:43.741097 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62277.3us
I20260709 13:11:43.653796 28262 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:43.745244 22632 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
W20260709 13:11:43.749576 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 135791us
W20260709 13:11:43.743991 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68016.3us
I20260709 13:11:43.757787 27978 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.673478 28194 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:43.735517 28199 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:43.673036 28223 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.796727 28179 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:43.798164 22631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2212) took 1880 ms (client timeout 2168 ms). Trace:
I20260709 13:11:43.677860 28185 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:43.760995 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.768318 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.798857 28069 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:43.802922 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:43.803313 28271 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:11:43.804337 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.771133 28063 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:43.773437 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:43.776448 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.823397 22627 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:43.777331 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.779421 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.740716 28118 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:43.751232 28058 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:43.834259 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.835031 22778 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 6.
I20260709 13:11:43.763428 28042 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:43.765028 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.075s (SENT)
I20260709 13:11:43.847903 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:43.766822 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:43.798743 22631 rpcz_store.cc:269] 0709 13:11:41.917645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.150498 (+232853us) service_pool.cc:224] Handling call
0709 13:11:43.797911 (+1647413us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b540034a154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:43.798077 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2135558400}
I20260709 13:11:43.801383 22632 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14.
W20260709 13:11:43.762326 22750 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 3.245s (SENT)
I20260709 13:11:43.830811 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.833782 22634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2853) took 1433 ms. Trace:
W20260709 13:11:43.840456 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 243717us
I20260709 13:11:43.841014 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:43.849854 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.842777 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:43.843688 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.844417 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.852330 22781 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:43.845392 22626 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:11:43.850099 22634 rpcz_store.cc:276] 0709 13:11:42.400286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.578983 (+178697us) service_pool.cc:224] Handling call
0709 13:11:42.579644 (+   661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:42.794477 (+214833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.794513 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:42.794557 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:42.794587 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:43.833400 (+1038813us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540006ccd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:43.833483 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:43.833694 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493435776}
I20260709 13:11:43.849956 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.743696 28264 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:43.855899 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.867727 22783 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
W20260709 13:11:43.869129 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 106870us
W20260709 13:11:43.856907 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 91918.4us
I20260709 13:11:43.856483 22598 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:43.864465 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.877955 22781 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14.
I20260709 13:11:43.879637 28179 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.858098 28042 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.880620 28042 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:43.880373 28179 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:43.886759 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:43.887447 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:43.895931 23384 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:43.836639 28194 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.767814 28187 consensus_queue.cc:1059] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:11:43.842006 28058 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:43.906399 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:43.867728 28216 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:43.865727 28264 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.884068 28248 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:43.833143 28185 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.906481 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:43.954427 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 103937us
I20260709 13:11:43.891933 28223 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:43.950026 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:43.901666 28282 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:43.964146 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57844.5us
W20260709 13:11:43.953003 28270 log.cc:927] Time spent T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:11:43.967828 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:43.942118 28189 raft_consensus.cc:2764] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:11:43.943392 28242 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11.
I20260709 13:11:43.952503 28211 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:43.959404 28225 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:43.904505 28227 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 13:11:43.957435 28267 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:11:43.963701 28118 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:44.037947 22750 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.664s (SENT)
I20260709 13:11:44.234938 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:44.025532 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 135137us
I20260709 13:11:43.973955 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:44.010268 28223 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.957881 28264 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:44.242128 28223 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.122200 28265 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.008121 28282 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.971626 28077 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.007786 28199 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:44.203747 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.245005 28042 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.156339 28185 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:43.966156 28124 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:44.246904 28266 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:44.248379 28290 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.248744 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 210830us
W20260709 13:11:44.095566 27781 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.002s
I20260709 13:11:44.249121 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:44.304520 22750 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.728s (SENT)
W20260709 13:11:44.217147 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 252578us
I20260709 13:11:44.279352 22702 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:44.273515 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 2168) took 2254 ms. Trace:
I20260709 13:11:44.277159 23416 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1176) took 1272 ms. Trace:
I20260709 13:11:44.260656 28189 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:44.306897 22782 rpcz_store.cc:276] 0709 13:11:42.018851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.545994 (+527143us) service_pool.cc:224] Handling call
0709 13:11:43.139508 (+593514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.174536 (+ 35028us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.273251 (+1098715us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54003408d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:44.273436 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364102400}
I20260709 13:11:44.307451 23416 rpcz_store.cc:276] 0709 13:11:43.004552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.006342 (+  1790us) service_pool.cc:224] Handling call
0709 13:11:43.006740 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.142248 (+135508us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.276835 (+1134587us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54006329d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:44.277080 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240076544}
I20260709 13:11:44.263175 22710 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:44.308425 22706 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 13:11:44.281283 28042 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.265205 28265 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:44.250296 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 276388us
I20260709 13:11:44.028076 28119 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.242s
I20260709 13:11:44.306115 28290 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:44.080252 27937 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.653s	user 0.013s	sys 0.000s
I20260709 13:11:44.315905 27937 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:44.047004 28058 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.318032 28290 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.274436 28267 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.319159 28267 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.298017 28262 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.321441 28267 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.233354 28225 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:44.321767 22701 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
I20260709 13:11:44.270025 28222 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.322779 28225 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.323503 28222 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:44.323884 28225 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:44.324424 28262 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.296690 27978 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:44.325590 27978 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:43.971233 28281 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:11:44.022382 28045 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.001s
W20260709 13:11:44.296373 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 345485us
I20260709 13:11:44.329262 28045 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.332345 28282 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.333786 22633 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:44.010602 28275 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:44.070927 28179 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.233366 28211 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.299329 22635 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
W20260709 13:11:44.301004 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 288723us
I20260709 13:11:44.223513 28069 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:44.358493 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:44.358440 28069 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.308930 22706 pending_rounds.cc:85] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:44.359892 22706 pending_rounds.cc:107] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:11:44.360208 22706 raft_consensus.cc:2889] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:44.361085 28069 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.347548 27781 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.310045 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.953s (SENT)
I20260709 13:11:44.311468 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:44.313781 28298 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:44.370805 28298 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.373731 28298 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.332592 28265 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.020565 28259 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.335170 28281 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.344388 28045 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.345355 22631 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:11:44.400748 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.401918 22631 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 19.
I20260709 13:11:44.346105 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.348376 23384 leader_election.cc:400] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14.
I20260709 13:11:44.407099 23384 leader_election.cc:403] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:11:44.407989 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 59778.6us
I20260709 13:11:44.357553 22600 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:44.364984 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53653.2us
I20260709 13:11:44.410913 28058 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14.
W20260709 13:11:44.366155 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56149.8us
I20260709 13:11:44.357529 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:44.369590 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:44.383189 27937 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:44.363165 27781 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.412806 27937 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.363634 28211 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:44.413446 27937 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:44.414675 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.334326 27978 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.386511 28199 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:11:44.304366 28077 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:44.418251 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60826.2us
W20260709 13:11:44.418638 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61142.7us
I20260709 13:11:44.419801 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:44.419934 22632 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:44.420320 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.421114 22675 leader_election.cc:400] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15.
I20260709 13:11:44.422174 22675 leader_election.cc:400] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:11:44.422055 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.423832 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:44.433423 28119 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:44.434454 28077 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.435485 28265 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:44.436223 28265 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:44.437305 28265 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.417716 28225 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:44.439004 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.439352 28225 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.439818 22707 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:44.441813 28225 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.438496 28209 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:44.443398 28209 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.444501 28209 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.446198 27978 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:44.446837 27978 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.447784 27978 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.452203 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:44.452584 23384 leader_election.cc:400] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14.
I20260709 13:11:44.452824 22707 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:44.455142 23418 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:11:44.458535 28281 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.456945 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.467691 28227 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:44.468816 28227 consensus_queue.cc:1243] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:11:44.472862 27937 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.476848 27781 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.479249 27937 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.479604 28210 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:44.455966 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:44.481438 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:44.489761 28255 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:44.449749 28045 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:11:44.500628 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.966s (SENT)
W20260709 13:11:44.501080 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45202.7us
I20260709 13:11:44.502054 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.435829 28267 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 13:11:44.504576 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 80814.4us
I20260709 13:11:44.507318 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.509021 22701 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:11:44.510177 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.511332 22707 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 7.
I20260709 13:11:44.512463 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.483548 28225 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:44.513350 22750 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.061s (SENT)
I20260709 13:11:44.528617 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.514793 22777 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:11:44.511468 28199 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:44.511561 28255 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.532183 28199 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.532907 28255 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.539703 28194 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:44.541139 28194 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:11:44.542865 28193 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:44.543304 28264 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.546756 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:44.548910 22703 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:44.491240 28210 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:44.549578 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:44.551599 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:44.526458 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.552297 22633 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:44.552634 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.552300 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.554137 22632 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:44.514464 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.526546 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.555631 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:44.513455 27937 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:44.558403 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.582592 22707 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:44.567528 22777 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:44.580500 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:44.583087 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69774.3us
I20260709 13:11:44.584903 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:44.513887 28179 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:44.556859 22701 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:44.584004 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:44.584928 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.062s (SENT)
W20260709 13:11:44.592607 22750 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.196s (SENT)
I20260709 13:11:44.592905 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.593716 28249 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:44.557410 28264 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:44.596874 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.221s (SENT)
W20260709 13:11:44.597869 22750 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.238s (SENT)
I20260709 13:11:44.598275 22750 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.550346 27781 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:44.600941 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:44.602730 22781 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:44.602888 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.514026 28045 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:44.555667 28159 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:11:44.542562 28194 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:44.586025 28201 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:44.601289 28069 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:44.602586 28179 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:44.587244 27937 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.606927 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:44.639271 22781 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:11:44.613379 28194 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.607704 28077 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:44.607950 28159 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:44.596583 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:44.594547 27942 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:44.585709 28199 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.589095 28258 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.543393 28193 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:44.603484 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:44.630102 28268 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:44.609417 22784 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:44.616173 28275 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:44.626392 28069 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.644659 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:44.624392 27878 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.665737 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.639972 27937 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:44.671314 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 74805.7us
W20260709 13:11:44.676084 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45797.8us
I20260709 13:11:44.681877 22632 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:44.672506 22701 raft_consensus.cc:2410] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:11:44.664081 28159 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:44.724098 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.724938 28211 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.725796 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.669173 28117 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:44.731281 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.673462 28069 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:44.693602 28147 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:11:44.745164 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 70912.8us
I20260709 13:11:44.743160 22707 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
W20260709 13:11:44.746302 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 148470us
I20260709 13:11:44.746206 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:44.693934 27878 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.748775 22781 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:11:44.748946 22750 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.296s (SENT)
I20260709 13:11:44.757159 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:44.667078 27942 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.673461 28290 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:11:44.695029 28296 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:11:44.747885 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.744621 28147 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.804157 28147 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:44.635022 28210 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.785327 28268 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:44.786391 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.788861 27878 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.709666 28179 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.794999 22701 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:44.674389 28275 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.824450 28275 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.797945 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:44.825320 27942 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.825434 22782 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:44.711261 28045 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:44.807756 28227 consensus_queue.cc:1175] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 17 status { last_received { term: 13 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:11:44.828159 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.813165 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.715160 28149 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:11:44.829517 27937 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.830528 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.831864 22707 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:11:44.832089 27937 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.832876 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.818722 28145 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:44.828001 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:44.670885 28258 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.836324 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.837149 22701 raft_consensus.cc:2393] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 7: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:44.837332 28258 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:44.838003 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.838637 22701 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
I20260709 13:11:44.838888 27878 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:44.839387 27878 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:44.839546 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.839826 27878 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:44.841243 28147 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.717001 28194 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.762163 28050 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:44.845216 28194 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:44.845338 28050 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.753403 28193 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.802340 22781 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:44.852079 28193 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:44.852972 23384 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:44.854027 28216 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:44.854589 28216 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:44.855551 28216 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.748577 28117 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.817082 28225 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:44.865707 22710 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
W20260709 13:11:44.866149 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:44.868037 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.869468 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.870057 22710 raft_consensus.cc:2376] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
I20260709 13:11:44.870913 28088 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:44.871506 28088 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:44.892277 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.895819 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:44.897130 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.898084 22701 raft_consensus.cc:2376] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15. Re-sending same reply.
I20260709 13:11:44.899724 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:44.912307 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.912504 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.917917 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.918838 22710 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:44.919688 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.921165 22710 raft_consensus.cc:2368] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12.
I20260709 13:11:44.922915 22710 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:44.924784 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.926121 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.927498 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.929015 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.934684 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.938894 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.894959 28211 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.916065 22707 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:44.935736 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.942241 22710 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:44.943521 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.945147 22707 raft_consensus.cc:2376] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9. Re-sending same reply.
I20260709 13:11:44.949955 22707 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: [17.14-18.15]
I20260709 13:11:44.946316 22701 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:11:44.956374 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:44.957312 22627 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:44.959677 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.961874 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:44.962918 22710 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:44.964178 22630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2219) took 2916 ms. Trace:
I20260709 13:11:44.964931 22630 rpcz_store.cc:276] 0709 13:11:42.054476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.566992 (+512516us) service_pool.cc:224] Handling call
0709 13:11:42.576566 (+  9574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:43.675780 (+1099214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.956458 (+1280678us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540022ea54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:44.956654 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2452241792}
I20260709 13:11:44.966183 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.967132 22701 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:11:44.962901 22707 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:11:44.977546 22627 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:44.979403 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:44.951198 28077 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:44.951442 28185 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:44.987030 28149 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:44.989311 28159 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.962663 28088 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:44.993158 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:44.995939 22710 raft_consensus.cc:2376] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14. Re-sending same reply.
I20260709 13:11:44.990409 28149 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:45.012295 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.013254 22710 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:45.018222 28205 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:45.018872 28205 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:45.019963 28205 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:44.998986 28179 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:44.987499 28077 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.025523 22776 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 20: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:44.989298 28147 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.026844 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:45.029610 22704 raft_consensus.cc:2376] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11. Re-sending same reply.
W20260709 13:11:45.012104 28222 consensus_meta.cc:287] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.688s	user 0.005s	sys 0.013s
I20260709 13:11:45.030686 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.032411 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.033085 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.036031 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.036736 22781 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:45.036950 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.021271 28205 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:45.042048 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.044809 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59768 (request call id 760) took 1305 ms. Trace:
I20260709 13:11:44.992625 28185 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.045261 22635 rpcz_store.cc:276] 0709 13:11:43.739256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.739419 (+   163us) service_pool.cc:224] Handling call
0709 13:11:43.759413 (+ 19994us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.356913 (+597500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.044593 (+687680us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540006dbd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:45.044753 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241115392}
I20260709 13:11:45.047564 22780 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:45.049341 22704 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:45.050258 22781 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:11:45.020684 28050 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:45.053634 22775 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:45.063447 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.065410 22704 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:45.066916 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:45.066483 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:45.082864 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.021641 28179 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.042918 28249 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.117782 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:45.118721 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.069990 22701 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:11:45.124598 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.062094 28225 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.131721 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.031579 28149 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.146471 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.151774 22701 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:11:45.155551 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.132663 28208 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:45.150538 28305 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.592s	user 0.002s	sys 0.000s
I20260709 13:11:45.157791 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:45.161646 22779 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:45.164371 28077 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.141415 28225 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.167526 28159 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:45.168486 28225 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:45.169673 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:45.160595 27878 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.180540 23418 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:45.180325 28179 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.185619 28248 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:45.186223 28248 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:45.187223 28248 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.188340 28248 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:45.192627 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3283) took 1002 ms. Trace:
I20260709 13:11:45.193118 22705 rpcz_store.cc:276] 0709 13:11:44.190395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.191392 (+   997us) service_pool.cc:224] Handling call
0709 13:11:44.191861 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.240699 (+ 48838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.240735 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.240782 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:44.240816 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.192344 (+951528us) spinlock_profiling.cc:243] Waited 48.6 ms on lock 0x7b54003d0ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:45.192389 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.192551 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111742720}
I20260709 13:11:45.186094 27878 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.195127 27878 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:45.195670 27878 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.197880 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.198683 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3282) took 1019 ms. Trace:
I20260709 13:11:45.205899 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.206089 22703 rpcz_store.cc:276] 0709 13:11:44.179512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.180420 (+   908us) service_pool.cc:224] Handling call
0709 13:11:44.180757 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.549369 (+368612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.198387 (+649018us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54003d09d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:45.198590 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":846181248}
I20260709 13:11:45.206775 22783 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:45.195840 23385 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.140683 28249 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.207960 28222 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:45.208386 28222 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.208736 28222 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:45.212144 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.214083 23385 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.220844 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.226528 22784 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:45.228044 22784 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:11:45.228739 27878 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.235737 28185 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.244196 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:45.250912 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.252727 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:45.217067 28227 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:45.249447 28185 raft_consensus.cc:2764] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:11:45.267591 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:45.270568 22784 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:45.276897 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.243681 22779 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:45.282099 28227 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:45.283732 27878 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:45.284231 27878 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:45.285429 27878 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.286502 27970 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:45.287050 27878 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:45.287139 27970 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.287829 27970 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:45.290171 28094 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:45.290936 28094 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.292399 27978 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:11:45.292752 28258 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:11:45.293411 28094 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:45.299242 28069 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.543s	user 0.000s	sys 0.011s
I20260709 13:11:45.299983 28069 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.302424 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:45.303386 22779 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:45.311190 28249 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.212327 28208 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.324397 28208 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:45.331421 22778 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:45.336746 22627 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:45.342077 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.343356 22784 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:45.344938 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:45.358156 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.365728 22779 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
W20260709 13:11:45.380018 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.383911 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 206029us
W20260709 13:11:45.385138 22750 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.767s (SENT)
I20260709 13:11:45.385582 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.401728 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.275952 28185 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:45.410920 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:45.416579 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:45.417634 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.412815 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.413242 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.411940 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.420747 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:45.421816 22627 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:45.421764 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.427744 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:45.430685 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46252 (request call id 1180) took 1993 ms. Trace:
I20260709 13:11:45.431339 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.431205 23417 rpcz_store.cc:276] 0709 13:11:43.437465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.437734 (+   269us) service_pool.cc:224] Handling call
0709 13:11:43.465663 (+ 27929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.362048 (+896385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.362104 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.362155 (+    51us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:11:44.362190 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.430320 (+1068130us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b54005e2254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:45.430391 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.430570 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060945664}
W20260709 13:11:45.432799 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.391163 28208 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:45.435662 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.437105 28310 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:45.437323 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.438319 23417 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:11:45.442188 28227 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.428546 28159 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:45.449442 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.450592 22627 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:45.440371 28208 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.433759 28069 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.469978 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.473026 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.427755 28248 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:45.451423 28310 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:45.483904 28310 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.455698 28227 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.489464 28227 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.498806 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3289) took 1288 ms. Trace:
I20260709 13:11:45.499542 22702 rpcz_store.cc:276] 0709 13:11:44.210905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.211080 (+   175us) service_pool.cc:224] Handling call
0709 13:11:44.211475 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.306533 (+ 95058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.498385 (+1191852us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b5400314a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:45.498708 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155520384}
I20260709 13:11:45.501228 22780 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:45.513851 23421 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:45.516111 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:45.463924 28045 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:45.529269 28332 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:11:45.453919 28147 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:45.531068 22782 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 19. Current term is 20. Ops: []
W20260709 13:11:45.533468 28324 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:11:45.533502 22629 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:45.536013 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2862) took 1726 ms. Trace:
I20260709 13:11:45.536099 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.536553 22629 rpcz_store.cc:276] 0709 13:11:43.809987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.813017 (+  3030us) service_pool.cc:224] Handling call
0709 13:11:45.532794 (+1719777us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b5800002d98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:11:45.533017 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.535652 (+  2635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.535938 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22052608}
I20260709 13:11:45.463867 28208 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:11:45.539816 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.476045 28309 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:11:45.550428 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 72738.8us
I20260709 13:11:45.477118 28088 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:45.554446 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:45.557247 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.564630 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.550915 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77974.5us
I20260709 13:11:45.551625 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:45.578683 22702 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:11:45.585863 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.589092 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.590102 22706 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:45.593291 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:45.593990 22703 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:45.611112 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.612365 22706 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:11:45.621456 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.623637 22705 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:45.624352 22705 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:45.625661 22705 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.628309 22705 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:45.631649 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.627270 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:45.635169 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
W20260709 13:11:45.639207 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 84859us
I20260709 13:11:45.643323 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:45.644220 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:11:45.642671 22780 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:11:45.645310 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 508205us
I20260709 13:11:45.553493 28247 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:11:45.650840 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:45.567101 28245 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:45.538172 28045 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.545710 28205 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
W20260709 13:11:45.646425 28324 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:11:45.665589 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 114085us
I20260709 13:11:45.649442 28268 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:45.668649 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:45.671293 22776 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:11:45.666973 28297 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:11:45.677098 28227 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:45.666646 28222 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.686691 28268 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:45.710281 22601 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.746845 28056 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:45.709168 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:45.718873 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 68107.1us
I20260709 13:11:45.689455 28249 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:45.753991 22703 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:45.756654 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:45.765256 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.770336 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.770893 22705 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:11:45.772099 22705 pending_rounds.cc:85] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:11:45.772653 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.772725 22705 pending_rounds.cc:107] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:11:45.773010 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.773505 22705 raft_consensus.cc:2889] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:45.773897 23412 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:11:45.774215 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:45.775516 28256 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:45.775367 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66360.6us
I20260709 13:11:45.734980 28227 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:45.735841 28316 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.701910 28225 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:11:45.712507 28257 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.747792 28077 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:45.794400 23419 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:11:45.761118 28315 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:45.795403 28316 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:45.800506 28333 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:11:45.801317 28330 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:11:45.796558 28225 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.789573 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:45.834190 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50069.2us
I20260709 13:11:45.835253 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:45.837539 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:45.821518 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62855.3us
I20260709 13:11:45.802101 28257 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.850245 28257 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.783635 28249 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:45.825065 28259 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:11:45.841424 28309 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:45.847873 28225 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:45.839010 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:45.852595 22750 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:45.871634 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:45.876402 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77503.8us
I20260709 13:11:45.877708 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:45.879962 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.866179 28256 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:45.834517 28279 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
I20260709 13:11:45.882800 28088 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:45.855505 28310 raft_consensus.cc:2764] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:11:45.869809 28309 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.903628 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.879738 28069 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:11:45.911509 28336 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:11:45.912827 28249 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:45.913252 28307 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.110s	user 0.003s	sys 0.001s
I20260709 13:11:45.907459 28205 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:45.952826 28205 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.915396 28309 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:45.953264 28205 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:45.916224 28310 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.954530 28316 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.954905 28310 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.916788 28345 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:45.958213 28345 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.958711 28345 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:45.960388 28319 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.834s	user 0.002s	sys 0.000s
I20260709 13:11:45.950035 28225 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.961066 28245 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:11:45.918210 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.963827 28310 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:45.963052 28225 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.972258 28256 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:45.918725 28340 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.920778 28088 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:45.890981 28045 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:45.904691 28227 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:45.930142 27970 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.004s
I20260709 13:11:45.932395 28117 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:45.935400 28349 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:45.937294 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3260) took 3394 ms. Trace:
I20260709 13:11:45.998317 22708 rpcz_store.cc:276] 0709 13:11:42.526767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:42.564353 (+ 37586us) service_pool.cc:224] Handling call
0709 13:11:42.564699 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:42.607009 (+ 42310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:42.607050 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:43.448202 (+841152us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18001c5940. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:11:45.027658 (+1579456us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b2c0006cb88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:45.030331 (+  2673us) log.cc:844] Serialized 52 byte log entry
0709 13:11:45.142361 (+112030us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:11:45.742061 (+599700us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.749772 (+  7711us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.917030 (+167258us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b18001c5940. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287
0709 13:11:45.917201 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:45.917282 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:45.920698 (+  3416us) raft_consensus.cc:1685] finished
0709 13:11:45.920717 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:45.920881 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":482733,"prepare.queue_time_us":548577,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":6879,"raft.queue_time_us":278978,"raft.run_cpu_time_us":631,"raft.run_wall_time_us":51479,"spinlock_wait_cycles":66196352,"thread_start_us":452308,"threads_started":3,"wal-append.queue_time_us":244102}
I20260709 13:11:45.921116 28341 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:45.951572 28222 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:45.976382 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 218150us
W20260709 13:11:45.978680 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.009681 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:46.011483 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.944970 28335 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:11:45.994177 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67153.1us
W20260709 13:11:46.014024 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:45.939435 28323 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.887s	user 0.002s	sys 0.000s
I20260709 13:11:46.018302 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:46.019076 22784 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.016611 28249 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:11:46.004475 28310 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.021113 28117 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.021675 28117 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.025597 28058 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.025990 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:46.026338 28058 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.021600 28310 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:45.994179 28088 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:45.925316 28193 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.088s
W20260709 13:11:46.029117 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 190194us
I20260709 13:11:46.032841 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:45.972208 28309 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.037619 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.038333 22710 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2729) took 1112 ms. Trace:
I20260709 13:11:46.039739 22784 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:46.041790 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:46.041790 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:46.044097 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:45.883474 28268 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.005515 27970 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.047403 28352 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:46.050289 22707 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:46.050787 22707 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:46.051084 28310 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.051659 22707 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.052858 22707 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:46.048163 28346 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:45.993366 28340 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.055552 22705 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:11:46.056056 28340 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.057577 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:46.059152 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.059854 28194 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:46.060686 28194 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:11:46.061070 28194 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:46.061293 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.061410 28194 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:46.049117 28343 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.031540 28222 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.050366 28309 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.063138 28343 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.063390 28222 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.065639 28343 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.062249 28345 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.039115 22710 rpcz_store.cc:276] 0709 13:11:44.926136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.047254 (+121118us) service_pool.cc:224] Handling call
0709 13:11:45.118109 (+ 70855us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:45.131266 (+ 13157us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:11:45.491742 (+360476us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b18002fa700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:11:45.704379 (+212637us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.917651 (+213272us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:45.918985 (+  1334us) log.cc:844] Serialized 52 byte log entry
0709 13:11:45.919135 (+   150us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:45.919396 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:11:45.920090 (+   694us) log.cc:844] Serialized 14 byte log entry
0709 13:11:46.030688 (+110598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.037728 (+  7040us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:46.037937 (+   209us) raft_consensus.cc:1685] finished
0709 13:11:46.037960 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.038253 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35075,"prepare.queue_time_us":535394,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":124,"raft.queue_time_us":174084,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":12699,"spinlock_wait_cycles":3036416,"thread_start_us":438982,"threads_started":3,"wal-append.queue_time_us":270210}
I20260709 13:11:46.066433 28058 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.065641 28222 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.081837 28210 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:46.082060 28207 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:11:46.077198 28208 consensus_meta.cc:287] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 13:11:46.072932 28341 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:11:46.084177 28208 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.073108 28349 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:46.076635 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.066504 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:46.091495 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.098119 22706 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:46.106652 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.115857 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.118379 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.050575 27970 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.158660 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.159471 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.164019 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:46.167281 22706 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:46.171289 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.171850 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.171288 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.179690 22701 raft_consensus.cc:2410] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:11:46.180812 22705 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:46.181236 22705 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:46.181077 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 139383us
I20260709 13:11:46.173390 22631 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:46.182503 22629 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:46.183164 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.183671 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.183882 22630 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:46.186235 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.188747 22633 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:46.184185 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.192272 22630 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 19. Current term is 20. Ops: []
I20260709 13:11:46.072070 28117 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.036773 28119 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 13:11:46.210757 22629 raft_consensus.cc:1240] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 20. Current term is 21. Ops: []
I20260709 13:11:46.158704 28310 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.079010 28344 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:46.169925 28058 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:46.160646 28343 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.170938 28315 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:46.242868 23415 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 1041) took 2434 ms. Trace:
W20260709 13:11:46.248229 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64099.8us
I20260709 13:11:46.249225 28353 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:46.258257 23415 rpcz_store.cc:276] 0709 13:11:43.779133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.780055 (+   922us) service_pool.cc:224] Handling call
0709 13:11:43.780391 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:44.013776 (+233385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:44.013830 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:44.013900 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:44.013964 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.213451 (+2199487us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54005163d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:46.213541 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.213735 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535959424}
I20260709 13:11:46.191576 28208 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.250267 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.191977 28341 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:46.258802 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.266710 22626 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:11:46.171514 28279 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 13:11:46.267362 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.270814 22626 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:11:46.270659 22706 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:46.265393 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.212337 28352 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:46.244212 28207 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.275269 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:46.278414 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:46.275214 28179 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:46.282130 28179 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:46.282591 28179 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.283016 28179 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:46.284021 28321 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 1.159s	user 0.004s	sys 0.000s
I20260709 13:11:46.285763 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3279) took 2875 ms. Trace:
I20260709 13:11:46.261461 28343 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.286221 22709 rpcz_store.cc:276] 0709 13:11:43.413741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:43.413888 (+   147us) service_pool.cc:224] Handling call
0709 13:11:43.414201 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:43.501055 (+ 86854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:43.501111 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:44.868003 (+1366892us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b2c0006cb88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:44.868494 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:11:44.987268 (+118774us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:11:44.987343 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:44.987423 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:46.283254 (+1295831us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:46.285455 (+  2201us) raft_consensus.cc:1685] finished
0709 13:11:46.285483 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:46.285683 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":149701,"raft.run_cpu_time_us":53,"raft.run_wall_time_us":54,"spinlock_wait_cycles":19223168,"thread_start_us":194844,"threads_started":2,"wal-append.queue_time_us":160662}
I20260709 13:11:46.289197 28343 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.289820 28343 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:46.278390 28045 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:46.291404 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 840) took 1012 ms. Trace:
I20260709 13:11:46.208879 28354 consensus_queue.cc:1059] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:11:46.291852 23421 rpcz_store.cc:276] 0709 13:11:45.279036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:45.279214 (+   178us) service_pool.cc:224] Handling call
0709 13:11:45.513614 (+234400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.514391 (+   777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.291153 (+776762us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54006b0150. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:46.291332 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538273024}
I20260709 13:11:46.255846 28310 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.290462 28343 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.293949 28225 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:11:46.268901 28339 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:11:46.297422 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.280320 28316 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:11:46.295622 28345 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.304152 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:46.297842 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.296296 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.308676 22629 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:46.308431 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:46.301481 23411 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:11:46.311105 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:46.299778 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.316035 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.318329 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:46.266077 28341 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.322357 28247 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.294567 28225 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:46.258492 28058 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.312346 27937 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:46.333356 27937 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:46.324874 28247 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.296787 27946 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:11:46.295367 28310 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.345041 23417 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:11:46.345587 23417 pending_rounds.cc:85] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 9
I20260709 13:11:46.345801 23417 pending_rounds.cc:107] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:11:46.346037 23417 raft_consensus.cc:2889] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:46.346769 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.346880 28247 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.304829 28353 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:11:46.347612 28194 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.351078 28179 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.352334 28058 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.353229 28179 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.297791 28259 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:46.355099 28259 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.326138 28341 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:46.362525 22782 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 849) took 1487 ms. Trace:
I20260709 13:11:46.363106 22782 rpcz_store.cc:276] 0709 13:11:44.875335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:44.875477 (+   142us) service_pool.cc:224] Handling call
0709 13:11:45.530836 (+655359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:45.531559 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:46.362272 (+830713us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b5400186fd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:46.362445 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1419177856}
I20260709 13:11:46.363173 28352 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.368083 28205 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.373399 28088 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:46.377671 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.280123 28207 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.388489 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:46.389402 23421 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:11:46.398005 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.296815 28346 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:11:46.395407 28222 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:46.418393 28222 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:46.419004 28222 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:46.298400 28315 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.424201 28225 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.427130 28045 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:46.427889 28045 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:46.428622 23421 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:46.414672 28343 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.429312 28045 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.432529 28045 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:46.396544 28207 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.437206 28205 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.438517 28088 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.427107 28341 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.445679 28341 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.439786 28344 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:46.492169 22628 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 20: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:46.502522 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:46.504951 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:11:46.508308 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:46.511648 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 233149us
W20260709 13:11:46.525103 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 205605us
I20260709 13:11:46.527042 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.529686 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:46.529539 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:46.530968 22701 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:46.530968 22709 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:46.539443 28066 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:46.539151 27624 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:46.540504 28066 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:46.543592 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.543124 28066 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.550104 28179 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:46.551180 28179 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:46.553011 28179 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.556278 28227 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:46.452419 28344 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.559219 28056 consensus_queue.cc:1059] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:46.560501 28343 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:46.567466 28058 raft_consensus.cc:3055] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:46.568179 28058 raft_consensus.cc:740] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:46.568830 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.557961 28341 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:11:46.589781 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:46.569934 28058 consensus_queue.cc:260] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.599071 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:46.612843 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.612944 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.614835 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.616426 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.607635 28088 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:46.567495 28343 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.624032 22701 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:46.625231 22704 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:46.620482 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.636934 22703 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:46.620918 28088 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.637637 22705 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.638641 28225 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:46.639921 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.639911 28225 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:46.650261 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 212990us
I20260709 13:11:46.652143 28069 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:46.657935 28225 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.641631 28088 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.644081 28367 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.645915 22705 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:46.646800 28343 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.609043 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.659484 28058 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:46.660228 22706 raft_consensus.cc:2410] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:11:46.667430 28194 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:46.664343 28370 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:11:46.669143 28259 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.691475 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:46.691185 28367 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.694965 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.697880 28056 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:11:46.675509 28069 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.715198 22632 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:46.715373 28069 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.641185 22703 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:46.703292 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.704677 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:46.718888 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.719472 22777 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:46.719921 22703 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:46.705894 28350 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:46.706851 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.708743 28056 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:46.698040 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 125416us
I20260709 13:11:46.723269 28056 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.726567 28088 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:46.727541 28088 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.730434 22782 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:46.730499 28279 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.723891 28259 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:11:46.739936 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.734421 22630 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:46.715405 22781 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:46.742365 22704 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8.
I20260709 13:11:46.724974 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:46.738236 22709 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:46.741071 28279 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.749928 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.818291 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:46.775884 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:46.769969 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45281.8us
I20260709 13:11:46.810261 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.749754 22703 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:11:46.810061 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.755900 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.759375 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:46.837800 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69854.6us
I20260709 13:11:46.793583 22708 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:46.811311 22635 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:46.783020 28077 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:46.756460 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:46.848208 28249 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.848980 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.849150 28249 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.850219 22631 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:46.851702 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.855448 27937 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.857069 27937 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:46.853518 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.860385 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.861629 22631 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:46.864379 28279 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.865322 22629 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:11:46.757447 28069 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.746780 28056 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:46.778903 28258 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:46.873211 22631 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:46.876427 22626 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:46.877139 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58257.4us
I20260709 13:11:46.772665 28257 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.759009 28367 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:46.892020 28316 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:46.892872 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.893105 28316 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:46.895030 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:46.874665 28258 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:46.922892 22704 raft_consensus.cc:1240] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:46.932992 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:46.934810 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:46.943056 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51890.3us
I20260709 13:11:46.936254 28185 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:46.936923 28365 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:46.937450 22705 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:11:46.939095 28316 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:46.954197 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:46.950120 22705 pending_rounds.cc:85] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:11:46.968237 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:46.970993 22779 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:46.973596 22782 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
W20260709 13:11:46.975117 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88005.6us
I20260709 13:11:46.976644 28256 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:46.977301 28069 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:46.978264 23419 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:46.982033 22781 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:11:46.982791 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:46.983429 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:46.990899 28272 consensus_queue.cc:1059] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:46.994196 28205 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:46.995764 28088 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:46.996407 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:46.996819 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:46.997556 22709 raft_consensus.cc:2410] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:11:46.998915 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:46.999378 27942 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.000402 22701 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:47.000211 27942 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:46.960789 28257 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.004676 22709 pending_rounds.cc:85] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:47.005097 22709 pending_rounds.cc:107] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:11:46.958041 22705 pending_rounds.cc:107] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:11:47.006912 22705 raft_consensus.cc:2889] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:47.005391 22709 raft_consensus.cc:2889] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:47.009151 22777 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:11:47.009526 22778 raft_consensus.cc:2368] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:11:47.017254 28029 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:11:47.021533 28354 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:11:47.022742 28205 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:47.023528 28258 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.025103 28258 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:47.005678 28257 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.027209 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:47.027987 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.017436 28376 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.157s	user 0.003s	sys 0.000s
I20260709 13:11:46.954406 28365 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:47.032104 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:47.032352 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:47.033241 28365 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.034646 22706 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:47.036337 28249 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:47.036751 22525 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 146) took 27214 ms (client timeout 15000 ms). Trace:
I20260709 13:11:47.040931 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:47.041853 22701 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:47.046550 28356 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:47.048950 28272 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:47.049510 28272 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:47.049774 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.050992 28272 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.052107 28147 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:47.052654 28272 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:47.053905 28147 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:11:47.048065 28356 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:11:47.062065 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.065692 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.068454 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.093955 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.112838 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:47.117291 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59474.4us
I20260709 13:11:47.115566 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.123750 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.125581 22781 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:47.127260 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.127422 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.128118 22781 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:47.128240 22776 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:47.115293 28088 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.129909 22776 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:11:47.125599 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.139371 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.152395 22781 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
W20260709 13:11:47.134876 28222 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.016s
I20260709 13:11:47.147746 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:47.139575 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.131359 28256 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.165074 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:47.144156 28340 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:47.135783 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.135469 28365 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:47.149829 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.149595 28179 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.172254 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.153787 22778 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:11:47.131543 28077 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.144603 28354 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.138604 28088 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.146603 28045 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.174789 22777 raft_consensus.cc:2410] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:11:47.176191 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:47.192417 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.193468 22776 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:47.194909 23419 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:47.197639 22775 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:47.199581 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.200387 22781 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:47.191538 28354 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.201838 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:47.152000 28147 consensus_queue.cc:1243] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:11:47.207755 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.170104 28259 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:11:47.214488 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.188443 28069 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.154059 27942 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.187681 28249 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.231693 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.238209 22784 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:11:47.238756 28209 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:47.239292 28344 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:47.239877 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:47.243974 22778 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:47.238767 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 91184.7us
I20260709 13:11:47.246323 22781 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:11:47.211017 28256 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.247525 22634 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:47.208539 28045 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.255292 22779 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:47.258726 28340 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.260565 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:47.266270 22777 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:11:47.189320 28222 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.258174 28267 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:47.262384 28088 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.265936 28209 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.279898 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.290825 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.218881 28179 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.219766 28077 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.289899 22627 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
W20260709 13:11:47.294474 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 82896.6us
I20260709 13:11:47.302052 28008 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.297049 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:47.287928 28267 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:11:47.316598 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56157.2us
I20260709 13:11:47.232481 28069 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.302928 28008 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.296659 28209 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:47.322324 22781 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:11:47.323873 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.307550 28316 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:47.323092 23385 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:47.336798 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.282202 28249 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.324615 28225 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:47.327190 28056 raft_consensus.cc:2764] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:11:47.398383 28384 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:11:47.398922 22782 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:47.400336 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.401262 28256 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.401895 28296 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:11:47.355149 28367 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:47.360394 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 54261.3us
I20260709 13:11:47.393576 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.420008 23418 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:47.421325 28267 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.421490 22529 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:47.422955 28267 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:47.392906 28354 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.426222 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.427281 28045 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.429072 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.420032 27624 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:47.430663 27624 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:47.431797 27624 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.434677 28008 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:47.437326 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:47.441648 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2231) took 1333 ms. Trace:
I20260709 13:11:47.442263 22627 rpcz_store.cc:276] 0709 13:11:46.108294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.108489 (+   195us) service_pool.cc:224] Handling call
0709 13:11:46.159066 (+ 50577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.277181 (+1118115us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b5400320120. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:11:47.322734 (+ 45553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:47.441274 (+118540us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540022db54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:47.441547 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2339910272}
W20260709 13:11:47.037331 22525 rpcz_store.cc:269] 0709 13:11:19.822368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:19.822744 (+   376us) service_pool.cc:224] Handling call
0709 13:11:47.036422 (+27213678us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:38.707100 (+     0us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a91216c
0709 13:11:39.014493 (+307393us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:39.015346 (+   853us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:39.015422 (+    76us) write_op.cc:435] Acquired schema lock
0709 13:11:39.015446 (+    24us) tablet.cc:662] Decoding operations
0709 13:11:39.016751 (+  1305us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:39.016891 (+   140us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:39.016928 (+    37us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:11:39.161368 (+144440us) tablet.cc:701] Row locks acquired
0709 13:11:39.203704 (+ 42336us) write_op.cc:260] PREPARE: finished
0709 13:11:39.283028 (+ 79324us) write_op.cc:270] Start()
0709 13:11:39.283367 (+   339us) write_op.cc:276] Timestamp: P: 1783602699282899 usec, L: 0
0709 13:11:39.283427 (+    60us) op_driver.cc:348] REPLICATION: starting
0709 13:11:39.284645 (+  1218us) log.cc:844] Serialized 4016 byte log entry
0709 13:11:40.360948 (+1076303us) op_driver.cc:464] REPLICATION: finished
0709 13:11:41.267337 (+906389us) write_op.cc:301] APPLY: starting
0709 13:11:41.280774 (+ 13437us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:41.751055 (+470281us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:41.751115 (+    60us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:42.132241 (+381126us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:42.132593 (+   352us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:11:42.132662 (+    69us) write_op.cc:312] APPLY: finished
0709 13:11:42.133099 (+   437us) log.cc:844] Serialized 97 byte log entry
0709 13:11:43.103491 (+970392us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.112372 (+  8881us) write_op.cc:454] Released schema lock
0709 13:11:43.123938 (+ 11566us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:43.249082 (+125144us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:43.254493 (+  5411us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":107440,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":893171,"apply.run_cpu_time_us":90315,"apply.run_wall_time_us":1951227,"cfile_cache_hit":42,"cfile_cache_hit_bytes":46007,"num_ops":10,"prepare.queue_time_us":1460241,"prepare.run_cpu_time_us":32141,"prepare.run_wall_time_us":550623,"raft.queue_time_us":613140,"raft.run_cpu_time_us":42846,"raft.run_wall_time_us":809048,"replication_time_us":1077232,"spinlock_wait_cycles":25822592,"thread_start_us":3708357,"threads_started":5,"wal-append.queue_time_us":1268412,"wal-append.run_cpu_time_us":33315,"wal-append.run_wall_time_us":2488244}]]}
W20260709 13:11:47.427935 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50588.5us
I20260709 13:11:47.455955 22777 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:47.458465 22778 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:11:47.460418 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 145820us
I20260709 13:11:47.463943 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.465555 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:47.466637 22779 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:47.468839 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.482964 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:47.488216 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:47.489050 22778 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:47.473382 28367 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:47.492535 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 133959us
I20260709 13:11:47.495426 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.476583 28179 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.491092 28350 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:47.521942 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.498867 28257 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:11:47.528470 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63063.8us
I20260709 13:11:47.510376 28056 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:47.499208 28077 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.536814 28008 raft_consensus.cc:2764] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:11:47.542835 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 90461.9us
I20260709 13:11:47.544615 28249 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.551113 22529 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 11, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 11 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:47.553606 28069 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:47.563158 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:47.564231 28179 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:47.566845 23414 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:11:47.567132 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 417404us
I20260709 13:11:47.574252 22707 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:47.579495 28340 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:47.582158 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:47.581066 22776 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:11:47.584170 22775 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 7.
I20260709 13:11:47.585589 22702 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
W20260709 13:11:47.530814 28258 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.505s	user 0.016s	sys 0.002s
I20260709 13:11:47.596659 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.596168 22784 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 6. Current term is 7. Ops: []
I20260709 13:11:47.534860 28222 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.515359 28344 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.079s
W20260709 13:11:47.605906 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 270954us
W20260709 13:11:47.636431 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.640376 22781 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:47.630890 28316 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:47.648769 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.849s (SENT) [suppressed 18 similar messages]
W20260709 13:11:47.646817 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.609794 28258 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:47.599731 28209 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.651510 22628 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:47.652566 28209 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:47.604529 28194 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:11:47.651180 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.650214 28316 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.622138 28029 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.655165 28029 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.657829 28029 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:47.666860 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.673555 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:47.647470 28392 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.675145 28392 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.676577 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.677563 28392 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:47.684976 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.698693 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:47.700357 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107603us
I20260709 13:11:47.693702 27624 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:47.704428 22529 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:47.707504 22529 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 11, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:47.709463 22529 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 10, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:47.743871 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:47.748433 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 265535us
I20260709 13:11:47.711179 28392 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:47.764634 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:47.766180 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 136132us
I20260709 13:11:47.767311 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:47.796099 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.803071 28008 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:47.803612 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:47.804360 22702 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:47.804661 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.807898 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.810726 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:47.804003 28008 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.811760 28392 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:47.817906 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.832764 28029 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.833776 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:47.833535 28029 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.835191 22780 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:11:47.836022 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 134734us
I20260709 13:11:47.837853 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.840101 27942 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:47.841142 28296 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:47.841614 28296 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:47.830996 28340 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:47.827291 28396 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:47.842586 28296 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:47.855167 28389 log.cc:927] Time spent T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 13:11:47.844085 28340 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:47.844244 28392 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.858531 28296 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:47.859508 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.860013 28392 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:47.860651 28257 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:11:47.861260 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.862169 28257 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:47.865504 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:47.866717 28088 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:47.866338 28340 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:47.867560 28340 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:47.870429 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:47.871234 23421 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:11:47.753872 28390 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 13:11:47.872618 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:47.844393 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:47.876606 28365 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.691s	user 0.011s	sys 0.004s
I20260709 13:11:47.872675 28045 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:47.825177 28194 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:47.853957 27970 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:47.895290 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:47.847891 28396 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:47.896166 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 154367us
I20260709 13:11:47.876888 28029 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:47.897485 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.852418 27942 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:11:47.893750 28272 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.892709 22529 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 3 to 5, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:47.920246 28008 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:47.852249 28258 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:11:47.921849 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 154647us
I20260709 13:11:47.896523 28077 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:11:47.901463 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 252750us
I20260709 13:11:47.942205 28356 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:11:47.942848 28356 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:47.943921 28356 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:47.945356 28356 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:11:47.948319 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:47.935724 27937 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:47.932829 28077 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.951968 28077 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:47.905705 28396 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:47.950470 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:47.961411 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:47.959749 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:47.953495 28267 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.006s
W20260709 13:11:47.980474 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47050.8us
I20260709 13:11:47.960144 28390 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:47.991155 23416 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:47.921056 28194 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.001263 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:48.004478 22750 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.579s (SENT)
I20260709 13:11:48.004921 28194 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.006845 22529 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.008728 28391 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.009402 28391 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.011059 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:48.012270 28391 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:48.012957 28257 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:48.013543 28257 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:47.983081 28045 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:11:48.015061 28045 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:48.015578 28257 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.016043 28045 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.018404 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.009207 22529 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 15, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:47.964198 28272 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.041975 28393 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.042888 28393 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:48.048521 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 98140.8us
I20260709 13:11:48.057525 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.058781 23416 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:48.079303 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:47.999975 28390 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:48.100392 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 45394.5us
W20260709 13:11:48.106201 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.139889 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.147071 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 261088us
W20260709 13:11:48.150051 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.035682 28259 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:48.124814 28347 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:11:48.164224 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141304us
I20260709 13:11:48.167850 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:48.170099 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:48.171962 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 72374.6us
I20260709 13:11:48.169117 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:48.182708 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:48.180867 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:48.183800 22631 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:48.186913 22632 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:11:48.192279 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:48.185454 22529 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.207952 28088 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:48.208230 28245 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:48.208815 28245 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.209357 28245 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:48.208760 28088 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.168483 28347 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:48.208856 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56269.3us
I20260709 13:11:48.177343 28194 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.185757 28029 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:48.201156 28249 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:11:48.220432 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:48.196450 28365 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:11:48.223853 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.246552 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45587.9us
I20260709 13:11:48.213466 28008 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.264997 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44685.7us
W20260709 13:11:48.268550 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 76665.3us
I20260709 13:11:48.278103 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:48.283018 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:48.287425 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 91215.2us
W20260709 13:11:48.293548 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.267791 28225 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:48.274857 28179 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:48.297587 22529 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:48.320263 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.328383 22703 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:11:48.290198 28272 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.268991 28194 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.305475 28392 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:48.336181 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55683.5us
I20260709 13:11:48.278944 28397 consensus_queue.cc:1059] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:11:48.315246 28179 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.284879 28365 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:11:48.284746 28008 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.307169 28354 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:11:48.360339 28423 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:11:48.333786 28391 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:48.316504 28225 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.374073 22780 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
W20260709 13:11:48.372344 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 89457.1us
I20260709 13:11:48.372998 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:48.270983 28390 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.353864 28179 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:48.380856 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:48.385396 22778 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:48.386468 22779 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:48.387730 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:48.376907 22780 pending_rounds.cc:85] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 9
I20260709 13:11:48.388656 22780 pending_rounds.cc:107] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:11:48.361227 28249 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.388938 22780 raft_consensus.cc:2889] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:48.391754 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:48.367089 28391 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:48.381484 28296 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.004s
I20260709 13:11:48.401916 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:48.361032 28077 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.409451 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:48.410341 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:48.411576 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.415771 22775 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:48.416000 23416 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:48.417615 22783 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:48.402379 28391 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:48.400432 28392 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.437069 28425 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 13:11:48.443403 28420 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.158s	user 0.001s	sys 0.001s
W20260709 13:11:48.443743 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56107.6us
I20260709 13:11:48.443920 28258 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:11:48.445437 28340 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.005s
I20260709 13:11:48.446249 22709 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:48.446636 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.447196 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.403020 28249 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:48.449080 28422 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:11:48.450829 28249 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:48.451412 28392 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.453639 28392 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:48.459017 22707 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:11:48.460402 22781 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:48.408557 28045 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:48.433386 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:48.437069 28419 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:11:48.399611 28354 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.481403 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.489523 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.483438 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.481030 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.429037 28257 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:48.482134 22708 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:48.490669 22705 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:48.490585 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.498430 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.500073 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.501202 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.502343 22702 raft_consensus.cc:2410] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:11:48.514796 28225 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.516265 22780 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:11:48.517402 28354 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.518787 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.492617 27937 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.542544 28179 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.544883 22703 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 9.
W20260709 13:11:48.545229 28420 log.cc:927] Time spent T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:48.546700 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.546916 28179 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:48.547721 22706 raft_consensus.cc:2410] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:11:48.520162 28397 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:48.554669 28088 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.517437 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.526432 22775 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:48.533679 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.557312 28008 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.538115 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.539425 22710 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:48.558856 23419 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:11:48.540086 28077 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.540414 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.541715 28194 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:48.541435 28279 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:48.571153 22707 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:48.542505 28431 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.559062 22710 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:48.570569 22701 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:48.570647 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.574914 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:48.576709 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:48.577536 23419 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:48.581152 22747 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:48.599201 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 174589us
W20260709 13:11:48.606880 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 304526us
I20260709 13:11:48.619616 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:48.558832 28397 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:48.633894 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52893.4us
I20260709 13:11:48.544024 27937 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.650885 22701 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:11:48.655689 22706 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:11:48.656049 22706 pending_rounds.cc:85] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:11:48.656271 22706 pending_rounds.cc:107] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:11:48.656494 22706 raft_consensus.cc:2889] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:11:48.615549 28425 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.128s	user 0.005s	sys 0.000s
W20260709 13:11:48.660547 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.661317 22628 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:48.624274 22710 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:48.662688 28438 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.105s	user 0.002s	sys 0.002s
I20260709 13:11:48.665870 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:48.652272 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:48.668849 23419 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:11:48.671862 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.674984 28434 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:48.675645 28434 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:48.676126 28434 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:48.688251 22710 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:48.621148 27942 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:11:48.650255 28397 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:48.705075 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.717307 22779 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:48.626518 28431 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:48.719705 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 286441us
I20260709 13:11:48.758217 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.759403 22703 raft_consensus.cc:2393] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:48.722245 28444 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:11:48.775111 27937 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:48.795171 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 184909us
W20260709 13:11:48.766376 28396 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.786s	user 0.013s	sys 0.002s
W20260709 13:11:48.810242 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47030.1us
I20260709 13:11:48.814997 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.872032 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:48.873106 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 152439us
I20260709 13:11:48.874504 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.880347 22706 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:48.882014 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:48.887764 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:48.887667 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.888952 22628 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:11:48.837004 28344 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:48.898507 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.894155 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:48.900676 28434 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:48.912654 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.928s (SENT)
I20260709 13:11:48.913767 23411 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 1075) took 1023 ms. Trace:
I20260709 13:11:48.925696 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:11:48.930183 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.932910 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.828528 28396 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.942970 22710 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:11:48.947077 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 132467us
I20260709 13:11:48.931684 28222 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:48.907749 28088 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:48.954066 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.957890 22707 raft_consensus.cc:2393] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:48.914207 23411 rpcz_store.cc:276] 0709 13:11:47.846222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:47.870742 (+ 24520us) service_pool.cc:224] Handling call
0709 13:11:47.871131 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:47.871418 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:48.010490 (+139072us) log.cc:844] Serialized 14 byte log entry
0709 13:11:48.868703 (+858213us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b18002d9be0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287
0709 13:11:48.868887 (+   184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:11:48.868981 (+    94us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:48.869057 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.869178 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:48.869405 (+   227us) inbound_call.cc:177] Queueing success response
0709 13:11:48.910144 (+ 40739us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b18002d9be0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
Metrics: {"prepare.queue_time_us":127580,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":89367,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":7611,"spinlock_wait_cycles":21101696,"thread_start_us":437814,"threads_started":3,"wal-append.queue_time_us":336134}
I20260709 13:11:48.948969 28069 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.966228 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:48.896808 28344 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:48.957294 28222 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.970894 28069 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:48.990934 28222 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:48.991402 28069 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:48.976995 28453 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:11:48.979661 28391 consensus_meta.cc:287] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.000s
I20260709 13:11:48.961426 28445 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:48.985988 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:48.987073 28245 consensus_meta.cc:287] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.774s	user 0.007s	sys 0.008s
I20260709 13:11:48.988394 28396 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:48.999306 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.007972 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:48.994275 28344 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:49.016371 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 50860.6us
I20260709 13:11:48.964946 28088 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.012622 28390 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.023741 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.024873 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.009174 28391 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.020831 28431 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:49.014586 28245 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.036190 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.016832 28445 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.026533 28459 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.041610 22778 raft_consensus.cc:2393] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:49.024919 28390 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.055051 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:49.058596 22708 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:49.069924 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:49.045796 28459 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.085000 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.083060 28045 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.087734 28045 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.089654 28088 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:49.088232 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:49.092283 22703 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:49.099924 28045 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.103721 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:49.104435 22706 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:49.113528 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:49.114545 22782 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:49.100481 28390 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.113674 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.101727 28433 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.117736 22783 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:49.120113 22703 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:49.126992 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:49.131138 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67636.5us
I20260709 13:11:49.129391 28433 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.142056 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.108989 28077 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:49.145203 28077 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:49.155939 22750 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 2.032s (SENT)
I20260709 13:11:49.164891 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.169900 22631 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
W20260709 13:11:49.121553 28457 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:11:49.172950 22631 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:49.162093 28445 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.174788 22635 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:11:49.154196 22782 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:49.159157 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.137897 28222 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.180557 22707 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:49.167903 28391 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:49.183066 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.205772 22635 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:11:49.184633 28459 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.184723 28088 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:49.185427 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.186233 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.186918 28390 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.189635 22783 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:49.189339 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.191259 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.216434 28390 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.217507 22633 raft_consensus.cc:2410] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:11:49.217523 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.218649 22783 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:49.219913 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.220896 22633 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:11:49.221419 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.222057 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:49.169579 28434 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.223023 22629 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:49.224710 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.225610 22775 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
W20260709 13:11:49.195457 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64706.9us
I20260709 13:11:49.196415 27624 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:11:49.200191 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68255.4us
I20260709 13:11:49.192492 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.244771 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.247162 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.248234 22629 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:49.222350 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:11:49.253511 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.243270 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.258599 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:49.231366 28397 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.000s
W20260709 13:11:49.264951 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.272564 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3384) took 1242 ms. Trace:
I20260709 13:11:49.275511 22706 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:49.256443 22635 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:11:49.273169 22709 rpcz_store.cc:276] 0709 13:11:48.015339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.018102 (+  2763us) service_pool.cc:224] Handling call
0709 13:11:49.257325 (+1239223us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000e5e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:49.257454 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":982165376}
I20260709 13:11:49.277405 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.278354 22706 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:49.252774 28476 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.281576 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.299162 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.299865 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:49.280489 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0919654529998297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
I20260709 13:11:49.265616 28431 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:49.213655 28445 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.061s
W20260709 13:11:49.301537 22675 connection.cc:256] RPC call timeout handler was delayed by -1.0586654529997759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:11:49.300557 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:49.314018 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:49.314352 22783 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:49.286127 28045 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:49.319500 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.272023 28391 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:11:49.302899 28447 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.023s	sys 0.040s
I20260709 13:11:49.318089 28179 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:49.324753 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:49.325762 22628 raft_consensus.cc:2393] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:11:49.328025 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44615.3us
I20260709 13:11:49.303395 28432 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:49.328049 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.329550 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.322697 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:49.323836 28447 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.024s	sys 0.041s
I20260709 13:11:49.333941 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.323468 28045 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.337611 28222 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.282076 28397 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.338939 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:49.338827 28347 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.340279 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.317714 28069 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:49.349298 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.286907 28316 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:11:49.355257 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:49.390873 22632 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:49.358497 28179 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:49.363046 22747 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:49.373808 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:49.382042 22703 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:49.392459 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.379083 22633 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
W20260709 13:11:49.394035 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53850us
W20260709 13:11:49.398272 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.385527 22781 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:49.391104 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52288.2us
I20260709 13:11:49.401808 22783 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:11:49.405165 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 105127us
I20260709 13:11:49.408906 23421 raft_consensus.cc:2368] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 21. Current term is 22.
I20260709 13:11:49.409857 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:49.279981 28476 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.412876 22706 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:11:49.412529 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.368556 28445 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.418519 23419 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:11:49.418566 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.419656 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:49.421885 22635 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:49.422911 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.408185 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.422945 22627 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:49.424587 23416 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
W20260709 13:11:49.431901 28466 log.cc:927] Time spent T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:11:49.424427 28445 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.434800 28472 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:49.435405 28472 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:49.435811 28472 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:49.436684 28445 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.437065 22747 leader_election.cc:400] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 21. Current term is 22.
I20260709 13:11:49.437420 22747 leader_election.cc:403] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:11:49.438719 28397 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.439167 28471 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:49.439863 28471 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.440433 28088 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:49.411950 28468 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:49.412264 28447 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:49.446170 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:49.423868 22781 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:49.455942 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.431628 28456 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:49.446826 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.453009 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.440517 28391 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.463572 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.461643 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.463701 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:49.460379 23416 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:49.473773 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 95985.9us
I20260709 13:11:49.477021 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.487200 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.444459 28069 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:49.502799 22675 leader_election.cc:400] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
W20260709 13:11:49.499444 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73647.5us
I20260709 13:11:49.444108 28045 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.451084 28468 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.454947 28222 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:49.507282 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:49.508095 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.462561 28397 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:11:49.510452 28397 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.510926 28397 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:49.521418 28397 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.524192 28471 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.525970 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:49.531627 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:49.532307 23421 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:11:49.535482 28390 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.536860 28258 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.537117 28452 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.537528 28258 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.537779 28452 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.540105 28258 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.540181 28452 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.480579 28456 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:49.542184 28456 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.542676 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:49.543448 23416 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:49.479158 28447 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.547585 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.508816 22779 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
W20260709 13:11:49.509613 28424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.007s	sys 0.006s
W20260709 13:11:49.537420 28430 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 1.046s	user 0.003s	sys 0.013s
I20260709 13:11:49.560037 22673 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:49.561498 22635 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:49.569820 22632 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
I20260709 13:11:49.575412 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.455228 28459 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:11:49.578500 23384 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:11:49.559908 28424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.007s	sys 0.006s
I20260709 13:11:49.595284 28476 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.584316 28475 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:49.584332 23414 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 858) took 1545 ms. Trace:
I20260709 13:11:49.573580 28045 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:11:49.481159 28391 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.563572 28222 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.591080 28077 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:49.590258 28427 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.680s	user 0.000s	sys 0.003s
I20260709 13:11:49.588138 28432 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:49.586861 28479 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.192s	user 0.002s	sys 0.000s
I20260709 13:11:49.560123 28397 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.508559 28468 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:49.573482 28205 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:49.577065 28069 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:11:49.620632 28088 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 21. Current term is 22.
I20260709 13:11:49.679672 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.680770 22709 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:49.624191 23421 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:11:49.612095 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.627167 28433 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:49.667166 28475 raft_consensus.cc:697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:49.692198 28391 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.691828 28475 consensus_queue.cc:237] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.628772 28486 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:49.693332 28486 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.693692 28222 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.629830 28467 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:11:49.584409 28465 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.509s	user 0.002s	sys 0.006s
W20260709 13:11:49.695194 28474 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.446s	user 0.000s	sys 0.004s
W20260709 13:11:49.633399 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72221.8us
I20260709 13:11:49.697625 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.697860 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.681212 28069 consensus_queue.cc:1243] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:11:49.698659 22705 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:49.632169 28447 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.644930 28485 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.700317 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.700433 28485 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.702272 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:49.648165 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 145642us
I20260709 13:11:49.668804 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:49.659868 23414 rpcz_store.cc:276] 0709 13:11:48.038916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.039120 (+   204us) service_pool.cc:224] Handling call
0709 13:11:48.081360 (+ 42240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.151892 (+ 70532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:48.151937 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:48.190769 (+ 38832us) log.cc:844] Serialized 26 byte log entry
0709 13:11:48.284912 (+ 94143us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:11:48.284970 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:48.285051 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.536526 (+1251475us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:49.583941 (+ 47415us) raft_consensus.cc:1685] finished
0709 13:11:49.583967 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.584231 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":795,"raft.queue_time_us":6145,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":7250,"spinlock_wait_cycles":45590400,"thread_start_us":88855,"threads_started":1,"wal-append.queue_time_us":193683}
I20260709 13:11:49.659842 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.678951 22782 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:11:49.705164 22701 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:49.693804 28486 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:49.703861 22709 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:49.710572 22704 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:49.679486 28029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.268s	user 0.016s	sys 0.051s
W20260709 13:11:49.712641 28029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.305s	user 0.017s	sys 0.051s
I20260709 13:11:49.715284 28467 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:49.717278 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:49.720568 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:49.721658 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.677136 28316 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.728096 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:49.700506 28069 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:49.743317 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:49.724845 28467 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:49.749445 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.752171 28485 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.754465 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:49.755141 28194 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:49.755028 28472 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.761529 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.828610 22781 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:49.731834 28316 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.750344 28433 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:49.738694 28045 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:11:49.771454 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49855.6us
I20260709 13:11:49.775499 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2763) took 1020 ms. Trace:
I20260709 13:11:49.778820 22784 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:49.783667 28467 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:49.748502 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:49.786988 28393 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:49.798925 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 70645.1us
I20260709 13:11:49.817914 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:49.832612 22702 rpcz_store.cc:276] 0709 13:11:48.755678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.755884 (+   206us) service_pool.cc:224] Handling call
0709 13:11:48.765157 (+  9273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:48.771172 (+  6015us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:11:48.818293 (+ 47121us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.360772 (+542479us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b180014c440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:11:49.477724 (+116952us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.676505 (+198781us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:49.722995 (+ 46490us) log.cc:844] Serialized 26 byte log entry
0709 13:11:49.732396 (+  9401us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:11:49.732802 (+   406us) log.cc:844] Serialized 14 byte log entry
0709 13:11:49.733587 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:49.740858 (+  7271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:49.775129 (+ 34271us) raft_consensus.cc:1685] finished
0709 13:11:49.775160 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:49.775415 (+   255us) inbound_call.cc:177] Queueing success response
0709 13:11:49.781653 (+  6238us) spinlock_profiling.cc:243] Waited 32.4 ms on lock 0x7b180014c440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
Metrics: {"mutex_wait_us":528,"prepare.queue_time_us":70316,"prepare.run_cpu_time_us":155,"prepare.run_wall_time_us":1718,"raft.queue_time_us":189127,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":33095,"spinlock_wait_cycles":133158272,"thread_start_us":288074,"threads_started":3,"wal-append.queue_time_us":538637}
I20260709 13:11:49.796561 28396 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:49.840651 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.836653 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:49.850284 22777 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:49.816658 28194 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.851262 22781 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:11:49.848774 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67021.5us
W20260709 13:11:49.852720 23385 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.603s (SENT)
I20260709 13:11:49.858500 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:49.862357 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113994us
I20260709 13:11:49.862926 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:49.853382 28194 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:49.871942 28472 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:49.873276 28497 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:11:49.875448 22778 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 3214) took 1509 ms (client timeout 1985 ms). Trace:
I20260709 13:11:49.876020 22776 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:49.853240 28045 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:49.861584 28498 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:49.860539 28467 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:49.870471 28316 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:49.875268 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:49.878264 22783 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:49.854437 28433 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:49.876062 22778 rpcz_store.cc:269] 0709 13:11:48.366042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:48.399318 (+ 33276us) service_pool.cc:224] Handling call
0709 13:11:49.875174 (+1475856us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b540033cf54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:49.875344 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1435561728}
W20260709 13:11:49.905205 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.690s (SENT)
I20260709 13:11:49.905973 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:49.897146 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:49.909423 28029 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.910148 28029 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:49.921432 28403 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.925007 28403 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:49.938308 28448 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 1.456s	user 0.014s	sys 0.020s
I20260709 13:11:49.907559 28390 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:11:49.911926 28069 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:11:49.968269 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:49.920492 28476 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:49.974829 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 220447us
I20260709 13:11:49.977212 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:49.978909 27970 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:49.979077 28506 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:11:49.983891 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86850.8us
I20260709 13:11:49.990355 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.978545 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:49.998543 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.999185 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.000429 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.000577 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.001524 22709 raft_consensus.cc:2376] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10. Re-sending same reply.
I20260709 13:11:50.008958 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.009075 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:50.011416 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 241428us
I20260709 13:11:50.010347 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:50.012408 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.013523 22701 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:11:50.013407 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.014762 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.015219 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.015424 22707 raft_consensus.cc:2376] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 8. Re-sending same reply.
I20260709 13:11:49.919186 28486 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.018564 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.018759 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.019693 22702 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:11:50.020155 22702 pending_rounds.cc:85] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:11:50.020402 22702 pending_rounds.cc:107] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:11:50.020653 22702 raft_consensus.cc:2889] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:50.022207 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:50.023042 22775 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:50.025116 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:50.027149 28502 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:11:49.995002 28396 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:49.970777 28045 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.028650 28396 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.032661 28468 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.033663 28245 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.033951 23420 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:11:49.981720 28069 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:50.035528 28069 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.036369 28468 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.037038 28194 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.042512 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.023042 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:49.985090 27970 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:49.996594 28506 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:49.974887 28029 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:49.995591 28393 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:50.047966 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:50.051108 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72649us
I20260709 13:11:50.052292 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.057170 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.060315 22708 raft_consensus.cc:1240] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:50.060840 22703 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:11:50.050952 27970 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:50.051672 28390 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:50.059464 28444 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:50.055368 28476 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:11:50.078152 22750 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.843s (SENT)
I20260709 13:11:50.055344 28393 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.070000 28347 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:50.056456 28506 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:50.068614 28469 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.103396 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.108525 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:50.111575 23416 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:11:50.112762 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44269.9us
I20260709 13:11:50.050735 28008 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:11:50.073158 28498 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:50.081533 28403 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.139729 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.361104 22704 raft_consensus.cc:2376] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15. Re-sending same reply.
I20260709 13:11:50.364063 28498 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.364662 22628 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:50.087610 28444 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.367092 28444 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:50.112534 28476 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.368887 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:50.104517 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
W20260709 13:11:50.145404 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.939s (SENT)
I20260709 13:11:50.359596 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:50.165175 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 112948us
I20260709 13:11:50.374111 23385 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.126524 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:50.359400 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 290798us
W20260709 13:11:50.380084 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.386276 28476 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.248129 28205 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:50.392107 28205 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.395632 28205 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.400060 28507 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:50.400928 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50386 (request call id 2204) took 1221 ms. Trace:
W20260709 13:11:50.197731 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 119601us
W20260709 13:11:50.401742 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.425495 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.426815 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.423890 22783 rpcz_store.cc:276] 0709 13:11:49.179334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:49.506101 (+326767us) service_pool.cc:224] Handling call
0709 13:11:49.878056 (+371955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:49.887285 (+  9229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:50.194633 (+307348us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b540033cf50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:50.400826 (+206193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854712576}
I20260709 13:11:50.419087 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.430244 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.402221 28511 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:50.214476 28194 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.416457 22704 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:11:50.425531 22750 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.773s (SENT)
I20260709 13:11:50.449661 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.419857 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:50.450173 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.410221 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.423210 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.454327 22708 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:50.454869 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.455042 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.455916 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.458195 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.459739 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:50.460281 22634 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:11:50.193396 28249 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.104511 28393 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:50.407812 28482 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:50.465481 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.467444 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.458990 28473 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:11:50.532588 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 387248us
I20260709 13:11:50.531635 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:50.520407 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.538874 28267 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:50.535264 28508 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:50.297050 28506 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.518898 28179 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:50.522758 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52688.7us
I20260709 13:11:50.472424 28463 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:11:50.526209 22705 pending_rounds.cc:85] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:11:50.126989 28469 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.471841 28511 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.600548 28469 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.600638 28511 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.523626 28249 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.603444 28249 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.603976 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:50.604925 22776 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:50.605549 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:50.552860 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127382us
I20260709 13:11:50.608309 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.609383 28456 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:50.610004 28456 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.612498 28456 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.551541 28512 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:11:50.360179 28390 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.614791 28029 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:50.615458 28029 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.618042 28029 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.618824 28506 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.619053 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:50.619895 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:50.623284 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:50.572434 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 107067us
I20260709 13:11:50.618302 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:50.552799 23420 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 19. Current term is 20. Ops: []
I20260709 13:11:50.590179 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:50.627264 22783 raft_consensus.cc:2410] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:11:50.576897 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.595395 28008 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:50.596597 22779 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:11:50.569000 28473 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:50.614331 28456 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:50.628298 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51531.7us
I20260709 13:11:50.647832 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:50.590690 22705 pending_rounds.cc:107] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 15.10
W20260709 13:11:50.639071 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54846.2us
I20260709 13:11:50.650403 22705 raft_consensus.cc:2889] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:50.654700 22702 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:50.649926 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.655730 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:50.644538 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.661741 22710 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:50.551945 28486 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.589843 28245 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:50.668767 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.660276 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:50.663631 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:50.520640 28507 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.634653 28205 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:50.678727 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:50.731813 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76200.9us
I20260709 13:11:50.732882 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:50.677402 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43618.1us
I20260709 13:11:50.682583 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.736851 28477 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 13:11:50.738817 28454 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.007s	sys 0.010s
W20260709 13:11:50.739285 28454 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.007s	sys 0.010s
W20260709 13:11:50.746192 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 89338.1us
W20260709 13:11:50.748096 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 65633.4us
W20260709 13:11:50.747750 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 66238us
I20260709 13:11:50.669811 28506 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:50.752566 28205 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:50.662797 28516 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:50.768580 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:50.775717 22632 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:11:50.673197 28249 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:50.675017 28257 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:50.787217 28507 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.730341 28403 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.789148 28403 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.792115 28403 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:50.662350 28008 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:50.793843 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46762.7us
I20260709 13:11:50.673420 28498 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:11:50.795145 28008 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.676155 28473 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.796617 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.734284 28512 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:50.761821 28205 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.800300 28449 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:50.787331 28257 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.749821 28475 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:50.802593 27624 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.803287 27624 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.804697 28257 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:50.805251 27624 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.798882 28512 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.676882 28511 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 13:11:50.769335 28424 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:50.811088 28424 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:50.811575 28424 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:50.744081 28222 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.079s
I20260709 13:11:50.767074 28451 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 13:11:50.771370 28477 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:50.775903 28486 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:50.755324 28444 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.772912 28397 consensus_queue.cc:1059] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 13:11:50.820242 28397 raft_consensus.cc:3055] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:50.820672 28397 raft_consensus.cc:740] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:50.820809 28444 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:50.821684 28397 consensus_queue.cc:260] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:50.824040 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:50.825399 22628 raft_consensus.cc:2368] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15.
I20260709 13:11:50.753974 28456 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:50.770900 28506 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:50.826854 28456 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.827282 28456 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:50.827575 28506 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.828955 28506 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:50.802748 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52031.1us
I20260709 13:11:50.806409 28512 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:50.756873 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.840221 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:50.842723 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85906.1us
I20260709 13:11:50.869238 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:50.838409 28468 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:50.845281 28222 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:50.897292 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49561.5us
I20260709 13:11:50.839221 28396 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
W20260709 13:11:50.888675 27970 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.002s
W20260709 13:11:50.939500 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:50.874760 28463 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:50.888715 28245 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:50.881270 28069 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:50.901397 28272 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:11:50.898219 28486 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:50.998487 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 154862us
I20260709 13:11:50.999608 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.003815 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
W20260709 13:11:51.012471 28523 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:11:51.012760 22634 raft_consensus.cc:606] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.017081 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:50.903120 28468 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.019148 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:51.021672 28468 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.018584 28456 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:50.971081 28272 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.034435 28456 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:51.036722 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:51.037869 28393 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:11:51.041350 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:50.960291 27970 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.067970 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.074959 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.075961 22781 raft_consensus.cc:2393] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:51.070606 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.039464 28272 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:50.972221 28463 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.090341 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.080840 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:50.980479 28486 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.074492 28249 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.091331 22783 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 7.
I20260709 13:11:51.082739 22784 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:51.092005 22781 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:11:51.071218 28467 consensus_meta.cc:287] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.184s	user 0.008s	sys 0.008s
I20260709 13:11:51.049649 28222 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.101168 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.105118 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.070147 28403 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.105325 22627 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:11:51.112090 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.038312 28393 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.097556 28477 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:51.087240 28475 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:51.119603 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.131543 22775 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:51.120287 28403 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.136389 28403 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:51.129982 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 397183us
I20260709 13:11:51.152998 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.150221 22775 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:11:51.113083 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 434461us
W20260709 13:11:51.111383 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 466915us
I20260709 13:11:51.163792 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.072752 28482 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:11:51.154742 22781 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:51.159235 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:51.187254 28540 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.188198 28540 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.190088 28456 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.191609 28390 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:51.192739 28390 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:51.193150 28390 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:51.200426 28540 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.202062 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:51.165829 28524 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:11:51.179709 28403 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:51.229779 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44687.2us
I20260709 13:11:51.184201 28516 raft_consensus.cc:2764] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:11:51.177109 28485 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:51.228824 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 285614us
I20260709 13:11:51.169364 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:51.236296 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.235036 22675 leader_election.cc:400] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15.
I20260709 13:11:51.239542 22675 leader_election.cc:403] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:51.241999 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:51.243577 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:51.247666 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78438.6us
I20260709 13:11:51.232647 22749 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:51.265758 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.267912 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.194188 28390 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.284893 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.249498 28468 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:51.315330 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50565.6us
I20260709 13:11:51.281162 27937 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:51.318540 27937 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.320422 27970 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.321954 27937 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.322788 28447 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:51.324112 28447 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:51.324585 28447 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:51.325809 28447 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.328542 28029 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.329200 28029 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.331547 28029 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:51.334187 28347 consensus_meta.cc:287] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.223s	user 0.012s	sys 0.002s
I20260709 13:11:51.255513 28205 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:11:51.314797 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 67516.7us
I20260709 13:11:51.251835 28485 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.371846 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:51.373553 22635 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:51.358237 28390 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:51.316320 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:51.313537 28530 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:11:51.285722 28434 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:51.278184 28356 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:51.250133 28541 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:11:51.335551 28393 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.289003 28494 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.338553 28486 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.339291 28468 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:51.355919 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 281075us
W20260709 13:11:51.350808 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 275651us
I20260709 13:11:51.289354 28507 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.288106 28511 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.339216 28463 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.283583 28508 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.347282 28471 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:51.416280 28471 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.417577 28468 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.291745 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:51.418466 28045 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:51.375555 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:51.336419 28529 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 13:11:51.377453 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.378294 28209 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.418584 28471 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.422444 28540 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.408337 28356 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.419100 28045 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.423108 28540 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.414214 28434 raft_consensus.cc:697] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:51.425052 28434 consensus_queue.cc:237] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.426657 28516 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:51.433467 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 148684us
I20260709 13:11:51.387818 28477 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:51.436128 28477 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:51.366314 28397 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:51.401731 22632 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:11:51.402199 27624 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:11:51.402457 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86271.2us
I20260709 13:11:51.414294 28511 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.440686 28485 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.443234 28356 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.444546 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:51.405999 22633 raft_consensus.cc:1240] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:11:51.374580 28528 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:11:51.442939 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:51.447057 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 155469us
I20260709 13:11:51.447755 28485 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:51.447981 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:51.448746 28485 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:51.450726 28485 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.439399 28508 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.450110 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.456350 28508 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.456321 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.458033 28508 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.422276 28209 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:51.395960 28527 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.415s	user 0.003s	sys 0.000s
I20260709 13:11:51.368402 28205 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:51.408970 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:51.463627 28205 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:11:51.464251 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 55374.3us
I20260709 13:11:51.411170 28494 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.463140 22631 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:51.414042 28507 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.469240 28494 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:51.470085 28552 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:51.471165 28552 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:51.472194 28552 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.473219 28222 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:51.448068 22704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2796) took 1020 ms. Trace:
I20260709 13:11:51.458930 28069 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:51.459905 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.461305 28511 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.451931 28456 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.372241 28447 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:51.474035 28222 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.470043 28507 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.474926 22704 rpcz_store.cc:276] 0709 13:11:50.427765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.467824 (+ 40059us) service_pool.cc:224] Handling call
0709 13:11:50.468218 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:50.524125 (+ 55907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.524514 (+   389us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:50.527089 (+  2575us) log.cc:844] Serialized 26 byte log entry
0709 13:11:50.973142 (+446053us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:11:51.008719 (+ 35577us) log.cc:844] Serialized 14 byte log entry
0709 13:11:51.009021 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.009133 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:51.447824 (+438691us) raft_consensus.cc:1685] finished
0709 13:11:51.447850 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.447999 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114539,"prepare.queue_time_us":48671,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":54940,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":19440,"thread_start_us":377518,"threads_started":1,"wal-append.queue_time_us":580891}
I20260709 13:11:51.482079 28508 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.481086 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:51.477399 28494 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.126s
I20260709 13:11:51.604682 28508 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:11:51.483095 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.504426 22627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 2960) took 1073 ms. Trace:
I20260709 13:11:51.605147 28508 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:51.606065 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 123181us
I20260709 13:11:51.607028 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.606462 28508 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.608332 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.608601 28508 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:51.609189 28511 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.609936 28511 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:51.612356 28511 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:51.613178 28494 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:11:51.616242 28403 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:51.616948 28403 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.510493 28045 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:51.515559 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 266842us
I20260709 13:11:51.619349 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.619705 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.620610 28456 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:51.621204 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:51.523694 28512 consensus_meta.cc:287] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.679s	user 0.014s	sys 0.000s
I20260709 13:11:51.549108 28222 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.622813 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.557199 28209 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.559269 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2798) took 1130 ms. Trace:
I20260709 13:11:51.626968 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.625782 22705 rpcz_store.cc:276] 0709 13:11:50.428316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.471778 (+ 43462us) service_pool.cc:224] Handling call
0709 13:11:50.520899 (+ 49121us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:50.679088 (+158189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:50.679121 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:50.810191 (+131070us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b18004399c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:11:50.835614 (+ 25423us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b2c0006cb88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:50.841164 (+  5550us) log.cc:844] Serialized 52 byte log entry
0709 13:11:51.234291 (+393127us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:11:51.280018 (+ 45727us) log.cc:844] Serialized 14 byte log entry
0709 13:11:51.417671 (+137653us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b18004399c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:11:51.418105 (+   434us) log.cc:844] Serialized 14 byte log entry
0709 13:11:51.418299 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.558906 (+140607us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:51.558960 (+    54us) raft_consensus.cc:1685] finished
0709 13:11:51.558985 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:51.559146 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11616,"prepare.queue_time_us":27691,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":104,"raft.queue_time_us":208393,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":14592,"spinlock_wait_cycles":133626752,"thread_start_us":354465,"threads_started":1,"wal-append.queue_time_us":515443}
W20260709 13:11:51.482534 28424 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.670s	user 0.014s	sys 0.001s
W20260709 13:11:51.628222 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.439462 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.628541 28424 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.577023 28249 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.629741 22784 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:51.629908 28249 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.630434 28467 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:11:51.630877 28467 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.578889 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.579188 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.583307 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.631237 28467 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:51.622457 28512 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.585052 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.635866 28512 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:51.638777 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 360730us
I20260709 13:11:51.641709 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:51.642285 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.642673 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.643989 28424 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:51.659734 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.474571 28272 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.477799 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:51.478878 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:51.481359 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:51.480451 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:51.505709 28534 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.414s	user 0.002s	sys 0.000s
I20260709 13:11:51.555325 28540 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:51.620648 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.578012 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.525484 28541 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15.
I20260709 13:11:51.600724 22782 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
W20260709 13:11:51.708777 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 269403us
W20260709 13:11:51.710789 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 91529.9us
I20260709 13:11:51.723222 22777 raft_consensus.cc:1275] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:11:51.725209 28403 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:51.727310 22634 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:11:51.727087 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.727491 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:51.728029 22631 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:11:51.723033 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.730995 22707 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:51.734764 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.740536 22776 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:51.735713 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.733348 22628 raft_consensus.cc:2468] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:11:51.738675 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.737648 28249 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:51.739315 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.746066 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.746901 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.733934 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.748034 28209 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.748749 28209 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:51.753919 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.605692 22627 rpcz_store.cc:276] 0709 13:11:50.430663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:50.430826 (+   163us) service_pool.cc:224] Handling call
0709 13:11:50.431199 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:51.114278 (+683079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:51.503993 (+389715us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b5400349754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:51.504316 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549877120}
I20260709 13:11:51.756088 22777 raft_consensus.cc:2376] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22. Re-sending same reply.
I20260709 13:11:51.757999 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.762974 28205 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.764421 22783 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 8. Current term is 9. Ops: []
I20260709 13:11:51.768669 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:51.770156 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:51.772006 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:51.772861 22529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 236) took 37021 ms (client timeout 15000 ms). Trace:
I20260709 13:11:51.759459 22628 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:11:51.775025 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:51.775841 22708 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:51.776476 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.761413 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:51.777112 22630 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:51.778134 22634 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:51.779136 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:51.779841 22709 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:11:51.780220 22709 pending_rounds.cc:85] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:51.760345 22528 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.780486 22709 pending_rounds.cc:107] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:11:51.780778 22709 raft_consensus.cc:2889] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:51.781431 22528 catalog_manager.cc:5697] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 11 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.784225 22528 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.785601 28194 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:51.786885 22703 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:51.787573 28194 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:51.788571 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.788780 28194 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:51.773257 22529 rpcz_store.cc:269] 0709 13:11:14.751691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:14.756687 (+  4996us) service_pool.cc:224] Handling call
0709 13:11:51.772743 (+37016056us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:48.981058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:49.041070 (+ 60012us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:49.041138 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:11:49.041152 (+    14us) tablet.cc:662] Decoding operations
0709 13:11:49.131694 (+ 90542us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:49.131813 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:49.131861 (+    48us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:11:49.133380 (+  1519us) tablet.cc:701] Row locks acquired
0709 13:11:49.133438 (+    58us) write_op.cc:260] PREPARE: finished
0709 13:11:49.133727 (+   289us) write_op.cc:270] Start()
0709 13:11:49.134002 (+   275us) write_op.cc:276] Timestamp: P: 1783602709133629 usec, L: 0
0709 13:11:49.134053 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:11:49.197709 (+ 63656us) log.cc:844] Serialized 3989 byte log entry
0709 13:11:49.693715 (+496006us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:49.876182 (+182467us) op_driver.cc:464] REPLICATION: finished
0709 13:11:49.975316 (+ 99134us) write_op.cc:301] APPLY: starting
0709 13:11:49.975444 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:50.029173 (+ 53729us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:50.029215 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:50.620255 (+591040us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:50.620621 (+   366us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 13:11:50.620705 (+    84us) write_op.cc:312] APPLY: finished
0709 13:11:50.621172 (+   467us) log.cc:844] Serialized 97 byte log entry
0709 13:11:50.621403 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:50.621637 (+   234us) write_op.cc:454] Released schema lock
0709 13:11:50.621718 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:50.622271 (+   553us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:50.622305 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":641596,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":98780,"apply.run_cpu_time_us":28404,"apply.run_wall_time_us":646932,"cfile_cache_hit":42,"cfile_cache_hit_bytes":46007,"num_ops":10,"prepare.queue_time_us":283866,"prepare.run_cpu_time_us":13935,"prepare.run_wall_time_us":268752,"raft.queue_time_us":165483,"raft.run_cpu_time_us":23223,"raft.run_wall_time_us":59335,"replication_time_us":741878,"spinlock_wait_cycles":8648576,"thread_start_us":345694,"threads_started":4,"wal-append.queue_time_us":50653,"wal-append.run_cpu_time_us":52342,"wal-append.run_wall_time_us":2374649}]]}
I20260709 13:11:51.789450 22709 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:51.792403 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.799314 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.794171 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.800911 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.799536 22627 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:51.786978 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:51.795033 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.801855 28403 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.802371 28434 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.786419 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.805883 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:51.811004 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.812803 28209 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:51.812832 22600 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.813704 28209 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:51.814365 28347 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:51.814904 28209 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.817771 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:51.817649 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.719686 28272 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:51.821430 28467 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:51.827751 22631 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 8: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:51.827975 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.739445 22635 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:51.830379 28347 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:51.830399 28475 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:51.834185 28475 consensus_queue.cc:1048] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:51.793593 22528 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.840720 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:51.848942 22783 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:51.850570 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:51.851523 22777 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:51.852063 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.852494 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.852725 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.855070 22776 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:11:51.857909 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.860946 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.861832 22707 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:11:51.862322 22707 pending_rounds.cc:85] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 12
I20260709 13:11:51.862396 22779 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:51.863178 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.863869 22777 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:11:51.864156 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:51.864487 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:51.865154 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:51.865314 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.866000 22777 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:11:51.867447 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.862586 22707 pending_rounds.cc:107] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:11:51.868523 22707 raft_consensus.cc:2889] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:51.872782 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:51.893863 22528 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:51.874055 22704 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:11:51.869082 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:51.923064 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58993.2us
I20260709 13:11:51.926049 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:51.904834 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.931876 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:51.948372 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45877.3us
I20260709 13:11:51.851049 28456 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.854357 28205 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:51.971664 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45693.9us
I20260709 13:11:51.972853 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:11:51.976766 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51443.2us
I20260709 13:11:51.978001 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:51.970945 22528 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 16 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.947813 28222 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:51.950812 28249 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:51.955197 28494 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:51.992084 22528 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 11 to 14, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:51.978255 28485 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.970930 28552 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:51.976330 28516 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:52.004889 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.007181 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 132064us
I20260709 13:11:52.020432 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:51.964465 28507 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:11:51.964821 28045 consensus_queue.cc:1059] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:11:52.022589 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:51.970160 28511 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:11:51.969939 28456 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.004005 28069 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.994119 28249 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:51.886698 28477 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.029488 22528 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 18 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:52.037257 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:51.987531 28403 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 13:11:52.026809 28447 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.004s
I20260709 13:11:52.056833 22528 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:52.061255 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 83405.8us
I20260709 13:11:52.065238 23419 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:11:52.074877 23415 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:11:52.047236 28249 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:52.081730 28507 raft_consensus.cc:3055] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:52.097514 28507 raft_consensus.cc:740] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:52.097856 28562 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:11:52.098121 28572 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:52.017971 28485 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:52.100030 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 168192us
I20260709 13:11:52.075713 28066 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:52.017504 28512 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:52.115954 23421 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 21. Current term is 22. Ops: []
I20260709 13:11:52.062224 28069 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.091631 28045 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.102900 28467 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:52.149848 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 250973us
I20260709 13:11:52.106441 28066 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.098667 28572 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.096009 28540 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:11:52.149574 22528 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:52.139828 28516 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.160856 22528 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 19, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.164768 22528 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 8 to 10, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 10 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:52.142118 28475 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:11:52.158471 28066 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:52.161566 28572 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:52.183904 28424 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:52.184563 28069 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:52.185485 28069 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:11:52.193521 28564 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.053s	user 0.002s	sys 0.000s
W20260709 13:11:52.174880 28390 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.770s	user 0.014s	sys 0.000s
W20260709 13:11:52.199916 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 135601us
I20260709 13:11:52.203114 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:52.203934 22630 raft_consensus.cc:606] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:52.204382 22630 raft_consensus.cc:614] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:52.098569 28507 consensus_queue.cc:260] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.206140 28507 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:52.171514 22528 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 8, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.168138 28516 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.155650 28045 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:52.213714 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 184057us
I20260709 13:11:52.198336 28475 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:11:52.224524 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.244458 22528 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:52.237535 28475 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:52.247700 28475 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.249068 28475 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:52.255884 28424 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:52.256491 28424 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:52.257112 28485 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:52.257493 28424 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.264386 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:52.265286 22630 raft_consensus.cc:606] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:52.267449 28045 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:52.268558 28477 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:52.218925 28456 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:52.277330 28516 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:52.279223 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:52.279336 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:52.280315 23419 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:52.281682 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:52.282737 22630 raft_consensus.cc:606] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:52.287315 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:52.288028 22630 raft_consensus.cc:606] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:52.288363 22630 raft_consensus.cc:614] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:11:52.289211 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69878.9us
I20260709 13:11:52.246363 22528 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 15, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:52.303716 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.321959 22528 catalog_manager.cc:5697] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 13, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 13 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:52.324011 22528 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.320988 22629 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:11:52.323132 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:52.334280 22528 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 13, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:11:52.337270 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 163347us
I20260709 13:11:52.332890 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:52.341107 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.319346 28540 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:52.345146 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.327628 28424 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:52.351023 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.353790 22777 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:11:52.356753 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.357100 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.358107 22781 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:52.362640 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.369567 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.370728 22778 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:52.372862 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.373926 22778 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:52.390538 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.391754 22784 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:52.397123 22629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2270) took 1026 ms. Trace:
W20260709 13:11:52.397903 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.397732 22629 rpcz_store.cc:276] 0709 13:11:51.370378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.371597 (+  1219us) service_pool.cc:224] Handling call
0709 13:11:52.396781 (+1025184us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54003d3454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:52.397019 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609023360}
W20260709 13:11:52.405557 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.416834 22778 raft_consensus.cc:2410] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:11:52.418213 28222 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.418911 28222 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.354167 28573 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.421491 28222 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:52.421878 28573 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:52.439257 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 466515us
I20260709 13:11:52.440639 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:52.443202 22784 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:11:52.449564 22777 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:52.466526 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:52.475620 22675 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:52.478834 28507 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:52.479490 28507 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.480005 28507 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:52.483542 28485 raft_consensus.cc:2804] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:52.484243 28485 raft_consensus.cc:697] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:52.485215 28485 consensus_queue.cc:237] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.478206 28066 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.489411 28066 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:52.494951 28572 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.495936 28045 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:52.497329 28045 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:52.497824 28045 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:52.499043 28045 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.499948 28573 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:52.500588 28045 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:52.516577 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:52.520983 28573 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:52.521581 28573 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.522060 28573 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:52.526934 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.523892 28463 raft_consensus.cc:993] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:52.357489 28540 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.535621 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.538308 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.473766 28516 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:52.497772 28572 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:52.556274 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.563191 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.564917 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:52.565431 22778 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:52.567973 22780 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:52.548467 28029 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:52.586596 28540 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:52.597369 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.601809 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:52.603902 22629 raft_consensus.cc:2393] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:52.606600 22635 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:11:52.621641 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 420896us
I20260709 13:11:52.629513 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:52.631165 22630 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: []
I20260709 13:11:52.633831 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:52.634886 22629 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:52.644287 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:52.666944 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:52.668093 22630 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:52.653995 28449 raft_consensus.cc:1081] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:52.673758 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:52.654991 28463 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:52.677793 22629 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 13:11:52.682015 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 167757us
W20260709 13:11:52.704721 28249 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.002s
I20260709 13:11:52.706362 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:52.707497 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 63357.7us
I20260709 13:11:52.708730 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:52.705348 28249 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.710824 23418 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:52.710909 28272 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:52.711516 28272 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:52.712011 28272 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:52.714934 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:52.715415 28194 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:52.716544 28587 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:52.716930 22600 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:52.717252 28587 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.738853 22600 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:52.740358 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 1112) took 1215 ms. Trace:
I20260709 13:11:52.741056 23421 rpcz_store.cc:276] 0709 13:11:51.524972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.525207 (+   235us) service_pool.cc:224] Handling call
0709 13:11:51.525678 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:52.116811 (+591133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.739900 (+623089us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400455454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:52.740242 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1356951936}
I20260709 13:11:52.750777 22630 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:11:52.750831 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.722154 28596 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:11:52.757788 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.755091 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:52.758687 22634 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:52.756096 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:52.756886 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:52.765281 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 92122.6us
I20260709 13:11:52.729543 28249 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:52.787663 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48936.1us
I20260709 13:11:52.788514 22600 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:52.652439 28573 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:52.770473 28511 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.782691 22634 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:52.795286 28194 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:52.796027 28511 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.792333 28259 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:11:52.793215 22749 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:52.764546 28541 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:52.761314 28508 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:52.766646 28390 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.808346 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:52.788089 28396 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:52.824740 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:52.825596 22634 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:11:52.832232 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:52.784004 28434 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:11:52.784222 28461 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:11:52.813084 28541 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:52.814291 28194 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:52.843325 22634 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:11:52.831357 28403 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:52.851940 28066 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:52.856848 28194 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.820346 28259 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:52.809844 28588 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
W20260709 13:11:52.891006 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:52.993539 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:52.839660 28516 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:52.821594 28508 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:52.847195 28469 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:11:52.825888 28390 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.012912 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.010360 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:52.993814 28449 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:53.002358 28045 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.005s
I20260709 13:11:52.995625 28583 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:52.996268 28485 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:11:53.023918 28477 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:11:53.071547 22634 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:53.043489 28403 raft_consensus.cc:697] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:53.052105 28390 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:53.087174 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 237868us
I20260709 13:11:53.014549 28516 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:53.088850 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70988us
I20260709 13:11:53.045634 28463 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:11:53.042718 28397 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:11:53.100569 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.060180 28587 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.071465 28069 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.094686 28586 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.071983 28259 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.087795 28608 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.091997 28045 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:53.123229 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.108553 28390 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:53.060568 28583 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.047986 28541 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.130582 28608 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:53.141965 28599 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 13:11:53.142665 28610 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.143359 28610 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.132802 28611 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:53.145982 28610 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.148267 28396 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:53.148883 28396 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:53.149137 23415 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:11:53.151075 28403 consensus_queue.cc:237] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.159029 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:53.130287 28249 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:11:53.139890 28566 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.981s	user 0.013s	sys 0.009s
I20260709 13:11:53.127873 28390 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:11:53.181726 28424 consensus_meta.cc:287] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.824s	user 0.003s	sys 0.013s
W20260709 13:11:53.188207 28600 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:11:53.192293 28573 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.194092 28209 raft_consensus.cc:993] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:53.195359 28516 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.128232 28586 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.196302 28583 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.129910 28475 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:11:53.198251 28586 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.200687 28508 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.130182 28045 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.168790 28249 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:53.166759 28541 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:53.120452 28069 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.208778 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.232079 22628 raft_consensus.cc:2393] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:53.229915 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.166260 28613 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.235764 22783 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:11:53.236946 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 141472us
I20260709 13:11:53.239889 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49444 (request call id 833) took 1579 ms. Trace:
I20260709 13:11:53.241127 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.241923 22704 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:53.231179 28249 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:53.246565 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.240442 22705 rpcz_store.cc:276] 0709 13:11:51.660895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.789135 (+128240us) service_pool.cc:224] Handling call
0709 13:11:51.789494 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:11:51.789969 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:51.790003 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:11:51.790716 (+   713us) log.cc:844] Serialized 52 byte log entry
0709 13:11:52.012718 (+222002us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:52.201442 (+188724us) log.cc:844] Serialized 14 byte log entry
0709 13:11:52.201956 (+   514us) log.cc:844] Serialized 14 byte log entry
0709 13:11:52.203874 (+  1918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:52.203968 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:52.322424 (+118456us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b48000ce358. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:11:53.114025 (+791601us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:53.239610 (+125585us) raft_consensus.cc:1685] finished
0709 13:11:53.239647 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:53.239826 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1610,"prepare.queue_time_us":306816,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":9373,"raft.run_cpu_time_us":3112,"raft.run_wall_time_us":128365,"spinlock_wait_cycles":8248960,"thread_start_us":392749,"threads_started":2,"wal-append.queue_time_us":310014}
I20260709 13:11:53.251421 22703 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:11:53.254832 28580 raft_consensus.cc:1081] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:53.258313 28540 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.259121 28540 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.260259 22780 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:53.261862 28571 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.262503 28571 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.268407 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.269507 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.269802 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.237056 28508 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.271037 28508 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.271831 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.272635 28511 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.273182 28508 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.235015 28613 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.275682 28397 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:53.276859 28613 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.279250 28615 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:53.237056 28608 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.281517 22705 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:53.282779 22705 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.283968 22705 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:53.284004 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.281503 28615 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:53.286708 28615 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:53.287647 28615 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.272652 22704 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:53.330258 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.331111 22632 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:53.293246 28595 log.cc:927] Time spent T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.493s	user 0.000s	sys 0.007s
I20260709 13:11:53.296017 28045 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.293260 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:53.301182 28272 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.589s	user 0.000s	sys 0.013s
I20260709 13:11:53.254238 28516 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.311821 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:53.319175 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:53.320081 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.321000 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:53.302307 22705 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.355832 22782 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:11:53.356913 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:53.331151 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.359150 23421 raft_consensus.cc:3060] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:53.327880 28507 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.847s	user 0.015s	sys 0.000s
I20260709 13:11:53.367872 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.380071 22705 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.327663 28508 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.366791 28045 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:53.394665 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:53.396287 23421 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:11:53.396679 23419 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:53.372738 28609 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.367645 28573 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.368258 28511 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.334707 28572 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:11:53.378993 28615 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:53.352607 28272 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.241547 28390 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.366375 28516 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.406525 28516 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.414984 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.402725 28609 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.423416 28604 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:11:53.424050 28604 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:53.379349 28507 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.401443 28045 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.433224 28045 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.412532 28485 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:53.437832 28390 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:53.436968 28272 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.437641 28485 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.413352 28573 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.456588 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50372 (request call id 3260) took 1116 ms. Trace:
I20260709 13:11:53.448849 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.450017 22703 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:11:53.390897 28508 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.412199 28618 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 13:11:53.457636 22703 pending_rounds.cc:85] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 8
I20260709 13:11:53.460813 22703 pending_rounds.cc:107] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:11:53.429471 28570 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:53.461195 22703 raft_consensus.cc:2889] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:53.457221 22783 rpcz_store.cc:276] 0709 13:11:52.340352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:52.340669 (+   317us) service_pool.cc:224] Handling call
0709 13:11:53.456377 (+1115708us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54003273d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:53.456510 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1909131008}
I20260709 13:11:53.412798 28511 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.423496 28609 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:53.472203 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.468170 23419 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:11:53.448972 28249 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.477154 22673 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:53.454248 28507 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.486265 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:53.487049 22706 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:53.487498 22706 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:53.489336 28272 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:53.490582 28573 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.439103 28396 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.493613 28396 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.463020 28570 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:53.425119 28604 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.498224 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.501943 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:53.502735 28615 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.495990 28604 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:53.503518 28615 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.504943 28249 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.483884 28609 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:53.508500 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:53.497354 28508 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.509454 28609 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:53.495643 28570 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.512609 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.512936 28609 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.514678 22702 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:53.516476 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.512961 28570 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:53.517701 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.518967 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.519116 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.519150 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.519930 22708 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:11:53.520725 22710 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:53.520874 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.521842 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.522284 22708 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:53.519989 22701 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:53.523476 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:53.519933 22704 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:53.533133 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.534169 22701 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:53.536974 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:53.537431 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:53.537806 22780 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:53.538118 22784 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:53.539265 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.531929 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:53.536935 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.543205 22704 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:53.539814 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.547295 28586 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:53.537613 28508 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.547829 28586 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.548342 28586 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:53.548346 28508 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.552419 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.553555 28508 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.554951 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.555475 22783 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:11:53.544921 22709 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:11:53.553323 22702 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:53.558094 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.559695 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.494542 28511 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.563561 22706 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.570519 28485 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.571873 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:53.574314 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.576340 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.576817 22784 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:53.580350 28511 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.580991 28511 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.583180 22710 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:53.587973 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.588972 22702 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:53.588006 28463 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:53.590971 28463 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:53.591369 28396 raft_consensus.cc:993] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:53.591954 28396 raft_consensus.cc:1081] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:53.592275 28463 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:53.594810 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.599157 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.607252 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.607301 22706 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:53.618840 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:53.728888 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.577503 28475 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:53.732971 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.733450 22780 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:53.738881 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:53.740460 22707 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:53.740803 28511 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:53.747191 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 156622us
W20260709 13:11:53.748040 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 142583us
I20260709 13:11:53.749584 28069 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.750375 28540 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.760113 22630 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:11:53.761328 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.761586 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:53.767170 28467 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:53.767876 28467 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:53.768954 28467 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.770546 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:53.775285 28347 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:53.776738 22600 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:53.777437 23419 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:53.782572 28571 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.787611 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:53.788023 28580 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:53.788869 28580 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:53.789875 28580 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.806200 23418 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 19. Current term is 20. Ops: []
I20260709 13:11:53.806380 28615 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:53.808351 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.810920 22783 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:11:53.815183 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.815021 28615 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:11:53.816483 28469 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:11:53.816982 28469 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:53.818154 28469 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.819702 28469 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:53.813871 28259 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Committing config change with OpId 17.11: config changed from index 5 to 11, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:53.821283 23416 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 5 to 11, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:53.780793 28586 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:53.830013 23415 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:53.833786 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:53.832247 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.838802 22783 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:53.839458 28449 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:53.839962 28449 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:53.845587 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:53.846552 22629 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:53.849979 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:53.852250 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98847.5us
I20260709 13:11:53.855072 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.856262 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:53.857023 22783 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:53.860364 28449 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:53.865128 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 126666us
I20260709 13:11:53.889614 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.884125 22629 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:53.891708 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:53.902916 23419 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:53.903891 23418 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:53.905050 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:53.870865 28616 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.907310 28616 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.831743 28586 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:53.904261 22747 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:53.920346 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.922353 22673 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:53.892504 28461 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:53.924185 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:53.891921 28477 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.897451 28609 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:53.892701 28574 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:11:53.912014 28390 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.949525 28390 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:53.950960 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:53.952746 28390 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:53.912612 28272 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:11:53.919849 28066 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:11:53.897514 28587 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.936914 28585 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:53.970781 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:53.941721 28449 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:53.945170 28467 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:53.947330 28461 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:53.965445 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.554s (SENT) [suppressed 30 similar messages]
W20260709 13:11:53.981837 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57730.1us
I20260709 13:11:53.965157 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:53.985220 23418 raft_consensus.cc:2393] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:53.892160 28511 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:11:53.920447 28602 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:53.926496 28618 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:53.926882 28508 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:11:53.979652 28625 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:11:53.926759 28194 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:53.932364 28507 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:11:53.973740 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62763.1us
I20260709 13:11:53.972716 28580 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:11:53.962872 28477 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.003777 28508 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:53.997546 28045 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:54.003557 28591 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:53.973040 28592 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 13:11:53.979038 28587 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.061499 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:54.080313 22702 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:11:54.063828 22704 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:54.066504 22706 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:11:54.009356 28194 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.036139 28507 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:54.077438 22710 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 19. Current term is 20. Ops: []
I20260709 13:11:54.084245 22704 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:54.090852 22704 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:54.001832 28602 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.015129 28609 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 13:11:54.074182 22701 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:11:54.070108 28397 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.092723 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:54.075840 28508 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:54.095644 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:54.106894 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 141477us
I20260709 13:11:54.087854 28194 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:54.070830 28066 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:11:54.135551 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:11:54.154506 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.158326 22701 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:54.096915 28616 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:54.166757 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:54.107213 28591 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:54.143886 28619 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:11:54.266892 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 105229us
I20260709 13:11:54.267870 22705 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:54.271106 28587 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:54.272337 28646 log.cc:927] Time spent T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:11:54.147169 28397 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.284883 28397 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.295857 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:54.296883 22709 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
W20260709 13:11:54.301666 23385 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.310524 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:54.310599 22701 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:54.311645 22709 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:54.106894 28461 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.326678 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:54.333205 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:54.333922 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.302219 28619 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.300232 28507 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.388962 28447 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:54.304702 22704 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:54.397343 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:54.397683 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 103698us
I20260709 13:11:54.397789 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:54.405825 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.406731 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.407527 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.405743 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.398155 22706 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:54.422539 22709 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:11:54.429045 22783 raft_consensus.cc:2393] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:54.429396 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.425236 22704 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.431593 22778 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:54.431445 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.433111 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.434567 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.389815 28591 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:54.445108 28259 consensus_meta.cc:287] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.624s	user 0.013s	sys 0.000s
I20260709 13:11:54.427656 28485 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.397169 28403 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:54.446447 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.448320 28602 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:54.441138 28586 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:11:54.430456 28477 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.463927 28485 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:54.479058 28469 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.659s	user 0.014s	sys 0.005s
I20260709 13:11:54.483703 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.485148 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.458411 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.487681 28463 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:54.489094 28570 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:11:54.489768 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:54.487689 28069 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:54.492025 28586 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:54.492493 28586 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:54.492929 28586 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:54.495114 28638 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.420s	user 0.000s	sys 0.003s
I20260709 13:11:54.494948 28403 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.497236 28605 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:54.497570 28403 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:54.497684 28605 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:54.500061 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.501612 22601 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:54.430660 28447 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.505033 28447 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:54.517499 22601 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:54.523658 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.507699 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3285) took 1221 ms. Trace:
I20260709 13:11:54.527972 22779 rpcz_store.cc:276] 0709 13:11:53.286524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:53.287172 (+   648us) service_pool.cc:224] Handling call
0709 13:11:53.287584 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:53.288022 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:11:53.288054 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:53.496834 (+208780us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b2c0006b798. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:11:53.497393 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:11:53.919620 (+422227us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:53.919665 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:53.919740 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:54.494456 (+574716us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:54.507417 (+ 12961us) raft_consensus.cc:1685] finished
0709 13:11:54.507443 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:54.507599 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1096,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":61,"spinlock_wait_cycles":62723584,"thread_start_us":421777,"threads_started":1,"wal-append.queue_time_us":501054}
I20260709 13:11:54.508724 28605 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.534376 28605 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:54.538956 28654 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:54.539640 28654 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:54.510635 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.540539 28654 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.541476 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:54.551460 22708 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:11:54.551910 22708 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:54.552783 22708 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.553743 22708 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:11:54.558094 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.511710 28508 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.513191 28469 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:54.513824 28656 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:54.486459 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:54.518190 28485 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.519972 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.463135 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.620869 22778 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:11:54.525738 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.622257 22783 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:54.461373 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:54.559454 22779 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
W20260709 13:11:54.582893 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 261324us
W20260709 13:11:54.629011 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 109114us
I20260709 13:11:54.630117 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.631315 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.632109 22778 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:54.603132 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:54.610088 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52999.6us
I20260709 13:11:54.634982 22709 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:54.639428 22783 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:11:54.640627 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:54.637125 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:54.648607 23421 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:54.644912 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:54.660102 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:54.661095 23419 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 22: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:54.666853 22710 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:54.611093 28656 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.452694 28194 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:54.673329 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.634306 28585 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:54.654968 28570 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:11:54.684823 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:54.686266 22781 raft_consensus.cc:1240] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:11:54.686424 22778 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:54.641397 28611 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:11:54.680961 28656 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:11:54.709970 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53483.5us
I20260709 13:11:54.705974 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:54.685707 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.681700 28571 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:54.719656 22675 connection.cc:256] RPC call timeout handler was delayed by -1.1971928640000442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 13:11:54.707751 28477 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:54.736485 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.737426 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.738086 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.739297 22631 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:54.739920 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.696870 28610 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:11:54.711210 28612 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:54.746152 28508 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.751015 22709 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:11:54.770514 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:54.752799 28066 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:54.720590 28570 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:54.756748 22708 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:11:54.774935 22705 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:54.775502 22705 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:54.776319 22705 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.777508 22705 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:54.707576 28259 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45654 (request call id 91) took 2922 ms. Trace:
I20260709 13:11:54.766311 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.767766 22702 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:54.755292 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:54.772136 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.775120 22708 pending_rounds.cc:85] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 4
I20260709 13:11:54.737352 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.780434 22708 pending_rounds.cc:107] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:11:54.780884 22708 raft_consensus.cc:2889] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:54.781033 22629 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:54.757762 28477 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:54.789942 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.790062 28654 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:54.803079 28654 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:54.803530 28654 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:54.767215 28612 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:54.812266 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:54.836213 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:54.861371 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:54.807739 28477 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:54.729378 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:54.898963 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 169724us
I20260709 13:11:54.865779 22632 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:11:54.822981 28612 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:54.903766 22705 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:11:54.904194 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:54.906854 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.909246 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.822870 28655 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:11:54.912127 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:54.914144 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:54.908072 28449 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:54.918316 28449 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.918740 28396 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:54.920758 28449 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.805596 28656 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:54.923804 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:54.930281 22702 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:11:54.942945 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 171473us
I20260709 13:11:54.896477 28194 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:54.908560 28508 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:54.914386 28655 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:54.938844 28485 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:11:54.950153 28663 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:11:54.941941 28507 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:54.994765 28507 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:54.995533 28507 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:54.996472 28249 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.997076 28249 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.997658 28396 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.976461 22627 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:54.913430 28609 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:11:54.988586 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:55.003618 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:54.998200 28396 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:54.985374 28485 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.951125 28449 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:11:54.984300 28475 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:55.019035 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:55.027093 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 390047us
I20260709 13:11:55.021787 28485 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:11:55.048032 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 124300us
I20260709 13:11:55.048272 22627 raft_consensus.cc:1275] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
W20260709 13:11:55.050043 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 408253us
I20260709 13:11:55.051748 22701 raft_consensus.cc:1275] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:11:55.052225 22701 pending_rounds.cc:85] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:11:55.052055 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:55.052466 22701 pending_rounds.cc:107] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:11:55.052742 22701 raft_consensus.cc:2889] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.055272 28656 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.056053 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:55.057030 22710 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:11:55.041785 28662 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:11:55.063208 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:55.066764 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:55.061878 28611 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:55.073397 22702 raft_consensus.cc:3055] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:55.073935 22702 raft_consensus.cc:740] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:55.074816 22702 consensus_queue.cc:260] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.075790 22702 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:55.050450 28249 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.079159 28249 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:55.063824 28403 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:11:55.080061 28403 consensus_queue.cc:1243] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:11:55.080585 28403 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:55.085609 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.087100 22627 raft_consensus.cc:2410] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:11:54.784376 28259 rpcz_store.cc:276] 0709 13:11:51.773827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.774764 (+   937us) service_pool.cc:224] Handling call
0709 13:11:51.780032 (+  5268us) log.cc:844] Serialized 494 byte log entry
0709 13:11:52.192521 (+412489us) spinlock_profiling.cc:243] Waited 2.52 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:52.594322 (+401801us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b180035a460. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f180adb158f 00007f1803df0571 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd
0709 13:11:54.696578 (+2102256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71349,"raft.queue_time_us":36789,"raft.run_cpu_time_us":1470,"raft.run_wall_time_us":1473,"spinlock_wait_cycles":11204096,"thread_start_us":242803,"threads_started":1,"wal-append.queue_time_us":302799,"wal-append.run_cpu_time_us":7025,"wal-append.run_wall_time_us":1165438}
I20260709 13:11:55.080910 28403 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:55.042994 28628 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:11:55.029215 28475 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.024958 28655 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.098932 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.099913 22627 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:55.100232 28670 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:11:55.093623 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 147414us
I20260709 13:11:55.087211 28609 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:55.110790 28586 consensus_meta.cc:287] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.617s	user 0.014s	sys 0.000s
I20260709 13:11:55.111459 28586 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.112023 28403 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.112637 22776 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
I20260709 13:11:55.113343 28403 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:55.119233 28673 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:11:55.120865 28396 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.121320 28654 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.125577 22784 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:11:55.126030 22784 pending_rounds.cc:85] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 8
I20260709 13:11:55.126226 22782 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:11:55.131901 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.133275 22706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3515) took 1024 ms. Trace:
I20260709 13:11:55.133673 22630 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
I20260709 13:11:55.136307 22783 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:11:55.126263 22784 pending_rounds.cc:107] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:11:55.137902 22784 raft_consensus.cc:2889] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.133739 22706 rpcz_store.cc:276] 0709 13:11:54.109002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:54.109282 (+   280us) service_pool.cc:224] Handling call
0709 13:11:54.109604 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:11:54.415947 (+306343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:55.133007 (+717060us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54000a8cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:11:55.133197 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663050496}
I20260709 13:11:55.133189 28586 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.131839 28507 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:55.144359 28605 consensus_meta.cc:287] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.609s	user 0.013s	sys 0.000s
I20260709 13:11:55.139367 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:55.146593 23418 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:11:55.147188 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.147891 22706 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:55.154500 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f4adfa257c34bb88d79bda9d39c5146 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:55.163906 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.164819 22784 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:55.165300 22634 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:11:55.166739 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:55.168325 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.172406 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
W20260709 13:11:55.177426 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.179512 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.180474 22777 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:55.171387 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:55.190161 22702 raft_consensus.cc:1275] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:11:55.190850 22702 pending_rounds.cc:85] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:55.192880 22702 pending_rounds.cc:107] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:11:55.194492 28475 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.195624 22702 raft_consensus.cc:2889] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.202184 22702 pending_rounds.cc:107] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:11:55.202518 22777 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:55.188079 28602 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.205199 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:55.204056 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 151860us
I20260709 13:11:55.210872 28477 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:55.212447 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 506589us
I20260709 13:11:55.231391 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:55.152774 28612 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.232015 22629 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:11:55.214428 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 290188us
I20260709 13:11:55.202883 22702 raft_consensus.cc:2889] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:55.217294 28424 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:55.218578 28069 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:55.219069 28396 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:11:55.220618 28688 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.245033 28688 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.247860 28688 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.248204 28619 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:55.249048 28069 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:55.250267 28069 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.251609 28069 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:55.229207 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 161555us
I20260709 13:11:55.213172 28683 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.234923 28681 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:11:55.260697 28680 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:11:55.260494 22710 raft_consensus.cc:2955] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Committing config change with OpId 17.11: config changed from index 5 to 11, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:11:55.234922 28424 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:55.263233 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.263962 22707 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:55.263764 28424 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.265218 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:55.265594 28424 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:55.255712 28683 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.267074 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:55.267593 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:55.267810 22704 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:11:55.268464 28396 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:55.268864 28396 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.269212 28396 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:55.269325 28683 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.272126 28612 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:55.274863 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.275851 22778 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:55.233619 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.235059 22784 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:55.278064 22701 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:11:55.240785 28586 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.241901 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:55.279896 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:55.280095 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46584.6us
I20260709 13:11:55.280993 22675 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:55.283195 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.286491 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:55.286739 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:55.287276 22704 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
W20260709 13:11:55.287681 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 211206us
I20260709 13:11:55.289433 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.260545 28272 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:11:55.292342 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.295382 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.291527 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:55.312271 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.311486 23385 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:55.313702 22709 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:11:55.314141 22709 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:55.313827 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.316148 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.317390 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:55.315266 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:55.315987 22707 raft_consensus.cc:2410] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:11:55.328577 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:55.327184 28613 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.330927 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:55.328835 22629 raft_consensus.cc:1217] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Deduplicated request from leader. Original: 11.7->[12.8-14.9]   Dedup: 12.8->[14.9-14.9]
I20260709 13:11:55.282738 28508 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:55.331687 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:55.290498 28612 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:55.350239 22630 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:55.351428 22630 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:55.286238 28683 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:55.288988 28602 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:55.307415 28688 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:55.331774 22780 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
I20260709 13:11:55.342958 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:55.287590 28654 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.342406 22710 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:55.356506 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:55.288872 28610 consensus_queue.cc:1048] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:11:55.350732 22777 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:55.349402 22629 raft_consensus.cc:1240] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: [12.8-14.9]
I20260709 13:11:55.349195 23418 raft_consensus.cc:2393] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 22: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:55.348506 28680 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:11:55.361125 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.333129 28690 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:55.331907 28660 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:55.368372 28249 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:55.380364 22776 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:11:55.348994 28668 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.384128 22777 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:11:55.302309 28655 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:55.337515 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:55.302277 28686 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 13:11:55.375296 22630 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.354462 28612 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:55.355544 28507 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.327767 28463 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:55.389192 28680 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:55.397424 28477 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.376433 28654 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:11:55.389654 28475 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:55.350230 28613 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.412338 28249 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.412695 28668 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.350243 28586 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 13:11:55.422789 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:55.481418 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58817.7us
W20260709 13:11:55.418715 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 106626us
I20260709 13:11:55.497224 22778 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:11:55.511540 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.513589 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:55.514067 28678 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:11:55.514962 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 174592us
I20260709 13:11:55.518656 28397 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:55.519237 28397 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:55.511585 28660 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:11:55.520298 28660 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:55.520848 28660 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:11:55.510637 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 118077us
I20260709 13:11:55.495623 28612 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:55.524016 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.532673 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.526888 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:55.534126 23419 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:55.535560 28396 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.504340 28469 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:11:55.538157 28396 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.508769 28671 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:55.539028 28671 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.504470 28508 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.541883 28671 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:55.543897 28680 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.519688 28397 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:55.547708 23411 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:55.547783 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:55.525472 23421 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:11:55.549551 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:55.550957 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 51452.9us
I20260709 13:11:55.550102 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:55.553411 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:55.553714 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.554397 23418 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:55.554352 22710 raft_consensus.cc:2376] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13. Re-sending same reply.
I20260709 13:11:55.555980 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.557034 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:55.557677 22707 raft_consensus.cc:2376] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21. Re-sending same reply.
I20260709 13:11:55.557682 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.557814 22783 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:55.509415 28628 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:55.481640 28272 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:11:55.479276 28194 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:11:55.511587 28463 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:55.593173 28463 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.600276 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.601069 22781 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:55.578019 28681 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:55.579496 28249 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.476864 28485 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.552606 28602 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:11:55.577184 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53237.7us
I20260709 13:11:55.604090 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:55.556185 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.562291 28690 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:55.505244 28475 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.615088 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.615996 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:55.623903 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67829.8us
I20260709 13:11:55.633308 22781 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
W20260709 13:11:55.635335 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 121945us
I20260709 13:11:55.607404 23418 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:11:55.566826 28628 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:55.585897 28552 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.643945 22701 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:55.639667 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:55.552320 28507 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:11:55.653354 22702 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:11:55.664400 28616 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:11:55.680424 28485 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:55.641929 28598 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:55.683647 28668 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.623275 28688 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:55.685431 28688 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.688109 28688 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.689881 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:55.684824 28613 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:55.709470 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 179116us
I20260709 13:11:55.655629 28690 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:55.740964 28690 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.706664 28467 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:55.738245 28681 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:11:55.743877 28681 consensus_queue.cc:1243] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer 1f99ad34cc414233929e0a92f8c3d8db log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:11:55.744825 28475 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:55.748008 28598 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:55.751559 28693 log.cc:927] Time spent T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:11:55.751919 28598 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:55.756538 28612 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:55.759174 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 161522us
W20260709 13:11:55.777740 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 128493us
I20260709 13:11:55.748194 28628 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.811646 28628 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:55.802471 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:55.817703 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 165530us
I20260709 13:11:55.819199 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.749689 28397 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:55.768842 28467 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:55.822415 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 194827us
W20260709 13:11:55.878209 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44381.1us
I20260709 13:11:55.884920 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.886291 28508 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:55.890837 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.892274 22778 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:11:55.892491 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:11:55.894074 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 254560us
I20260709 13:11:55.897796 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.877446 28613 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:11:55.908522 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:55.823410 28686 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:55.917976 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:55.898263 28485 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:55.919626 28508 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:55.912778 28688 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.920754 28683 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.949795 28485 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:55.889469 28701 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.113s
I20260709 13:11:55.931981 28612 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:55.936534 28613 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:55.934813 28508 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:55.982865 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.983805 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:55.984745 22627 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:11:55.931646 28702 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 13:11:55.997850 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.554s (SENT)
I20260709 13:11:55.998513 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.000589 28424 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:55.972256 28692 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:55.997905 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:55.951848 28467 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.024639 28613 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.029332 22634 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:11:56.030717 28668 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:56.031298 28668 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.031741 28668 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:56.032676 28397 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:56.045481 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 127594us
W20260709 13:11:56.046120 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.047281 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.047845 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:55.981794 28508 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:56.052578 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.054968 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.055341 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.056650 28688 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:56.057209 28485 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:56.055773 22703 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:56.062282 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.073421 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:56.061888 28690 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:56.073526 28688 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:56.106104 28660 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.009s
I20260709 13:11:56.106848 28715 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:56.061452 28424 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:56.107988 28424 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.112672 22630 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:11:56.125308 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.636s (SENT)
I20260709 13:11:56.125871 23385 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.127806 28249 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:56.129206 28477 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:11:56.129709 28477 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:56.130795 28477 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.131378 28668 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.132000 28477 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:56.120523 28660 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.123664 28424 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:56.124467 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.163302 22777 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 12. Current term is 13. Ops: []
W20260709 13:11:56.134361 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 61251.5us
I20260709 13:11:56.168606 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:56.170511 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.136059 22703 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:56.136374 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:56.141167 22709 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.175299 22709 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:56.143149 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.178543 22780 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:11:56.144562 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:56.182389 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66200.3us
I20260709 13:11:56.184123 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.187408 28704 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:56.184201 28690 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:11:56.187970 28704 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:56.190044 28688 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:56.177186 28507 consensus_queue.cc:1048] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 13:11:56.193620 23415 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:11:56.193622 23416 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 20. Current term is 21. Ops: [20.17-20.17]
I20260709 13:11:56.183256 28715 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:56.226284 22632 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:56.211716 22709 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:11:56.232093 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.232954 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.234694 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:56.238072 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:56.242280 22776 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:56.244318 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:56.243558 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:56.246580 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60579.2us
I20260709 13:11:56.209123 28249 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:56.251300 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.253495 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:56.254252 22632 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:11:56.255364 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:56.255756 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:56.256407 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:56.258021 22776 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:11:56.260380 22633 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:11:56.262254 22630 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
W20260709 13:11:56.249591 28602 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:11:56.260371 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:56.271164 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.273353 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.273835 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.274791 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.221258 28690 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 13:11:56.275460 22708 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:11:56.275604 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.276257 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.281356 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.283546 22706 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 13: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:56.284451 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:56.284682 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.287873 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.288187 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.223196 28701 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:11:56.288698 22703 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:56.293591 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.297281 22704 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:56.241995 28424 consensus_queue.cc:1048] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:11:56.307075 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.308127 28619 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:56.308621 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.309818 28540 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:56.310024 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.271987 28613 raft_consensus.cc:2764] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:11:56.311295 28507 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:56.311508 28602 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:56.311884 28602 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:56.311908 28507 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.312812 28540 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:56.312891 28602 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.314309 28507 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:56.211759 28704 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.315920 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.283304 28611 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:56.316941 28629 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:56.317255 28611 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:56.317603 28629 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:56.317708 28611 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:56.318989 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:56.318701 28629 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.320492 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.259522 28249 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.321529 28692 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:11:56.321982 28540 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:56.323647 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:56.325254 22703 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:56.325618 22750 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.323136 28692 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:11:56.328035 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.325886 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:56.332173 22708 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:11:56.335597 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.336544 22703 raft_consensus.cc:2376] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12. Re-sending same reply.
I20260709 13:11:56.337805 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.338860 22703 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:56.341240 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.259327 28578 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:11:56.351070 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.314407 28602 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:56.347658 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:56.352222 22703 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 11.
I20260709 13:11:56.357349 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:56.359668 28710 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.326s	user 0.001s	sys 0.003s
I20260709 13:11:56.361132 22703 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:11:56.349170 28592 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:56.368784 28592 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:56.371034 22705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3550) took 1090 ms. Trace:
I20260709 13:11:56.371490 22705 rpcz_store.cc:276] 0709 13:11:55.280584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.280719 (+   135us) service_pool.cc:224] Handling call
0709 13:11:56.370820 (+1090101us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54000e6d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:56.370968 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329791872}
I20260709 13:11:56.372464 28249 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:56.373564 28660 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:56.323076 28668 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:56.349841 28507 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:56.278868 28570 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:11:56.393554 28508 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.351853 28692 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:11:56.459832 22704 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:56.344683 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.354974 28540 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:56.492560 28508 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:56.486585 22777 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:11:56.496896 22673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:56.498378 28259 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:56.499125 28259 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.502063 28259 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:56.519972 22706 raft_consensus.cc:1240] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:56.342278 28066 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.118s
W20260709 13:11:56.525429 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 281187us
W20260709 13:11:56.539057 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 194457us
I20260709 13:11:56.546806 22777 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:11:56.543540 28249 consensus_queue.cc:1059] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:11:56.543776 28692 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:56.556080 22784 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:11:56.560823 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 317705us
W20260709 13:11:56.563547 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 229596us
I20260709 13:11:56.567080 23385 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:56.568256 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.571741 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:56.571806 22784 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:11:56.572256 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:56.572615 22778 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:56.573786 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:56.570191 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:56.562263 28660 raft_consensus.cc:3055] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:11:56.594997 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:11:56.642783 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52445.9us
W20260709 13:11:56.692839 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 364867us
I20260709 13:11:56.694001 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.585387 28477 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:56.700901 22778 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 23.
I20260709 13:11:56.728817 28610 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.597854 28660 raft_consensus.cc:740] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:56.729560 28610 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.696029 28668 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:56.726095 28477 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:56.763887 28610 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:56.773186 28730 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:11:56.775099 28612 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:56.759209 28668 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:56.782353 28477 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:11:56.815346 28729 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:11:56.823153 28611 consensus_meta.cc:287] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.000s
W20260709 13:11:56.819967 28728 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:11:56.890533 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 196415us
I20260709 13:11:56.903373 28029 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:56.905005 28029 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.902112 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:56.824069 28611 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:56.916044 28720 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.873s	user 0.001s	sys 0.002s
W20260709 13:11:56.916335 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 224855us
I20260709 13:11:56.924696 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47414 (request call id 3570) took 1014 ms. Trace:
I20260709 13:11:56.935802 28674 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:56.925318 22701 rpcz_store.cc:276] 0709 13:11:55.910552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.977313 (+ 66761us) service_pool.cc:224] Handling call
0709 13:11:55.977844 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:55.981106 (+  3262us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:55.981152 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:55.981809 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:11:56.042522 (+ 60713us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:56.042584 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:56.042705 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:56.914597 (+871892us) spinlock_profiling.cc:243] Waited 9.83 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:56.923991 (+  9394us) raft_consensus.cc:1685] finished
0709 13:11:56.924015 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:56.924468 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6306,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":22608256,"thread_start_us":60244,"threads_started":1,"wal-append.queue_time_us":60643}
I20260709 13:11:56.939833 28654 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:56.941228 28654 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.945549 28654 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:56.948462 22626 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:55.159909 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:56.952659 28029 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:56.953570 28592 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:56.954389 28592 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:56.956399 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:56.960669 22634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 2312) took 1643 ms (client timeout 1669 ms). Trace:
I20260709 13:11:56.961045 22626 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
W20260709 13:11:56.961287 22634 rpcz_store.cc:269] 0709 13:11:55.316883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.317063 (+   180us) service_pool.cc:224] Handling call
0709 13:11:56.960168 (+1643105us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540006e0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:56.960392 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510564736}
I20260709 13:11:56.969249 28592 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:56.972138 28685 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:56.987176 28668 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:56.986845 28397 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:56.988238 28628 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:56.994431 28611 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:56.989774 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:57.000682 22781 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:57.001417 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:57.013099 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:57.016860 28660 consensus_queue.cc:260] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.023409 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:57.027441 28704 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:57.060992 22626 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 11: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:57.059342 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 69871.8us
I20260709 13:11:57.063025 28610 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:57.066903 22707 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:11:57.067098 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 43821us
I20260709 13:11:57.070369 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:57.071570 22634 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:57.072989 28461 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:57.074200 28461 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.076987 28681 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.077509 28461 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.078482 22708 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:57.078162 28681 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.083022 28508 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.084183 28508 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.088173 28692 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.092473 28685 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.095041 28605 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:57.095773 28605 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:57.096190 23417 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:11:57.096726 28592 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:57.098719 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:57.099936 22777 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:11:57.083504 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.100934 23418 raft_consensus.cc:2368] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14.
I20260709 13:11:57.101477 28660 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:57.102079 23421 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23. Ops: []
I20260709 13:11:57.105531 28685 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:57.112383 28692 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.116673 28508 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.117866 28508 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.096253 28605 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:57.118803 28508 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.121614 28508 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.118827 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.117973 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:57.124095 28681 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:57.133205 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 231454us
I20260709 13:11:57.134680 22777 raft_consensus.cc:1240] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:11:57.136090 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:57.136308 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:57.137048 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:57.138360 22528 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 147) took 16321 ms (client timeout 15000 ms). Trace:
I20260709 13:11:57.138988 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:57.143249 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:57.134529 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:57.149237 28570 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:57.149940 28570 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.150472 28570 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:57.150861 28685 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:11:57.124223 28585 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:57.153090 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:57.153885 28585 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:57.122315 28508 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:57.183365 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:57.197867 22708 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:57.199170 22626 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:11:57.200300 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:57.201046 28649 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:11:57.180509 28585 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:57.204533 28397 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:57.205245 28397 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:57.205854 28397 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:57.211393 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60960.2us
I20260709 13:11:57.216588 23384 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:57.219115 28508 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.220512 28209 consensus_queue.cc:1059] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:57.221710 28209 raft_consensus.cc:3055] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:11:57.222187 28209 raft_consensus.cc:740] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:11:57.139019 22528 rpcz_store.cc:269] 0709 13:11:40.817109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:40.817443 (+   334us) service_pool.cc:224] Handling call
0709 13:11:41.191194 (+373751us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:11:57.137996 (+15946802us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:52.783746 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:53.055003 (+271257us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:53.057591 (+  2588us) write_op.cc:435] Acquired schema lock
0709 13:11:53.061054 (+  3463us) tablet.cc:662] Decoding operations
0709 13:11:53.142012 (+ 80958us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:53.142115 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:53.142143 (+    28us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:11:53.143694 (+  1551us) tablet.cc:701] Row locks acquired
0709 13:11:53.143717 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:11:53.143929 (+   212us) write_op.cc:270] Start()
0709 13:11:53.144076 (+   147us) write_op.cc:276] Timestamp: P: 1783602713143856 usec, L: 0
0709 13:11:53.144101 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:11:53.206457 (+ 62356us) log.cc:844] Serialized 7048 byte log entry
0709 13:11:53.506814 (+300357us) op_driver.cc:464] REPLICATION: finished
0709 13:11:53.755033 (+248219us) write_op.cc:301] APPLY: starting
0709 13:11:53.755128 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:53.917888 (+162760us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:53.918037 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:54.607218 (+689181us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:54.703475 (+ 96257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 13:11:54.709554 (+  6079us) write_op.cc:312] APPLY: finished
0709 13:11:54.729890 (+ 20336us) log.cc:844] Serialized 164 byte log entry
0709 13:11:54.983621 (+253731us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:55.000368 (+ 16747us) write_op.cc:454] Released schema lock
0709 13:11:55.002808 (+  2440us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:55.126875 (+124067us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18002da960. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:11:55.127010 (+   135us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:55.127037 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130387,"reported_tablets":28,"spinlock_wait_cycles":798420736,"child_traces":[["op",{"apply.queue_time_us":247911,"apply.run_cpu_time_us":48373,"apply.run_wall_time_us":1264743,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79511,"num_ops":18,"prepare.queue_time_us":255728,"prepare.run_cpu_time_us":22331,"prepare.run_wall_time_us":622300,"raft.queue_time_us":72919,"raft.run_cpu_time_us":58398,"raft.run_wall_time_us":223251,"replication_time_us":362523,"spinlock_wait_cycles":6102784,"thread_start_us":718472,"threads_started":5,"wal-append.queue_time_us":515740,"wal-append.run_cpu_time_us":10274,"wal-append.run_wall_time_us":2131808}]]}
I20260709 13:11:57.227643 28209 consensus_queue.cc:260] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.230293 28477 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.231937 28209 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:57.233776 28477 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.235327 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.236281 23415 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:57.228340 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.222209 28508 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.218914 28611 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:11:57.264038 22634 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
W20260709 13:11:57.266119 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47564.1us
I20260709 13:11:57.279081 22707 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
W20260709 13:11:57.242024 28602 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.835s	user 0.012s	sys 0.010s
W20260709 13:11:57.281643 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.288262 23421 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:57.289182 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3567) took 1376 ms. Trace:
I20260709 13:11:57.296530 22709 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:11:57.283571 28692 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.285634 28611 raft_consensus.cc:697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:57.289696 22707 rpcz_store.cc:276] 0709 13:11:55.909514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:55.910916 (+  1402us) service_pool.cc:224] Handling call
0709 13:11:57.022874 (+1111958us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b58001c5698. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:11:57.285357 (+262483us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3827584}
I20260709 13:11:57.325029 28508 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.325601 28611 consensus_queue.cc:237] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.326004 28508 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.328753 28508 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.328933 28692 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.329710 28692 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.332288 28692 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.333699 22598 leader_election.cc:400] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14.
I20260709 13:11:57.335947 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:57.323354 28029 consensus_queue.cc:1059] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:11:57.339910 28592 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:57.340732 28029 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:11:57.341166 28592 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:11:57.341609 28592 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:57.339560 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:57.341619 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:57.304402 28475 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:11:57.341866 28674 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:57.326359 22709 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49444 (request call id 859) took 1020 ms. Trace:
I20260709 13:11:57.347621 28737 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:57.348322 28737 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:57.348006 22709 rpcz_store.cc:276] 0709 13:11:56.305445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.347271 (+ 41826us) service_pool.cc:224] Handling call
0709 13:11:57.326039 (+978768us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400087d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:57.326265 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149622528}
I20260709 13:11:57.349540 28737 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.351605 28744 consensus_queue.cc:1059] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:57.353284 28742 raft_consensus.cc:3055] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:57.353726 28742 raft_consensus.cc:740] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:57.354897 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.299777 28628 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:11:57.367304 22784 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:11:57.341194 28029 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:57.361523 28692 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:57.384554 22709 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:57.388378 28508 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:57.394126 28508 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.397956 28508 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.398780 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:57.399516 28752 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.400122 22701 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:11:57.400309 28752 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.369539 28742 consensus_queue.cc:260] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.412078 28742 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:57.383262 28692 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:57.437662 22709 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:11:57.410112 28605 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.463382 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.360289 28592 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.454401 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:57.467029 22708 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:11:57.443513 28692 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:57.469913 22708 pending_rounds.cc:85] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 7
I20260709 13:11:57.479912 22708 pending_rounds.cc:107] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:11:57.498906 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:57.499750 22635 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:57.498983 22708 raft_consensus.cc:2889] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:57.484962 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:57.455003 28508 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.473915 28752 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.514071 22701 raft_consensus.cc:1240] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 13:11:57.537894 28609 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.538846 28609 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.541641 28609 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:11:57.543443 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58476.1us
I20260709 13:11:57.545580 23384 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:57.546645 28668 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:57.547148 28747 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:57.547914 28747 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:57.549372 28747 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.551215 28508 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.530187 28467 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.516778 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:57.524322 28681 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.607447 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.605436 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:57.559242 28570 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.619640 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:57.631711 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86270.9us
I20260709 13:11:57.633548 22635 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:57.595283 28752 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:57.640470 22783 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:57.600276 28508 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:57.644150 28704 consensus_meta.cc:287] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.005s
W20260709 13:11:57.666280 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 109488us
I20260709 13:11:57.667972 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:57.671981 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.603473 28467 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.699664 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:57.649444 28681 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.709044 22703 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:11:57.719429 28757 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:11:57.721910 28681 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.723614 28762 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.724342 28762 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.730998 22708 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:11:57.732594 22701 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:57.740229 28029 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.741634 28029 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:57.730742 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:57.686722 28752 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.758299 22710 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:57.758927 22710 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:57.758162 28681 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:57.760066 28752 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.759977 22710 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:57.760984 28762 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:57.744096 28467 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:57.770840 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:57.771852 22708 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:11:57.775437 28578 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:11:57.778795 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:11:57.787125 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 190406us
I20260709 13:11:57.788640 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:57.788759 28396 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:11:57.761116 22710 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:57.798913 22783 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:11:57.761664 22526 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:57.750094 28610 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.686s	user 0.010s	sys 0.007s
I20260709 13:11:57.808146 28761 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:11:57.809301 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
W20260709 13:11:57.815771 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 165035us
I20260709 13:11:57.829455 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:11:57.830807 22629 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:11:57.823872 22526 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 9 to 11, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:57.787185 28570 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:57.856159 22776 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:57.824822 28610 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:57.861734 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:57.828001 28762 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:57.852766 28752 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:57.855772 28761 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:57.857656 28602 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:57.874910 22526 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 10 to 13, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:57.907840 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 155994us
I20260709 13:11:57.830794 28396 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.873027 28668 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:57.934165 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:57.864195 28628 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:57.935420 22526 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:57.972638 22526 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:57.950989 28720 log.cc:927] Time spent T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.077s	user 0.003s	sys 0.003s
I20260709 13:11:57.875612 28692 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:57.949848 28753 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:57.927788 28762 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:58.003999 28585 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.801s	user 0.017s	sys 0.000s
I20260709 13:11:57.946324 28752 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:58.025605 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 90812.1us
W20260709 13:11:58.029246 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 174989us
I20260709 13:11:57.966670 28602 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:58.032434 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57137.3us
W20260709 13:11:58.036943 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.038129 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.038297 28761 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.039739 22526 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:58.051579 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102404us
I20260709 13:11:58.050920 28668 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
W20260709 13:11:58.059996 28759 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:11:58.060570 28775 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.064625 28696 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:58.068301 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:57.990548 28628 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.073269 22635 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 13:11:58.045313 28649 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:58.088344 22635 pending_rounds.cc:85] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting all ops after (but not including) 13
I20260709 13:11:58.093789 22635 pending_rounds.cc:107] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:11:58.094231 22635 raft_consensus.cc:2889] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.062425 28592 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:58.099128 23416 raft_consensus.cc:1240] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:58.027253 28585 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.096616 22710 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:11:58.101303 22710 pending_rounds.cc:85] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 10
I20260709 13:11:58.109007 22710 pending_rounds.cc:107] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:11:58.109542 22710 raft_consensus.cc:2889] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.034353 28194 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.229s
I20260709 13:11:58.121130 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:58.113507 28742 consensus_meta.cc:287] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.701s	user 0.017s	sys 0.000s
I20260709 13:11:58.123807 22780 raft_consensus.cc:2368] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14.
I20260709 13:11:58.127880 22526 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:58.119336 28766 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:11:58.111254 28649 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.131723 28779 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:58.132902 28747 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:58.133525 28649 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:58.137463 22778 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 3400) took 1006 ms. Trace:
W20260709 13:11:58.138808 28758 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 13:11:58.139117 28768 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:11:58.138018 22778 rpcz_store.cc:276] 0709 13:11:57.130329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.130754 (+   425us) service_pool.cc:224] Handling call
0709 13:11:57.131228 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.132346 (+  1118us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:11:57.370961 (+238615us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.634658 (+263697us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.662451 (+ 27793us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.676831 (+ 14380us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:11:57.677068 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.677713 (+   645us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.136798 (+459085us) raft_consensus.cc:1685] finished
0709 13:11:58.136824 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.137002 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23057,"prepare.queue_time_us":194706,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":28,"raft.queue_time_us":29600,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":6100,"thread_start_us":414487,"threads_started":2,"wal-append.queue_time_us":406150}
W20260709 13:11:58.141960 28769 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:11:58.102785 28744 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:58.143018 28744 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.145694 28744 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.095429 28775 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.155753 28692 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.157138 28775 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:58.157823 28752 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:58.158497 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.159651 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.160463 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.161134 23421 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:58.157048 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:11:58.152320 28397 consensus_meta.cc:287] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.898s	user 0.017s	sys 0.001s
I20260709 13:11:58.162060 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.162328 28397 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.163275 28602 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.164866 28397 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.165004 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.147429 28762 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.173872 23412 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:58.176849 28396 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.180284 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:58.185729 22673 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:58.157163 22526 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:58.094445 28696 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.195950 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.197510 28696 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.196866 22526 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 15, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:58.198108 28751 raft_consensus.cc:476] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Not starting pre-election -- already a leader
I20260709 13:11:58.193214 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.195950 28776 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:58.204562 28776 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:58.205281 28696 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:11:58.205667 28776 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.205327 28612 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.207381 28612 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.207922 28674 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:11:58.208500 28668 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.210500 28612 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:58.192234 28585 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:58.220625 28272 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:58.222080 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:58.224885 22780 raft_consensus.cc:2368] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18.
I20260709 13:11:58.222266 28272 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:58.138028 28628 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.212040 22526 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 11 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:58.234683 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 199447us
I20260709 13:11:58.227941 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:11:58.235524 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 140536us
I20260709 13:11:58.244978 28605 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.192313 28762 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:11:58.226120 28272 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:58.258236 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.262110 28272 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.263808 28272 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:58.263856 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:58.247890 28762 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.265865 28762 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:58.252662 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:58.269698 22526 catalog_manager.cc:5697] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:11:58.276208 28771 log.cc:927] Time spent T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.237s	user 0.002s	sys 0.003s
I20260709 13:11:58.282037 22775 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:58.287667 28467 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:58.288300 28467 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.288658 28467 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:58.327847 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:58.333128 22526 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 12 to 15, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:58.312491 28397 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.345328 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.345402 28784 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:58.346508 28751 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.363126 28751 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.363765 22780 raft_consensus.cc:1275] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:11:58.365566 22631 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:11:58.365715 28751 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.367275 28628 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:11:58.377480 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.383562 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.383562 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.385805 28397 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.389510 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.388739 22780 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:58.390177 22777 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:11:58.390893 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:58.403949 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 44608.1us
I20260709 13:11:58.417168 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.426389 22526 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:11:58.425870 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:58.402201 28784 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.460282 22627 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:58.462149 22703 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:11:58.464344 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61869.9us
I20260709 13:11:58.469094 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.470436 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.471429 22702 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:58.405630 28611 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 13:11:58.478781 28784 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.480674 28397 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.481434 28784 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.481235 22526 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 4 to 5, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:58.483903 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:58.487224 22629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45666 (request call id 3040) took 2261 ms (client timeout 1542 ms). Trace:
W20260709 13:11:58.487725 22629 rpcz_store.cc:269] 0709 13:11:56.225846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:56.226000 (+   154us) service_pool.cc:224] Handling call
0709 13:11:58.486979 (+2260979us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540022db54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:11:58.487126 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3317001984}
I20260709 13:11:58.490715 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:58.491101 28766 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:11:58.502532 22775 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:58.498811 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:58.526439 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.528137 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.534277 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.536954 22600 leader_election.cc:400] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14.
I20260709 13:11:58.537560 22600 leader_election.cc:403] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:11:58.538971 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.541424 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.544922 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.549034 22600 leader_election.cc:400] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18.
I20260709 13:11:58.549626 22600 leader_election.cc:403] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:11:58.558876 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:58.564849 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66246.9us
I20260709 13:11:58.568095 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.540931 28668 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:11:58.573844 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.578373 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.580636 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.582242 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.583079 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:58.583516 28764 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.583s	user 0.001s	sys 0.002s
I20260709 13:11:58.563499 28628 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:58.584745 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.585515 22781 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:11:58.585660 22784 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42226 (request call id 961) took 1092 ms. Trace:
I20260709 13:11:58.586946 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.583973 22780 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:11:58.589588 23385 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:58.563894 28605 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14.
I20260709 13:11:58.599649 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:11:58.600138 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.602910 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.603893 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.604884 22775 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:11:58.607322 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.601881 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.611485 23411 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:11:58.608237 22626 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 21.
I20260709 13:11:58.571338 28668 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:58.612076 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.615928 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.617684 22780 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:11:58.586141 22784 rpcz_store.cc:276] 0709 13:11:57.493129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:57.502468 (+  9339us) service_pool.cc:224] Handling call
0709 13:11:57.519397 (+ 16929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:11:57.592627 (+ 73230us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:11:57.623073 (+ 30446us) log.cc:844] Serialized 14 byte log entry
0709 13:11:57.716061 (+ 92988us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:11:57.830463 (+114402us) log.cc:844] Serialized 26 byte log entry
0709 13:11:57.831939 (+  1476us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:11:57.831999 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:11:57.832106 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:11:58.572509 (+740403us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:11:58.585270 (+ 12761us) raft_consensus.cc:1685] finished
0709 13:11:58.585412 (+   142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:11:58.585571 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11987,"prepare.queue_time_us":33154,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":33452,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":43888,"spinlock_wait_cycles":4644096,"thread_start_us":92132,"threads_started":1,"wal-append.queue_time_us":250996}
I20260709 13:11:58.622502 28424 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.623162 28424 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.625808 28424 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:11:58.631575 28795 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:11:58.637470 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.608894 28780 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:11:58.587419 28477 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:11:58.621277 28668 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:58.688211 28784 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:58.689806 28751 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.690472 28751 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.692925 28751 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.693570 28751 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:58.689018 28784 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.665692 28737 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:58.700481 28737 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.701460 28762 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.704023 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.705345 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:58.706252 22635 raft_consensus.cc:606] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:58.706671 28737 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:58.717638 23418 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:11:58.694089 28751 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.722066 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.722532 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.724025 28779 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:58.724318 28784 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.725754 28751 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.727717 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.683143 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:11:58.730229 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47242.1us
W20260709 13:11:58.708851 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.730775 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:58.708740 28761 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:58.710114 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:58.731482 22626 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
W20260709 13:11:58.731587 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 94791us
I20260709 13:11:58.754631 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.755964 23411 raft_consensus.cc:2393] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 12: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:58.745626 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:58.767530 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.764223 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:58.756673 28751 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.789157 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.761076 28696 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:11:58.809372 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.809629 28777 consensus_queue.cc:1048] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:11:58.811563 28805 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:58.821373 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:58.825373 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:58.784497 28508 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:58.817756 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:58.822121 22709 raft_consensus.cc:2376] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16. Re-sending same reply.
I20260709 13:11:58.819401 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.833813 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.834570 22707 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:11:58.802539 28696 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.761179 28761 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.819644 28773 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:58.848917 28762 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.855432 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.855755 22777 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
W20260709 13:11:58.839638 28467 consensus_meta.cc:287] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.003s
I20260709 13:11:58.857453 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:58.863437 23418 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:58.869989 22706 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:11:58.845909 28696 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:58.846155 28585 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:58.867074 28685 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:58.903371 28685 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.868292 28467 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.907591 22630 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:11:58.908890 28467 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:58.910351 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:58.911795 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.912415 22709 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:11:58.913257 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.917758 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.920759 22709 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:11:58.922945 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.923694 22704 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:11:58.929396 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:58.930575 28773 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.931102 28773 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:58.936079 28747 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:58.936475 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.812350 28805 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:58.938867 28759 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:11:58.940750 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.839835 28508 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:58.942736 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:58.840380 28751 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.914983 28585 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:58.948446 28585 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:58.953632 22703 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 17. (term mismatch)
I20260709 13:11:58.954150 22703 pending_rounds.cc:85] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 16
I20260709 13:11:58.954404 22703 pending_rounds.cc:107] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 20.17
I20260709 13:11:58.954663 22703 raft_consensus.cc:2889] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:11:58.906376 28685 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:58.985082 28668 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:58.987655 23411 raft_consensus.cc:1240] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18. Ops: []
I20260709 13:11:58.995173 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:58.996006 23411 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:11:58.998006 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:59.019155 28761 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.019381 28591 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:11:59.019807 28591 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:59.055919 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.017952 28628 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18.
I20260709 13:11:59.062312 28591 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:11:59.120703 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 328534us
I20260709 13:11:59.124881 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.127449 23385 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:11:59.128124 28762 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:11:59.129849 28696 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.137878 22626 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:11:59.147181 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.150058 23418 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:11:59.151659 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 367959us
I20260709 13:11:59.132048 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.157084 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.168701 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:11:59.169279 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:59.169787 22601 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.173054 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.170364 22633 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 9.
I20260709 13:11:59.177790 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:59.182152 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.189026 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.133013 28685 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:59.199640 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.152633 28767 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:59.171130 28592 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:11:59.221055 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73962.9us
I20260709 13:11:59.187656 28779 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.189275 28784 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:11:59.189090 28737 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:59.198454 28783 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:59.221283 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51585.5us
I20260709 13:11:59.205664 28692 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.230223 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:11:59.211978 28764 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:11:59.194278 28786 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:59.153882 28029 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:11:59.197296 28776 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:59.193485 28804 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:59.213115 28767 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.233717 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.236928 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:59.264902 22782 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:11:59.250679 28784 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:11:59.211076 28685 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:11:59.281901 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91491.9us
I20260709 13:11:59.288367 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:11:59.288955 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 53470.1us
I20260709 13:11:59.242669 28761 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:59.250437 28767 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:59.237489 28811 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:59.267432 28779 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:59.318877 22634 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:11:59.319731 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:59.291561 28508 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.311338 28696 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.311234 28786 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:59.440037 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:59.319964 28668 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:59.342900 28751 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:59.356412 28808 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
W20260709 13:11:59.419361 28825 log.cc:927] Time spent T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 13:11:59.359488 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 137130us
I20260709 13:11:59.369007 28824 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:11:59.377936 28773 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.385282 22635 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:11:59.389559 22782 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:11:59.417775 22629 raft_consensus.cc:2410] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:11:59.458271 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.253199 28776 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.446776 28761 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:59.460266 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.464892 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:59.468959 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.474164 28785 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:11:59.474853 28824 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:59.475142 28785 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:11:59.475683 28785 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:59.476135 28785 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:11:59.476357 28824 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.429011 28461 consensus_queue.cc:1059] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:11:59.480224 28685 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:59.483341 28773 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.451741 28786 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.493824 22631 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:11:59.503453 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:59.522864 22629 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:59.523746 22629 raft_consensus.cc:606] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.524147 22629 raft_consensus.cc:614] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:59.438849 28585 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:11:59.527652 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 96568.6us
I20260709 13:11:59.551699 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:59.552567 22782 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:11:59.553704 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:11:59.557364 28817 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:11:59.561163 28272 raft_consensus.cc:993] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:59.561861 28272 raft_consensus.cc:1081] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:11:59.564819 28816 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:11:59.529558 28592 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.574746 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:59.579455 23411 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 9.
I20260709 13:11:59.580142 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:59.583391 22630 raft_consensus.cc:606] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:11:59.584914 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116064us
I20260709 13:11:59.575963 28803 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:11:59.592131 28803 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:59.593119 28803 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.602895 28761 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.610029 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:59.614157 28778 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:11:59.614701 28778 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:59.614984 22750 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.615073 28778 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:11:59.619174 22777 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:11:59.574520 28477 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:11:59.628067 28831 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:11:59.631788 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:11:59.594568 28803 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:59.649194 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:59.649313 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:59.651743 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:11:59.657020 22776 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 19: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:11:59.648138 28591 consensus_meta.cc:287] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
I20260709 13:11:59.646136 28029 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:11:59.667217 28029 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.668347 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:11:59.650703 28786 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.580704 28784 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.695526 22777 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:11:59.702795 22709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:59.705399 28785 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.714135 28785 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:59.718988 22780 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 14. Current term is 15. Ops: []
I20260709 13:11:59.735296 28797 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:11:59.736418 28066 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.737296 28066 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.736176 28797 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.689805 28805 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.746124 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:59.762557 28424 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:11:59.763381 28424 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:59.764413 28424 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.765805 28424 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:59.767385 22749 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.768515 28585 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:59.769265 28585 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:11:59.769845 28585 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:59.764060 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:11:59.775174 22600 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:59.777390 28029 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:59.673151 28591 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.779376 22781 raft_consensus.cc:2368] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:11:59.779759 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:11:59.780117 28591 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:11:59.780488 23417 raft_consensus.cc:1813] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.781077 23417 raft_consensus.cc:2393] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 18: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:11:59.781558 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:59.783372 28066 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.783866 22635 raft_consensus.cc:1813] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Handling vote request from an unknown peer a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.784238 28776 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.784598 22635 raft_consensus.cc:2410] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:11:59.762558 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:59.789608 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:11:59.790360 23418 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:11:59.792058 22633 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 19: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:11:59.804750 22630 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:11:59.803879 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:11:59.806901 22783 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:11:59.750674 28797 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:11:59.811487 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.000s	sys 0.022s
W20260709 13:11:59.817304 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.001s	sys 0.023s
W20260709 13:11:59.818564 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 584908us
I20260709 13:11:59.819645 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.802917 28461 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:11:59.818639 28477 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:11:59.826924 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.832890 28783 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:59.833262 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.834749 22709 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.835224 22709 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:59.837550 28838 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.838109 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.839869 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.839870 28838 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:59.840984 28838 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:59.841521 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.809938 28811 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:11:59.814477 28786 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:59.747026 28767 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.829488 28779 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:59.830289 22750 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.982s (SENT) [suppressed 12 similar messages]
I20260709 13:11:59.855461 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.852166 22709 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.859264 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.870448 23418 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:11:59.874017 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:59.875077 22635 raft_consensus.cc:606] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.875569 22635 raft_consensus.cc:614] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:11:59.877034 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:59.801120 28585 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.860127 28783 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:11:59.878978 28591 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:11:59.879768 28591 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:11:59.881024 28591 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.844012 28461 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:11:59.883682 28461 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.886735 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.870070 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.896453 23411 raft_consensus.cc:2368] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:59.858963 28784 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:11:59.903031 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.904282 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.906067 28826 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:11:59.905189 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.886735 28776 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:59.916775 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86531.1us
I20260709 13:11:59.912726 28628 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:11:59.919024 28779 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:59.919266 22709 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.919584 28779 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.920085 28779 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:59.920881 28773 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:11:59.923501 28696 raft_consensus.cc:993] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:59.897528 28783 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:11:59.923698 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:11:59.924690 28696 raft_consensus.cc:1081] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:11:59.925971 28696 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:11:59.926381 28696 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:59.857878 28786 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.926771 28696 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:11:59.928784 28786 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.931603 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:11:59.932060 28767 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:11:59.932343 22630 raft_consensus.cc:606] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:11:59.936756 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.938547 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.940014 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:59.941006 22630 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:11:59.943820 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:11:59.954581 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:11:59.957149 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:59.957932 22702 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:11:59.937731 28776 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:59.965739 23385 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Timed out: RequestConsensusVote RPC to 127.21.5.130:34681 timed out after 1.991s (SENT)
I20260709 13:11:59.943020 28784 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:59.949796 28838 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:11:59.974858 28508 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:59.967986 22598 leader_election.cc:400] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:11:59.968892 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:59.976804 22598 leader_election.cc:403] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:11:59.980135 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:59.982376 28778 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.951538 28751 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:11:59.990015 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:11:59.969154 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:11:59.948249 28773 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:11:59.992116 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:11:59.992810 22703 raft_consensus.cc:2393] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:11:59.992690 28773 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:11:59.993552 28475 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:11:59.992810 22706 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:59.994022 28475 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:11:59.994283 22706 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:11:59.995129 28773 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:11:59.995087 28475 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:11:59.996665 28475 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:00.003077 28803 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:00.003822 28803 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.007277 28778 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.011291 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.018704 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.019815 22782 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:12:00.022677 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.030340 28762 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:00.032025 28508 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:00.032580 28508 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:00.034468 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:00.035487 28751 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:00.036082 28751 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:00.037101 28751 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.040781 28761 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:00.034022 28508 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.041787 28761 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.048463 28605 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:00.049187 28605 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:00.049676 28605 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:00.049768 28751 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:00.056164 28803 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.061807 28508 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:00.062975 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.063882 22705 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:00.064888 28826 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:00.068341 22705 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:00.069530 28761 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.074191 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:00.062745 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 105860us
W20260709 13:12:00.087823 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61785.7us
I20260709 13:12:00.089293 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.091624 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.093823 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.094570 28605 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.095626 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.095606 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.096784 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.098098 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.100700 22630 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:00.103649 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.104523 22702 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:00.103969 22631 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:12:00.090448 22706 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.108260 22776 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 13:12:00.108568 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.099251 22750 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:00.111043 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.114145 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.118170 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.118011 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.118914 22780 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.120369 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:00.121474 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.122087 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.122855 22630 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:12:00.123451 22783 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 10.
I20260709 13:12:00.118032 23384 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:00.124778 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.125092 22776 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:00.125177 22782 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:00.125792 22706 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.123492 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.123956 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.108117 28397 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:12:00.128248 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.130370 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.130901 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.125406 22784 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:12:00.128854 22706 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:00.135133 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.135452 22675 leader_election.cc:400] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:12:00.135882 22675 leader_election.cc:403] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:00.136454 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.138511 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:00.139595 22633 raft_consensus.cc:2393] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:00.141471 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.142163 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.142845 22635 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:00.143234 22635 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:00.145428 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.146371 22709 raft_consensus.cc:2393] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 10: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:00.146792 28605 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.150698 28762 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:00.151016 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.117724 28591 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:00.115317 28424 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:00.154702 28591 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.155252 28591 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.157794 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.158605 22627 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:00.141255 28824 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:00.159914 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.160277 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.145483 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.161504 28475 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:00.117355 28668 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 13:12:00.169090 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.171033 28475 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:00.171886 28778 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:00.172184 28475 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.170461 22706 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:00.170233 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:00.176249 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.141027 28761 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:12:00.141000 28830 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:00.178365 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.179186 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.182415 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.182473 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.183228 22627 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:00.183504 22627 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:00.186681 22780 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:00.188076 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.180590 22635 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.190142 22635 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.180285 28762 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:00.190267 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:00.193637 28737 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:00.196283 28779 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.196779 28762 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.191905 22627 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.198830 28762 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.200713 28783 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:00.201193 28783 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:00.201541 28783 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:00.187338 28776 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:00.202633 28578 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:00.203346 28578 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.189370 28830 raft_consensus.cc:697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:00.204835 28830 consensus_queue.cc:237] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.205619 28578 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:00.208309 22705 raft_consensus.cc:2410] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:12:00.212767 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.213725 22703 raft_consensus.cc:2393] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:00.214088 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.217038 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:00.198906 28784 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:00.224251 28696 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.227087 22631 raft_consensus.cc:2368] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:00.186044 28838 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:00.241079 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:00.244683 22635 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:00.246867 28779 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:00.251752 28696 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:00.254886 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:00.280150 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.280809 22781 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.258919 22627 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.283543 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:00.260923 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.270714 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:00.290372 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 100187us
I20260709 13:12:00.291337 22782 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.292846 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.296315 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.297091 22631 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:00.307420 22675 leader_election.cc:400] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:00.321256 22675 leader_election.cc:403] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:00.295744 28779 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:00.324613 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:00.328302 22706 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:12:00.329394 28850 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:00.330071 28850 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:00.332434 23411 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:00.336588 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:00.336853 28831 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:00.333667 28850 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.338809 28785 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:12:00.339409 22784 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:00.340626 28780 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:12:00.341270 28838 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.341513 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.341696 28780 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:12:00.342919 28780 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:00.343931 28780 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.345484 28780 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:00.349012 22635 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:12:00.348869 28751 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:00.276494 28591 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.307087 28508 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:00.359831 22782 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:00.364269 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.352087 28783 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.307529 28396 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:00.411334 22709 raft_consensus.cc:2393] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 17: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:00.412410 28783 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.412907 22705 raft_consensus.cc:1240] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:00.321097 28824 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:12:00.414278 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:00.415683 28396 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:00.416307 28396 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:00.377013 22776 raft_consensus.cc:2368] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16.
I20260709 13:12:00.357931 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:00.379740 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.419014 22635 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.419767 28783 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:12:00.420998 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 63241.9us
I20260709 13:12:00.418306 28396 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.424278 22600 leader_election.cc:400] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16.
I20260709 13:12:00.424659 22600 leader_election.cc:403] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:12:00.388054 28762 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.430194 28762 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:00.362636 28029 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:12:00.432510 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:00.382066 28851 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:12:00.378198 28784 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:00.435520 22701 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:12:00.375787 28591 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:00.440538 28784 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:00.441265 28784 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.444212 28784 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:00.444725 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.399371 28761 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 12. Current term is 13.
I20260709 13:12:00.448002 28824 consensus_queue.cc:1059] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:00.449069 28824 raft_consensus.cc:3055] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:00.449398 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:00.449445 28824 raft_consensus.cc:740] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:00.450105 22701 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:00.450765 28824 consensus_queue.cc:260] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.452167 28824 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:00.461701 22704 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:00.467907 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.476374 28585 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:00.477874 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:00.478566 22634 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.479022 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:00.479899 22629 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:00.486929 22781 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:00.487928 22635 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:00.496973 22627 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:00.515497 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:00.516217 22781 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:00.505304 22634 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:00.514551 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:00.517524 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:00.518790 22784 raft_consensus.cc:2410] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:12:00.520964 22709 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:12:00.521358 22709 pending_rounds.cc:85] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 6
I20260709 13:12:00.521585 22709 pending_rounds.cc:107] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:12:00.521871 22709 raft_consensus.cc:2889] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:00.516412 22749 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:00.527263 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.538265 22701 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:00.519623 28810 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16.
I20260709 13:12:00.509789 28826 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:12:00.525041 28508 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:00.525518 28838 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:00.541081 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:00.530112 28592 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:00.531834 28850 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:00.560179 23411 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:00.565649 22781 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:00.613870 22782 raft_consensus.cc:2410] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
W20260709 13:12:00.614823 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73893.4us
I20260709 13:12:00.586211 28592 raft_consensus.cc:697] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:00.589305 28508 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:00.625126 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:00.583324 28784 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:12:00.619179 28804 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:00.642023 22707 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:00.647426 28592 consensus_queue.cc:237] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.640475 28762 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:00.657500 28762 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:00.658664 28762 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:00.662242 28397 raft_consensus.cc:993] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:00.691152 28508 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:00.674010 28209 raft_consensus.cc:1081] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:00.736882 28850 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:00.767607 28852 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.187s	user 0.003s	sys 0.000s
I20260709 13:12:00.772385 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:00.774910 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 137225us
I20260709 13:12:00.780478 28784 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:12:00.803103 28853 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.223s	user 0.000s	sys 0.003s
W20260709 13:12:00.804267 28858 log.cc:927] Time spent T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.223s	user 0.002s	sys 0.000s
I20260709 13:12:00.823881 22630 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:00.847189 22703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:00.848165 22703 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:00.848716 22703 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:00.866498 22528 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:00.846592 28850 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 13:12:00.870839 28855 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.291s	user 0.000s	sys 0.005s
I20260709 13:12:00.992473 22777 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:12:00.872146 28859 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
I20260709 13:12:00.880422 22782 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
W20260709 13:12:00.980021 28860 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.121s	user 0.002s	sys 0.000s
I20260709 13:12:00.996622 28784 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:00.985854 28804 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:00.856076 28674 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:12:01.014259 22702 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:12:01.016269 22703 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.022010 22709 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:01.047775 22703 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:01.063449 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:01.064467 23417 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:01.038321 28477 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:12:01.061277 28397 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.060288 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:01.094043 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 222620us
I20260709 13:12:01.087875 22528 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 5 to 9, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.054502 28805 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:12:01.131974 22528 catalog_manager.cc:5697] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 17, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.135074 22528 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: config changed from index 5 to 11, term changed from 13 to 17, VOTER a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) evicted. New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.137948 22528 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 18 to 20, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 20 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.113642 28397 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.153316 28397 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:01.124570 28668 consensus_queue.cc:1059] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:12:01.169006 28668 consensus_queue.cc:1243] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer 91ebb78bf12f41baae84669c1effc423 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:12:01.174400 23411 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:01.179297 23411 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:01.180428 23411 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.195207 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:01.196372 22703 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:01.199247 22707 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:01.200567 23411 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:01.200865 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:01.201854 22631 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:01.202309 22631 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:01.205789 22634 raft_consensus.cc:1275] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:12:01.207448 22777 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:12:01.134138 28737 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.218551 23417 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:01.221823 22673 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:01.222918 28508 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:01.223738 28508 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:01.225463 28508 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.234392 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:01.217803 22627 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:01.145412 22528 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.234651 22708 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:01.238057 22634 raft_consensus.cc:1240] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:01.241385 22528 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.260390 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:01.273721 22631 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.287274 23417 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:01.250970 28461 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:12:01.303200 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 316083us
I20260709 13:12:01.304638 22781 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:01.295300 22528 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:01.316877 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:01.247555 28737 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:01.315634 22528 catalog_manager.cc:5697] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 22. New cstate: current_term: 22 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:01.334995 22528 catalog_manager.cc:5697] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:01.341813 22775 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:12:01.353403 22775 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:01.299731 28461 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.358240 22635 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:01.358420 22634 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:01.355542 22528 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:01.371047 22528 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.373940 22528 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 4 to 13. New cstate: current_term: 13 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:01.382126 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 327252us
I20260709 13:12:01.296824 28850 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:12:01.388302 22528 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:12:01.389569 22526 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 239) took 22507 ms (client timeout 15000 ms). Trace:
I20260709 13:12:01.344945 28810 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:12:01.371452 28461 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:01.332831 28805 consensus_queue.cc:1048] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:12:01.395601 22528 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 9 to 13, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:01.399194 28838 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.380970 28863 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:12:01.442538 28876 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:01.445618 28508 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:01.418851 28797 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:12:01.450558 28737 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:01.465360 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 65199.5us
W20260709 13:12:01.475818 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 65783.1us
I20260709 13:12:01.402087 28762 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 13:12:01.478241 28838 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:01.516434 28797 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.517120 28797 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:01.520684 28875 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:12:01.523397 28863 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:01.535049 28838 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:01.549676 28461 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.552335 22631 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:01.437683 28784 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:01.576858 23411 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 13:12:01.591634 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 169164us
I20260709 13:12:01.598199 23411 pending_rounds.cc:85] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 8
I20260709 13:12:01.615567 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:01.577832 28784 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:01.633816 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 234772us
I20260709 13:12:01.612643 23411 pending_rounds.cc:107] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:12:01.636330 23411 raft_consensus.cc:2889] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:01.634335 22635 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:01.643479 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:01.651423 22702 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:01.682499 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:01.683445 22634 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
W20260709 13:12:01.688632 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52714.2us
I20260709 13:12:01.689023 28885 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:01.690128 28885 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:01.694216 28885 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:01.703150 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:01.704147 22777 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:01.704877 28784 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:01.686900 28830 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:12:01.633886 22631 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:01.719516 22706 raft_consensus.cc:1275] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:01.739319 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:01.743876 28877 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:01.744511 28877 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.744916 28877 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:01.747434 28797 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:01.752403 28889 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.099s	user 0.001s	sys 0.001s
I20260709 13:12:01.763408 28877 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:01.764389 28461 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:01.767689 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:01.768643 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 44957.2us
I20260709 13:12:01.771324 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:01.768486 22776 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:01.769722 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:01.772223 23418 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:01.776245 28784 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:01.776031 28761 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:12:01.776767 28784 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:01.777252 28784 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:01.778213 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:01.778923 23421 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:01.781035 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:01.786136 22778 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:01.800971 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:01.801762 22702 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
W20260709 13:12:01.806025 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 368973us
I20260709 13:12:01.806116 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:01.394408 22526 rpcz_store.cc:269] 0709 13:11:38.878931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:38.879284 (+   353us) service_pool.cc:224] Handling call
0709 13:12:01.380196 (+22500912us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:59.543194 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:59.566350 (+ 23156us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:59.566418 (+    68us) write_op.cc:435] Acquired schema lock
0709 13:11:59.566430 (+    12us) tablet.cc:662] Decoding operations
0709 13:11:59.570495 (+  4065us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:59.570639 (+   144us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:59.570669 (+    30us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:11:59.624549 (+ 53880us) tablet.cc:701] Row locks acquired
0709 13:11:59.646413 (+ 21864us) write_op.cc:260] PREPARE: finished
0709 13:11:59.670833 (+ 24420us) write_op.cc:270] Start()
0709 13:11:59.671033 (+   200us) write_op.cc:276] Timestamp: P: 1783602719664394 usec, L: 0
0709 13:11:59.671078 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:11:59.757719 (+ 86641us) log.cc:844] Serialized 5539 byte log entry
0709 13:11:59.987669 (+229950us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed8e28 00007f180a913581 00007f180a912d58 00007f180a9179f1 00007f180a9179a9 00007f180a917939 00007f180a917901 00007f180a916b2d 000000000038f354 000000000038f199
0709 13:12:00.092594 (+104925us) op_driver.cc:464] REPLICATION: finished
0709 13:12:00.327280 (+234686us) write_op.cc:301] APPLY: starting
0709 13:12:00.327407 (+   127us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:00.367118 (+ 39711us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:00.367164 (+    46us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:00.402562 (+ 35398us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:00.402986 (+   424us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=0,mrs_lookups=0
0709 13:12:00.403060 (+    74us) write_op.cc:312] APPLY: finished
0709 13:12:00.403567 (+   507us) log.cc:844] Serialized 129 byte log entry
0709 13:12:00.404919 (+  1352us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:00.405138 (+   219us) write_op.cc:454] Released schema lock
0709 13:12:00.405237 (+    99us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:00.536025 (+130788us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:00.539298 (+  3273us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":418803,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":212024,"apply.run_cpu_time_us":22155,"apply.run_wall_time_us":191769,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62759,"num_ops":14,"prepare.queue_time_us":858735,"prepare.run_cpu_time_us":24047,"prepare.run_wall_time_us":480588,"raft.queue_time_us":180254,"raft.run_cpu_time_us":9675,"raft.run_wall_time_us":100042,"replication_time_us":421272,"spinlock_wait_cycles":7241600,"thread_start_us":694898,"threads_started":4,"wal-append.queue_time_us":144025,"wal-append.run_cpu_time_us":33053,"wal-append.run_wall_time_us":1506443}]]}
I20260709 13:12:01.851202 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:01.852166 22631 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:01.874862 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 69735.2us
I20260709 13:12:01.885637 22631 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
W20260709 13:12:01.989557 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 182533us
W20260709 13:12:01.994171 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 116640us
I20260709 13:12:01.995411 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:01.992249 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 186289us
I20260709 13:12:01.856124 28885 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:01.864737 28785 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.020722 28785 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:01.998564 28877 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:01.862529 28029 consensus_queue.cc:1048] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.163s
I20260709 13:12:02.020108 28885 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.024127 28907 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:02.064254 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 238610us
W20260709 13:12:02.110987 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 113359us
I20260709 13:12:02.049860 28885 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:02.162932 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.019s	sys 0.019s
W20260709 13:12:02.163921 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.019s	sys 0.019s
W20260709 13:12:02.164467 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 562309us
I20260709 13:12:02.069002 28838 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:02.184471 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:02.192216 22631 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:02.191823 28838 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:02.221597 28838 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:02.202009 28785 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:02.065233 28907 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.214689 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:02.217811 28903 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 13:12:02.219045 28902 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:12:02.247642 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 252302us
I20260709 13:12:02.258665 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:02.259688 22782 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:02.259689 23412 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
W20260709 13:12:02.268692 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 63764.2us
I20260709 13:12:02.072230 28909 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.147s
I20260709 13:12:02.242738 23421 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
W20260709 13:12:02.297008 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 305934us
W20260709 13:12:02.300953 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86400.5us
I20260709 13:12:02.301856 22749 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:02.321828 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:02.322861 22708 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:12:02.327665 28886 log.cc:927] Time spent T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 13:12:02.326826 28892 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:02.331538 28892 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.332003 28892 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:02.369874 28918 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.370831 28918 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.373713 28918 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:02.375347 28922 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:12:02.354130 28784 consensus_meta.cc:287] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.576s	user 0.017s	sys 0.000s
I20260709 13:12:02.351485 28878 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:02.360333 28924 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.390674 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:02.391531 22702 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:02.398981 28907 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:02.386673 28784 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.405467 28885 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.407116 28784 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:02.408100 28885 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:02.416275 22708 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:02.425503 28908 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:02.426333 28908 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.429229 28908 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:02.398360 28066 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:02.434307 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:02.441923 22709 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:02.445056 22598 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:02.449556 28914 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:02.450325 28914 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:02.451399 28914 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.393172 28878 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:02.439460 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:02.464515 22702 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:02.461572 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:02.465538 22709 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
I20260709 13:12:02.469831 22627 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 13:12:02.470199 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:02.392623 28924 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.473671 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:02.474972 23411 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:12:02.441788 28908 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:02.561316 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 74059.4us
I20260709 13:12:02.476730 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:02.563112 28885 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.563385 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:02.564126 28885 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:02.565408 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88767.6us
I20260709 13:12:02.566591 28907 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:02.567242 28907 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:02.567746 28907 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:02.584434 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:02.585701 22630 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:02.588980 28885 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:02.595247 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:02.463959 28066 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.662662 22630 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:02.663997 22631 raft_consensus.cc:2410] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 13:12:02.668673 28926 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:02.669371 28926 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.671756 28926 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:02.618484 28908 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:02.662266 28931 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.240s	user 0.000s	sys 0.003s
I20260709 13:12:02.687369 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:02.659561 28824 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.701351 28762 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.709131 28881 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.711930 28881 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.695798 28785 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:02.684613 28838 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.704924 28892 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.805840 22702 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
W20260709 13:12:02.805751 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 173935us
I20260709 13:12:02.698167 28824 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.649976 28878 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.811311 22708 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:02.709439 28762 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:02.866886 22708 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:02.804708 28785 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:02.951953 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 139680us
W20260709 13:12:02.942586 28944 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.080s	user 0.000s	sys 0.007s
I20260709 13:12:02.947188 28918 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:02.966312 22601 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:02.964485 28908 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:02.961313 28066 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:02.981752 28918 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:02.956996 28892 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:02.992444 28824 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:03.030658 28066 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:02.966943 28914 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.092813 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.093674 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.093473 28914 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.095754 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.096520 22707 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:12:03.097141 28914 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.032821 28762 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.033398 28397 raft_consensus.cc:2764] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:12:03.034787 28940 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.225s	user 0.000s	sys 0.003s
W20260709 13:12:03.036350 28932 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.379s	user 0.001s	sys 0.000s
I20260709 13:12:03.099562 28918 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:03.030843 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 64643.8us
W20260709 13:12:03.039134 28939 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.095s	user 0.004s	sys 0.000s
W20260709 13:12:03.039932 28943 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.095s	user 0.004s	sys 0.000s
I20260709 13:12:03.066501 28826 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:03.103294 28826 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.105918 28826 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:03.084306 28907 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.008s
I20260709 13:12:03.084659 28850 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:03.107947 28838 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.108173 28850 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.108994 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.109813 23412 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:03.087396 28908 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:03.111066 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.110940 28908 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.113269 28908 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.113132 28761 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:12:03.113487 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.114951 28907 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.115901 22635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 2372) took 1296 ms. Trace:
I20260709 13:12:03.116411 28850 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.116353 22635 rpcz_store.cc:276] 0709 13:12:01.819335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:01.819760 (+   425us) service_pool.cc:224] Handling call
0709 13:12:01.820133 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:01.856215 (+ 36082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:01.856261 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:01.856844 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:12:02.409954 (+553110us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b18003375a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:12:02.690697 (+280743us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:12:02.690755 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:02.690856 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:03.115589 (+424733us) raft_consensus.cc:1685] finished
0709 13:12:03.115611 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:03.115817 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41942,"raft.queue_time_us":168934,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":20431,"spinlock_wait_cycles":12803840,"thread_start_us":280484,"threads_started":1,"wal-append.queue_time_us":813773}
I20260709 13:12:03.118947 28907 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:03.120779 28866 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.121464 28866 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.124300 28866 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.126402 28881 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.087275 28066 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.089555 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:03.127461 22705 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:03.090852 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.133598 22708 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:03.091341 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.134555 22706 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:03.137660 28785 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.116364 22703 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:03.153812 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.154894 23421 raft_consensus.cc:2368] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23.
I20260709 13:12:03.156090 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.156873 23412 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
W20260709 13:12:03.159574 28942 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:12:03.160465 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:03.161288 23416 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:03.083608 28924 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.165346 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.167389 28871 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:03.168084 28871 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.170269 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:03.174142 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73030.3us
I20260709 13:12:03.176608 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.177103 28895 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:03.177605 22707 raft_consensus.cc:2393] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 22: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:03.177747 28895 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.186401 28915 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:03.187299 28915 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.190058 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:03.191852 22782 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:03.194439 28863 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.198501 28863 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.199855 22673 leader_election.cc:400] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:03.200374 22673 leader_election.cc:403] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:12:03.201116 28863 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.197372 28467 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:03.209278 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.201238 28896 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:03.191123 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.201874 28066 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.211226 22783 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 20.
I20260709 13:12:03.204859 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.212867 28871 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.229082 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.233525 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.234675 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.158954 22705 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:03.241776 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.244156 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:03.244944 22776 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:03.116533 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:03.252490 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.225154 28896 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.256817 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:03.255025 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 138187us
I20260709 13:12:03.262238 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.234373 28810 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:03.265101 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:03.279105 22782 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:03.283440 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:03.284525 22704 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:03.288090 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:03.283345 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:03.271313 28896 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.305083 28578 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:03.306804 28896 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.291086 23385 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:03.291633 28810 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.310025 28810 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.316063 28895 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.317157 28885 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:03.317837 28885 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:03.318818 28885 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.323923 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 23.
I20260709 13:12:03.326965 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:03.328949 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.345469 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.348122 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:03.259933 28762 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:03.343379 28896 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:03.329491 28924 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:03.339377 28872 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.362282 28907 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:12:03.364207 28762 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:03.384487 28907 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:03.369493 28773 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:03.390370 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.418891 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.419360 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.419806 22631 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:12:03.377485 28872 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.385890 28907 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.435352 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:03.443564 22635 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:12:03.444885 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:03.442662 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 115782us
I20260709 13:12:03.448823 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.447196 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:03.454162 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.455605 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:03.460639 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.467818 22778 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:03.456050 22750 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:03.447023 28881 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:03.446604 28878 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:03.475883 22635 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 23.
I20260709 13:12:03.396164 28773 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.481523 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.469633 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:03.479027 22599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.055s	sys 0.071s
W20260709 13:12:03.492769 22599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.055s	sys 0.071s
I20260709 13:12:03.465791 28924 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:03.443307 28880 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:03.480228 28592 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:03.476961 22750 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:03.492162 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.511130 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.519600 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.520555 22635 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:03.496166 28924 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:03.527882 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.497619 28592 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:03.531345 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 62105.9us
I20260709 13:12:03.532572 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.480921 28881 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.539664 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.530207 28924 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:03.543751 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:03.547540 22634 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:03.519245 28878 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:03.551766 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.553126 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.562278 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.563449 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.563673 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.563488 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.567111 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:03.468644 28762 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.499577 28880 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.529762 28797 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:03.543180 28592 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:03.587373 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48748.8us
I20260709 13:12:03.587855 22776 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:03.589718 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.592574 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.600673 22598 leader_election.cc:400] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23.
I20260709 13:12:03.601227 22598 leader_election.cc:403] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:03.581408 28923 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:03.607038 22634 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:03.589691 28878 raft_consensus.cc:697] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:03.608565 28914 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23.
I20260709 13:12:03.617343 28923 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.652566 28881 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:03.652774 28959 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
W20260709 13:12:03.656936 28955 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:12:03.660933 28592 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.636760 28872 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.675087 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.663372 28923 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:03.685878 28953 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 13:12:03.671016 28951 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 13:12:03.706522 28592 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:03.719455 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 91708us
W20260709 13:12:03.716974 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 126436us
I20260709 13:12:03.733430 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:03.734138 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:03.738727 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:03.730006 28954 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.209s	user 0.000s	sys 0.002s
I20260709 13:12:03.739694 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:03.743893 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 161769us
I20260709 13:12:03.744264 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:03.748119 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:03.748495 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 164309us
I20260709 13:12:03.751497 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:03.738538 28916 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:03.749804 28914 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:03.768050 23416 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:03.748984 28396 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:03.788764 22703 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:12:03.791308 28885 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:03.794212 28916 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:03.694865 28878 consensus_queue.cc:237] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.804711 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:03.755942 28872 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:03.777240 28971 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:03.774135 28880 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:03.825605 28915 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:03.813784 22777 raft_consensus.cc:1240] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:03.817625 28508 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:03.830145 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.818333 28918 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:03.837147 28918 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.840087 28918 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.843894 28508 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.769927 28397 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.847579 28397 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.857887 28773 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.839850 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:03.879763 28924 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:03.881704 22782 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 20. Current term is 21. Ops: []
W20260709 13:12:03.891917 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52209.8us
I20260709 13:12:03.894845 28915 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:03.894848 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.895936 28915 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:03.896515 28915 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:03.871685 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:03.910590 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:03.912340 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 150377us
I20260709 13:12:03.869872 28508 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:03.937199 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.943312 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:04.000340 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46377.6us
I20260709 13:12:04.001603 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:03.966087 28961 log.cc:927] Time spent T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.231s	user 0.005s	sys 0.000s
I20260709 13:12:03.962285 28876 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.018580 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:03.963323 28767 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.025779 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.027886 22626 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:12:04.026760 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:04.040396 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:03.947044 28773 consensus_queue.cc:1059] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:12:04.049893 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:04.051420 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:04.036799 28950 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.011s	sys 0.031s
W20260709 13:12:04.065779 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61952.6us
I20260709 13:12:04.071710 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:04.122622 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 50905us
I20260709 13:12:04.124011 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:04.124578 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.016769 28895 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:12:04.125604 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:04.132129 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:04.050001 28908 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:04.068434 28950 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.011s	sys 0.031s
I20260709 13:12:04.122869 28915 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.042129 28767 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:04.322956 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 197443us
I20260709 13:12:04.068812 28881 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.198s
I20260709 13:12:04.133401 28877 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:04.136914 28969 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:12:04.349460 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 327407us
I20260709 13:12:04.289871 28885 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:04.035300 28876 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } attrs { replace: true } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.355018 28885 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.358678 28880 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:04.384829 28973 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
W20260709 13:12:04.390178 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 258359us
I20260709 13:12:04.392063 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:04.411603 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.390522 28885 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:12:04.490288 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 488760us
I20260709 13:12:04.491698 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.503608 22675 leader_election.cc:400] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:04.505198 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:04.496517 28972 log.cc:927] Time spent T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.461s	user 0.000s	sys 0.002s
W20260709 13:12:04.509619 23418 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.000s	sys 0.013s
W20260709 13:12:04.514299 23418 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.000s	sys 0.013s
W20260709 13:12:04.515581 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.000s	sys 0.012s
W20260709 13:12:04.516531 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.000s	sys 0.012s
W20260709 13:12:04.516899 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 577377us
I20260709 13:12:04.501168 28762 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:04.507920 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.500200 28880 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:04.523275 28983 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:04.526260 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.527235 28994 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:04.528013 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.528880 28994 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:04.531260 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.532187 28983 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:04.510722 28896 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:04.509621 28924 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:04.547289 28977 log.cc:927] Time spent T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
W20260709 13:12:04.548276 28984 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:12:04.549712 28876 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:04.550885 28975 log.cc:927] Time spent T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.298s	user 0.000s	sys 0.001s
W20260709 13:12:04.566120 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 146198us
I20260709 13:12:04.570509 28995 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:04.570914 28974 log.cc:927] Time spent T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:12:04.571295 28995 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.574007 28995 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.584677 28767 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:04.588395 28923 consensus_meta.cc:287] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.872s	user 0.012s	sys 0.002s
I20260709 13:12:04.588981 28923 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.590241 22707 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:12:04.592550 28986 log.cc:927] Time spent T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:12:04.592600 28923 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.569727 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.605505 28885 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.611662 28810 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:04.612557 28397 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.612670 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.613758 28885 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.607082 22778 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:12:04.609170 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.617436 28896 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:04.619386 28925 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:04.621124 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.613950 28397 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:04.640575 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.641569 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:04.643157 22778 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:12:04.645944 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:04.650947 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:04.660571 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:04.663551 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:04.620005 28925 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.680792 28508 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:04.687161 28925 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.687875 28925 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.688011 28880 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:04.695303 28767 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.691164 28880 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.712149 28767 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.698571 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.716936 28877 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:04.717470 28826 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:04.688439 28925 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.717628 28877 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.718046 28826 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.720005 28877 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.699535 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:04.695189 28910 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.722584 28910 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.699478 22633 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:12:04.724692 28910 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.725778 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.726424 28871 raft_consensus.cc:3055] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:04.726876 28871 raft_consensus.cc:740] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:04.727265 28925 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.729032 28767 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.729653 28871 consensus_queue.cc:260] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.731101 22749 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:04.732179 28877 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:04.732995 28877 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.733489 28877 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.737112 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:04.738070 28592 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:12:04.738142 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.739384 28899 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:12:04.739817 28899 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:04.740754 28899 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.741451 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:04.729144 28901 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:04.744596 28901 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.746832 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.743848 28965 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:04.748697 28965 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.752732 28871 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:04.738401 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.759367 22707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 3775) took 1167 ms. Trace:
I20260709 13:12:04.760532 28901 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.760591 22702 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:04.721839 28826 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.765079 28826 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:04.765197 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63934.7us
I20260709 13:12:04.765709 28826 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.763872 28965 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.769677 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.715873 28880 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.771701 28880 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.772399 28880 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.776947 28880 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.783450 28826 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:04.786906 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116198us
I20260709 13:12:04.798132 28785 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:04.799062 28785 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.802318 28785 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.802027 28989 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:04.802971 28592 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.814795 28592 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.816036 22776 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 20. Current term is 21. Ops: []
I20260709 13:12:04.818186 28592 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.818305 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.805150 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:04.821066 28785 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:04.822854 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.824775 28871 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:04.839808 28877 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.843845 28901 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.844602 28901 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.759902 22707 rpcz_store.cc:276] 0709 13:12:03.592153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.592334 (+   181us) service_pool.cc:224] Handling call
0709 13:12:04.759150 (+1166816us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400315954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:04.759300 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279587584}
I20260709 13:12:04.806461 22702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 2936) took 1077 ms. Trace:
I20260709 13:12:04.847440 22702 rpcz_store.cc:276] 0709 13:12:03.728774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:03.728921 (+   147us) service_pool.cc:224] Handling call
0709 13:12:04.760357 (+1031436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:04.765625 (+  5268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:04.806213 (+ 40588us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400315950. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:04.806355 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2367526784}
I20260709 13:12:04.808490 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.860634 28850 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.861421 28850 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.863858 28850 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.866353 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.867964 28850 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:04.868638 28850 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.871841 28850 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.860729 28899 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:04.893836 22628 raft_consensus.cc:1275] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:12:04.895820 28592 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:04.902560 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.904217 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.896648 22702 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:12:04.897377 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:04.898101 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:04.898090 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.899662 28761 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:04.907527 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.907986 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:04.908066 28761 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:04.908763 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.909116 22776 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:04.910775 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:04.911358 28761 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:04.914875 28901 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:04.915958 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.916596 29002 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:04.921025 28066 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:04.922659 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.922952 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.924402 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.924961 28880 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:04.927774 23416 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 20. Current term is 21. Ops: []
I20260709 13:12:04.927361 28880 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:04.932032 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.936412 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.933254 28880 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.942459 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.943832 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:04.947189 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.949400 28066 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:04.948324 28880 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:04.963557 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:04.976295 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:04.983698 28396 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:04.984565 28396 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:04.987742 28396 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:04.995074 23385 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:04.997922 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:05.001211 28877 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:05.007474 22634 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:12:05.008808 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:05.010236 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:05.016494 28066 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:05.017225 28982 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:12:05.017763 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.026747 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:05.027443 28850 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:05.028151 28850 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.030422 28850 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:05.044708 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:05.047955 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.050750 22701 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:12:05.052300 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:05.052794 22704 raft_consensus.cc:1275] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:12:05.066183 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.066715 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.067606 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.069731 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.078594 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.087033 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.089725 22627 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 21.
I20260709 13:12:05.091709 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.106796 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.111348 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.086009 28872 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:05.093092 29002 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.107803 28965 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.093420 29003 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
I20260709 13:12:05.120764 28879 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:05.147595 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.161835 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.125697 28872 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:05.168398 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.177518 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.187120 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.187120 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.190546 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.191298 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.191854 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.191938 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.193857 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:05.194002 28982 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:05.195991 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.197373 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:05.197765 28982 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.197932 29002 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 13:12:05.189922 28992 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:05.200281 28805 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.230691 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:05.231417 22628 raft_consensus.cc:606] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.232141 22628 raft_consensus.cc:614] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:05.197374 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:05.240487 28872 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:05.246889 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:05.258998 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:05.261365 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 59052.1us
W20260709 13:12:05.291697 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46248.7us
I20260709 13:12:05.298139 28989 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:12:05.309943 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.313328 28914 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:05.326357 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.330016 28508 consensus_queue.cc:1048] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:05.340169 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:05.340955 22631 raft_consensus.cc:606] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.341308 22631 raft_consensus.cc:614] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:05.314771 28826 consensus_queue.cc:1048] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:12:05.346011 28971 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:05.369469 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.393711 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.396153 28878 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.409044 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.405136 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:05.410843 22631 raft_consensus.cc:606] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.414412 28880 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:05.430388 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:05.431420 22632 raft_consensus.cc:606] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.438341 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:05.439409 22632 raft_consensus.cc:606] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.440475 22632 raft_consensus.cc:614] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:05.444248 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:05.487437 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:05.490022 22524 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:05.495766 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 195501us
I20260709 13:12:05.503448 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.504520 28508 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:05.503908 22524 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:05.504974 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.505885 28508 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:05.507184 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:05.506585 22524 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.508797 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.510536 28971 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:05.511963 22675 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.511407 28918 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:05.515215 29019 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:05.519218 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 110276us
I20260709 13:12:05.509742 28508 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:05.539201 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:05.554216 28508 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:05.576152 22524 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.578817 22524 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.581782 22524 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.602533 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.604390 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.605440 28971 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:05.606000 28592 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:05.609205 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.611761 28901 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:05.613602 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:05.616158 28901 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:05.620146 28872 raft_consensus.cc:993] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:05.620734 28872 raft_consensus.cc:1081] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:05.622469 22710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:05.623147 22710 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:05.623597 22710 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:05.631894 22524 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.634366 22524 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.667698 22524 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.671188 22524 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.673178 22524 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.677253 22524 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 6 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:05.702657 28810 raft_consensus.cc:993] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:05.703399 28810 raft_consensus.cc:1081] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:05.708950 22701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:05.709734 22701 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:05.710276 22701 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:05.719223 22710 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.747172 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 148) took 7452 ms. Trace:
I20260709 13:12:05.750568 22528 rpcz_store.cc:276] 0709 13:11:58.275236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.275826 (+   590us) service_pool.cc:224] Handling call
0709 13:12:05.727103 (+7451277us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:01.777548 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:01.778186 (+   638us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:01.778229 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:12:01.778251 (+    22us) tablet.cc:662] Decoding operations
0709 13:12:01.828087 (+ 49836us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:01.843065 (+ 14978us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:01.848902 (+  5837us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:12:02.246919 (+398017us) tablet.cc:701] Row locks acquired
0709 13:12:02.246941 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:12:02.247132 (+   191us) write_op.cc:270] Start()
0709 13:12:02.247294 (+   162us) write_op.cc:276] Timestamp: P: 1783602722247061 usec, L: 0
0709 13:12:02.247322 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:12:02.248152 (+   830us) log.cc:844] Serialized 5974 byte log entry
0709 13:12:03.267026 (+1018874us) op_driver.cc:464] REPLICATION: finished
0709 13:12:03.827022 (+559996us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f180a913ef0
0709 13:12:04.114937 (+287915us) write_op.cc:301] APPLY: starting
0709 13:12:04.115090 (+   153us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:04.717432 (+602342us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18002f09e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806c1c0ec 00007f1806c68c0a 00007f1806c684f7 00007f1806c68490 00007f1806c1ff48 00007f180a9341bb 00007f180a934532 00007f180a97bb93 00007f180a828de3
0709 13:12:04.780095 (+ 62663us) spinlock_profiling.cc:243] Waited 4.92 ms on lock 0x7b18002f09e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:12:04.856767 (+ 76672us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:04.856799 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:05.355413 (+498614us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:05.355705 (+   292us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 13:12:05.355755 (+    50us) write_op.cc:312] APPLY: finished
0709 13:12:05.356163 (+   408us) log.cc:844] Serialized 139 byte log entry
0709 13:12:05.398701 (+ 42538us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:05.414788 (+ 16087us) write_op.cc:454] Released schema lock
0709 13:12:05.428999 (+ 14211us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:05.429475 (+   476us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:05.429513 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":584566,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":847462,"apply.run_cpu_time_us":40410,"apply.run_wall_time_us":1314556,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66947,"num_ops":15,"prepare.queue_time_us":324896,"prepare.run_cpu_time_us":51912,"prepare.run_wall_time_us":662777,"raft.queue_time_us":461338,"raft.run_cpu_time_us":51292,"raft.run_wall_time_us":1067635,"replication_time_us":1019487,"spinlock_wait_cycles":61607424,"thread_start_us":1990777,"threads_started":5,"wal-append.queue_time_us":250902,"wal-append.run_cpu_time_us":100315,"wal-append.run_wall_time_us":1845220}]]}
I20260709 13:12:05.768682 22710 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:05.776401 22701 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.810352 22701 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:05.821081 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:05.821902 22635 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:05.822293 22635 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:05.824139 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:05.824806 22776 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:05.826491 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:05.827195 22631 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:05.827495 22631 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:05.828275 22631 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.829337 22635 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.833267 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:05.833881 22782 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:05.836741 22635 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:05.843660 22631 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:05.863257 22631 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:05.876536 22782 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:05.895252 22776 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:05.896171 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:05.901316 28508 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:05.922930 28508 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:05.924125 28508 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:05.929956 22635 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:05.932519 22675 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:05.935731 28918 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:05.952420 28918 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:06.007278 28918 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.107407 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 with delete_type TABLET_DATA_TOMBSTONED (TS a0e9f7dc2a2e4bdb9287d910e4b491f7 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:37270
I20260709 13:12:06.149969 29034 tablet_replica.cc:333] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:12:06.163465 29034 raft_consensus.cc:2243] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:06.164709 29034 raft_consensus.cc:2272] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:06.172534 29034 ts_tablet_manager.cc:1916] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:12:06.195478 28761 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:06.196237 28761 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.198644 28761 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:06.197315 22528 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.200167 22528 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 9 to 11, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.202482 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:06.203372 22631 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:06.203570 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:06.204414 22784 raft_consensus.cc:2393] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 16: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:06.203327 22528 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 13, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 13 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.212713 22528 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 10 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.214581 22528 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.216181 22528 catalog_manager.cc:5697] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 17, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 17 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.217805 22528 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 8 to 13, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 13 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.223474 22528 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.225237 22528 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:06.227494 22528 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.216543 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:06.242465 28761 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:06.243047 28761 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:06.243467 28761 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:06.259792 28901 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.260629 28901 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.273376 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:06.274149 22635 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:06.275791 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:06.277134 28877 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:06.277669 28877 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.278105 28877 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:06.281255 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70268.2us
I20260709 13:12:06.303959 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:06.306953 28901 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:06.336619 22532 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 8 to 9, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 9 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:06.337549 28877 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.341719 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:06.348297 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:06.349359 22631 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:06.352473 22776 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:06.373665 22631 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:06.379624 22749 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:06.364389 28877 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:06.386875 28785 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:06.367197 28918 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:12:06.398734 28785 raft_consensus.cc:697] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:06.404690 28785 consensus_queue.cc:237] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.405089 28761 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.406889 22704 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:06.407418 28761 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:06.421296 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:06.508049 28761 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:06.505000 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:06.525826 28901 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.541471 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:06.546244 22631 raft_consensus.cc:2368] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:12:06.541348 28761 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:06.607888 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88346.9us
I20260709 13:12:06.578064 28901 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:06.616150 28761 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:06.617799 29034 ts_tablet_manager.cc:1929] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.10
I20260709 13:12:06.619568 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:06.628974 22635 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:06.617580 28901 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:06.632838 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:06.627660 23416 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:06.605108 28878 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:06.641067 29034 log.cc:1199] T 2f4adfa257c34bb88d79bda9d39c5146 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/wals/2f4adfa257c34bb88d79bda9d39c5146
I20260709 13:12:06.640592 28878 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:06.703235 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:06.704382 22631 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:06.686383 28878 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:06.714857 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:06.715863 22635 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:06.729262 22675 leader_election.cc:400] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
I20260709 13:12:06.730088 22675 leader_election.cc:403] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:12:06.748732 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:06.733961 28918 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:06.755923 29051 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:06.767699 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:06.785982 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:06.787622 28878 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:06.788260 28878 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.788692 28878 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:06.783139 28880 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:06.790104 28880 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:06.790807 28880 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:06.818499 22784 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:12:06.822332 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 46525.2us
I20260709 13:12:06.826974 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:06.827780 22701 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:06.832623 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:06.833527 22703 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:06.837057 28878 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:06.857609 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 44494.2us
I20260709 13:12:06.872082 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:06.876123 22631 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:06.886433 22635 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:12:06.963070 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 239230us
I20260709 13:12:06.934724 28918 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283) with higher term. Message: Invalid argument: T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17.
W20260709 13:12:06.981402 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 77794.5us
W20260709 13:12:07.054446 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 93991.9us
W20260709 13:12:07.054864 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52163.9us
I20260709 13:12:07.057735 28878 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:07.066725 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81867.5us
I20260709 13:12:06.933773 28880 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:06.988427 28908 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 13:12:07.112047 22776 raft_consensus.cc:2368] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 21. Current term is 22.
I20260709 13:12:07.091970 28508 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:07.116835 28880 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:07.140625 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:07.141506 23421 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:07.093449 22675 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:07.095520 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:07.144049 22631 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:07.057140 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 2f4adfa257c34bb88d79bda9d39c5146 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
W20260709 13:12:07.159031 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 65702.7us
I20260709 13:12:07.162253 28508 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:07.163738 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:07.164448 22704 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:07.165889 22519 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 271019us
W20260709 13:12:07.171676 22532 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 251) took 40387 ms (client timeout 15000 ms). Trace:
W20260709 13:12:07.175668 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 74440.9us
I20260709 13:12:07.177110 28508 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:12:07.172291 22532 rpcz_store.cc:269] 0709 13:11:26.775411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:26.775654 (+   243us) service_pool.cc:224] Handling call
0709 13:12:07.162530 (+40386876us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:06.349365 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:06.349987 (+   622us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:06.350031 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:12:06.350043 (+    12us) tablet.cc:662] Decoding operations
0709 13:12:06.350476 (+   433us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:06.350557 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:06.350584 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:06.350819 (+   235us) tablet.cc:701] Row locks acquired
0709 13:12:06.350831 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:12:06.351003 (+   172us) write_op.cc:270] Start()
0709 13:12:06.351133 (+   130us) write_op.cc:276] Timestamp: P: 1783602726350957 usec, L: 0
0709 13:12:06.351150 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:12:06.351850 (+   700us) log.cc:844] Serialized 538 byte log entry
0709 13:12:06.354164 (+  2314us) op_driver.cc:464] REPLICATION: finished
0709 13:12:06.355635 (+  1471us) write_op.cc:301] APPLY: starting
0709 13:12:06.383844 (+ 28209us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5400357ba8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180d2999e1 00007f180d29864c 00007f180a928e10 00007f180a91497c 00007f180a918d61 00007f180a918d19 00007f180a918ca9 00007f180a918c71 00007f180a917e9d 000000000038f354 000000000038f199
0709 13:12:06.383985 (+   141us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:06.384920 (+   935us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:06.384945 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:06.385737 (+   792us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:06.385917 (+   180us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:06.385968 (+    51us) write_op.cc:312] APPLY: finished
0709 13:12:06.386244 (+   276us) log.cc:844] Serialized 25 byte log entry
0709 13:12:06.386461 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.392071 (+  5610us) write_op.cc:454] Released schema lock
0709 13:12:06.393119 (+  1048us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:06.416454 (+ 23335us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.416489 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":399890,"reported_tablets":51,"spinlock_wait_cycles":3584,"child_traces":[["op",{"apply.queue_time_us":582,"apply.run_cpu_time_us":12187,"apply.run_wall_time_us":46821,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4188,"num_ops":1,"prepare.queue_time_us":4548,"prepare.run_cpu_time_us":2791,"prepare.run_wall_time_us":2790,"replication_time_us":2856,"spinlock_wait_cycles":4870528}]]}
W20260709 13:12:07.181198 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 102840us
W20260709 13:12:07.182356 22524 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 237) took 20924 ms (client timeout 15000 ms). Trace:
I20260709 13:12:07.193218 23421 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 22.
I20260709 13:12:07.210798 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:07.211618 22706 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:07.219975 22702 raft_consensus.cc:1275] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:12:07.224296 22706 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 22.
I20260709 13:12:07.193892 22704 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:07.226321 22631 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:07.247977 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:07.248713 22632 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:07.283556 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:07.284960 29040 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:07.285710 29040 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:07.287177 29040 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:07.190451 22524 rpcz_store.cc:269] 0709 13:11:46.257745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:46.488083 (+230338us) service_pool.cc:224] Handling call
0709 13:12:07.182243 (+20694160us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:05.856199 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:05.908705 (+ 52506us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:05.908750 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:12:05.908772 (+    22us) tablet.cc:662] Decoding operations
0709 13:12:05.910096 (+  1324us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:05.910178 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:05.910217 (+    39us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:12:05.941137 (+ 30920us) tablet.cc:701] Row locks acquired
0709 13:12:05.941164 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:12:05.941359 (+   195us) write_op.cc:270] Start()
0709 13:12:05.941490 (+   131us) write_op.cc:276] Timestamp: P: 1783602725941304 usec, L: 0
0709 13:12:05.941510 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:12:05.980883 (+ 39373us) log.cc:844] Serialized 4774 byte log entry
0709 13:12:06.027741 (+ 46858us) op_driver.cc:464] REPLICATION: finished
0709 13:12:06.108042 (+ 80301us) write_op.cc:301] APPLY: starting
0709 13:12:06.108143 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:06.122190 (+ 14047us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:06.122370 (+   180us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:06.133056 (+ 10686us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:06.133375 (+   319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:12:06.133432 (+    57us) write_op.cc:312] APPLY: finished
0709 13:12:06.133846 (+   414us) log.cc:844] Serialized 113 byte log entry
0709 13:12:06.134080 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.134290 (+   210us) write_op.cc:454] Released schema lock
0709 13:12:06.134362 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:06.166766 (+ 32404us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:06.166800 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":459341,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":79911,"apply.run_cpu_time_us":18839,"apply.run_wall_time_us":40856,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":99747,"prepare.run_cpu_time_us":6484,"prepare.run_wall_time_us":142787,"replication_time_us":86000,"spinlock_wait_cycles":37760,"thread_start_us":137656,"threads_started":2}]]}
W20260709 13:12:07.359731 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98424.9us
I20260709 13:12:07.370839 22632 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 22.
W20260709 13:12:07.387344 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 106461us
I20260709 13:12:07.405227 22750 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:07.407325 28877 consensus_queue.cc:1048] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:07.413941 29040 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:12:07.415369 29040 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:07.458499 29040 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:07.479430 22675 leader_election.cc:400] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 21. Current term is 22.
W20260709 13:12:07.528628 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55115.2us
W20260709 13:12:07.775108 22527 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 250) took 62954 ms (client timeout 15000 ms). Trace:
W20260709 13:12:07.786100 22527 rpcz_store.cc:269] 0709 13:11:04.820917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:04.821242 (+   325us) service_pool.cc:224] Handling call
0709 13:12:07.774809 (+62953567us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":714291,"reported_tablets":51}
I20260709 13:12:07.812000 22627 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:07.813982 29040 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:07.776518 22525 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.822301 22525 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.853484 22525 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 11 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.864269 22702 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:07.870939 29060 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:07.863759 22525 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.901243 22525 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.919271 22525 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 9 to 10, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 10 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.935025 22702 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:12:07.931171 22525 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 15 to 16, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.936839 28877 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:07.955268 22525 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.965878 22525 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.973825 22525 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 9 to 10, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:07.987740 22525 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 12 to 13, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 13 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.992319 22525 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.069308 22627 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:12:08.072968 28877 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:08.094202 23418 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:12:08.099169 29060 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:08.345402 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:08.398209 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44151.1us
I20260709 13:12:08.498121 22527 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 20 to 22, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:08.566188 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:08.567114 22627 raft_consensus.cc:606] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:08.573546 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:08.574347 22627 raft_consensus.cc:606] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:08.581687 29044 raft_consensus.cc:993] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:08.583333 29044 raft_consensus.cc:1081] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:08.612767 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:08.613533 22628 raft_consensus.cc:606] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:08.613910 22628 raft_consensus.cc:614] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:08.619516 22776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:12:08.620450 22776 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.620929 22776 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:08.661442 22526 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.665336 22526 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 17, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.672158 22526 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 16, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.690317 22526 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.695896 22526 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:08.677459 22525 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 252) took 17342 ms (client timeout 15000 ms). Trace:
I20260709 13:12:08.705111 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:08.718091 22628 raft_consensus.cc:606] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:08.718523 22628 raft_consensus.cc:614] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:08.721838 22776 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:08.724953 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:08.725723 22627 raft_consensus.cc:606] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:08.726132 22627 raft_consensus.cc:614] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:08.727487 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:12:08.719974 22525 rpcz_store.cc:269] 0709 13:11:51.335555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:51.335797 (+   242us) service_pool.cc:224] Handling call
0709 13:12:08.677348 (+17341551us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:08.087991 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:08.088769 (+   778us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:08.088941 (+   172us) write_op.cc:435] Acquired schema lock
0709 13:12:08.088978 (+    37us) tablet.cc:662] Decoding operations
0709 13:12:08.105979 (+ 17001us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:08.106119 (+   140us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:08.106169 (+    50us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:12:08.126746 (+ 20577us) tablet.cc:701] Row locks acquired
0709 13:12:08.126806 (+    60us) write_op.cc:260] PREPARE: finished
0709 13:12:08.136778 (+  9972us) write_op.cc:270] Start()
0709 13:12:08.136997 (+   219us) write_op.cc:276] Timestamp: P: 1783602728136681 usec, L: 0
0709 13:12:08.137041 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:12:08.137934 (+   893us) log.cc:844] Serialized 4774 byte log entry
0709 13:12:08.260499 (+122565us) op_driver.cc:464] REPLICATION: finished
0709 13:12:08.262887 (+  2388us) write_op.cc:301] APPLY: starting
0709 13:12:08.263078 (+   191us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:08.366914 (+103836us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:08.367063 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:08.468376 (+101313us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:08.468728 (+   352us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 13:12:08.468796 (+    68us) write_op.cc:312] APPLY: finished
0709 13:12:08.469258 (+   462us) log.cc:844] Serialized 113 byte log entry
0709 13:12:08.477219 (+  7961us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:08.477401 (+   182us) write_op.cc:454] Released schema lock
0709 13:12:08.477479 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:08.537131 (+ 59652us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:08.537174 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":51,"spinlock_wait_cycles":61056,"child_traces":[["op",{"apply.queue_time_us":1942,"apply.run_cpu_time_us":22389,"apply.run_wall_time_us":274241,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54383,"num_ops":12,"prepare.queue_time_us":40049,"prepare.run_cpu_time_us":13966,"prepare.run_wall_time_us":141231,"raft.queue_time_us":57860,"raft.run_cpu_time_us":12617,"raft.run_wall_time_us":18739,"replication_time_us":123224,"thread_start_us":107512,"threads_started":4,"wal-append.queue_time_us":61624}]]}
I20260709 13:12:08.729915 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:08.729751 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:08.730746 22627 raft_consensus.cc:3055] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:08.731134 22627 raft_consensus.cc:740] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:08.732023 22627 consensus_queue.cc:260] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:08.733364 22627 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:08.732407 22526 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.743597 22526 catalog_manager.cc:5697] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 11 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.764832 22526 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.783399 22776 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:08.782742 22526 catalog_manager.cc:5697] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.790761 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:08.791522 22710 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:08.793542 22526 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 12, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.805150 22526 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:08.809779 22627 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:08.808305 22526 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 10 to 12, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.821497 22749 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:08.825958 22710 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:08.829178 29040 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:08.840914 22526 catalog_manager.cc:5697] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:08.881296 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51636us
I20260709 13:12:08.852526 29040 raft_consensus.cc:697] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:08.883149 22526 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 10 to 11, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:08.902171 29040 consensus_queue.cc:237] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:08.959095 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48762.7us
W20260709 13:12:08.990348 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 62559us
W20260709 13:12:08.997553 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70333.8us
I20260709 13:12:09.033346 22526 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.035543 22526 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.037729 22526 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.044670 22526 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 5 to 8, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 8 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:09.081135 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53646.1us
I20260709 13:12:09.099931 22526 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:09.108726 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 102516us
I20260709 13:12:09.196561 29039 raft_consensus.cc:993] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:09.197208 29039 raft_consensus.cc:1081] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:09.201781 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:09.202432 22706 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:09.202843 22706 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:09.213665 22706 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:09.216673 22706 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:09.219302 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:09.220049 22635 raft_consensus.cc:3055] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:09.220363 22635 raft_consensus.cc:740] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:09.221282 22635 consensus_queue.cc:260] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:09.222576 22635 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:09.228523 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:09.229290 23421 raft_consensus.cc:3060] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:09.234638 22635 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:09.236513 22675 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:09.239154 29062 raft_consensus.cc:2804] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:09.239678 29062 raft_consensus.cc:697] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:09.240710 29062 consensus_queue.cc:237] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:09.253964 23421 raft_consensus.cc:2468] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:09.436362 22710 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:09.440551 29092 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:09.459339 22635 raft_consensus.cc:1275] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:09.461812 28877 consensus_queue.cc:1048] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:09.698244 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52529.1us
I20260709 13:12:09.706667 22526 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 238) took 4555 ms. Trace:
I20260709 13:12:09.701436 22529 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 16, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:09.731312 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:09.720618 22526 rpcz_store.cc:276] 0709 13:12:05.151556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:05.152256 (+   700us) service_pool.cc:224] Handling call
0709 13:12:09.706549 (+4554293us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:09.194758 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:09.196620 (+  1862us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:09.196668 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:12:09.196686 (+    18us) tablet.cc:662] Decoding operations
0709 13:12:09.204650 (+  7964us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:09.204751 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:09.204794 (+    43us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:12:09.206271 (+  1477us) tablet.cc:701] Row locks acquired
0709 13:12:09.206287 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:12:09.206488 (+   201us) write_op.cc:270] Start()
0709 13:12:09.218449 (+ 11961us) write_op.cc:276] Timestamp: P: 1783602729206423 usec, L: 0
0709 13:12:09.218486 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:12:09.219360 (+   874us) log.cc:844] Serialized 7469 byte log entry
0709 13:12:09.257112 (+ 37752us) op_driver.cc:464] REPLICATION: finished
0709 13:12:09.303103 (+ 45991us) write_op.cc:301] APPLY: starting
0709 13:12:09.303220 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:09.347889 (+ 44669us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:09.347991 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:09.429221 (+ 81230us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:09.429596 (+   375us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 13:12:09.429656 (+    60us) write_op.cc:312] APPLY: finished
0709 13:12:09.438133 (+  8477us) log.cc:844] Serialized 172 byte log entry
0709 13:12:09.439173 (+  1040us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:09.442427 (+  3254us) write_op.cc:454] Released schema lock
0709 13:12:09.442522 (+    95us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:09.443206 (+   684us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:09.443247 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":109672,"reported_tablets":51,"child_traces":[["op",{"apply.queue_time_us":45631,"apply.run_cpu_time_us":27900,"apply.run_wall_time_us":140052,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83699,"num_ops":19,"prepare.queue_time_us":23406,"prepare.run_cpu_time_us":6128,"prepare.run_wall_time_us":62162,"replication_time_us":38415,"thread_start_us":44310,"threads_started":2}]]}
I20260709 13:12:09.763806 22635 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:12:09.790989 23418 raft_consensus.cc:1275] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:12:09.810892 29091 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:12:09.826738 29109 consensus_queue.cc:1048] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:10.013836 22524 catalog_manager.cc:5697] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 19 to 20, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:12:10.101153 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57675.8us
W20260709 13:12:10.197443 22529 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 240) took 11388 ms (client timeout 15000 ms). Trace:
W20260709 13:12:10.210803 22529 rpcz_store.cc:269] 0709 13:11:58.809246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:58.819753 (+ 10507us) service_pool.cc:224] Handling call
0709 13:11:58.894703 (+ 74950us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b5400002438. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180cc9c173 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:10.197348 (+11302645us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:09.826049 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:09.839493 (+ 13444us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:09.839533 (+    40us) write_op.cc:435] Acquired schema lock
0709 13:12:09.839551 (+    18us) tablet.cc:662] Decoding operations
0709 13:12:09.839953 (+   402us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:09.840021 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:09.840055 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:09.840254 (+   199us) tablet.cc:701] Row locks acquired
0709 13:12:09.840273 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:12:09.840427 (+   154us) write_op.cc:270] Start()
0709 13:12:09.840586 (+   159us) write_op.cc:276] Timestamp: P: 1783602729840382 usec, L: 0
0709 13:12:09.840603 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:12:09.841374 (+   771us) log.cc:844] Serialized 538 byte log entry
0709 13:12:09.950834 (+109460us) op_driver.cc:464] REPLICATION: finished
0709 13:12:09.987084 (+ 36250us) write_op.cc:301] APPLY: starting
0709 13:12:09.987190 (+   106us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:09.988629 (+  1439us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:09.988737 (+   108us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:09.989526 (+   789us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:09.989723 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:09.989769 (+    46us) write_op.cc:312] APPLY: finished
0709 13:12:09.990000 (+   231us) log.cc:844] Serialized 25 byte log entry
0709 13:12:09.990195 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:09.990301 (+   106us) write_op.cc:454] Released schema lock
0709 13:12:09.990347 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:10.025495 (+ 35148us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:10.025529 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38644,"reported_tablets":51,"spinlock_wait_cycles":4658560,"child_traces":[["op",{"apply.queue_time_us":35943,"apply.run_cpu_time_us":3719,"apply.run_wall_time_us":38408,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8315,"num_ops":1,"prepare.queue_time_us":48663,"prepare.run_cpu_time_us":3043,"prepare.run_wall_time_us":15683,"replication_time_us":109990,"spinlock_wait_cycles":250752,"thread_start_us":69348,"threads_started":2}]]}
I20260709 13:12:10.242410 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37270
I20260709 13:12:10.300475 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 2f4adfa257c34bb88d79bda9d39c5146 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:12:10.357757 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37270
I20260709 13:12:10.364341 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 2f4adfa257c34bb88d79bda9d39c5146 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
W20260709 13:12:10.391095 22533 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 237) took 74859 ms (client timeout 15000 ms). Trace:
W20260709 13:12:10.391826 22533 rpcz_store.cc:269] 0709 13:10:55.511597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.512562 (+   965us) service_pool.cc:224] Handling call
0709 13:12:10.371256 (+74858694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26612,"reported_tablets":51}
I20260709 13:12:10.417872 22526 catalog_manager.cc:5697] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.420449 22526 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.422106 22526 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.429970 22526 catalog_manager.cc:5697] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.431768 22526 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.433374 22526 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.441165 22526 catalog_manager.cc:5697] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:10.451325 22526 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:10.463230 22531 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 249) took 94902 ms (client timeout 15000 ms). Trace:
W20260709 13:12:10.464017 22531 rpcz_store.cc:269] 0709 13:10:35.561255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:35.562418 (+  1163us) service_pool.cc:224] Handling call
0709 13:12:10.462985 (+94900567us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":328615,"reported_tablets":50,"spinlock_wait_cycles":1169408}
I20260709 13:12:10.746810 22769 tablet_service.cc:1558] Processing DeleteTablet for tablet f33fee9d64384b499e55f8bc58339f69 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 5)) from {username='slave'} at 127.0.0.1:37270
I20260709 13:12:10.747516 22771 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f4adfa257c34bb88d79bda9d39c5146 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 5 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:37270
I20260709 13:12:10.750308 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet f33fee9d64384b499e55f8bc58339f69 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
W20260709 13:12:10.751020 22530 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 236) took 104281 ms (client timeout 15000 ms). Trace:
I20260709 13:12:10.751160 22519 catalog_manager.cc:5028] TS a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): tablet 2f4adfa257c34bb88d79bda9d39c5146 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
W20260709 13:12:10.751536 22530 rpcz_store.cc:269] 0709 13:10:26.623680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.624103 (+   423us) service_pool.cc:224] Handling call
0709 13:12:10.750796 (+104126693us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6315,"reported_tablets":48}
I20260709 13:12:10.816821 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 253) took 3994 ms. Trace:
I20260709 13:12:10.822515 22528 rpcz_store.cc:276] 0709 13:12:06.823816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:06.828195 (+  4379us) service_pool.cc:224] Handling call
0709 13:12:10.816576 (+3988381us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45205,"reported_tablets":51}
I20260709 13:12:10.869313 22702 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:10.958498 23421 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:10.960584 29091 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:10.964896 22781 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:10.966852 29090 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:11.025710 29090 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Committing config change with OpId 11.10: config changed from index 4 to 10, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:11.068207 22781 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 4 to 10, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:11.114717 22528 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 4 to 10, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New cstate: current_term: 11 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:11.129712 22520 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a937bd6b17c04d5199f368b1d595cf8b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:12:11.132846 23421 raft_consensus.cc:2955] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Committing config change with OpId 11.10: config changed from index 4 to 10, VOTER 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) evicted. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:11.204227 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.323642 22619 tablet_service.cc:1558] Processing DeleteTablet for tablet a937bd6b17c04d5199f368b1d595cf8b with delete_type TABLET_DATA_TOMBSTONED (TS 1f99ad34cc414233929e0a92f8c3d8db not found in new config with opid_index 10) from {username='slave'} at 127.0.0.1:45654
I20260709 13:12:11.515219 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:11.516196 22634 raft_consensus.cc:606] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:11.605633 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 52012.1us
I20260709 13:12:11.607609 29140 tablet_replica.cc:333] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:11.609017 29140 raft_consensus.cc:2243] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:11.609835 29140 raft_consensus.cc:2272] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:11.642510 29140 ts_tablet_manager.cc:1916] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:12:11.662535 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 110808us
I20260709 13:12:11.688263 22784 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:12:11.689105 22784 raft_consensus.cc:606] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:11.708081 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:11.708822 22634 raft_consensus.cc:606] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:11.774953 29044 raft_consensus.cc:993] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:11.775552 29044 raft_consensus.cc:1081] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:12:11.790932 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53938.4us
I20260709 13:12:11.845592 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:11.852550 22628 raft_consensus.cc:606] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:11.857882 22710 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:11.858644 22710 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:11.859047 22710 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:11.844313 29140 ts_tablet_manager.cc:1929] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 11.9
W20260709 13:12:11.869537 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68207.3us
W20260709 13:12:11.870437 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69790.5us
I20260709 13:12:11.937006 22628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:11.938048 22628 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:11.934944 29140 log.cc:1199] T a937bd6b17c04d5199f368b1d595cf8b P 1f99ad34cc414233929e0a92f8c3d8db: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/wals/a937bd6b17c04d5199f368b1d595cf8b
W20260709 13:12:11.963460 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 96112.3us
I20260709 13:12:11.959368 22710 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:11.972816 22710 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:11.993109 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:11.995150 22628 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:11.995574 22628 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:12:12.028543 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50708.9us
I20260709 13:12:12.000419 22628 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.031986 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:12.062436 22783 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:12.023072 22519 catalog_manager.cc:5028] TS 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): tablet a937bd6b17c04d5199f368b1d595cf8b (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:12:12.082793 22628 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:12.090917 22632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:12.091707 22632 raft_consensus.cc:606] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:12.107319 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:12.109625 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:12.112530 29150 raft_consensus.cc:993] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:12.113373 29044 raft_consensus.cc:1081] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:12.131312 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:12:12.132045 22782 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.132441 22782 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:12.145612 29147 raft_consensus.cc:993] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:12.142055 22782 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:12.146240 29147 raft_consensus.cc:1081] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:12.149580 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:12.150277 23421 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:12.151582 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:12.152307 22632 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:12.152597 22632 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:12:12.155997 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:12.160526 23421 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:12.162081 22782 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:12.164368 22747 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:12.164551 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:12:12.165227 22634 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.165594 29101 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:12.165716 22634 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:12.166234 29101 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:12:12.167614 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 223398us
I20260709 13:12:12.153671 22632 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:12.173111 22783 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:12.177160 22632 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:12.167759 29101 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:12.199270 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:12.200821 29090 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:12.202373 29090 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:12.208194 29090 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.222363 22634 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.224295 22628 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:12.277463 22632 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:12.236881 22634 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:12.286170 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:12.287063 22783 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:12.287427 22783 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:12.311470 22783 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.338662 22783 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:12.351933 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60855us
I20260709 13:12:12.353351 22524 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:12.360117 29149 raft_consensus.cc:993] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:12:12.365754 22748 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:12.366747 29162 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:12:12.367590 29149 raft_consensus.cc:1081] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:12.368584 29104 raft_consensus.cc:993] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:12.370118 29160 raft_consensus.cc:1081] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:12.380837 29158 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:12.381408 29158 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:12.388341 22783 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:12.396345 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:12.397324 22710 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:12.397133 22530 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:12.409132 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 56202.2us
I20260709 13:12:12.412616 22702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:12.413578 22702 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.414029 22702 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:12.424347 22701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:12.426388 22701 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.426939 22701 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:12.428907 22710 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:12.425874 22703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:12.430548 22703 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.430958 22703 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:12.441473 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 138336us
I20260709 13:12:12.441679 22702 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.447372 22702 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:12.449064 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:12.449787 22634 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:12.444785 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:12.451387 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:12.452070 22783 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:12.463542 22703 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.450659 22634 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:12.482808 22634 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.484452 22634 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:12.485090 22701 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.491541 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:12.492293 22628 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:12.492631 22628 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:12.493017 22703 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:12.493456 22628 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.495476 22628 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:12.496296 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:12.496927 22632 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:12.497207 22632 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:12.498035 22632 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.499228 22632 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:12.499497 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54813us
I20260709 13:12:12.502056 29044 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:12.502712 29044 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:12.503763 29044 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.518947 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:12.519683 22781 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:12.541003 22701 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:12.543710 22632 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:12.545953 22783 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:12.550751 22781 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:12.552242 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:12.553043 22779 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:12.569278 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:12.572372 29165 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:12.574924 22779 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:12:12.573338 29165 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:12.581785 29165 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.584252 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:12.588272 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:12.588711 29135 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:12.589335 29135 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:12.589632 22628 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:12:12.589375 29136 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:12.591353 29136 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:12.592382 29136 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.594723 22634 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:12.590477 29135 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:12.637990 23418 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:12:12.643637 29101 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:12.692529 22632 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:12.695079 22781 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:12.700572 29165 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:12.709250 22632 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:12:12.715222 29092 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:12:12.716111 29165 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:12:12.776546 22528 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 12 to 13, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:12.778512 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57713.3us
W20260709 13:12:12.836941 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 68723.2us
I20260709 13:12:12.973351 22703 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:12:12.984947 29158 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:13.001386 22781 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:12:13.015817 29165 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:13.035794 22783 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:12:13.037741 29146 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:13.070142 22634 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:12:13.091774 29174 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:12:13.147980 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:13.175787 22530 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 17 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 18 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:13.290496 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48059.3us
I20260709 13:12:13.350811 22524 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 19, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.353158 22524 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 15 to 16, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:13.359069 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 72649.9us
I20260709 13:12:13.448133 22634 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:12:13.505241 22634 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:13.515992 22779 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:13.571875 29165 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:13.574438 29165 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:13.603695 29136 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:13.629899 22779 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:12:13.694980 29174 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:12:13.739281 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48159.4us
W20260709 13:12:13.831728 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81308.3us
W20260709 13:12:13.957784 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54146.9us
W20260709 13:12:13.962945 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54105.7us
W20260709 13:12:14.048085 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 179151us
W20260709 13:12:14.048552 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 86646.4us
I20260709 13:12:14.609935 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:14.758184 22776 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:12:14.758972 22776 raft_consensus.cc:606] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:14.924002 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 123046us
I20260709 13:12:15.273221 22702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:15.274032 22702 raft_consensus.cc:606] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:15.306244 22631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:15.307077 22631 raft_consensus.cc:606] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:15.308970 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:12:15.310096 29198 raft_consensus.cc:993] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:15.311188 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:15.310767 29198 raft_consensus.cc:1081] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:15.320541 29223 raft_consensus.cc:993] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:15.321276 29223 raft_consensus.cc:1081] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:15.325876 22701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47418
I20260709 13:12:15.326653 22701 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:15.327087 22701 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:15.331730 22635 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:12:15.332535 22635 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:15.332950 22635 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.346978 22701 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.356339 22701 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:15.357357 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:15.360522 22782 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:15.360919 22782 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:15.361704 22782 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.362879 22782 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:15.357446 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:15.368352 22631 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:15.379657 22635 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.383670 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:15.384321 22701 raft_consensus.cc:3055] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:15.384636 22701 raft_consensus.cc:740] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:15.388176 22701 consensus_queue.cc:260] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.388651 22782 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:15.389737 22701 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.396708 22635 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:15.401750 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:15.389844 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:15.403378 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 46006.3us
I20260709 13:12:15.403765 22776 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.405045 29223 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:15.412761 29223 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:15.413772 29223 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.432567 22701 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
W20260709 13:12:15.461731 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 56186.3us
I20260709 13:12:15.461779 22601 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:15.469985 22631 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:15.472324 29226 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:15.522051 29226 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:15.526104 22776 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:15.589212 22526 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 19, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:15.602855 29226 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:15.630650 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 138676us
I20260709 13:12:15.687463 29234 raft_consensus.cc:993] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:12:15.709496 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 47330.5us
I20260709 13:12:15.712481 29234 raft_consensus.cc:1081] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
W20260709 13:12:15.709539 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43912.8us
I20260709 13:12:15.745841 22707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:15.746590 22707 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:15.747025 22707 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.762573 22530 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 16 to 17, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:15.773950 22707 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.786566 22707 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:15.787804 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:15.795148 22628 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:15.795564 22628 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:15.796482 22628 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.787887 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:15.798981 22782 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.798265 22628 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:15.845669 22782 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:15.860725 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:15.862280 29228 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:15.862907 29228 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:15.863879 29228 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:15.882248 22776 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:15.895500 29244 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:15.857968 22628 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:15.938437 22631 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:15.960421 29240 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:16.080976 22528 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:16.098762 22776 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:16.100477 29234 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:16.151889 22706 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:16.159636 29217 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:12:16.312533 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:16.443507 22781 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:12:16.445660 29244 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:16.481339 22628 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:12:16.487412 29244 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:16.806730 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86806.2us
W20260709 13:12:16.814846 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 82917.2us
W20260709 13:12:17.465385 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 88687.8us
I20260709 13:12:17.574744 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:17.655495 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 132041us
W20260709 13:12:17.733587 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 57244.6us
I20260709 13:12:18.153851 22634 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:18.155098 22634 raft_consensus.cc:606] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:18.208336 22704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:18.209100 22704 raft_consensus.cc:606] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:18.212844 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:18.215510 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:18.359632 29269 raft_consensus.cc:993] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:18.360540 29268 raft_consensus.cc:1081] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:18.371949 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf401fd06552416592129b334cdf8167"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:12:18.372869 22634 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.373414 22634 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:18.378839 29273 raft_consensus.cc:993] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:18.379817 29277 raft_consensus.cc:1081] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:18.397459 22781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:12:18.398957 22781 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:18.411928 22781 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:18.451545 22634 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.455556 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:18.456432 22702 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:18.456267 22781 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.456849 22702 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:18.457770 22702 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.459216 22634 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:18.459332 22702 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:18.460377 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:18.461052 22783 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:18.462920 22781 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:18.468974 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:18.474555 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:18.475365 22704 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:18.488500 22634 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:18.490417 22634 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:18.492745 22634 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.502089 22634 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:18.516149 22783 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:18.525192 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:18.496400 22702 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:18.526818 29277 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:18.527150 22704 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:18.527451 29277 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:18.528548 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:18.529271 29277 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.531677 29282 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:18.532331 29282 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:18.534407 29282 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:18.549309 22634 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:18.613472 22530 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:18.605636 22528 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 18 to 19, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.942684 22704 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:18.958844 29277 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:18.971606 22707 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:19.009229 29280 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:19.019090 22781 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:12:19.156898 22627 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:12:19.171059 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83540.7us
I20260709 13:12:19.290972 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:19.325529 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102790us
I20260709 13:12:19.327272 29293 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:19.328889 29298 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 13:12:19.421456 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 215911us
I20260709 13:12:19.460774 29303 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:12:19.475582 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 280075us
W20260709 13:12:19.556797 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 219391us
W20260709 13:12:19.591262 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 95702.7us
W20260709 13:12:19.649314 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48082.2us
W20260709 13:12:19.927802 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 186114us
W20260709 13:12:19.937569 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 196724us
W20260709 13:12:19.995066 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66771.8us
W20260709 13:12:20.040869 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43764.1us
W20260709 13:12:20.675449 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 165892us
W20260709 13:12:20.916707 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52197.4us
I20260709 13:12:21.045960 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:21.277722 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:21.278735 22627 raft_consensus.cc:606] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:21.298178 22706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:21.298894 22706 raft_consensus.cc:606] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:21.304046 22782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:12:21.304816 22782 raft_consensus.cc:606] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:21.306694 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:12:21.308990 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:21.383874 29306 raft_consensus.cc:993] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:21.384573 29306 raft_consensus.cc:1081] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:21.389127 22707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47418
I20260709 13:12:21.389822 22707 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.390213 22707 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.422978 22707 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:21.441468 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:21.442303 22779 raft_consensus.cc:3055] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:21.442689 22707 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:21.442653 22779 raft_consensus.cc:740] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:21.445644 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:21.445488 22779 consensus_queue.cc:260] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:21.446399 23418 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.447110 22779 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.485491 22779 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:21.462216 23418 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:21.507485 22673 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:21.509701 29268 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:21.510380 29268 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:21.511412 29268 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:21.615115 29322 raft_consensus.cc:993] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:21.616607 29268 raft_consensus.cc:1081] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:21.622911 29309 raft_consensus.cc:993] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:21.623617 29309 raft_consensus.cc:1081] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:21.624539 22526 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.629205 22630 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:12:21.631937 22630 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.632754 22630 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:21.639043 22706 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:21.640029 22706 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:21.640501 22706 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.662647 22630 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.665105 22630 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:21.666474 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:21.666882 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:21.667894 22701 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:21.668252 22701 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:21.669179 22701 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.671474 22701 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:21.672096 22779 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:21.697584 22779 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:21.698792 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:21.700238 29334 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:21.700819 29334 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:21.701871 29334 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.704653 22701 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:21.720407 22706 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.789137 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:21.790022 22630 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:21.790392 22630 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:21.791489 22630 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.799173 22706 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:21.800447 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:21.801012 22779 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.804430 22528 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 19 to 20, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 20 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.805851 22630 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:21.830828 22779 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:21.832427 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:21.834925 22630 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:21.842487 29326 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:21.843627 29326 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:21.844636 29326 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:21.898764 22528 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:21.958637 23421 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:21.963584 29268 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:22.125036 22782 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:12:22.128698 29336 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:22.131494 22779 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:22.134363 29268 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:22.195990 22701 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:12:22.200428 29336 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:22.300778 22779 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:22.302556 29268 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:22.314769 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:22.357066 22630 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:22.366954 29326 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:12:22.598512 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 58879.8us
W20260709 13:12:22.616190 29355 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:12:22.720158 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102512us
I20260709 13:12:23.542246 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:23.645076 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50122.9us
I20260709 13:12:24.150878 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:24.151664 22627 raft_consensus.cc:606] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:24.219322 22707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:24.220099 22707 raft_consensus.cc:606] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:24.299811 22707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:24.300666 22707 raft_consensus.cc:606] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:24.323659 29361 raft_consensus.cc:993] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:24.324294 29361 raft_consensus.cc:1081] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:24.346910 22630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:24.347674 22630 raft_consensus.cc:606] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:24.350806 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:12:24.353358 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:24.360034 22776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50386
I20260709 13:12:24.360919 22776 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:24.361404 22776 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.442386 22776 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:24.484634 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:24.485370 23421 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.487077 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:24.487798 22706 raft_consensus.cc:3055] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:24.488137 22706 raft_consensus.cc:740] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:24.488941 22706 consensus_queue.cc:260] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:24.490245 22706 raft_consensus.cc:3060] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:24.511233 22706 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:24.519346 22750 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:24.520567 29293 raft_consensus.cc:2804] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:24.521108 29293 raft_consensus.cc:697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:24.522105 29293 consensus_queue.cc:237] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:24.535457 22776 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:24.547137 23421 raft_consensus.cc:2468] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:24.582887 29352 raft_consensus.cc:993] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:24.587113 29352 raft_consensus.cc:1081] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:24.589277 22782 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:12:24.589951 22782 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:24.590346 22782 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:24.616211 22782 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:24.621389 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:24.622373 22630 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:24.622812 22630 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:24.623860 22630 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:24.625578 22630 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:24.632217 29352 raft_consensus.cc:993] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:24.632812 29352 raft_consensus.cc:1081] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:24.634684 22707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:24.635316 22707 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:24.635672 22707 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:24.637950 22782 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:24.644119 22707 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:24.659011 22630 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:24.659837 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:24.660857 22706 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:24.664610 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:24.665407 22627 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:24.665721 22627 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:24.666543 22627 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:24.667935 22627 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:24.670979 22706 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:24.673096 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:24.674865 29293 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:24.676072 29293 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:24.689703 22707 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:24.692013 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: true dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:24.703505 23421 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:24.695379 22526 catalog_manager.cc:5697] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 11 to 12, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.700344 29293 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:24.740520 22627 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:24.742295 23421 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:24.796336 22673 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:24.802981 29361 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:24.805528 29361 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:24.811944 29361 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:24.932658 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 76352.6us
I20260709 13:12:24.945453 23421 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:24.948148 29360 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:24.937642 22528 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:25.010486 29374 raft_consensus.cc:993] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:25.011250 29374 raft_consensus.cc:1081] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:25.048354 22627 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:12:25.049140 22627 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:25.049639 22627 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:25.081966 22627 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:25.084193 22627 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:25.095034 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:25.095767 22776 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:25.097599 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:25.098351 22706 raft_consensus.cc:3055] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:25.098884 22706 raft_consensus.cc:740] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:25.122401 22776 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:25.099661 22706 consensus_queue.cc:260] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:25.125566 22706 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:25.133868 22706 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:25.137262 22601 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:25.154323 29332 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:12:25.186317 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44736.5us
W20260709 13:12:25.218160 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79568.1us
I20260709 13:12:25.239331 29332 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:25.240839 29332 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:25.269122 22526 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 14 to 15, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.303359 22706 raft_consensus.cc:1275] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:25.306210 22627 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:25.310019 23418 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:25.312059 29385 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:25.318732 22706 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:12:25.339484 29326 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:25.353572 22634 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:25.354667 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:25.430009 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 78036.4us
I20260709 13:12:25.445715 29382 consensus_queue.cc:1048] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:25.476681 29383 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:25.530375 22530 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:25.686882 29373 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:12:25.843758 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 272657us
I20260709 13:12:25.947412 29400 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:12:25.954929 22707 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:12:25.974639 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 164210us
I20260709 13:12:26.078261 22782 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:12:26.087095 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 248292us
I20260709 13:12:26.144435 29395 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:26.149412 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 54649.1us
I20260709 13:12:26.158768 22776 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:26.191273 29396 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:26.253594 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60923.3us
W20260709 13:12:26.291215 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 72404.3us
W20260709 13:12:26.316051 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49759us
W20260709 13:12:26.470293 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 65631.6us
W20260709 13:12:26.545421 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102001us
W20260709 13:12:26.704862 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 149375us
W20260709 13:12:26.732256 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 187069us
W20260709 13:12:26.737901 29424 log.cc:927] Time spent T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.307s	user 0.003s	sys 0.000s
W20260709 13:12:26.797040 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 77138.1us
I20260709 13:12:26.798830 23415 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:12:26.802529 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:26.821070 29365 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:12:26.905418 22634 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:12:26.918902 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 65544.1us
I20260709 13:12:26.940277 29366 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Committing config change with OpId 22.16: config changed from index 5 to 16, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:26.957351 29427 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 13:12:26.956553 23415 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 5 to 16, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:12:26.984472 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43620.9us
I20260709 13:12:27.008342 22519 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53c258bc80ce4d88bac804cb01a34b32 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 13:12:27.063844 22526 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 5 to 16, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New cstate: current_term: 22 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.097900 22627 raft_consensus.cc:2955] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Committing config change with OpId 22.16: config changed from index 5 to 16, VOTER 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:27.328845 22696 tablet_service.cc:1558] Processing DeleteTablet for tablet 53c258bc80ce4d88bac804cb01a34b32 with delete_type TABLET_DATA_TOMBSTONED (TS 91ebb78bf12f41baae84669c1effc423 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:49918
I20260709 13:12:27.348259 29445 tablet_replica.cc:333] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:27.354055 29445 raft_consensus.cc:2243] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:27.363448 29445 raft_consensus.cc:2272] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:27.395903 22782 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:12:27.396626 22782 raft_consensus.cc:606] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:27.429066 29445 ts_tablet_manager.cc:1916] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:12:27.478909 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70283.1us
I20260709 13:12:27.536211 29432 raft_consensus.cc:993] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:27.537083 29432 raft_consensus.cc:1081] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:27.539770 22707 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47418
I20260709 13:12:27.540557 22707 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:27.540963 22707 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:27.549038 29445 ts_tablet_manager.cc:1929] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 22.15
I20260709 13:12:27.549772 29445 log.cc:1199] T 53c258bc80ce4d88bac804cb01a34b32 P 91ebb78bf12f41baae84669c1effc423: Deleting WAL directory at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/wals/53c258bc80ce4d88bac804cb01a34b32
I20260709 13:12:27.567888 22707 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:27.569991 22520 catalog_manager.cc:5028] TS 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): tablet 53c258bc80ce4d88bac804cb01a34b32 (table test-workload [id=9cbe73e0e42446e1aa17adfa4b74cc2f]) successfully deleted
I20260709 13:12:27.578660 22707 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:27.911731 22600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.011s	sys 0.003s
W20260709 13:12:27.912673 22600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.011s	sys 0.003s
W20260709 13:12:27.913156 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 510573us
I20260709 13:12:28.084841 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:28.088708 22630 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:28.098951 22709 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:28.128414 29449 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:28.130472 29449 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:28.159889 29449 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:12:28.420936 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 58183us
I20260709 13:12:28.462317 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:28.464100 29336 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:28.470670 29336 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:28.463619 22635 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:28.488713 22635 raft_consensus.cc:606] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Received request to transfer leadership to TS 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:28.502732 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 279750us
I20260709 13:12:28.507300 29336 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:28.511065 22630 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
W20260709 13:12:28.517791 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 69483us
I20260709 13:12:28.518254 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:28.552784 29449 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:28.565423 29449 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:28.578603 29336 raft_consensus.cc:993] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:28.579269 29336 raft_consensus.cc:1081] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Signalling peer 91ebb78bf12f41baae84669c1effc423 to start an election
I20260709 13:12:28.581374 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:28.611927 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:28.615262 29429 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:28.627295 22656 debug-util.cc:398] Leaking SignalData structure 0x7b0800366720 after lost signal to thread 21529
W20260709 13:12:28.628947 22656 debug-util.cc:398] Leaking SignalData structure 0x7b0800366780 after lost signal to thread 22567
W20260709 13:12:28.630106 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 90874.3us
W20260709 13:12:28.631609 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002d4c80 after lost signal to thread 22659
W20260709 13:12:28.633455 22656 debug-util.cc:398] Leaking SignalData structure 0x7b080003f880 after lost signal to thread 22734
W20260709 13:12:28.636135 22656 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0800 after lost signal to thread 22808
W20260709 13:12:28.638736 22656 debug-util.cc:398] Leaking SignalData structure 0x7b0800386040 after lost signal to thread 23445
I20260709 13:12:28.643024 22709 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
 from {username='slave'} at 127.0.0.1:47414
I20260709 13:12:28.643867 22709 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:28.644275 22709 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.696463 22709 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:28.700369 22709 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:28.714532 22782 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:28.772444 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:28.783080 29393 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:28.783838 29393 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:28.785447 29393 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:28.814395 22701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:28.815259 22701 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:28.823570 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:28.824450 22631 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:28.824781 22631 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:28.825732 22631 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:28.823598 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:28.827605 22782 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.831965 22631 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:28.862011 22524 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 16 to 17, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.863752 22778 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:37284
I20260709 13:12:28.864518 22778 raft_consensus.cc:606] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:28.866596 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:12:28.871749 22782 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:12:28.877043 22631 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:12:28.882145 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:28.884369 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:28.906566 29393 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:28.912124 29393 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:28.913313 29393 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.119844 29401 raft_consensus.cc:993] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:29.120630 29401 raft_consensus.cc:1081] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:29.123346 22628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45666
I20260709 13:12:29.124188 22628 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:29.124882 22628 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:29.135474 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55395.1us
I20260709 13:12:29.143714 22524 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 12 to 13, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 13 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:29.182174 29393 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: : Instructing follower 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:29.184950 29393 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Signalling peer 1f99ad34cc414233929e0a92f8c3d8db to start an election
I20260709 13:12:29.201941 22628 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.204546 22628 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:29.205749 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:29.208333 22634 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:45674
I20260709 13:12:29.209923 22782 raft_consensus.cc:3055] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:29.210274 22634 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:29.210465 22782 raft_consensus.cc:740] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:29.210813 22634 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:29.212759 22782 consensus_queue.cc:260] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.214077 22782 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:29.212368 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:29.216042 22702 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:29.225076 22634 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.234752 22628 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:29.240983 22702 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:29.249511 22776 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:12:29.250221 22601 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:29.251720 29384 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:29.252270 29384 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:29.253208 22634 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:29.254400 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:29.253252 29384 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.255079 22701 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:29.255489 22701 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:29.255156 29476 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:29.256491 22701 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.257794 22701 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:29.241096 22782 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:29.270363 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:29.271270 22776 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:29.285904 22782 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:29.285916 22634 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:12:29.288090 29447 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:29.293404 29326 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:29.296190 22702 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:29.298719 29426 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:29.320729 22598 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 79ed72c313bf419bb01d45a59391a262. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:29.325692 22701 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:29.327374 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:29.331302 29336 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:29.332026 22776 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:29.334213 29336 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:29.338466 29336 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:29.401196 22524 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:29.439306 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 48890.3us
I20260709 13:12:29.509300 22707 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:12:29.518643 22530 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 16 to 17, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:29.523559 22782 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:12:29.527835 29487 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Committing config change with OpId 12.8: config changed from index -1 to 8, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
I20260709 13:12:29.540916 22627 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:29.551388 29453 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:12:29.597112 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 102560us
W20260709 13:12:29.612778 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 110813us
I20260709 13:12:29.688697 22706 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:29.691303 22776 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:12:29.703473 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 200873us
I20260709 13:12:29.708240 29426 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:12:29.725941 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 120410us
W20260709 13:12:29.726068 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 118111us
I20260709 13:12:29.723577 22530 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: config changed from index -1 to 8, NON_VOTER 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) added. New cstate: current_term: 12 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } }
W20260709 13:12:29.807665 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 53449.9us
W20260709 13:12:29.844874 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 97939.6us
I20260709 13:12:29.853454 29501 ts_tablet_manager.cc:933] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
I20260709 13:12:29.873814 22701 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:29.903882 29336 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:29.910970 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:29.905108 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 96151.8us
I20260709 13:12:29.924117 29501 tablet_copy_client.cc:323] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.129:37283
I20260709 13:12:29.991125 22645 tablet_copy_service.cc:140] P 1f99ad34cc414233929e0a92f8c3d8db: Received BeginTabletCopySession request for tablet 79ed72c313bf419bb01d45a59391a262 from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:59768)
I20260709 13:12:29.991887 22645 tablet_copy_service.cc:161] P 1f99ad34cc414233929e0a92f8c3d8db: Beginning new tablet copy session on tablet 79ed72c313bf419bb01d45a59391a262 from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:59768: session id = 6a31ac6451ff4f73bee634487d84d001-79ed72c313bf419bb01d45a59391a262
I20260709 13:12:30.006948 29476 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:30.041316 22780 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:30.047570 22777 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:12:30.063854 29488 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:30.077430 29447 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:30.080194 22645 tablet_copy_source_session.cc:215] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:30.085747 29501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79ed72c313bf419bb01d45a59391a262. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.121079 22707 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
W20260709 13:12:30.179792 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51685.7us
I20260709 13:12:31.368208 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:12:31.722343 21531 debug-util.cc:398] Leaking SignalData structure 0x7b0800370360 after lost signal to thread 22675
W20260709 13:12:31.723331 21531 kernel_stack_watchdog.cc:198] Thread 22675 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 221ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 13:12:31.884724 22673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.686s	user 0.011s	sys 0.007s
W20260709 13:12:31.885471 22673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.687s	user 0.012s	sys 0.007s
I20260709 13:12:31.899171 29524 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:31.899976 29524 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:31.902287 29524 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:31.903039 29524 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:31.903682 29524 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:31.905292 29525 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:31.910295 29525 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:31.913255 29525 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:31.913945 29525 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:31.915643 29524 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:31.919170 29524 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:31.919956 29524 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:31.926159 22779 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.725s	user 0.018s	sys 0.014s
W20260709 13:12:31.927476 22779 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.729s	user 0.020s	sys 0.014s
W20260709 13:12:31.928592 23385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.052s	sys 0.039s
W20260709 13:12:31.929706 23385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.075s	user 0.052s	sys 0.040s
I20260709 13:12:31.935001 29526 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:31.935865 29526 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:31.938675 29524 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:31.939401 29524 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:31.914520 29525 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:31.951651 29527 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:31.952792 29527 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:31.955523 29527 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:31.939929 29524 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:31.958283 29524 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:31.960727 23384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.041s	user 0.018s	sys 0.035s
W20260709 13:12:31.968405 23384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.019s	sys 0.035s
I20260709 13:12:31.960922 29526 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:31.962639 29525 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:31.977108 29532 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:31.977331 29529 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:31.997676 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.000757 23415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.002319 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.002342 22779 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 4315) took 1903 ms. Trace:
I20260709 13:12:32.004477 22779 rpcz_store.cc:276] 0709 13:12:30.098424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.098558 (+   134us) service_pool.cc:224] Handling call
0709 13:12:30.098875 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:30.104574 (+  5699us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:30.104608 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:30.105191 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:12:30.187071 (+ 81880us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:12:31.928087 (+1741016us) log.cc:844] Serialized 14 byte log entry
0709 13:12:32.001864 (+ 73777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:32.001989 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:32.002054 (+    65us) raft_consensus.cc:1685] finished
0709 13:12:32.002072 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:32.002237 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5213,"prepare.queue_time_us":44260,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":6901,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":1883516,"threads_started":4,"wal-append.queue_time_us":83732,"wal-append.run_cpu_time_us":1983,"wal-append.run_wall_time_us":1007496}
I20260709 13:12:32.015434 29531 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:31.991087 29529 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:31.991212 29532 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.019560 29529 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.021241 29529 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.021730 29538 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.021893 29529 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.016444 29531 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.026085 29531 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.027268 29536 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.028017 29536 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.022367 29538 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:32.035089 22707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.834s	user 0.006s	sys 0.067s
W20260709 13:12:32.035539 22707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.836s	user 0.006s	sys 0.067s
I20260709 13:12:32.033449 29532 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.037858 29536 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.038822 29538 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.043541 29530 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.044232 29530 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.049237 29530 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.052806 29531 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.053633 29531 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.056421 29531 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.061792 29533 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.062600 29533 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.071385 29535 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.073714 29535 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:32.076632 22674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.881s	user 0.017s	sys 0.016s
I20260709 13:12:32.077319 29535 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.078372 29539 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.079051 29539 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.081432 29539 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.081916 29539 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.082427 29539 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.083287 29533 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.083935 29531 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.084563 29531 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.087073 29531 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.087582 29531 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.088114 29531 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:32.092375 23383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.120s	user 0.024s	sys 0.034s
W20260709 13:12:32.092963 22674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.898s	user 0.018s	sys 0.017s
I20260709 13:12:32.090363 29531 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:32.097507 23383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.221s	user 0.024s	sys 0.034s
I20260709 13:12:32.105471 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.110989 29534 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.111413 29496 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.112111 29538 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.112784 29538 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.104879 29527 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.113778 29541 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.113909 29527 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.115473 29540 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.115986 29536 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.116129 29540 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.116624 29536 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.117509 29530 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.118105 29530 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.106810 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.120303 29536 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.120679 29530 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.121297 29534 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.122298 29536 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.122676 22634 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 12.
I20260709 13:12:32.122918 29536 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.123776 29532 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.124365 29532 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.111555 29458 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.125556 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.125375 29458 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:32.125838 23382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.013s	sys 0.018s
W20260709 13:12:32.126430 23382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.246s	user 0.013s	sys 0.018s
I20260709 13:12:32.115370 29538 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.127240 29534 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.116271 29527 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.132817 29540 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.112169 29496 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.101560 29531 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.146062 29531 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.132530 29525 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.114387 29541 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.135573 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.097992 29535 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.146584 29536 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.152661 29540 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.153281 29540 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.155825 29540 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.148344 23421 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:32.137225 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.136035 29539 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.130877 29532 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.128127 29538 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.162442 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.161366 29535 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.212342 22634 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
W20260709 13:12:32.215598 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800345620 after lost signal to thread 21529
W20260709 13:12:32.217193 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800346b60 after lost signal to thread 22567
I20260709 13:12:32.148432 29531 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.144002 29458 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.223567 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.224783 22634 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:12:32.147718 29496 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.158638 29540 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.149509 29530 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.317377 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.149230 29541 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.160390 29533 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.322530 29527 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.323477 29544 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.147290 29525 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.326416 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.333858 22634 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
W20260709 13:12:32.314482 29501 consensus_meta.cc:287] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 2.108s	user 0.014s	sys 0.010s
I20260709 13:12:32.379220 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.220844 29538 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.381191 29533 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.375523 29541 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.372798 29540 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.416144 29549 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.417127 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.379902 29546 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.414126 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.391597 29535 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.422924 29547 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.423617 29547 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.424453 29458 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:32.424896 29458 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.426112 29547 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.427680 29501 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46252 (request call id 1608) took 2728 ms. Trace:
I20260709 13:12:32.428164 29501 rpcz_store.cc:276] 0709 13:12:29.699147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:29.699330 (+   183us) service_pool.cc:224] Handling call
0709 13:12:29.853987 (+154657us) ts_tablet_manager.cc:934] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283)
0709 13:12:30.084116 (+230129us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:30.182783 (+ 98667us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:30.186859 (+  4076us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:32.389598 (+2202739us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:32.427018 (+ 37420us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:32.427471 (+   453us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:32.427555 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":39817,"tablet-copy.queue_time_us":112403,"thread_start_us":57672,"threads_started":2}
I20260709 13:12:32.430219 29501 tablet_copy_client.cc:806] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:32.395277 29534 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.443194 29501 tablet_copy_client.cc:670] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:32.429216 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 283) took 2189 ms. Trace:
I20260709 13:12:32.443099 29534 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.443881 22528 rpcz_store.cc:276] 0709 13:12:30.239550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.239728 (+   178us) service_pool.cc:224] Handling call
0709 13:12:32.428977 (+2189249us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:12:32.402120 29526 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.445766 29526 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.446165 29534 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.432353 29525 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.447028 29525 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.447738 29525 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.448552 29526 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.379011 29530 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.434199 29546 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.416862 29549 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.451958 29546 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.425323 29458 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:32.452435 29549 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.453644 29525 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.436316 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.377696 29545 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.401877 29550 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.395057 29524 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.433144 29538 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.399483 29537 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.395608 29544 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.381052 29527 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.482570 29548 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.483603 29548 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.485265 29527 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.436874 29533 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.486604 29533 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.486923 29548 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.487495 29533 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.489207 29548 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.465613 22598 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.489939 29548 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.400944 22634 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:32.493753 29548 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.494825 29548 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.495046 29546 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:32.482048 29552 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.499329 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.499884 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.437853 29541 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.454972 29530 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.527757 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.476040 29549 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:32.509380 22630 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:32.510157 22634 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:32.422175 29540 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:32.517066 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.322s	user 0.015s	sys 0.010s
I20260709 13:12:32.480386 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:12:32.536005 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55786.6us
W20260709 13:12:32.537652 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.343s	user 0.016s	sys 0.010s
W20260709 13:12:32.538229 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 2.34351e+06us
I20260709 13:12:32.534837 22627 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 22.
I20260709 13:12:32.542912 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.482133 29547 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.507678 29546 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.547168 29525 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:32.533638 29549 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.495597 29548 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.537056 29530 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.545832 29533 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.546685 29492 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:32.576390 29492 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.577023 29492 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:32.555912 29546 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:32.572038 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.579885 29533 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.580570 29533 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:32.586757 29536 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:32.587522 29536 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.588047 29536 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:32.589986 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.591429 29533 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.592087 29533 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.592638 29533 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.476744 29550 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.595356 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.595084 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.531005 29524 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.561581 29534 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:32.569689 29549 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:32.600520 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.476464 29545 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.555269 29547 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.496021 29537 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.613615 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:32.530050 29552 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.622272 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.594954 29525 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.625296 29543 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:32.625851 29543 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.626241 29543 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:32.604034 29534 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.607575 29538 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:32.627307 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.633411 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.638125 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.611851 29539 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.641511 22780 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:32.647055 29557 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.647814 29557 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.650722 29551 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.651460 29551 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.651887 29548 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.653399 29544 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.634388 29540 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.653962 29551 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.654803 29555 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.654558 29540 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.655435 29555 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.657071 29540 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.640473 29538 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.657979 29555 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.658212 29538 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:32.658738 29553 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.659343 29553 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.662094 29553 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.642109 29539 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.663100 29558 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.663453 29535 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.663805 29558 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.664081 29535 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.665172 29539 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.666167 29558 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:32.667246 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800346b20 after lost signal to thread 22581
I20260709 13:12:32.668987 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.669677 22709 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
W20260709 13:12:32.672107 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94795.2us
I20260709 13:12:32.654023 29557 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.670948 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.677662 29533 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.678308 29554 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.603010 29546 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.599951 29530 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.777118 29553 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.734704 29542 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.782425 29553 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.783071 29542 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.783684 22777 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:12:32.688103 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.784708 29530 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.784859 29553 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.785539 29553 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.723941 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:32.681075 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.796206 29525 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.799613 29524 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.803639 29458 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.803963 29536 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.806063 29458 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.757427 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.638990 29534 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:32.640637 29496 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:32.786070 29553 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.676961 29558 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:32.816398 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 202923us
I20260709 13:12:32.791936 29535 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.781450 29562 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.821398 29530 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.822880 29531 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.822402 29557 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:32.819888 29555 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.828969 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.952505 29555 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.952876 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.953344 29531 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.955626 22629 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:32.820814 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.828920 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.830292 23419 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:32.950786 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.958328 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:32.969015 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.957263 22701 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:12:32.951735 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.953074 29530 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.953653 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:32.824347 29561 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:32.953140 29562 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.954442 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:32.969878 22629 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:12:32.972959 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:32.971315 29557 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.975863 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.974599 22710 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:12:32.808847 29492 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.890518 29546 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.948774 29536 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.976675 29538 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.811666 29554 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.968833 29559 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.988791 29549 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:32.975934 29531 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:32.990653 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.991135 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.819113 29496 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.992461 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:33.016023 29533 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:12:32.977638 29561 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.021059 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:33.022022 29561 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:32.993121 29560 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:32.993278 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:32.993983 29538 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.022822 29560 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:32.994709 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:32.996055 29554 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.997691 29553 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:32.998507 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 232134us
I20260709 13:12:33.024734 22601 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:33.024775 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.025349 29560 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:32.999472 22673 consensus_peers.cc:597] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 -> Peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877): Couldn't send request to peer 6a31ac6451ff4f73bee634487d84d001. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b8291ea7e33a40b5afaebc02d6ce7e9d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:33.026219 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.027067 22629 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:33.028045 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.028791 29546 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:33.029192 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.000145 23415 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.001125 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.031404 22627 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:33.033015 23411 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:33.001758 29529 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:33.034770 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08000f5ac0 after lost signal to thread 22659
I20260709 13:12:33.002108 29492 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:33.036325 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080018e560 after lost signal to thread 22734
I20260709 13:12:33.002396 29555 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.003139 29564 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:33.003566 23412 raft_consensus.cc:1240] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:32.989815 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.004271 29557 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.007102 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:33.008191 29550 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.009532 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.004307 29559 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.012769 29530 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.014643 29562 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.949743 29558 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.029237 29546 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:33.056831 29558 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:32.938055 29543 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.057955 29545 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:33.058151 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.018857 29496 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:12:33.060171 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08001a2ca0 after lost signal to thread 22808
I20260709 13:12:33.061659 23421 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.063016 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.019428 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.017668 23417 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
W20260709 13:12:33.065440 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08003a25a0 after lost signal to thread 23445
I20260709 13:12:33.023898 29541 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.039017 23420 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:33.030441 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.047613 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.041734 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.066471 22629 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:33.067447 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:33.065842 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800376f60 after lost signal to thread 29336
I20260709 13:12:33.075049 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:33.078853 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4300 after lost signal to thread 29460
W20260709 13:12:33.079735 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080040f640 after lost signal to thread 29461
I20260709 13:12:33.079483 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:33.080056 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08002a5800 after lost signal to thread 29463
W20260709 13:12:33.080487 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08001abee0 after lost signal to thread 29474
W20260709 13:12:33.080952 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080032b580 after lost signal to thread 29489
W20260709 13:12:33.081208 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08003fabe0 after lost signal to thread 29490
W20260709 13:12:33.081470 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800328100 after lost signal to thread 29494
W20260709 13:12:33.081691 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08002e13a0 after lost signal to thread 29495
I20260709 13:12:33.066772 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:33.069475 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.082937 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.064498 29550 consensus_queue.cc:1059] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:12:33.084286 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.084772 29550 raft_consensus.cc:3055] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:33.085265 29550 raft_consensus.cc:740] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:33.085508 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.086891 29557 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:33.087353 29557 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.087738 29557 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:33.082228 22564 debug-util.cc:398] Leaking SignalData structure 0x7b08003c06e0 after lost signal to thread 29508
I20260709 13:12:33.069901 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:33.092483 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 100071us
I20260709 13:12:33.093739 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.095057 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:33.092562 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.096324 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.092562 22635 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:33.098255 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:33.100351 22627 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:12:33.101035 22630 raft_consensus.cc:1240] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:12:33.101389 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.101984 22627 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.061661 29561 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:33.056870 29546 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.091045 29530 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:33.105144 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.102802 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.093103 29541 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:33.106932 23384 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:33.112370 29530 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.136601 29530 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:33.138022 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.118966 29549 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.139501 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.120363 29559 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.106999 29544 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:33.051100 29564 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.140749 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.141906 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.142874 29564 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.142975 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.122706 29540 consensus_queue.cc:1059] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:33.123126 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.144224 29540 raft_consensus.cc:3055] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:33.144276 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.144796 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.145032 22709 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:12:33.144774 29540 raft_consensus.cc:740] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.122761 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.146764 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.147205 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.146996 29540 consensus_queue.cc:260] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.129374 29501 tablet_copy_client.cc:538] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:33.086359 29550 consensus_queue.cc:260] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.131767 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.132062 29534 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.151448 29550 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.153079 29534 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.128834 29542 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.134872 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.105048 29538 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:33.138238 29529 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:33.147408 22709 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:33.105440 29561 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.155972 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.108316 29533 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:33.119766 22627 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:33.156741 22635 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.165184 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.157620 23414 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:33.165952 22706 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 19.
I20260709 13:12:33.165994 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.052443 29560 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:12:33.168236 22627 raft_consensus.cc:2393] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 18: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:33.159260 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.145195 29552 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.168610 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.168804 22780 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:12:33.156040 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 50956.5us
I20260709 13:12:33.166303 23384 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:33.200582 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.201318 29536 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:33.158195 29538 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.169924 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.201928 29536 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:33.202136 29538 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.173249 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.206153 22629 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.162878 29561 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.184293 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.184639 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.206992 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.185148 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.186784 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.188339 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.189404 29540 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:33.161263 29544 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:33.190292 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.213769 29544 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.214269 29544 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:33.221771 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.224328 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.231163 29546 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.202209 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.259531 22626 raft_consensus.cc:1240] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:33.259708 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.260911 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.261242 29546 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.171255 29559 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:33.202484 22710 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:33.203099 29536 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.263054 29559 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:33.264078 29559 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.265401 29559 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.203863 29537 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.204016 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.262105 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.164862 29529 raft_consensus.cc:697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:33.268792 29529 consensus_queue.cc:237] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.270767 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.272459 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.279803 29561 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.282606 29561 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.191987 29496 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.165195 29533 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.245091 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.246392 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.303921 22775 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:33.247890 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.248000 29541 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:12:33.249289 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.249562 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:33.249367 29552 consensus_queue.cc:1059] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:33.309204 22780 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 11.
I20260709 13:12:33.250816 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.251093 23414 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:33.252740 22630 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:33.305481 29533 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.321327 22633 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.253050 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.253705 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.323566 23418 raft_consensus.cc:1240] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 22. Current term is 23. Ops: []
I20260709 13:12:33.324719 22627 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:33.255587 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.257016 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.205202 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.284273 29530 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.287197 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.142544 22705 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:12:33.288106 29557 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.200748 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:33.289649 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:33.291096 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 49220.2us
I20260709 13:12:33.224627 29501 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:12:33.295492 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.303073 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.303277 22783 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:33.307294 22635 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:33.310209 22781 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:33.311718 29547 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.316232 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.281934 29538 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.325940 22629 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:33.327471 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.337275 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.364215 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.374872 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.375893 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.377245 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.378255 29551 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:33.379057 29560 consensus_queue.cc:1059] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:12:33.379851 29530 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.380497 29540 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:33.411387 29555 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:33.411998 29555 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.412999 29496 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.422629 29561 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:33.423199 29561 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.382946 29545 raft_consensus.cc:3055] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:33.386216 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.424984 29545 raft_consensus.cc:740] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.425985 29545 consensus_queue.cc:260] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.427585 29545 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:33.428483 29533 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.347859 29544 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.427066 29555 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.432112 29544 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.373090 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.433573 22776 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:33.433848 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.434513 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.435066 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.387606 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.437651 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.439826 29566 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:33.396596 22629 raft_consensus.cc:1240] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
W20260709 13:12:33.442227 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54756.6us
I20260709 13:12:33.398132 22627 raft_consensus.cc:1240] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.399870 29560 raft_consensus.cc:3055] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:33.401131 29543 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.448411 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.401597 22779 raft_consensus.cc:2393] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:33.402148 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.424186 29561 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.456179 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.445188 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:33.448432 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:33.449536 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 100900us
W20260709 13:12:33.450402 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 249739us
I20260709 13:12:33.459306 22749 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.466005 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.442178 29542 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:33.447739 29560 raft_consensus.cc:740] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:33.466565 29561 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:33.466773 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.477427 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.466982 29567 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:12:33.482774 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 98490.7us
I20260709 13:12:33.483029 29546 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:33.484251 29546 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.485046 29533 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:33.467043 29542 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.486166 29542 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.468966 29564 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:33.469058 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.470055 29560 consensus_queue.cc:260] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.493914 29560 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:33.490909 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.497052 22628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.492349 29555 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.506400 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.507433 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.507686 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.508013 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.502274 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.508831 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.510589 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:33.511469 23418 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:33.512640 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.512905 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.514696 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.494827 29563 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:33.515353 22702 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:33.515486 29557 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.516251 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.516542 29530 consensus_queue.cc:1059] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:12:33.516846 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.517316 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.518177 29530 raft_consensus.cc:3055] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:12:33.518309 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.518676 29530 raft_consensus.cc:740] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.516897 29539 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:33.519790 29530 consensus_queue.cc:260] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.521320 29530 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:33.522459 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.511482 29575 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:33.524945 29575 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.525285 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.520179 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.526755 29565 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:33.527994 22630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.528316 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.528867 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.529479 22633 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:33.529529 23384 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:33.520259 29539 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.531435 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.533890 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.518528 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.535046 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.513517 22708 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:33.530540 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.536230 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.532929 29539 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:33.537947 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.538647 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.530465 23384 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
I20260709 13:12:33.539848 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.540904 29549 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:33.541415 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.510289 29453 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:33.530846 29496 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:33.543043 29496 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.542796 29453 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.543458 29496 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:33.543692 29492 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:33.544389 29492 raft_consensus.cc:697] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:33.545495 29492 consensus_queue.cc:237] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.547399 29572 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:33.547787 29549 raft_consensus.cc:697] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:33.548170 29453 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:33.528458 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.549206 29549 consensus_queue.cc:237] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:33.550817 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 67150.7us
I20260709 13:12:33.549572 29577 consensus_queue.cc:1059] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:12:33.552111 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:33.553112 29544 raft_consensus.cc:3055] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:33.553532 29544 raft_consensus.cc:740] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:33.553673 29541 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:33.554358 29541 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:33.547915 29572 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.554793 29541 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:33.525344 29575 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:33.555225 29572 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:33.553629 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:12:33.557061 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60275.9us
I20260709 13:12:33.554778 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.559137 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.559705 22634 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.562597 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.561200 22627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.564029 22627 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:33.564788 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.564028 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:33.566995 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.569089 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.576148 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.575096 22776 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.573616 22709 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:33.574361 22633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.578018 22633 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:33.554719 29544 consensus_queue.cc:260] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.580123 29544 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:33.572785 29564 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.582938 29540 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.572438 29577 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:33.584717 29577 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.585150 29577 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:33.568893 29557 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:12:33.586990 29557 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.587435 29557 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:33.573449 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:33.593206 29453 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:33.567589 29535 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:33.599022 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:33.576326 29543 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.608570 29543 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.611729 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.613379 29567 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:33.613814 29567 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.614156 29567 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:33.614719 22528 catalog_manager.cc:5697] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 12 to 13, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 13 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:33.617019 29539 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.616866 22528 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 15, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:33.618216 22628 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:33.619279 29565 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:33.603681 29533 consensus_queue.cc:1175] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ebb78bf12f41baae84669c1effc423" responder_term: 15 status { last_received { term: 15 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 15 index: 9 } }
I20260709 13:12:33.622205 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.623833 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:33.624866 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45880.4us
I20260709 13:12:33.628211 22634 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:33.623833 23385 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.626999 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:33.632470 29532 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:33.637499 29536 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:33.633214 29530 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:33.642473 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.645092 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.647221 29534 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:33.650071 29530 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:33.650401 29538 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.650636 29530 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.651110 29530 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:33.651167 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.652551 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.653340 29545 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:33.654055 29545 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:33.655004 29553 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:33.655280 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.664966 22633 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:33.667637 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:33.671777 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.673388 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:33.674928 22627 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:33.674010 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:33.692896 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:33.691378 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 122320us
I20260709 13:12:33.695731 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.696084 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:33.701128 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.701426 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.700990 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.705219 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.707641 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.715215 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:33.719441 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45439.3us
I20260709 13:12:33.720434 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.676723 29561 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:33.721451 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:33.722442 22627 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:33.678962 29545 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:33.724534 22709 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:12:33.725550 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:33.734381 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.736414 22627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
mode: GRACEFUL
new_leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:59422
I20260709 13:12:33.737160 22627 raft_consensus.cc:606] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Received request to transfer leadership to TS a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.740422 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:33.742547 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 176083us
I20260709 13:12:33.742578 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:33.743644 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:33.706291 29453 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:33.749209 22628 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:33.744596 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:33.719777 29535 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:33.724261 29563 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.759788 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.755002 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.736505 29533 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:33.732808 29545 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.764432 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:33.741525 29558 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.735960 29576 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:33.714758 29530 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:33.773962 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68183.7us
I20260709 13:12:33.778573 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.778859 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.754223 29543 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.758364 29535 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:33.767463 29561 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:33.736275 29496 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.768723 29545 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:33.781457 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.812172 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:33.750077 29559 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:12:33.826114 22710 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:33.783604 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.814999 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:33.815336 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:33.818490 22783 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:33.819332 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.820151 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.822973 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.824671 29530 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.826855 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.822208 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.781847 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.763065 29533 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.773432 29547 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:33.766039 29581 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.837752 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.842830 22777 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:33.840042 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.813208 29535 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:33.844496 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.807328 29539 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.845825 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.845899 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.854975 29553 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.856279 29496 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.857710 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.844486 29557 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.861039 29557 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.842483 29533 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:33.847352 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:33.869951 29538 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:33.828631 29561 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.881744 29561 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:33.886868 29582 ts_tablet_manager.cc:933] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.891347 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:33.892452 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45186.5us
I20260709 13:12:33.815062 29534 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:12:33.781322 29542 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.899120 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.899259 29542 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:33.908365 23418 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.910351 22776 raft_consensus.cc:1240] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:33.861784 29541 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:33.920385 22626 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:12:33.923038 22634 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:33.924942 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:33.925628 29560 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:33.842290 29547 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:33.930202 29547 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:33.934353 29453 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:33.948853 22629 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.926260 29560 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:33.877833 29586 consensus_queue.cc:1059] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:12:33.904090 29575 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.965724 23421 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:33.968201 22631 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:12:33.968873 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 223077us
I20260709 13:12:33.977658 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.986795 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:33.996589 22775 raft_consensus.cc:1240] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:33.966805 29550 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:33.996709 22781 raft_consensus.cc:1240] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:34.010495 29547 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.995594 29453 consensus_queue.cc:1059] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:12:34.014045 29547 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.018112 29541 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:33.930230 29577 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:33.981004 29565 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:12:34.022466 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:33.971906 29543 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:12:34.024968 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:34.031872 29545 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.034204 29545 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:33.975239 29562 consensus_queue.cc:1059] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:12:34.043536 29562 raft_consensus.cc:3055] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:34.043985 29562 raft_consensus.cc:740] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.044973 29575 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.036299 22776 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:34.002511 29550 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.044994 29562 consensus_queue.cc:260] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.041428 29533 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.030157 29572 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.047696 29550 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.049028 29550 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:34.049291 29533 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.049551 29556 raft_consensus.cc:3055] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:34.050048 29556 raft_consensus.cc:740] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.020212 29560 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.052009 29560 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:34.053786 29561 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.054212 22634 raft_consensus.cc:1240] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
I20260709 13:12:34.054924 29572 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:34.061506 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.014122 29535 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.066689 22778 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:34.075232 22633 raft_consensus.cc:1240] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 20. Current term is 21. Ops: []
I20260709 13:12:34.076740 22784 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:34.102110 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:34.105196 29545 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:34.111632 29545 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.115144 29577 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.119371 29561 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.120525 29559 raft_consensus.cc:3055] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:34.121258 29559 raft_consensus.cc:740] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:12:34.127693 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 158104us
I20260709 13:12:34.130505 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.126749 29556 consensus_queue.cc:260] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.131417 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.132354 29556 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:34.130525 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.134989 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.135141 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.136266 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.019918 29582 tablet_copy_client.cc:323] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Beginning tablet copy session from remote peer at address 127.21.5.130:34681
I20260709 13:12:34.137526 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.137827 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.141402 29535 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:34.141964 22710 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:12:34.142940 22706 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 13:12:34.144043 29453 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:34.146639 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.153082 22784 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 23.
W20260709 13:12:34.154632 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.155894 22778 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:34.160327 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.160676 22634 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:34.162475 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.164887 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:34.166376 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:34.177042 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 211277us
I20260709 13:12:34.141463 29562 raft_consensus.cc:3060] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:34.179087 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.157145 29545 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:34.182287 29453 consensus_queue.cc:1059] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:34.188412 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.188966 22779 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:34.189235 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.188445 22776 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:34.203097 22781 raft_consensus.cc:2393] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:34.191701 22784 raft_consensus.cc:1240] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 20. Current term is 21. Ops: []
I20260709 13:12:34.195600 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.198164 22775 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:12:34.203239 22783 raft_consensus.cc:1275] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:12:34.200793 22782 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:34.191701 22706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e"
dest_uuid: "91ebb78bf12f41baae84669c1effc423"
mode: GRACEFUL
new_leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
 from {username='slave'} at 127.0.0.1:48938
I20260709 13:12:34.122509 29559 consensus_queue.cc:260] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.203205 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.204770 22706 raft_consensus.cc:606] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Received request to transfer leadership to TS 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:34.191195 22780 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 19. Current term is 20. Ops: []
I20260709 13:12:34.176422 29533 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:34.208745 22706 raft_consensus.cc:614] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:12:34.218513 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.174252 29538 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:12:34.216727 22600 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.177757 29586 consensus_queue.cc:1059] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:12:34.180555 29565 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:12:34.245988 29579 raft_consensus.cc:993] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: : Instructing follower a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:34.246721 29579 raft_consensus.cc:1081] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Signalling peer a0e9f7dc2a2e4bdb9287d910e4b491f7 to start an election
I20260709 13:12:34.235301 29554 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:34.253580 22600 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.225659 29575 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.255597 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.256623 29558 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:34.257194 29558 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.257819 29587 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.258180 29558 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:34.261073 29592 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.080s	user 0.000s	sys 0.002s
I20260709 13:12:34.268502 22634 raft_consensus.cc:1275] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 13:12:34.268467 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.280695 22629 raft_consensus.cc:1275] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 15. Preceding OpId from leader: term: 16 index: 16. (index mismatch)
I20260709 13:12:34.242058 29598 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:12:34.236685 29559 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:34.303653 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.308157 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.312769 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.312837 22749 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:34.315197 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.314846 29538 consensus_queue.cc:1048] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:34.315280 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.316924 29538 consensus_queue.cc:1048] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:12:34.318406 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.318859 29537 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:34.319698 29544 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.321276 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.321534 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.322494 29544 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.324398 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.325461 29544 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:34.325819 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.327204 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.328267 29544 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:12:34.329072 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.332479 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:34.333034 22629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.333566 29544 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:34.334905 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.335248 22749 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.336372 29544 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.336524 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.335045 22631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.338081 22749 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.339010 29544 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:34.339695 29544 raft_consensus.cc:697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:34.340255 29492 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:34.340981 29544 consensus_queue.cc:237] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.342162 29536 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:34.342732 29536 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:34.343851 29536 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.347036 23384 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:34.349278 22749 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.283583 29543 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:34.356213 23384 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:34.360634 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.356901 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.363252 29602 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:34.370788 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.356329 29555 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:12:34.426460 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 69654.6us
I20260709 13:12:34.351784 29545 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.428884 22749 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.353538 29577 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:34.353509 29554 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:34.430338 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
W20260709 13:12:34.440405 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 79862.8us
I20260709 13:12:34.428524 22631 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
W20260709 13:12:34.436307 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 80177.9us
I20260709 13:12:34.355198 29587 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:12:34.374397 29602 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:12:34.449707 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.375725 29538 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:34.440577 29577 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:34.306488 29579 consensus_queue.cc:1059] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:12:34.351161 29580 consensus_queue.cc:1059] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:12:34.467314 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.437806 29551 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:34.447587 29546 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:34.463023 29601 consensus_queue.cc:1059] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:12:34.428330 29549 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:12:34.462230 29577 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:34.452945 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:34.468926 29586 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:34.443102 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:34.471805 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 279382us
I20260709 13:12:34.442217 29555 raft_consensus.cc:697] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:12:34.470036 29567 consensus_meta.cc:287] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.791s	user 0.014s	sys 0.005s
I20260709 13:12:34.453805 29547 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.509572 29567 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.510053 22719 tablet_copy_service.cc:140] P 91ebb78bf12f41baae84669c1effc423: Received BeginTabletCopySession request for tablet b8291ea7e33a40b5afaebc02d6ce7e9d from peer 6a31ac6451ff4f73bee634487d84d001 ({username='slave'} at 127.0.0.1:49444)
I20260709 13:12:34.495421 29549 raft_consensus.cc:697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:34.510804 22719 tablet_copy_service.cc:161] P 91ebb78bf12f41baae84669c1effc423: Beginning new tablet copy session on tablet b8291ea7e33a40b5afaebc02d6ce7e9d from peer 6a31ac6451ff4f73bee634487d84d001 at {username='slave'} at 127.0.0.1:49444: session id = 6a31ac6451ff4f73bee634487d84d001-b8291ea7e33a40b5afaebc02d6ce7e9d
I20260709 13:12:34.512213 29567 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:34.513080 22633 raft_consensus.cc:1240] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 19. Current term is 20. Ops: []
I20260709 13:12:34.515009 29555 consensus_queue.cc:237] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.506055 29602 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.517701 29602 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:34.506412 29587 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:12:34.519403 29549 consensus_queue.cc:237] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.503937 29586 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:34.524534 29586 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:34.497807 29597 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:12:34.525964 29597 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:34.526381 29597 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.491885 29590 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:34.489425 29551 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:34.539803 29590 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.540715 29551 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.540987 29590 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:34.542466 29590 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:34.527851 29553 consensus_queue.cc:1175] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 13 status { last_received { term: 13 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 13 index: 11 } }
W20260709 13:12:34.553321 29550 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.504s	user 0.014s	sys 0.000s
I20260709 13:12:34.497675 29537 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:12:34.528921 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85990.7us
I20260709 13:12:34.559203 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.560702 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:34.561805 29565 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.496781 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:34.565133 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68467.7us
I20260709 13:12:34.566502 22749 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:34.569761 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.577929 22749 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:34.580361 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 120768us
I20260709 13:12:34.572741 29492 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:34.584100 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.562943 29532 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:34.572971 29537 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:12:34.583993 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 116780us
I20260709 13:12:34.596213 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.579188 29561 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:12:34.599481 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.588693 29603 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.606981 29615 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:34.607750 29615 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.610246 29615 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:34.576669 29565 consensus_queue.cc:1048] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:12:34.527405 29597 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.611497 22708 raft_consensus.cc:2468] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:34.612545 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.612991 29577 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.619728 29550 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.595468 29533 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:12:34.616276 29560 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:12:34.617502 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:34.622539 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 194455us
I20260709 13:12:34.629066 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:34.635313 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.635191 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.638911 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:34.642061 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.640012 29501 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:34.657583 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.662410 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.626381 29538 raft_consensus.cc:697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:34.664537 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.650311 29603 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:34.667037 29617 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:34.677520 29501 tablet_bootstrap.cc:492] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:12:34.679100 29608 raft_consensus.cc:3055] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:34.679630 29608 raft_consensus.cc:740] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.679133 29501 ts_tablet_manager.cc:1403] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 1.455s	user 0.046s	sys 0.016s
I20260709 13:12:34.668363 29560 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.665578 29550 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:34.687633 29553 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.681880 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.683272 22778 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:34.684826 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:34.686332 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:34.687464 22780 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 13. Current term is 14. Ops: []
I20260709 13:12:34.699131 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 179) took 1095 ms. Trace:
I20260709 13:12:34.702551 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:34.660065 29453 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.707662 29566 raft_consensus.cc:3055] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:12:34.708177 29566 raft_consensus.cc:740] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.708280 29577 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.709187 29566 consensus_queue.cc:260] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.688411 22781 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c"
dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
 from {username='slave'} at 127.0.0.1:50372
I20260709 13:12:34.711715 22781 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.712273 22781 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:34.664074 29579 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:34.652123 29492 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:34.689219 29597 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:34.697569 29581 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:34.715641 29581 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.716542 29581 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.718040 29581 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:34.680636 29608 consensus_queue.cc:260] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.722467 29608 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:34.724102 29579 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.731716 29579 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:34.699599 22528 rpcz_store.cc:276] 0709 13:12:33.604197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.604357 (+   160us) service_pool.cc:224] Handling call
0709 13:12:34.699040 (+1094683us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:33.880496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:33.901558 (+ 21062us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:33.901617 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:12:33.901641 (+    24us) tablet.cc:662] Decoding operations
0709 13:12:33.902132 (+   491us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:33.902190 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:33.902222 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:12:33.903046 (+   824us) tablet.cc:701] Row locks acquired
0709 13:12:33.903065 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:12:33.903239 (+   174us) write_op.cc:270] Start()
0709 13:12:33.903386 (+   147us) write_op.cc:276] Timestamp: P: 1783602753903187 usec, L: 0
0709 13:12:33.903409 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:12:33.904296 (+   887us) log.cc:844] Serialized 913 byte log entry
0709 13:12:34.144620 (+240324us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b1800358a20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:12:34.315069 (+170449us) op_driver.cc:464] REPLICATION: finished
0709 13:12:34.341726 (+ 26657us) write_op.cc:301] APPLY: starting
0709 13:12:34.341833 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:34.344004 (+  2171us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:34.344030 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:34.346893 (+  2863us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:34.347150 (+   257us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:12:34.347197 (+    47us) write_op.cc:312] APPLY: finished
0709 13:12:34.347467 (+   270us) log.cc:844] Serialized 33 byte log entry
0709 13:12:34.348428 (+   961us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:34.348566 (+   138us) write_op.cc:454] Released schema lock
0709 13:12:34.348615 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:34.358778 (+ 10163us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:34.358817 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":26374,"apply.run_cpu_time_us":5428,"apply.run_wall_time_us":7372,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":168458,"prepare.run_cpu_time_us":20723,"prepare.run_wall_time_us":263430,"raft.queue_time_us":52228,"raft.run_cpu_time_us":11235,"raft.run_wall_time_us":31020,"replication_time_us":411473,"spinlock_wait_cycles":19926272,"thread_start_us":378102,"threads_started":4,"wal-append.queue_time_us":222831}]]}
I20260709 13:12:34.741915 22634 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.742929 22634 raft_consensus.cc:2393] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:34.752529 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:34.760841 29594 raft_consensus.cc:3055] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:12:34.763743 22781 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.761602 29594 raft_consensus.cc:740] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:34.768131 29594 consensus_queue.cc:260] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.770731 22781 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:34.771260 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db"
I20260709 13:12:34.772082 22632 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:34.772480 22632 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.777122 22634 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:34.782630 29572 raft_consensus.cc:3055] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:12:34.783226 29572 raft_consensus.cc:740] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.787547 29566 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:34.784267 29572 consensus_queue.cc:260] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.773540 22632 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.791302 29572 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:12:34.791730 22632 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:34.770289 29594 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:34.801442 29554 raft_consensus.cc:3055] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:34.802029 29554 raft_consensus.cc:740] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.803052 29554 consensus_queue.cc:260] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:34.804014 22719 tablet_copy_source_session.cc:215] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:12:34.804244 29554 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:34.808125 29536 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:34.811266 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 129516us
I20260709 13:12:34.811937 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.812950 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.813928 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:34.814652 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.815138 29610 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:34.820115 22781 raft_consensus.cc:1240] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 8. Current term is 9. Ops: []
I20260709 13:12:34.815861 29562 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:34.816331 29598 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.816517 29551 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:12:34.848752 22530 catalog_manager.cc:5697] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 14 to 15, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:34.851059 22528 catalog_manager.cc:5697] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 15 to 16, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:34.851468 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.822726 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.854442 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.856153 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.862758 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.868840 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.861109 29590 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:34.870455 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.860015 29539 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:34.823276 29610 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:34.882813 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.903530 29539 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:34.904223 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:34.905614 29564 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:12:34.886803 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:34.887732 29553 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:34.923189 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:34.912361 29550 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:34.924790 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.926879 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.868975 29587 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:12:34.927407 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:34.860638 29538 consensus_queue.cc:237] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.861130 29562 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:34.858202 29598 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:34.886361 22708 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:34.934091 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: true dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.904994 29539 raft_consensus.cc:3055] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:34.912112 22782 raft_consensus.cc:1240] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:34.880326 29453 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:34.914568 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45196.2us
I20260709 13:12:34.916644 29590 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:34.930943 22781 raft_consensus.cc:2393] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:34.877184 29563 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:34.934859 22704 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:34.935395 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:34.997462 22780 raft_consensus.cc:1240] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:34.936033 22707 raft_consensus.cc:1275] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:34.930500 29560 consensus_queue.cc:1059] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
W20260709 13:12:34.878508 29558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.020s	sys 0.007s
I20260709 13:12:34.976436 29501 raft_consensus.cc:359] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:34.832787 29492 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:34.917886 29564 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.941334 29539 raft_consensus.cc:740] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:34.989490 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:34.989976 22750 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:34.995311 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:35.012398 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.005630 22632 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:35.014001 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:35.016776 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:35.005362 29558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.020s	sys 0.008s
I20260709 13:12:35.037019 22704 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:35.038172 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:35.040452 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:35.007613 29501 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:12:35.050104 29501 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:35.052421 29501 ts_tablet_manager.cc:1434] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.372s	user 0.003s	sys 0.007s
I20260709 13:12:35.044422 29547 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:35.027529 29561 raft_consensus.cc:2804] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:35.054469 29561 raft_consensus.cc:697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:35.056157 29561 consensus_queue.cc:237] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.044915 29542 consensus_queue.cc:1059] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:35.060494 29577 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:12:35.061721 29582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8291ea7e33a40b5afaebc02d6ce7e9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:35.046950 29560 consensus_queue.cc:1059] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:35.027187 29598 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:35.067322 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53425us
I20260709 13:12:35.026647 29564 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.076885 29564 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:35.077224 29598 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.081830 22710 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:35.081192 29575 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:35.087195 22635 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:12:35.098716 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:35.103344 22634 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:12:35.069721 29615 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:35.089165 29558 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:35.161943 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:35.163033 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57903.2us
I20260709 13:12:35.164752 22645 tablet_copy_service.cc:342] P 1f99ad34cc414233929e0a92f8c3d8db: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-79ed72c313bf419bb01d45a59391a262 received from {username='slave'} at 127.0.0.1:59768
I20260709 13:12:35.165390 22645 tablet_copy_service.cc:434] P 1f99ad34cc414233929e0a92f8c3d8db: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-79ed72c313bf419bb01d45a59391a262 on tablet 79ed72c313bf419bb01d45a59391a262 with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:35.113476 29603 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:35.174153 29546 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:35.186252 29545 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:12:35.205001 29610 raft_consensus.cc:3055] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:12:35.205562 29610 raft_consensus.cc:740] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:35.196190 29546 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.175016 29558 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.217715 29558 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.213732 29598 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:35.101698 29539 consensus_queue.cc:260] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.208881 29546 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.206580 29610 consensus_queue.cc:260] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.153827 29551 consensus_queue.cc:1048] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:12:35.223627 22635 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:35.232676 29594 raft_consensus.cc:3055] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:35.233560 29594 raft_consensus.cc:740] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:35.238909 29594 consensus_queue.cc:260] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.240523 29594 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:35.246557 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.247210 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.247838 22781 raft_consensus.cc:2393] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 23: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:35.247922 22782 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:35.250865 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:35.252367 29587 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.252964 29587 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.260079 29581 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:35.260695 29581 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:35.261157 29581 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:35.265985 29610 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:35.280079 23421 raft_consensus.cc:1217] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEARNER]: Deduplicated request from leader. Original: 12.7->[12.8-12.8]   Dedup: 12.8->[]
I20260709 13:12:35.282225 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:35.282241 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:35.283299 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.285045 22782 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:35.283293 23419 raft_consensus.cc:2393] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 16: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:35.299335 22635 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:12:35.318173 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.285166 29539 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:12:35.282187 29608 consensus_meta.cc:287] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.559s	user 0.011s	sys 0.002s
I20260709 13:12:35.354293 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:35.368682 22635 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:35.366173 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:35.387463 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 296139us
I20260709 13:12:35.342075 29575 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:35.377820 29542 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:35.379138 29560 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:35.400236 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 298832us
I20260709 13:12:35.401352 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:35.384368 29567 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:35.403218 29543 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.389559 29496 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:12:35.416257 29575 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:35.441344 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 342153us
I20260709 13:12:35.413164 29567 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:35.455884 22778 raft_consensus.cc:1275] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:12:35.503181 22750 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:35.515888 29587 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.426321 29561 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:35.502741 29567 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:35.415095 29560 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.561625 29560 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:35.562772 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:35.567849 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 64784.9us
I20260709 13:12:35.580186 29551 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:35.581312 29551 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:35.583609 29635 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:12:35.556057 29628 log.cc:927] Time spent T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 13:12:35.591782 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:35.596026 29551 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.596761 29582 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:46262 (request call id 1339) took 1999 ms. Trace:
I20260709 13:12:35.593381 29545 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:12:35.597241 29582 rpcz_store.cc:276] 0709 13:12:33.597850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:33.598024 (+   174us) service_pool.cc:224] Handling call
0709 13:12:33.887592 (+289568us) ts_tablet_manager.cc:934] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Initiating tablet copy from peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
0709 13:12:35.030040 (+1142448us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:12:35.515221 (+485181us) tablet_metadata.cc:665] Metadata flushed
0709 13:12:35.515388 (+   167us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:12:35.532678 (+ 17290us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:12:35.533048 (+   370us) tablet_replica.cc:206] Creating consensus instance
0709 13:12:35.592584 (+ 59536us) ts_tablet_manager.cc:989] Replica registered.
0709 13:12:35.596672 (+  4088us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":40973,"tablet-copy.queue_time_us":166829,"thread_start_us":154576,"threads_started":2}
I20260709 13:12:35.600555 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.601430 29545 raft_consensus.cc:697] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:35.606004 29582 tablet_copy_client.cc:806] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 0 data blocks...
I20260709 13:12:35.606724 29582 tablet_copy_client.cc:670] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Starting download of 1 WAL segments...
I20260709 13:12:35.607661 22781 raft_consensus.cc:1240] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 13. Current term is 14. Ops: []
W20260709 13:12:35.626157 29564 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.549s	user 0.004s	sys 0.014s
I20260709 13:12:35.480024 29543 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.640945 22708 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:12:35.641709 22708 pending_rounds.cc:85] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 11
I20260709 13:12:35.649174 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:35.650832 22705 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
W20260709 13:12:35.666807 29615 consensus_meta.cc:287] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:12:35.667656 29615 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:35.681593 29543 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:35.606725 29545 consensus_queue.cc:237] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.686250 29639 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:35.695060 22703 raft_consensus.cc:1275] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:12:35.704701 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 57377.2us
I20260709 13:12:35.711676 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:35.712566 23421 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:35.641958 22708 pending_rounds.cc:107] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.12
I20260709 13:12:35.716125 22708 raft_consensus.cc:2917] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Aborting config change with OpId 17.12: Aborted: Op aborted by new leader
I20260709 13:12:35.716590 22708 raft_consensus.cc:2889] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:35.661180 29610 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.730466 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:35.715411 29537 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.019s	sys 0.044s
W20260709 13:12:35.732455 22675 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.731698 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:55634 (request call id 0) took 5616 ms. Trace:
W20260709 13:12:35.742403 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:35.740243 29529 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.742156 22708 rpcz_store.cc:276] 0709 13:12:30.108190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:30.118768 (+ 10578us) service_pool.cc:224] Handling call
0709 13:12:30.185889 (+ 67121us) log.cc:844] Serialized 499 byte log entry
0709 13:12:35.724508 (+5538619us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4850,"raft.queue_time_us":10184,"raft.run_cpu_time_us":2039,"raft.run_wall_time_us":2038,"spinlock_wait_cycles":187136,"thread_start_us":1836434,"threads_started":1,"wal-append.queue_time_us":1209745,"wal-append.run_cpu_time_us":2595,"wal-append.run_wall_time_us":1007489}
I20260709 13:12:35.758670 22781 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:12:35.760170 22632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:35.740597 29537 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.020s	sys 0.044s
I20260709 13:12:35.766310 22628 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:35.771770 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
W20260709 13:12:35.782334 29641 log.cc:927] Time spent T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.224s	user 0.004s	sys 0.000s
I20260709 13:12:35.783430 29543 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:35.772619 29596 consensus_queue.cc:1048] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 13:12:35.741839 29610 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:35.797266 29581 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.536s	user 0.014s	sys 0.000s
I20260709 13:12:35.797866 29581 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.800333 29581 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.753820 29529 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.803267 22626 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
W20260709 13:12:35.845227 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 119394us
W20260709 13:12:35.850054 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 141361us
I20260709 13:12:35.851871 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:35.841315 29567 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.880648 29567 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:35.861093 29653 log.cc:927] Time spent T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:12:35.871752 29546 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:35.876147 29529 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:35.852468 29597 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:35.856472 29658 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:12:35.886960 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 153210us
I20260709 13:12:35.890529 29610 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.888810 29546 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.888921 29567 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:35.889139 29658 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:35.889997 29597 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.902585 29658 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:35.884873 29632 consensus_queue.cc:1048] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:12:35.910154 29546 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.910954 29567 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:35.912887 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 286) took 1165 ms. Trace:
I20260709 13:12:35.913380 29658 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:35.916558 29658 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:35.917666 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:35.921988 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.922567 23419 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:35.924513 22783 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:35.922730 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:35.926393 22778 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:35.926452 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:35.944665 22782 raft_consensus.cc:1275] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:35.945118 22632 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:35.946882 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:35.948464 29538 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:12:35.932340 22631 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:36.004618 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:36.009665 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:36.012041 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:36.021335 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 273608us
I20260709 13:12:36.023197 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.010802 29532 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:36.032114 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:36.034327 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:36.023955 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:35.913334 22528 rpcz_store.cc:276] 0709 13:12:34.747439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.754788 (+  7349us) service_pool.cc:224] Handling call
0709 13:12:35.912805 (+1158017us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:35.013805 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:35.014451 (+   646us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:35.014489 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:12:35.014508 (+    19us) tablet.cc:662] Decoding operations
0709 13:12:35.042766 (+ 28258us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:35.068671 (+ 25905us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:35.090128 (+ 21457us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:35.108429 (+ 18301us) tablet.cc:701] Row locks acquired
0709 13:12:35.115248 (+  6819us) write_op.cc:260] PREPARE: finished
0709 13:12:35.170421 (+ 55173us) write_op.cc:270] Start()
0709 13:12:35.180521 (+ 10100us) write_op.cc:276] Timestamp: P: 1783602755159622 usec, L: 0
0709 13:12:35.183179 (+  2658us) op_driver.cc:348] REPLICATION: starting
0709 13:12:35.262675 (+ 79496us) log.cc:844] Serialized 538 byte log entry
0709 13:12:35.597969 (+335294us) op_driver.cc:464] REPLICATION: finished
0709 13:12:35.848316 (+250347us) write_op.cc:301] APPLY: starting
0709 13:12:35.848491 (+   175us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:35.856296 (+  7805us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:35.860634 (+  4338us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:35.862104 (+  1470us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:35.862415 (+   311us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:35.862488 (+    73us) write_op.cc:312] APPLY: finished
0709 13:12:35.862841 (+   353us) log.cc:844] Serialized 25 byte log entry
0709 13:12:35.863097 (+   256us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:35.863220 (+   123us) write_op.cc:454] Released schema lock
0709 13:12:35.863283 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:35.863813 (+   530us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:35.863845 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":18491,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":217036,"apply.run_cpu_time_us":5627,"apply.run_wall_time_us":18671,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8315,"num_ops":1,"prepare.queue_time_us":154444,"prepare.run_cpu_time_us":4236,"prepare.run_wall_time_us":251583,"replication_time_us":388397,"thread_start_us":249492,"threads_started":2}]]}
I20260709 13:12:36.027789 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.013528 29594 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:36.029718 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:36.044651 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:36.045892 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.024425 29550 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:36.048617 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.022755 29567 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.049952 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.051859 22709 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:36.027261 29586 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.061599 22702 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:36.031047 29532 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.031342 29610 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:36.069334 29579 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:36.109002 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:36.109234 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43493.2us
I20260709 13:12:36.121660 22777 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:12:36.122705 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70736.4us
I20260709 13:12:36.085072 29532 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:36.128082 29610 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:36.146226 29643 log.cc:927] Time spent T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.588s	user 0.007s	sys 0.000s
I20260709 13:12:36.043998 29645 consensus_queue.cc:1059] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 13:12:36.059165 29598 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:36.072808 29536 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:36.105854 29579 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.153462 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:36.153445 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 112852us
I20260709 13:12:36.156895 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.157315 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.166168 22777 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:36.166298 22782 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:36.124869 29544 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.125636 29597 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.169080 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:36.072628 29586 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.106078 29565 consensus_queue.cc:1048] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:12:36.190297 22704 raft_consensus.cc:2468] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:36.158046 29536 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.166070 29579 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:36.170787 29529 raft_consensus.cc:3055] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:36.154023 29555 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:12:36.065783 29594 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.211100 22601 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.215557 22777 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:36.193157 29597 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:36.202895 29536 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.197268 29669 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:36.199193 29598 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:36.173202 29544 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.206512 29529 raft_consensus.cc:740] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:36.266292 22707 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:12:36.250190 29572 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:12:36.259112 22581 auto_rebalancer.cc:629] Failed to schedule move for tablet b8291ea7e33a40b5afaebc02d6ce7e9d: Aborted: Op aborted by new leader
W20260709 13:12:36.270715 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 69226.8us
I20260709 13:12:36.267788 29669 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.251539 29535 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.248656 29603 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:36.290526 22750 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:36.300767 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86777.1us
I20260709 13:12:36.277195 29572 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:36.304841 29572 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:36.293057 29537 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 13:12:36.316329 29662 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 13:12:36.321940 29663 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.258s	user 0.002s	sys 0.000s
I20260709 13:12:36.276309 29560 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:36.296691 29561 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.293779 29672 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.269428 29598 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.375953 22626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "1f99ad34cc414233929e0a92f8c3d8db" is_pre_election: true
I20260709 13:12:36.297317 29535 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.390305 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:36.387831 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 124321us
I20260709 13:12:36.351142 29546 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.406031 22600 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:36.300323 29603 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.383481 29558 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.418694 29677 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:36.418555 29558 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.419605 29665 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.420207 29665 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.421164 29666 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.421514 29558 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.421763 29666 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.422649 29586 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.346748 29560 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.407454 29567 raft_consensus.cc:2804] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:36.364578 29561 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.407874 29639 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.442066 29665 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.446803 29586 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:36.452093 29535 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.371817 29672 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.518515 22703 raft_consensus.cc:1275] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:12:36.519654 29672 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.451318 29536 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.349917 29529 consensus_queue.cc:260] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.575982 29529 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:36.577478 29536 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.511776 29610 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:36.434722 29597 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:36.512563 29677 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:36.411058 29546 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.585235 29597 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.514899 29680 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:36.516215 29669 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:36.503443 29555 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.583096 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.588204 22705 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:36.587160 29546 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.454682 29567 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:36.503404 29598 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:36.592182 29592 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:12:36.621380 29544 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:36.622131 29544 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.622694 29544 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.630883 29544 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:36.585237 29677 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:36.636802 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 228508us
I20260709 13:12:36.595559 29567 raft_consensus.cc:3060] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:36.507923 29639 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.645380 29620 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.646078 29620 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.646826 29639 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:36.648990 29561 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:36.642472 29535 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.587774 29555 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:36.688719 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 45106.5us
I20260709 13:12:36.689865 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:36.549469 29586 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.695744 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.696421 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.699761 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.614899 29680 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:36.696977 22706 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:36.651501 29535 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.710340 29535 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.724427 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.725479 22709 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:36.735828 29683 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.736620 29683 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.726315 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:36.739094 22701 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.649022 29679 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.741793 29679 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.650372 29666 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.654078 29555 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:36.744400 29679 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 2
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:12:36.749702 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
I20260709 13:12:36.750911 29603 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.754632 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 306) took 1969 ms. Trace:
I20260709 13:12:36.751993 22701 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:36.765488 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:36.766273 23421 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:36.649856 29620 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.703838 29588 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.785207 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.785288 22778 raft_consensus.cc:1240] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 13:12:36.789657 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.790354 22705 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:36.788352 23385 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.682937 29579 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.755386 22530 rpcz_store.cc:276] 0709 13:12:34.784866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:34.785047 (+   181us) service_pool.cc:224] Handling call
0709 13:12:36.754509 (+1969462us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:35.265667 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:35.266504 (+   837us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:35.266545 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:12:35.266567 (+    22us) tablet.cc:662] Decoding operations
0709 13:12:35.267060 (+   493us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:35.267122 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:35.267154 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:35.355513 (+ 88359us) tablet.cc:701] Row locks acquired
0709 13:12:35.355552 (+    39us) write_op.cc:260] PREPARE: finished
0709 13:12:35.385028 (+ 29476us) write_op.cc:270] Start()
0709 13:12:35.394819 (+  9791us) write_op.cc:276] Timestamp: P: 1783602755378529 usec, L: 0
0709 13:12:35.394861 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:12:35.536843 (+141982us) log.cc:844] Serialized 533 byte log entry
0709 13:12:36.026681 (+489838us) op_driver.cc:464] REPLICATION: finished
0709 13:12:36.055107 (+ 28426us) write_op.cc:301] APPLY: starting
0709 13:12:36.064921 (+  9814us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:36.258257 (+193336us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:36.274656 (+ 16399us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:36.397017 (+122361us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:36.417831 (+ 20814us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:36.432900 (+ 15069us) write_op.cc:312] APPLY: finished
0709 13:12:36.456753 (+ 23853us) log.cc:844] Serialized 25 byte log entry
0709 13:12:36.520296 (+ 63543us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:36.520425 (+   129us) write_op.cc:454] Released schema lock
0709 13:12:36.520480 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:36.520939 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:36.520972 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48432,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":9717,"apply.run_cpu_time_us":7004,"apply.run_wall_time_us":470279,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4188,"num_ops":1,"prepare.queue_time_us":400140,"prepare.run_cpu_time_us":4698,"prepare.run_wall_time_us":291216,"replication_time_us":624589}]]}
I20260709 13:12:36.799710 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.799880 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.800426 22775 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:36.800591 22784 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:36.781373 29555 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.802143 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.793413 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.803610 22705 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:12:36.794363 23385 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.806210 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.807982 22705 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 8.
I20260709 13:12:36.795038 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.796761 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.797080 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.798123 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:36.792315 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 386357us
I20260709 13:12:36.789091 29582 tablet_copy_client.cc:538] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:12:36.802865 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.809157 22706 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:36.808909 22701 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 8.
I20260709 13:12:36.811537 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.812347 22775 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 9: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:36.798360 29543 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:36.813396 22779 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:36.809844 22709 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:12:36.816941 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.817579 29537 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:36.817160 29678 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.813550 29581 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:36.817616 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.804319 29683 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.820160 29594 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:36.820341 29587 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.820878 29587 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:36.815929 22749 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:36.788638 29588 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.827967 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.827697 22779 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:36.804819 29685 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:36.826647 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:36.827837 29679 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.833766 29542 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.837175 22601 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.822716 29543 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.818446 29678 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.851878 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.847093 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:36.818570 29581 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.859480 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.818764 29549 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:36.805368 29555 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.830035 29683 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:36.858719 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:36.853976 29672 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:36.854908 29543 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.885351 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:36.858994 29566 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:36.886372 22783 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:36.823315 29537 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.866762 29546 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:12:36.888564 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76242.1us
I20260709 13:12:36.867221 29581 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:36.896613 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.897424 22779 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:36.898345 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.899012 22779 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:36.899978 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.893671 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.881856 29665 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:36.910108 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:36.881026 29610 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:36.900650 22779 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:36.900766 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:36.854243 29679 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.908354 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:36.908787 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.880052 29549 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:36.859838 29542 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:36.906517 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 90599.2us
I20260709 13:12:36.879667 29683 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:36.830490 29680 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.938287 22775 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
W20260709 13:12:36.956660 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:36.975355 22776 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 9.
I20260709 13:12:36.884099 29672 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.987231 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:36.890442 29566 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.900481 29546 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.987695 22673 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:36.959156 29572 consensus_meta.cc:287] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.002s
I20260709 13:12:36.995168 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:36.867439 29685 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:36.910416 29665 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.996563 22709 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:12:36.889843 29690 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:36.914381 29610 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:36.941609 29587 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:36.940989 29666 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:36.997506 29546 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.959663 29594 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:36.989609 29672 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:37.028226 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.005053 29610 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:36.992050 29566 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:37.001870 29691 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
W20260709 13:12:37.039778 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 129806us
I20260709 13:12:37.005132 29665 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:37.048197 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.006662 29603 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:37.005797 29690 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.052968 22776 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:37.028812 29579 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.037972 29587 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:37.062053 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.063267 22776 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:37.035459 29670 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.069307 22709 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:37.002908 29572 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.075152 29695 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.075968 29695 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.076498 29572 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.078915 29678 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:37.079100 29555 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.079244 22631 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:37.080549 29542 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.043057 29677 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.045281 29577 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.089023 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.056618 29586 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.089895 23421 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:37.035887 29666 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.070801 29587 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.093760 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.042518 29594 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.094483 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:37.094426 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:37.096273 22702 raft_consensus.cc:2468] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:37.060725 29690 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.095853 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:37.108517 22706 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:37.091120 29588 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.058749 29549 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.098407 29587 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:37.094002 29577 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.112943 29676 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:37.116750 29566 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.118000 29537 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:37.118831 29700 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.120376 29666 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.091316 29678 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.128427 22776 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
W20260709 13:12:37.127693 29532 consensus_meta.cc:287] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.981s	user 0.008s	sys 0.008s
I20260709 13:12:37.147224 29700 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.129495 29676 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.098335 29542 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:37.155316 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:37.150736 29535 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:12:37.096771 29677 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.086915 29670 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.162740 29695 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.212828 29537 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:37.230983 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 116878us
W20260709 13:12:37.165004 29567 consensus_meta.cc:287] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.002s
I20260709 13:12:37.151208 29564 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.278000 29564 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.279938 29701 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.280956 29564 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.280805 29701 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.282891 29634 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:37.283627 29634 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.284024 29701 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.284586 29582 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Bootstrap starting.
I20260709 13:12:37.286113 29634 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:37.286382 29597 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:37.286151 29535 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:37.288625 29535 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:37.264231 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.154568 29579 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.213788 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.163878 29676 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:37.274407 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:37.296305 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 141171us
W20260709 13:12:37.300415 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:37.302150 22749 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.158924 29678 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.270829 29542 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.304875 29579 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.307966 29546 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.308993 29566 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:37.310246 29705 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:12:37.310648 29704 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.314992 29708 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.316299 29677 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.320256 29670 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.321573 29594 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.298018 29537 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.362432 29708 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.298391 29567 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.323781 29588 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.158576 29532 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.327558 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.327832 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.290506 29683 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.332312 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:37.314038 22524 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:37.342048 22749 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:37.349555 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 75215.6us
I20260709 13:12:37.354635 29685 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.355647 29542 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.357568 29710 raft_consensus.cc:2804] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:37.294754 29700 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.342506 29701 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:37.359396 29564 raft_consensus.cc:1064] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: attempting to promote NON_VOTER 6a31ac6451ff4f73bee634487d84d001 to VOTER
I20260709 13:12:37.369234 29708 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.367542 29665 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.377460 29709 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:12:37.375936 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:37.333952 29543 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.374893 22601 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:37.379403 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.387892 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.388788 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.377909 29710 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.390676 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.413763 22779 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.414202 29579 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.413295 29537 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:37.391160 23419 raft_consensus.cc:2368] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17.
I20260709 13:12:37.415196 29709 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.415678 29709 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:37.371891 29577 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.386849 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.374568 22524 catalog_manager.cc:5697] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 15 to 16, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:37.401106 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.403609 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.389667 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.353134 29581 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.416388 23421 raft_consensus.cc:1240] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:37.418270 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.426080 22780 raft_consensus.cc:2468] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:37.417227 22778 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:37.362423 29704 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:37.410367 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.392796 29597 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.426810 22782 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 14: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:37.419443 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.373485 29588 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.415199 29543 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.412147 29567 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.395875 29683 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.428826 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.444841 22781 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:37.430028 29581 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:37.431977 22749 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:37.437350 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:37.441169 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 66365.9us
I20260709 13:12:37.437356 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.413069 29678 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.412993 29532 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.393015 29701 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.451462 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.452075 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.452593 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.454335 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:37.457720 22749 leader_election.cc:336] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:37.471505 22709 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:37.407883 29560 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.463951 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.419725 29710 raft_consensus.cc:3060] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:37.449338 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:37.471802 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:37.471650 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:37.435772 29702 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.659044 22710 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:12:37.575819 23419 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46252 (request call id 1641) took 1622 ms (client timeout 1928 ms). Trace:
W20260709 13:12:37.478998 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:37.573541 22703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 3589) took 1236 ms. Trace:
I20260709 13:12:37.435484 29615 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:37.662689 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.562733 29654 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.555119 29666 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.576277 23414 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:37.661612 22703 rpcz_store.cc:276] 0709 13:12:36.337145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.337505 (+   360us) service_pool.cc:224] Handling call
0709 13:12:36.338038 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:36.518963 (+180925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:37.573068 (+1054105us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54002bf7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:12:37.573381 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414083968}
I20260709 13:12:37.511463 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.692078 22782 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:37.493520 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.570269 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:37.693768 22706 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:37.694094 22704 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.568463 22709 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:37.446734 29542 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:37.669752 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:37.664199 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.700814 22710 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
W20260709 13:12:37.701411 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61277.4us
W20260709 13:12:37.659848 23419 rpcz_store.cc:269] 0709 13:12:35.952993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.011025 (+ 58032us) service_pool.cc:224] Handling call
0709 13:12:37.575513 (+1564488us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54004f4f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:37.575715 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3077259648}
W20260709 13:12:37.704720 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 255454us
W20260709 13:12:37.708177 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 250489us
W20260709 13:12:37.709553 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:37.458420 29566 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:12:37.709890 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:37.658740 29690 consensus_meta.cc:287] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 13:12:37.447360 29579 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.457268 29558 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:12:37.710300 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:37.718709 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.779014 22775 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:12:37.779134 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68848.2us
I20260709 13:12:37.625674 29581 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.690940 29546 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.580000 29685 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:12:37.770725 21526 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 22646)
I20260709 13:12:37.689671 29654 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.704025 29542 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.779289 22781 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:37.720438 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:37.711226 29587 consensus_meta.cc:287] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.002s
I20260709 13:12:37.785257 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:37.790779 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.791368 22776 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.457356 29683 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.786438 22704 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:37.444967 29597 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:37.721227 29610 consensus_meta.cc:287] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.666s	user 0.007s	sys 0.006s
I20260709 13:12:37.711517 29715 raft_consensus.cc:2804] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:37.714634 29695 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:37.781402 29603 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:37.787635 29566 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:37.794214 29542 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:37.703653 29666 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.809674 22776 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:37.575202 29564 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.824857 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.826483 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:37.827035 22704 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:37.712039 29676 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.703456 29615 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.828645 22782 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:37.804960 29546 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.804262 29665 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.784741 29579 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.805327 29715 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.784360 29581 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.785915 29690 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.806437 29718 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.810702 29566 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.788030 29558 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.797843 29587 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.692111 29702 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.842983 22779 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:37.844928 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:37.849684 23419 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:37.792699 29685 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.831697 29535 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:37.800278 29610 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.827332 29719 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:37.846834 29672 consensus_meta.cc:287] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.794s	user 0.011s	sys 0.002s
I20260709 13:12:37.813119 29603 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.848074 29566 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.842010 29715 raft_consensus.cc:3060] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:37.882385 22705 raft_consensus.cc:2368] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15.
I20260709 13:12:37.843130 29709 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.890094 29685 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.891666 29610 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.891897 29690 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:37.892781 29713 log.cc:927] Time spent T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:12:37.887250 29701 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.896739 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:12:37.917346 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 509032us
I20260709 13:12:37.918449 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:37.890146 29558 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.864010 29594 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:37.920064 29594 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.853977 29718 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.921669 29577 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:37.922111 29577 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.862489 29535 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.922462 29577 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.924208 29535 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.870528 29672 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.926227 29672 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.900509 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.843984 29546 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.922470 29594 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.866401 29719 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:37.920445 29547 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:37.929157 29547 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:37.929628 29547 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:37.936055 22701 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:37.937474 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 4788) took 1052 ms. Trace:
I20260709 13:12:37.937847 22701 rpcz_store.cc:276] 0709 13:12:36.884811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.885023 (+   212us) service_pool.cc:224] Handling call
0709 13:12:37.937233 (+1052210us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400315e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:37.937375 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414252416}
I20260709 13:12:37.939270 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:37.939978 29603 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.915947 29588 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.940835 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:37.941867 29597 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.942209 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:37.943297 29695 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:37.951644 29695 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:12:37.952282 29695 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.952746 29695 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:12:37.954598 22783 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50386 (request call id 2888) took 1169 ms. Trace:
I20260709 13:12:37.955106 22783 rpcz_store.cc:276] 0709 13:12:36.784741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.784941 (+   200us) service_pool.cc:224] Handling call
0709 13:12:37.954339 (+1169398us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54000cb054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:37.954490 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":230402304}
I20260709 13:12:37.958932 29704 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.959547 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:37.960855 29684 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:37.961519 29684 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:37.961890 29684 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:37.944171 29683 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:37.904388 29720 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:12:37.947568 29676 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:37.951558 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 155441us
I20260709 13:12:37.945256 29654 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:37.983922 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.969542 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:37.984665 22783 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:37.974888 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:37.949322 29566 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:37.986040 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:37.988969 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:37.993914 22779 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:37.989789 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:37.990103 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:37.996938 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:38.000715 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:37.951298 29710 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.016966 22779 raft_consensus.cc:2468] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
W20260709 13:12:38.043416 29680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.004s	sys 0.020s
W20260709 13:12:38.052914 29680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.005s	sys 0.020s
I20260709 13:12:38.048821 29666 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.065070 29710 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.065958 29695 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.069403 29695 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:38.071873 29716 log.cc:927] Time spent T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.279s	user 0.003s	sys 0.001s
W20260709 13:12:38.115111 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 285161us
I20260709 13:12:38.117681 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.124166 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:38.145273 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:38.146355 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 140839us
W20260709 13:12:38.147826 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:38.148823 22749 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:38.153213 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.130043 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:38.169044 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:38.172873 29558 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:38.173661 29558 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.174118 29558 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:38.174238 29535 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.175680 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.043025 29547 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.180240 29529 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.181205 29529 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.181854 29547 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.042667 29542 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.175688 29535 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:38.188521 22749 leader_election.cc:336] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:38.189404 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:38.191790 29570 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:38.192972 29570 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:38.195001 29542 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.196225 29577 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.198145 29588 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:38.199551 29594 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.200609 29594 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.203707 29577 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.204830 29594 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.205736 29594 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.198518 29542 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:38.206837 29594 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.216385 29594 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.210968 29577 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:38.196280 29570 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.200620 29529 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:38.250373 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 81622.9us
I20260709 13:12:38.264983 22708 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:55634 (request call id 1) took 1957 ms. Trace:
W20260709 13:12:38.271849 23418 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46262 (request call id 1358) took 1363 ms (client timeout 1807 ms). Trace:
W20260709 13:12:38.272673 23418 rpcz_store.cc:269] 0709 13:12:36.907871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.908044 (+   173us) service_pool.cc:224] Handling call
0709 13:12:38.271253 (+1363209us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54006b0154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:38.271566 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657077504}
W20260709 13:12:38.278945 29736 log.cc:927] Time spent T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 13:12:38.279456 29666 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.265776 22708 rpcz_store.cc:276] 0709 13:12:36.307353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.307541 (+   188us) service_pool.cc:224] Handling call
0709 13:12:38.264211 (+1956670us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54002bf7d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee3735 00007f180bea3d29 00007f18054221d4 00007f1805422162 00007f1805422091 00007f180542200c 00007f18054212c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:38.264646 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502499584}
I20260709 13:12:38.280670 29666 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.281980 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:38.283080 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 55991.7us
I20260709 13:12:38.284196 22600 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.284255 29701 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:38.285246 29701 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:38.287241 29704 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:12:38.288048 29704 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:38.288391 29654 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.288904 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.289388 29654 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.290014 22780 raft_consensus.cc:2468] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:38.289742 29704 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.284581 29547 consensus_queue.cc:1048] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:12:38.292580 29654 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.294355 29594 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.295190 29555 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.295723 29603 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.285324 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:38.298888 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.303598 29603 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:38.305464 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 117177us
I20260709 13:12:38.306113 29587 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.307197 29724 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:12:38.308362 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 268650us
I20260709 13:12:38.309019 29535 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.307251 29726 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.314710 29579 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.315160 29701 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.317293 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.317387 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:38.318526 22779 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:38.319067 29666 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.319559 29535 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:38.312361 29724 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:38.324259 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 121048us
I20260709 13:12:38.328833 29533 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:38.329006 29726 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.331815 29672 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.327641 29724 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:38.333241 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.312183 29594 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.337205 23421 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46274 (request call id 1570) took 1038 ms. Trace:
I20260709 13:12:38.338434 29603 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.339001 23421 rpcz_store.cc:276] 0709 13:12:37.297925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:37.298082 (+   157us) service_pool.cc:224] Handling call
0709 13:12:37.298523 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.430413 (+131890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.334961 (+904548us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400354b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:12:38.336259 (+  1298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270556032}
I20260709 13:12:38.341197 29726 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.341604 29594 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.342635 29546 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.342403 29672 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.343623 29654 consensus_queue.cc:1059] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:38.344936 29654 raft_consensus.cc:3055] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:38.345357 29672 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.345386 29654 raft_consensus.cc:740] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:38.346052 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.333446 29555 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.350800 29566 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.351649 29555 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.324113 29558 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:38.348052 22749 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:38.346856 29533 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.362131 22779 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 10: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:38.363425 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.366168 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.355822 29615 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.372205 29533 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.373204 29654 consensus_queue.cc:260] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.375089 29654 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:38.376034 29706 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 1.126s	user 0.000s	sys 0.011s
I20260709 13:12:38.367091 23419 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:38.379701 29709 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.380278 29718 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.381632 22631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45666 (request call id 3890) took 2032 ms. Trace:
I20260709 13:12:38.377602 29724 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.384375 29702 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:38.386394 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.382292 22631 rpcz_store.cc:276] 0709 13:12:36.349493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.349656 (+   163us) service_pool.cc:224] Handling call
0709 13:12:36.350137 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:37.079932 (+729795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:38.380847 (+1300915us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540006b3d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:12:38.381263 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1676208384}
I20260709 13:12:38.391847 29558 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.396260 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.398855 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.409953 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:38.370448 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:38.418092 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 100920us
I20260709 13:12:38.438026 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.444819 29724 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.446532 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.447675 22784 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:12:38.467212 22601 leader_election.cc:400] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15.
I20260709 13:12:38.467923 22601 leader_election.cc:403] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:38.469184 29563 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:12:38.469630 29679 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.435216 29558 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.473456 29558 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.477545 29558 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.435936 22783 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
I20260709 13:12:38.481761 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.487018 22777 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:38.490383 22779 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:38.495306 29581 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.497861 29581 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.498739 29581 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.518920 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.522363 29640 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.523306 29640 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:38.527756 29719 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.527894 29700 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.523813 22673 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:38.526392 29640 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.535310 29700 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.534060 29661 consensus_queue.cc:1059] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:12:38.539994 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.538046 29661 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:38.545207 29672 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.531494 29658 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.549098 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.542968 29726 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.556649 29726 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.542204 29594 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.559340 29594 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:38.558267 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 141629us
I20260709 13:12:38.538093 29700 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.554838 29658 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:38.555160 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 118410us
W20260709 13:12:38.557173 29715 consensus_meta.cc:287] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.669s	user 0.015s	sys 0.009s
I20260709 13:12:38.549633 29661 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:38.552222 29672 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:38.566071 29666 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:38.572403 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.574226 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.574554 29672 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.566139 29603 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.575233 29715 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.575757 29603 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.577889 29715 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.571831 29581 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.571926 29594 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.571308 29661 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.574908 29558 raft_consensus.cc:2804] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:38.578681 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:38.579481 29658 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.579910 29610 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.595295 29610 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.598205 29610 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.579962 29700 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.599100 29700 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.600991 29744 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.589016 29666 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:38.586305 29603 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.603713 29603 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.603832 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.604097 23384 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.604848 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 22.
I20260709 13:12:38.604559 29603 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.588179 29558 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.607051 29558 raft_consensus.cc:3060] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:38.590233 29719 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:38.610896 29719 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.611578 29719 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:38.612879 29726 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.613191 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.613817 29603 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.614046 29661 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:38.615643 29603 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.616488 29603 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.601589 29700 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.583017 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.627367 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.602816 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.588110 29672 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.616729 22528 catalog_manager.cc:5697] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 13 to 14, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:38.619838 29739 log.cc:927] Time spent T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.172s	user 0.000s	sys 0.004s
I20260709 13:12:38.586229 29594 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:38.620904 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:38.621703 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.622941 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.623342 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.626329 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.618165 29666 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.635715 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.636595 23419 raft_consensus.cc:2393] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:38.635911 29697 raft_consensus.cc:2804] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:38.642475 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:38.643016 29594 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.645834 29594 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.646670 29679 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:38.647194 29679 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.647750 29679 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:38.654245 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.659189 22705 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:12:38.659514 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.660198 22780 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:38.660868 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.661732 22703 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:12:38.663352 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.664111 29603 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.668519 29697 raft_consensus.cc:697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:38.677522 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:38.677989 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.680042 23418 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 LEARNER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:38.682520 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.665941 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.686625 22710 raft_consensus.cc:2468] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:38.684257 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:38.689294 29697 consensus_queue.cc:237] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:38.690897 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 70103.7us
I20260709 13:12:38.692358 29594 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:38.692800 29594 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.693190 29594 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:38.698222 22528 catalog_manager.cc:5697] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 22 to 23, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:38.700215 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.701511 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.701907 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.703344 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.639010 29744 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.711786 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.712777 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.725530 22710 raft_consensus.cc:2468] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:38.666791 22524 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 286) took 3360 ms. Trace:
I20260709 13:12:38.730242 22703 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
I20260709 13:12:38.731138 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.732578 22777 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:38.731420 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.735105 22709 raft_consensus.cc:2468] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:38.742388 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.743938 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.746475 22701 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 18: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:38.744532 22704 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:38.745021 22705 raft_consensus.cc:2468] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:38.754317 22783 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:38.748208 22528 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 17 to 18, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:38.759615 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.763616 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.765048 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.765793 22704 raft_consensus.cc:2468] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:38.735327 29719 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.742327 29672 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.759035 29603 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:38.740208 29594 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.777894 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.758906 29700 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:38.783066 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.789399 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.792227 22701 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:12:38.795539 22707 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:38.797541 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.798698 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.799659 22704 raft_consensus.cc:2468] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:38.800412 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.804088 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.794991 29744 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.811111 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.811578 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.812392 22782 raft_consensus.cc:2393] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:38.813666 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.789008 29603 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.794970 29558 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:38.817656 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.819921 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.808118 29700 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.820355 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.822631 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.726264 22524 rpcz_store.cc:276] 0709 13:12:35.305856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:35.306088 (+   232us) service_pool.cc:224] Handling call
0709 13:12:35.704578 (+398490us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:12:38.666483 (+2961905us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:37.805944 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:37.912666 (+106722us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:37.912736 (+    70us) write_op.cc:435] Acquired schema lock
0709 13:12:37.912767 (+    31us) tablet.cc:662] Decoding operations
0709 13:12:37.913425 (+   658us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:37.913523 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:37.913568 (+    45us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:12:37.914065 (+   497us) tablet.cc:701] Row locks acquired
0709 13:12:37.914093 (+    28us) write_op.cc:260] PREPARE: finished
0709 13:12:37.914333 (+   240us) write_op.cc:270] Start()
0709 13:12:37.914513 (+   180us) write_op.cc:276] Timestamp: P: 1783602757914260 usec, L: 0
0709 13:12:37.914544 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:12:37.931551 (+ 17007us) log.cc:844] Serialized 918 byte log entry
0709 13:12:38.251792 (+320241us) op_driver.cc:464] REPLICATION: finished
0709 13:12:38.363415 (+111623us) write_op.cc:301] APPLY: starting
0709 13:12:38.402942 (+ 39527us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:38.457267 (+ 54325us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:38.457397 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:38.458519 (+  1122us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:38.467187 (+  8668us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:12:38.467260 (+    73us) write_op.cc:312] APPLY: finished
0709 13:12:38.467706 (+   446us) log.cc:844] Serialized 33 byte log entry
0709 13:12:38.469579 (+  1873us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:38.469801 (+   222us) write_op.cc:454] Released schema lock
0709 13:12:38.469875 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:38.470647 (+   772us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:38.470679 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"spinlock_wait_cycles":350883072,"child_traces":[["op",{"apply.queue_time_us":87186,"apply.run_cpu_time_us":7263,"apply.run_wall_time_us":113252,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":356970,"prepare.run_cpu_time_us":25929,"prepare.run_wall_time_us":259219,"raft.queue_time_us":19536,"raft.run_cpu_time_us":23669,"raft.run_wall_time_us":59656,"replication_time_us":336911,"thread_start_us":288821,"threads_started":4,"wal-append.queue_time_us":128393}]]}
I20260709 13:12:38.824556 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.829067 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.831193 22705 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:38.833508 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.831025 22706 raft_consensus.cc:2468] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:38.838322 22707 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:12:38.838861 22784 raft_consensus.cc:2393] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 16: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:38.840441 22709 raft_consensus.cc:2368] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:12:38.841706 22704 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:38.835884 29719 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.851315 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.852255 22782 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:38.853269 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.848541 22598 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:38.862453 22783 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:38.863597 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:38.864691 29563 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15.
I20260709 13:12:38.865304 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.863874 22709 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:12:38.863663 29672 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.868593 29603 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.870046 29700 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:38.871728 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:38.878996 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.888192 22709 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:38.888394 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:38.855603 29684 consensus_meta.cc:287] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.843s	user 0.016s	sys 0.008s
I20260709 13:12:38.890709 22780 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:38.890875 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.889899 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:38.857486 29594 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.895056 22784 raft_consensus.cc:1240] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:12:38.892216 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:38.897050 22777 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:38.899693 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:38.900436 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.901017 22703 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:12:38.901742 29715 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:38.909431 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.917300 22704 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 22: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:38.910858 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.918223 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.920661 22784 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 18: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
W20260709 13:12:38.912096 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 63666.5us
I20260709 13:12:38.914846 22709 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:12:38.909011 29746 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:12:38.922932 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:38.923647 22703 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 10.
W20260709 13:12:38.924458 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:38.898375 29684 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.916218 29558 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:38.938675 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.941514 22779 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:38.941943 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:38.951545 22707 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:12:38.952476 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:38.940909 29654 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.967600 29654 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:38.976212 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 68842.6us
I20260709 13:12:38.978477 29551 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.979358 29551 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:38.984356 29551 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.988843 29684 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:38.971011 29594 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:38.995756 29594 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.047827 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:39.053941 22709 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:39.079046 22704 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:39.133487 22705 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:39.144837 22701 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
I20260709 13:12:39.102243 29679 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:39.156545 29594 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:39.157953 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 226181us
I20260709 13:12:39.163013 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.170895 22707 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
W20260709 13:12:39.170068 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:39.177022 22776 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:12:39.178325 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:39.182516 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:39.169629 29654 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:39.190662 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 187719us
I20260709 13:12:39.202243 22781 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:39.203433 22778 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:39.206789 22780 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:12:39.218540 22707 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:12:39.242058 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:39.215169 29558 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:12:39.248198 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.249257 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.250207 22703 raft_consensus.cc:2393] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:39.250855 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.251449 22705 raft_consensus.cc:2393] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 10: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:39.248396 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.218297 29697 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:12:39.252760 22600 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:39.249284 22708 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:12:39.240307 29679 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:39.259832 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 85733.3us
W20260709 13:12:39.260725 22749 leader_election.cc:336] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.262894 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:39.264990 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.258821 29719 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:39.265586 29661 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:39.261802 29715 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:12:39.267057 29715 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.268226 29715 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.265827 22701 raft_consensus.cc:2368] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18.
I20260709 13:12:39.272652 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:39.273036 22749 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.273597 22708 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:39.273746 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:39.276474 29654 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:39.279922 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.283963 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.284559 22600 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:39.285920 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.289050 22710 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:12:39.289050 22704 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:39.291865 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.292515 22709 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:39.294405 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:39.282225 22749 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.295079 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.295696 22709 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:12:39.296885 29679 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:12:39.297518 29679 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.298650 29679 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.303220 22600 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:39.303398 29719 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:12:39.303222 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.306066 22600 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:39.306674 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:39.307487 22707 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:12:39.308439 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 68606.7us
I20260709 13:12:39.310708 22709 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:39.296859 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:39.319317 29558 raft_consensus.cc:2804] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:39.319940 29558 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.343792 29759 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:39.323583 29563 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:39.335080 29661 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:39.335676 29558 raft_consensus.cc:3060] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:39.318001 29719 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 13:12:39.354957 29679 raft_consensus.cc:2804] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:12:39.366902 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60355.5us
I20260709 13:12:39.368147 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:39.369168 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86981.2us
W20260709 13:12:39.376245 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:39.385707 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67948.9us
I20260709 13:12:39.365717 29719 consensus_queue.cc:1059] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:12:39.367230 29679 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:39.407529 29679 raft_consensus.cc:3060] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:12:39.439098 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43864.9us
W20260709 13:12:39.443356 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 60933.7us
I20260709 13:12:39.453676 22784 raft_consensus.cc:1275] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:12:39.456828 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88749.1us
I20260709 13:12:39.442108 29759 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.442937 29563 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.458819 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.459211 22601 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.467155 29759 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.476171 29761 raft_consensus.cc:3055] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:39.476797 29761 raft_consensus.cc:740] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.477751 29761 consensus_queue.cc:260] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:39.484431 29558 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.486182 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:39.487104 22707 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:39.482348 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.490752 22701 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:12:39.495786 29761 raft_consensus.cc:3060] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:39.453362 29715 raft_consensus.cc:2804] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:12:39.500545 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 117971us
W20260709 13:12:39.499395 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:39.504446 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 61198.9us
W20260709 13:12:39.502719 22749 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:39.508980 22749 leader_election.cc:336] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.509631 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:39.508579 29719 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:12:39.515013 29563 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.519073 29614 consensus_queue.cc:1048] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:39.500520 29715 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.523725 29715 raft_consensus.cc:3060] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:39.524121 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:39.526302 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:39.539358 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 80251.6us
I20260709 13:12:39.540503 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.526170 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:39.544471 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.545068 29684 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:39.545774 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 180) took 2818 ms. Trace:
I20260709 13:12:39.546875 29766 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:12:39.548251 29558 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:39.549499 22598 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.550581 29719 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:39.562671 29765 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:39.563280 29765 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.563657 29765 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:39.555269 22528 rpcz_store.cc:276] 0709 13:12:36.727178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.727363 (+   185us) service_pool.cc:224] Handling call
0709 13:12:39.545693 (+2818330us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:39.214270 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:39.226816 (+ 12546us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:39.226885 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:12:39.226898 (+    13us) tablet.cc:662] Decoding operations
0709 13:12:39.227627 (+   729us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:39.227761 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:39.227804 (+    43us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:12:39.228300 (+   496us) tablet.cc:701] Row locks acquired
0709 13:12:39.228351 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:12:39.228666 (+   315us) write_op.cc:270] Start()
0709 13:12:39.228842 (+   176us) write_op.cc:276] Timestamp: P: 1783602759228465 usec, L: 0
0709 13:12:39.228865 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:12:39.229726 (+   861us) log.cc:844] Serialized 1293 byte log entry
0709 13:12:39.345462 (+115736us) op_driver.cc:464] REPLICATION: finished
0709 13:12:39.487176 (+141714us) write_op.cc:301] APPLY: starting
0709 13:12:39.487275 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:39.493904 (+  6629us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:39.494071 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:39.518671 (+ 24600us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:39.518958 (+   287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:12:39.519009 (+    51us) write_op.cc:312] APPLY: finished
0709 13:12:39.519264 (+   255us) log.cc:844] Serialized 41 byte log entry
0709 13:12:39.519471 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:39.519591 (+   120us) write_op.cc:454] Released schema lock
0709 13:12:39.519637 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:39.523111 (+  3474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:39.523153 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":57414,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":141340,"apply.run_cpu_time_us":7632,"apply.run_wall_time_us":35933,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16691,"num_ops":3,"prepare.queue_time_us":301183,"prepare.run_cpu_time_us":4372,"prepare.run_wall_time_us":30674,"replication_time_us":116322,"thread_start_us":143763,"threads_started":2}]]}
I20260709 13:12:39.570964 29760 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:39.573004 29763 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:39.576041 29767 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:39.576797 29767 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:39.584113 29767 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:39.584000 29679 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:39.586763 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46385.8us
I20260709 13:12:39.596686 29700 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:39.593122 22749 consensus_peers.cc:597] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:39.592357 29766 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:39.607867 29766 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:39.610769 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 61373.2us
I20260709 13:12:39.611536 29558 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:39.612502 22598 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:39.603153 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.615994 22707 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
W20260709 13:12:39.618535 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56924.9us
W20260709 13:12:39.614735 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:39.620970 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 94947.9us
I20260709 13:12:39.619004 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:39.624868 29558 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:12:39.624646 29719 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:39.627789 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:39.630172 29679 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:39.633201 29582 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=11 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
W20260709 13:12:39.625866 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.648258 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.593277 29759 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:12:39.634724 29582 tablet_bootstrap.cc:492] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Bootstrap complete.
I20260709 13:12:39.655329 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:39.639089 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:39.663383 22749 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.695317 29765 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.698125 29765 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:39.700819 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:39.710160 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62100.8us
I20260709 13:12:39.686829 29684 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.685170 29582 ts_tablet_manager.cc:1403] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Time spent bootstrapping tablet: real 2.401s	user 0.074s	sys 0.015s
I20260709 13:12:39.711890 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:39.709324 29773 log.cc:927] Time spent T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:12:39.722860 23412 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:12:39.709530 29603 raft_consensus.cc:2764] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:12:39.725093 29772 raft_consensus.cc:2804] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:39.725145 29768 raft_consensus.cc:2804] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:39.725775 29772 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:39.725898 29768 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.726330 29772 raft_consensus.cc:3060] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:39.726404 29768 raft_consensus.cc:3060] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:39.738161 29654 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.741174 29654 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.743781 29654 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:39.756855 29700 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:39.731168 29766 raft_consensus.cc:2804] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:39.767504 29766 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.768105 29766 raft_consensus.cc:3060] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:39.774935 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 70186.8us
I20260709 13:12:39.720136 29684 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:39.753999 29715 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.781844 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:39.807735 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 95986.2us
I20260709 13:12:39.824621 22530 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 17 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:39.825423 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:39.810237 29603 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:39.826566 29603 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.827005 22530 catalog_manager.cc:5697] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 17 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:39.829224 29603 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:39.831764 29768 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.827055 22601 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.847227 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:39.854573 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:39.856511 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.816574 29684 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:39.809319 22581 auto_rebalancer.cc:629] Failed to schedule move for tablet 12b8ec59bdc04cd8b5dae5bcc35466f6: Illegal state: Replica 91ebb78bf12f41baae84669c1effc423 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:39.857452 22749 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:39.805399 29784 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:39.854060 29760 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:39.858440 23414 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:39.872171 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:39.875921 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48951.3us
I20260709 13:12:39.862666 29763 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:12:39.883693 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.865926 29784 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.871495 29760 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:39.894524 22747 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:39.900606 29654 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:39.901101 29654 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:39.901480 29654 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:39.907773 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:39.897708 29760 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:39.895172 29784 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:39.903272 29779 raft_consensus.cc:2804] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:39.910297 29768 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:39.917327 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:39.916405 29772 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:39.945751 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:39.904747 29765 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:12:39.968729 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:39.965983 29563 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:39.935408 29715 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:39.917052 29766 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:39.935920 29779 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:39.979149 29563 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:39.992164 22581 auto_rebalancer.cc:629] Failed to schedule move for tablet b907163bc2174a2aa11df858ccae00ac: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:39.982972 29779 raft_consensus.cc:3060] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:39.990557 29563 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:40.024890 29700 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.025733 29700 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.028702 29700 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.029881 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.031124 22707 raft_consensus.cc:2393] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 22: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:40.031947 29772 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.032895 29684 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:40.036628 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.037698 22701 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 19.
I20260709 13:12:40.043535 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.044811 22701 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 15.
I20260709 13:12:40.052664 29582 raft_consensus.cc:359] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.054535 29582 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Initialized, Role: LEARNER
I20260709 13:12:40.055351 29582 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } }
I20260709 13:12:40.058126 29563 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.059423 29582 ts_tablet_manager.cc:1434] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Time spent starting tablet: real 0.337s	user 0.011s	sys 0.001s
I20260709 13:12:40.061172 29563 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:40.052129 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 147641us
W20260709 13:12:40.064435 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:40.066381 22750 leader_election.cc:400] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:12:40.066854 22750 leader_election.cc:403] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.068675 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:40.070724 29760 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.077099 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.085367 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.089553 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.086194 22709 raft_consensus.cc:2376] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15. Re-sending same reply.
I20260709 13:12:40.087608 29661 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:40.097537 29661 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:40.098075 29661 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:40.125308 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 155281us
I20260709 13:12:40.133535 29796 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:40.139372 29766 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.137773 29654 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.149669 22601 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.070140 29784 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:40.157840 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 185480us
I20260709 13:12:40.164443 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.152045 29654 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.151842 29760 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:40.164732 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 84951.2us
W20260709 13:12:40.170986 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 74401.4us
I20260709 13:12:40.171387 23419 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:40.174345 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:40.172921 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.166018 29783 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:40.188832 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.168860 29684 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 12. Current term is 13.
I20260709 13:12:40.190376 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.190419 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.193646 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.194259 29766 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:12:40.198110 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.189352 29563 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:40.205999 29563 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:40.206548 29563 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:40.209549 29784 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.211624 29779 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.193747 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.223266 29779 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.217468 22781 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:12:40.217461 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:40.233799 22776 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:40.234790 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.218402 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:40.228088 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:40.238242 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46882.6us
I20260709 13:12:40.231266 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:40.241004 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.241859 22782 raft_consensus.cc:2368] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:12:40.241822 29783 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:40.228482 29661 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.239085 29763 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:12:40.231290 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.244782 22778 raft_consensus.cc:2368] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:40.232378 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.249243 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.248970 29763 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:40.251716 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:40.264629 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76261.7us
I20260709 13:12:40.253304 23418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:40.253182 29772 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:40.253541 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.253897 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.256176 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.267040 22702 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:40.256053 29790 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:40.257133 29783 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.257815 22704 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23. Ops: []
I20260709 13:12:40.259002 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.317493 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.318832 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.320192 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.322110 22708 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:12:40.257135 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.258427 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.260524 22706 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:12:40.262424 29563 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.263160 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.263453 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.216522 23419 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:40.277084 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.285499 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.285650 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.286363 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.287177 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.281786 29661 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.287495 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.291900 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:40.292716 22784 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:12:40.293094 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.287949 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.307318 22601 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.301566 22702 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:40.289851 23385 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.290450 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:40.337997 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 64505.9us
I20260709 13:12:40.335381 23417 pending_rounds.cc:85] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 15
I20260709 13:12:40.342666 22701 raft_consensus.cc:2393] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db in current term 12: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:40.344686 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:40.347754 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 95176.3us
I20260709 13:12:40.348773 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.339725 23417 pending_rounds.cc:107] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 13:12:40.351007 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:40.359136 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 51958.5us
I20260709 13:12:40.351075 23417 raft_consensus.cc:2889] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:40.367807 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:40.369356 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79000.8us
I20260709 13:12:40.370632 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.382704 22601 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.389267 22778 raft_consensus.cc:1240] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:40.389802 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:40.356518 29603 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:40.386107 22747 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:40.392535 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.393868 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:40.396189 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45363.3us
I20260709 13:12:40.396965 22601 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.397915 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.399467 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.402297 29785 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:40.403091 29603 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:40.402493 29796 raft_consensus.cc:2804] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:40.405287 29796 raft_consensus.cc:697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:40.406301 29796 consensus_queue.cc:237] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.406867 29759 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:40.362231 29766 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:40.361598 29684 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:12:40.413842 29551 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:40.414306 29765 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:40.411279 29661 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:40.415437 29781 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:40.381207 29784 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:40.417016 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 127250us
I20260709 13:12:40.368052 29803 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:12:40.378504 29679 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:40.395277 29654 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:40.430250 22601 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.428277 22750 leader_election.cc:400] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18.
I20260709 13:12:40.445829 22750 leader_election.cc:403] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.410387 22777 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:40.449121 29781 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:40.449792 29781 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:40.450860 29781 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:40.453745 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 56364.6us
W20260709 13:12:40.454890 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 122236us
I20260709 13:12:40.455139 22675 leader_election.cc:400] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:12:40.455634 22675 leader_election.cc:403] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.456766 22675 leader_election.cc:400] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:40.457201 22675 leader_election.cc:403] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:12:40.425228 29661 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:40.457906 29700 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12.
I20260709 13:12:40.458376 29818 raft_consensus.cc:3060] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:40.461174 29679 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.462699 29603 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:40.463253 29603 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.463804 29679 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.465991 29603 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.467100 29799 raft_consensus.cc:2764] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:12:40.468008 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.468650 22719 tablet_copy_service.cc:342] P 91ebb78bf12f41baae84669c1effc423: Request end of tablet copy session 6a31ac6451ff4f73bee634487d84d001-b8291ea7e33a40b5afaebc02d6ce7e9d received from {username='slave'} at 127.0.0.1:49444
I20260709 13:12:40.469305 22719 tablet_copy_service.cc:434] P 91ebb78bf12f41baae84669c1effc423: ending tablet copy session 6a31ac6451ff4f73bee634487d84d001-b8291ea7e33a40b5afaebc02d6ce7e9d on tablet b8291ea7e33a40b5afaebc02d6ce7e9d with peer 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:40.458009 29661 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:12:40.476982 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52513.3us
I20260709 13:12:40.479764 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:40.500790 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:40.421932 29766 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.512814 22675 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.442572 29761 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:12:40.502650 29684 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:40.519435 29684 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:40.428285 29759 consensus_queue.cc:1059] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:12:40.522810 22601 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:40.520133 23384 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:40.519958 29684 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:40.482402 29654 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681) with higher term. Message: Invalid argument: T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 17. Current term is 18.
I20260709 13:12:40.526533 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.529886 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.527091 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.549685 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:40.551486 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.574424 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:40.586195 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 66116.4us
I20260709 13:12:40.558467 29765 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:40.557911 29803 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:12:40.585793 29697 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:40.565748 29761 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:12:40.630475 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 119806us
I20260709 13:12:40.633193 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.632938 29809 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:40.639319 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.600359 29563 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.615415 29603 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:40.661202 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 128600us
W20260709 13:12:40.669765 29813 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 13:12:40.670498 29819 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:40.671190 29819 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.673799 29819 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.690305 29766 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:40.691829 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:40.692817 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 19.
I20260709 13:12:40.694643 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:40.695705 29766 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:40.696194 29766 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:40.696573 29766 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:40.697930 29684 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:40.701329 29816 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.213s	user 0.000s	sys 0.005s
I20260709 13:12:40.709765 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:40.715456 29817 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.266s	user 0.002s	sys 0.000s
I20260709 13:12:40.717021 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:40.718827 29551 consensus_queue.cc:1059] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:40.721345 29558 raft_consensus.cc:3055] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:12:40.652493 29796 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:40.662889 29826 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:12:40.734058 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 307) took 3888 ms. Trace:
W20260709 13:12:40.802249 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85333.9us
I20260709 13:12:40.737300 22528 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 18 to 19, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:40.741374 29661 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.750245 22776 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 12. Current term is 13. Ops: []
W20260709 13:12:40.757584 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 195321us
I20260709 13:12:40.805442 22705 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:12:40.758379 29818 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075) with higher term. Message: Invalid argument: T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 15. Current term is 16.
I20260709 13:12:40.776800 29766 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:40.795898 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:40.804998 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:40.799115 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89500.9us
I20260709 13:12:40.812456 22703 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:40.817595 29829 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:12:40.818012 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:40.819818 22528 catalog_manager.cc:5697] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 17, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 17 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.799970 29558 raft_consensus.cc:740] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:12:40.831660 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 183539us
I20260709 13:12:40.822139 29797 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:40.811590 29779 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:40.832717 22528 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:40.802428 22530 rpcz_store.cc:276] 0709 13:12:36.845588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:36.845783 (+   195us) service_pool.cc:224] Handling call
0709 13:12:40.733978 (+3888195us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:39.991128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:40.027308 (+ 36180us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:40.027449 (+   141us) write_op.cc:435] Acquired schema lock
0709 13:12:40.027469 (+    20us) tablet.cc:662] Decoding operations
0709 13:12:40.027986 (+   517us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:40.028054 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:40.028087 (+    33us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:12:40.028464 (+   377us) tablet.cc:701] Row locks acquired
0709 13:12:40.028484 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:12:40.028656 (+   172us) write_op.cc:270] Start()
0709 13:12:40.028783 (+   127us) write_op.cc:276] Timestamp: P: 1783602760028594 usec, L: 0
0709 13:12:40.028804 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:12:40.029592 (+   788us) log.cc:844] Serialized 923 byte log entry
0709 13:12:40.203674 (+174082us) op_driver.cc:464] REPLICATION: finished
0709 13:12:40.317168 (+113494us) write_op.cc:301] APPLY: starting
0709 13:12:40.317277 (+   109us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:40.369011 (+ 51734us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:40.382345 (+ 13334us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:40.406224 (+ 23879us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:40.406560 (+   336us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 13:12:40.412259 (+  5699us) write_op.cc:312] APPLY: finished
0709 13:12:40.432416 (+ 20157us) log.cc:844] Serialized 33 byte log entry
0709 13:12:40.475750 (+ 43334us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:40.482680 (+  6930us) write_op.cc:454] Released schema lock
0709 13:12:40.483649 (+   969us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:40.490965 (+  7316us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:40.491000 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":113205,"apply.run_cpu_time_us":7249,"apply.run_wall_time_us":173748,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12503,"num_ops":2,"prepare.queue_time_us":135411,"prepare.run_cpu_time_us":3584,"prepare.run_wall_time_us":46784,"replication_time_us":174661,"thread_start_us":175601,"threads_started":2}]]}
I20260709 13:12:40.867578 22705 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:12:40.841346 29797 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:40.845289 29661 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.831288 29772 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:12:40.855780 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.840387 29829 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:40.857237 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:40.858862 29684 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:40.861353 29832 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:40.862506 29819 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:40.886595 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:40.821853 29768 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:12:40.889647 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 80201.1us
W20260709 13:12:40.945528 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58981.4us
W20260709 13:12:40.964607 22675 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:40.876364 29797 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:40.969429 29830 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:40.970634 29830 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.973269 22776 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:12:40.973821 29830 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:40.961408 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 208785us
I20260709 13:12:40.975528 22783 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:12:40.894429 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.965520 22598 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.968806 29829 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:40.969518 22675 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:40.981151 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 86861.2us
I20260709 13:12:40.982031 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.983053 22750 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:40.984018 29781 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:12:40.984490 29781 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:40.984858 29843 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:40.985458 29843 raft_consensus.cc:697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:40.985569 29781 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:40.986986 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:40.987794 29781 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:40.988320 29796 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:40.990242 29792 consensus_queue.cc:1059] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 13:12:40.990288 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:40.992017 22749 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:40.993510 29796 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:12:40.995129 29801 consensus_queue.cc:1175] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "91ebb78bf12f41baae84669c1effc423" responder_term: 12 status { last_received { term: 12 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 12 index: 9 } }
I20260709 13:12:40.986495 29843 consensus_queue.cc:237] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:41.016944 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 94309.4us
I20260709 13:12:41.018527 22598 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.025471 22598 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.036005 22598 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.020174 29768 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:41.037456 29760 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:12:41.050422 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:40.894971 29819 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.037515 29783 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:41.066584 22598 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:41.068061 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 85118.5us
I20260709 13:12:41.044059 29826 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:12:41.071978 22598 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:40.893321 29832 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.081128 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:41.062492 29715 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:12:40.883373 29558 consensus_queue.cc:260] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:41.097611 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 128113us
I20260709 13:12:41.070252 29761 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:12:41.098809 22675 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.099354 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.100464 22709 raft_consensus.cc:1240] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 22. Current term is 23. Ops: []
I20260709 13:12:41.103122 22598 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:41.114768 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 71973.9us
I20260709 13:12:41.112983 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:41.089116 29719 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:41.140102 22704 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:12:41.146179 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 47566.5us
W20260709 13:12:41.146852 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 43894.9us
W20260709 13:12:41.147500 22675 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.148162 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.100261 29783 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:41.105822 29654 raft_consensus.cc:3055] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:12:41.159255 23384 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:41.161571 29848 log.cc:927] Time spent T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.176s	user 0.005s	sys 0.000s
W20260709 13:12:41.162223 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.162969 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.163522 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.165685 22675 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.126202 29826 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:41.173566 29765 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:41.174427 29765 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.177065 29765 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.178062 23411 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:12:41.162580 29832 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:41.188585 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.192075 29558 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:41.162410 29838 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:41.166474 29839 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:41.200129 29819 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.156287 29772 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:12:41.209971 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 44296.1us
W20260709 13:12:41.211396 22675 consensus_peers.cc:597] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.213796 22675 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:41.168321 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.216996 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48784.7us
I20260709 13:12:41.153954 29654 raft_consensus.cc:740] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:41.214537 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.220531 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.189404 29759 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:12:41.164469 29803 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:41.223935 29839 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
W20260709 13:12:41.238489 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.266331 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 193685us
W20260709 13:12:41.266417 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 76048us
I20260709 13:12:41.269174 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:41.271212 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.272281 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58544us
W20260709 13:12:41.273068 22675 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.236280 29772 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.274264 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.283514 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 83031.2us
I20260709 13:12:41.270428 29826 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:41.280506 29803 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:41.286016 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.293994 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.239542 29855 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:12:41.300298 29684 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:41.301565 29849 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:41.302424 29849 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.279628 29760 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.307905 22675 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.313257 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.309528 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.315075 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.274832 29563 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:12:41.308851 29825 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:12:41.333364 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 48141.1us
I20260709 13:12:41.281489 29772 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.293772 29783 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 13:12:41.334808 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49069.8us
W20260709 13:12:41.333774 22675 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.348187 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.345600 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:41.349236 22779 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:41.331250 29819 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:41.355222 22776 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:12:41.358958 22783 raft_consensus.cc:3055] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:12:41.359596 22783 raft_consensus.cc:740] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:41.318857 29760 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.306634 29654 consensus_queue.cc:260] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.378223 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:41.363214 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94141.2us
I20260709 13:12:41.382480 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:41.360551 22783 consensus_queue.cc:260] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.384119 29765 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:41.392623 29865 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:41.392452 29765 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.393321 29865 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.395273 29765 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:41.395938 29865 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:41.396262 29837 raft_consensus.cc:3055] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:41.396890 29837 raft_consensus.cc:740] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:41.400771 29760 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.401211 29863 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.402364 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 68640.2us
W20260709 13:12:41.404047 22675 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.404413 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.402501 22783 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:41.360270 29826 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:12:41.402263 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 107645us
I20260709 13:12:41.359529 29849 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.418277 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:41.342386 29797 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.358958 29792 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:12:41.412725 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.422387 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:41.423123 22675 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.432945 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.399125 29837 consensus_queue.cc:260] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.431149 29858 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:41.427361 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:41.411921 29654 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:12:41.433730 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:41.411576 29819 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:12:41.442041 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.443497 22675 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.443596 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 79632us
W20260709 13:12:41.444343 22675 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.446930 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.448904 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.449695 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:41.450253 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.452685 22775 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 21. Current term is 22. Ops: []
I20260709 13:12:41.453293 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.454347 23384 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.457510 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:41.442263 29700 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:41.460247 22783 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:12:41.460678 22783 pending_rounds.cc:85] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 12
I20260709 13:12:41.460505 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.461261 22703 raft_consensus.cc:2468] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 18.
I20260709 13:12:41.462586 22783 pending_rounds.cc:107] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:12:41.463004 22783 raft_consensus.cc:2889] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:41.466146 22709 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:12:41.401844 29863 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.462215 29819 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:12:41.459363 29866 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.475243 22675 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.475952 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:41.470815 29855 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.477137 29856 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:41.477744 29856 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:41.477516 29855 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.478247 29856 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:41.480310 29849 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:41.474047 29866 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:41.482731 21526 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 22660)
I20260709 13:12:41.483978 29866 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:41.485661 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.486654 22783 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
I20260709 13:12:41.487301 22784 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:41.487960 29855 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:41.453065 29826 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:41.510135 22779 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
W20260709 13:12:41.471050 29857 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.181s	user 0.000s	sys 0.002s
W20260709 13:12:41.557943 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 82750.8us
W20260709 13:12:41.592020 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.560122 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:41.595148 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:41.599506 22675 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:41.600370 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:41.606099 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.611438 22707 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:12:41.594172 29825 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:41.607821 22675 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:41.560034 22599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.616874 22777 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:12:41.616405 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.559693 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 65784.4us
I20260709 13:12:41.562980 29772 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:41.611545 29826 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:41.638235 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.562016 29863 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.644737 29839 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:41.646593 29759 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:41.687533 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.687997 29759 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.688673 29826 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:41.691447 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.650157 29765 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:41.651057 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.695766 22707 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:41.653060 29837 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:41.653544 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.657889 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:41.713102 22704 raft_consensus.cc:2468] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:41.664073 29797 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:41.665166 29766 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:41.721141 22781 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:12:41.615869 29803 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:41.682931 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 75342.3us
I20260709 13:12:41.670213 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:41.686805 29839 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.728807 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.698406 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:41.706744 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.730952 29779 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:41.721009 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63217.8us
I20260709 13:12:41.690407 29759 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:41.728389 29783 raft_consensus.cc:2804] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:41.686656 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:41.747646 22779 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:41.743130 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:41.744223 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 104684us
I20260709 13:12:41.759497 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:41.675827 29856 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.763227 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:41.765539 29856 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:41.771772 29772 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:41.777463 22600 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:41.780483 29715 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:41.781484 29873 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:41.782241 29873 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:41.746212 29839 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:41.783385 29873 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:41.732023 29849 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:41.746726 29783 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:41.784776 29783 raft_consensus.cc:3060] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:41.784713 29849 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.789660 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:41.793133 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 47862us
I20260709 13:12:41.796391 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:41.802944 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.804240 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:41.818120 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:41.827557 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 327048us
W20260709 13:12:41.828526 23384 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 140 similar messages]
W20260709 13:12:41.896468 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67963.5us
I20260709 13:12:41.755648 29779 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:41.797783 29759 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
W20260709 13:12:41.907166 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89150.6us
W20260709 13:12:41.944619 22750 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:41.825191 29863 consensus_queue.cc:1059] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:12:41.906446 29785 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:41.996191 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887cfff73fab4d52b2e39f5b6089133a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:41.998044 22706 raft_consensus.cc:1240] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:12:41.998730 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:41.821971 29719 raft_consensus.cc:2804] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:42.024858 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:42.004868 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 175158us
W20260709 13:12:42.005278 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 161202us
I20260709 13:12:41.821722 29558 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:41.756640 29803 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 13:12:42.013418 29860 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:42.022573 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.025969 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:42.027554 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.031045 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.041549 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:42.014469 29779 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.043417 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:42.044042 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.046391 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:42.002413 29849 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.045889 22777 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:42.016167 29829 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:42.027313 29796 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:42.027485 29558 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:42.065395 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:42.038892 29719 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:42.085456 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.082989 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:42.063453 29882 log.cc:927] Time spent T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:12:42.093479 29860 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:42.094851 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:42.102968 29783 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.075103 29558 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:42.063323 29779 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:42.054109 29835 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:42.085523 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.115814 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 181) took 2409 ms. Trace:
I20260709 13:12:42.050493 29759 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:42.091504 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47536.8us
I20260709 13:12:42.092115 29811 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:42.037331 29785 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.118893 22704 raft_consensus.cc:2468] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:42.088460 29719 raft_consensus.cc:3060] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:42.138173 29765 raft_consensus.cc:3055] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:12:42.145061 22524 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:42.056221 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.067623 29829 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.084633 29679 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:12:42.215500 29781 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.216220 29781 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.222123 22708 raft_consensus.cc:1240] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:42.224424 29829 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.182197 29779 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:12:42.236706 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.238545 22701 raft_consensus.cc:1275] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:12:42.200703 29765 raft_consensus.cc:740] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:42.218531 29781 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.207386 22524 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:42.240299 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.252760 22524 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:42.251679 22710 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:42.241160 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.152858 29783 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.072052 29796 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.171875 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.179741 22777 raft_consensus.cc:1240] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 19. Current term is 20. Ops: []
I20260709 13:12:42.273314 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.123287 29700 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:42.185079 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:42.254667 22524 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db reported cstate change: term changed from 11 to 13, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 13 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:42.174067 22528 rpcz_store.cc:276] 0709 13:12:39.714504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:39.717584 (+  3080us) service_pool.cc:224] Handling call
0709 13:12:42.115732 (+2398148us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:41.076149 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:41.077105 (+   956us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:41.077161 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:12:41.077192 (+    31us) tablet.cc:662] Decoding operations
0709 13:12:41.077981 (+   789us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:41.078107 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:41.078151 (+    44us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:12:41.078536 (+   385us) tablet.cc:701] Row locks acquired
0709 13:12:41.078561 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:12:41.088145 (+  9584us) write_op.cc:270] Start()
0709 13:12:41.096886 (+  8741us) write_op.cc:276] Timestamp: P: 1783602761081639 usec, L: 0
0709 13:12:41.114016 (+ 17130us) op_driver.cc:348] REPLICATION: starting
0709 13:12:41.169722 (+ 55706us) log.cc:844] Serialized 1293 byte log entry
0709 13:12:41.353096 (+183374us) op_driver.cc:464] REPLICATION: finished
0709 13:12:41.756241 (+403145us) write_op.cc:301] APPLY: starting
0709 13:12:41.774064 (+ 17823us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:41.776646 (+  2582us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:41.776760 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:41.798525 (+ 21765us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:41.834011 (+ 35486us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 13:12:41.859674 (+ 25663us) write_op.cc:312] APPLY: finished
0709 13:12:41.937365 (+ 77691us) log.cc:844] Serialized 41 byte log entry
0709 13:12:41.996564 (+ 59199us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:41.996682 (+   118us) write_op.cc:454] Released schema lock
0709 13:12:41.996739 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:42.027067 (+ 30328us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:42.036244 (+  9177us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":241523,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":352467,"apply.run_cpu_time_us":9864,"apply.run_wall_time_us":265897,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16691,"num_ops":3,"prepare.queue_time_us":179973,"prepare.run_cpu_time_us":4777,"prepare.run_wall_time_us":96086,"replication_time_us":224072,"thread_start_us":463347,"threads_started":2}]]}
I20260709 13:12:42.191071 22709 raft_consensus.cc:1275] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:12:42.205897 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 149755us
I20260709 13:12:42.202726 29863 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.261021 23420 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:42.276424 22750 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:42.189719 29811 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:42.274377 23411 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 12.
I20260709 13:12:42.279657 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.278043 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:12:42.242323 29884 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.091s	user 0.003s	sys 0.000s
W20260709 13:12:42.282457 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.293948 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:42.244321 29679 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:42.288820 22675 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:42.305676 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.305938 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:42.302050 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:42.251482 29785 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:42.306563 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:42.291265 29811 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:42.325016 22782 raft_consensus.cc:1240] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 21. Current term is 22. Ops: []
W20260709 13:12:42.326462 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 261205us
I20260709 13:12:42.296414 29772 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:42.304629 29856 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:42.314860 29829 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:42.290652 29700 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.248332 29765 consensus_queue.cc:260] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.336588 29772 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:42.399194 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 101686us
W20260709 13:12:42.394395 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 52313.4us
I20260709 13:12:42.365625 29715 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.289390 29863 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.316636 29679 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.404845 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:42.409982 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 98102.1us
W20260709 13:12:42.417085 22675 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.417888 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.411790 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:42.421613 22781 raft_consensus.cc:1240] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 19. Current term is 20. Ops: []
W20260709 13:12:42.424957 22675 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.390606 29772 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:42.414207 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:42.425753 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.363054 22710 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:12:42.440490 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:42.442502 22675 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.445815 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.389494 29799 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:12:42.431080 29825 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:42.448647 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.405750 29839 consensus_queue.cc:1059] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
W20260709 13:12:42.455747 22675 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.458580 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.439777 29766 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:42.461969 22675 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:42.462774 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.415516 29715 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.463626 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:42.459273 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:42.483343 21526 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 22660)
I20260709 13:12:42.447306 29826 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:42.488770 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:42.457926 29785 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:42.504384 29891 log.cc:927] Time spent T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:12:42.505156 29889 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:12:42.506093 29888 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:12:42.461514 29830 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:42.512609 29829 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:42.514593 29700 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:42.544971 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 330060us
I20260709 13:12:42.547596 22702 raft_consensus.cc:3055] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:42.548019 22702 raft_consensus.cc:740] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:42.548795 22702 consensus_queue.cc:260] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.549809 22702 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:42.519484 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.537858 29796 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.516500 29715 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.559667 29863 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.560931 29697 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:42.561421 29697 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:42.564795 29679 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.567052 29558 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.570487 29558 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:42.574252 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 126930us
W20260709 13:12:42.579674 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 117732us
W20260709 13:12:42.580031 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 52019.6us
W20260709 13:12:42.580672 22675 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.581478 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:42.582056 22675 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.582257 22707 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:42.582692 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.584813 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:42.588054 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 147652us
I20260709 13:12:42.585757 23419 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:12:42.588943 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:42.589977 22750 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:42.596103 22675 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.596683 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:42.632418 22675 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:42.633615 22675 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.634063 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.562458 29697 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.628363 29799 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.569648 29719 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.640161 29783 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.706059 29783 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.709254 29783 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.640321 29768 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.710666 29768 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.713255 29768 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.637004 29910 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:42.717288 29894 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.652380 29781 raft_consensus.cc:3055] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:12:42.733515 29781 raft_consensus.cc:740] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:42.733502 29894 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.735002 29781 consensus_queue.cc:260] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.736478 29894 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:42.737155 29781 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:42.659543 29799 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.660209 29829 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:42.741487 29799 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.694814 29860 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.743065 29860 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:42.650185 29772 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:42.703576 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:42.680728 29697 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:42.675318 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:42.763639 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 88434.4us
W20260709 13:12:42.644975 22675 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.765733 22675 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:42.767050 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 122103us
W20260709 13:12:42.768060 22675 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.768795 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:42.669262 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:42.646703 29830 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:42.773818 29783 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:12:42.776665 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:42.777653 29832 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:12:42.778559 29832 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:42.780323 22675 leader_election.cc:336] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.781170 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.655298 29765 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:12:42.797626 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 94127us
W20260709 13:12:42.799039 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:42.747701 29860 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:42.820150 22675 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.820729 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:42.822269 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:42.824333 22675 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.824894 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:42.831809 22675 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.832552 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:42.834182 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:42.835207 22675 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:42.835582 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:42.848589 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:42.843509 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 225149us
I20260709 13:12:42.859190 22707 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:42.823804 29719 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:42.864526 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.120424 22705 raft_consensus.cc:2468] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 12.
I20260709 13:12:43.121588 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.123023 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:42.860986 22702 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:12:42.864895 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:43.125548 22702 pending_rounds.cc:85] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 12
I20260709 13:12:43.125841 22702 pending_rounds.cc:107] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 19.13
I20260709 13:12:42.825779 29910 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:43.126247 22702 raft_consensus.cc:2889] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:43.099720 29908 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:43.132186 29759 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.132876 29759 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.133399 29765 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.133981 29765 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:42.829035 29829 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:43.104504 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:42.861469 29832 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:43.107062 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 517168us
I20260709 13:12:42.854318 29830 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:43.141741 29759 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.113255 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.146492 29835 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:42.876578 29768 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.152367 29835 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.152801 29768 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.154923 29768 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.125746 22710 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:43.156587 29835 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.136008 29719 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.163892 29719 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.165216 29900 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:43.136152 29783 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.166415 29783 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.167936 29765 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.168670 29783 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.148021 29910 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.138172 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:43.166450 29898 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:43.170197 23420 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:12:43.100817 29811 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.758s	user 0.007s	sys 0.007s
I20260709 13:12:43.175012 29811 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.176771 29850 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.177912 29811 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.177626 29850 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:43.102041 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 266830us
I20260709 13:12:43.143446 29870 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:43.169448 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 469491us
W20260709 13:12:43.189977 22747 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.134356 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:43.180421 22600 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.102044 29772 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.187326 22710 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:43.188339 23417 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 16. Current term is 17. Ops: []
W20260709 13:12:43.194280 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 51563.5us
W20260709 13:12:43.195993 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 61681.5us
I20260709 13:12:43.177942 29907 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.219117 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.224948 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.224403 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.211215 29783 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:43.215060 29898 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.270207 23417 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46262 (request call id 1403) took 1162 ms. Trace:
I20260709 13:12:43.210994 29900 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.228360 29829 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.272608 23417 rpcz_store.cc:276] 0709 13:12:42.103428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.103603 (+   175us) service_pool.cc:224] Handling call
0709 13:12:42.103930 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:43.193629 (+1089699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.265205 (+ 71576us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54006ee454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:12:43.265411 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2473124736}
W20260709 13:12:43.278721 22674 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.281539 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:43.267441 29697 consensus_meta.cc:287] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.007s
I20260709 13:12:43.222939 29760 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.170269 29910 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.286309 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:43.287679 29781 consensus_meta.cc:287] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.000s
I20260709 13:12:43.303938 29910 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:43.304862 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.304682 29760 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.305644 29830 consensus_queue.cc:1059] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:12:43.306567 29830 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:43.306952 29830 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:43.307818 29760 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.307799 29830 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.288813 22601 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:43.293963 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.196460 29870 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.299762 22708 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:43.304715 29766 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.267278 29783 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.297552 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.339564 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.346678 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.324070 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.326295 29772 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:43.270740 29719 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.318404 29863 raft_consensus.cc:2804] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:12:43.332414 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.309456 29830 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:43.327157 29870 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.344691 22601 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:43.351838 22704 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:43.317227 23420 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 24.
I20260709 13:12:43.296118 29829 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:43.361791 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 216468us
I20260709 13:12:43.341795 29783 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.341226 29850 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:43.369527 22598 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001; no voters: 
W20260709 13:12:43.353204 29909 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:12:43.395676 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:43.347890 29803 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.352240 29765 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.362231 29863 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.341598 29766 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.374994 29719 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:12:43.424285 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 245346us
I20260709 13:12:43.423856 22776 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 4541) took 1238 ms. Trace:
I20260709 13:12:43.371148 29896 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:43.418231 29781 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:12:43.378387 29910 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.414917 29906 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
W20260709 13:12:43.433739 22675 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.423655 29863 raft_consensus.cc:3060] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:12:43.452852 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.432912 22776 rpcz_store.cc:276] 0709 13:12:42.178513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.180184 (+  1671us) service_pool.cc:224] Handling call
0709 13:12:42.198084 (+ 17900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.201245 (+  3161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.201296 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.201993 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.240771 (+ 38778us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b480012c258. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:12:42.519649 (+278878us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:12:43.118572 (+598923us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.370369 (+251797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.370479 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.370530 (+    51us) raft_consensus.cc:1685] finished
0709 13:12:43.411133 (+ 40603us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.416944 (+  5811us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62250,"prepare.queue_time_us":123672,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":99649,"raft.run_cpu_time_us":2829,"raft.run_wall_time_us":32364,"spinlock_wait_cycles":27022080,"thread_start_us":817655,"threads_started":3,"wal-append.queue_time_us":360031}
I20260709 13:12:43.422881 29803 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.459442 23384 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:43.433203 29920 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:43.460426 22675 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:43.460652 22749 leader_election.cc:336] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.473399 29803 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.473985 29918 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:43.477651 29919 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:12:43.452194 29719 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:43.479830 29719 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.481603 22708 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 24.
I20260709 13:12:43.463675 29923 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.469899 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:43.486689 21526 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 22660)
I20260709 13:12:43.486951 29924 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:12:43.488950 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:43.494628 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:43.496196 29924 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:12:43.498081 29924 consensus_queue.cc:1048] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:12:43.500489 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:43.506675 29896 consensus_queue.cc:1048] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:43.507532 29918 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.464526 29910 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.509541 29766 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.509388 29832 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.510450 29832 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.512079 29829 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.423467 29765 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.515249 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.516276 23420 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:43.459754 29781 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:43.522275 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.518646 22673 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:43.529744 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.531445 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.533890 29908 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.296006 29900 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.534687 29908 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.535058 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.535925 22779 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:43.537374 29900 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.537431 29908 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.321063 29898 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.538187 29908 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:43.508351 29918 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.482816 29923 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.263460 29907 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.538929 29910 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.547031 29832 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.539834 29898 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.547571 29907 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.538596 29908 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.540984 29918 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:43.540411 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:43.550431 22778 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:43.520238 29829 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.551822 29829 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.548660 29908 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:43.553380 29783 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.554222 29829 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.513561 29924 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:12:43.541779 29923 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:43.528369 29909 log.cc:927] Time spent T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:12:43.562040 29765 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.562779 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.564193 22778 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:43.576699 22776 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 13: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:43.567078 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.559079 29781 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.465359 29906 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:43.590588 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 176556us
W20260709 13:12:43.600001 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:43.601302 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.587654 29850 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.605778 29925 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.606511 29925 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.608927 29925 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.611016 29919 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:43.613605 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.615427 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 4553) took 1172 ms. Trace:
I20260709 13:12:43.615821 22775 rpcz_store.cc:276] 0709 13:12:42.443453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.443625 (+   172us) service_pool.cc:224] Handling call
0709 13:12:42.444015 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:42.490784 (+ 46769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:42.490816 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:42.498207 (+  7391us) log.cc:844] Serialized 26 byte log entry
0709 13:12:42.527219 (+ 29012us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:12:42.632185 (+104966us) log.cc:844] Serialized 14 byte log entry
0709 13:12:43.164915 (+532730us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18003be7e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:12:43.166261 (+  1346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:43.166478 (+   217us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:43.598966 (+432488us) raft_consensus.cc:1685] finished
0709 13:12:43.603204 (+  4238us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:43.615315 (+ 12111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356683,"prepare.queue_time_us":116360,"prepare.run_cpu_time_us":178,"prepare.run_wall_time_us":177,"raft.queue_time_us":580004,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":12588,"spinlock_wait_cycles":26295680,"thread_start_us":74663,"threads_started":2,"wal-append.queue_time_us":380427}
I20260709 13:12:43.618000 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.619117 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.619859 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:43.598428 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:43.623236 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.626451 22781 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:43.627962 29829 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.628579 29829 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.625655 22780 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 13.
I20260709 13:12:43.631436 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:43.636332 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 117848us
I20260709 13:12:43.634328 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:43.597805 29900 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.641311 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:43.641366 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.645751 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:43.646473 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 48176.1us
I20260709 13:12:43.649869 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.651064 23420 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 13.
I20260709 13:12:43.652319 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.655067 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.655766 22701 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:43.657771 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:43.662864 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:43.663218 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 184728us
I20260709 13:12:43.662864 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:43.664923 22675 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.666869 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.668459 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a937bd6b17c04d5199f368b1d595cf8b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.668906 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:43.671878 22775 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 22: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:43.665462 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:43.677511 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.679940 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.680723 22709 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
W20260709 13:12:43.684244 22676 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:43.685492 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.686560 22782 raft_consensus.cc:2393] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:43.687703 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.687947 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.647748 29910 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:43.654460 29766 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:12:43.695823 22675 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.706903 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:43.705194 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.707939 22706 raft_consensus.cc:2468] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:43.710275 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:43.630296 29850 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.723461 22601 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:43.731148 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 227515us
I20260709 13:12:43.734968 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:43.730638 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:43.706462 29766 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:43.740969 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45139.9us
I20260709 13:12:43.689867 29934 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:43.665673 29900 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.739780 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:43.742072 22675 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:43.746470 29935 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:43.710947 29926 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.713186 29931 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:43.714512 29898 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.757205 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:43.758548 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:43.721947 29715 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.743794 29760 raft_consensus.cc:2804] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:43.751400 29719 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:43.738245 29772 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:43.887049 29772 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.744097 29832 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:43.747905 29766 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:12:43.740025 29941 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.779757 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.895256 29941 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.896584 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.898123 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.777257 29898 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.787182 29943 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:43.788852 29928 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:43.777820 29922 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:43.776023 29926 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.798586 29863 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.800982 29783 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.802282 29829 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:43.812124 29908 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:43.815850 22784 raft_consensus.cc:1275] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:12:43.818436 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:43.816268 29935 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:43.826495 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.840230 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:43.855046 29896 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:43.859138 29715 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:43.769500 29931 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:43.869760 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.780979 29760 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting leader election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:43.880100 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.899325 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:44.079910 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.933143 22710 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:43.943128 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:44.065268 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.989022 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:43.998258 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49090.3us
W20260709 13:12:44.098578 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 313166us
I20260709 13:12:44.082401 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:44.140203 22708 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:43.987145 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:43.926246 22781 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
W20260709 13:12:44.159163 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 319025us
I20260709 13:12:44.141458 23414 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:44.141194 23420 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:44.091648 23384 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:44.164700 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73226.7us
I20260709 13:12:44.142210 22776 raft_consensus.cc:2468] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:44.143571 22783 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:44.144568 22780 raft_consensus.cc:2468] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:44.156309 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.155539 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.173681 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:43.890477 29772 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.172233 23417 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:44.190784 22708 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
W20260709 13:12:44.204353 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:43.899641 29850 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.200644 22747 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:44.205334 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.214161 22673 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:43.898587 29941 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:44.157749 29760 raft_consensus.cc:3060] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:43.984683 29906 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.069078 29898 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:44.140686 29931 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:44.015790 29900 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:44.221220 29870 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:44.174304 29935 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:44.232478 29939 log.cc:927] Time spent T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.238s	user 0.004s	sys 0.000s
W20260709 13:12:44.237215 22701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.077s	sys 0.104s
I20260709 13:12:44.246060 22708 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:12:44.191151 29781 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:44.216208 29938 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:44.222234 29943 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:44.254218 29930 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:43.947893 29928 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.174505 29896 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:44.290686 29870 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.329874 23421 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 24 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:12:44.303049 29938 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.343101 29563 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:44.306041 29943 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:44.308400 29930 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.364944 29943 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:44.369957 29930 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:44.347070 29938 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:44.412003 29908 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.414585 29863 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:44.471139 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 260692us
W20260709 13:12:44.473034 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 214442us
I20260709 13:12:44.475103 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:44.474782 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.480160 22701 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:44.476436 29925 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:44.482596 29894 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.483762 29926 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:44.483925 29894 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:44.486577 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:44.487224 21526 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 22660)
I20260709 13:12:44.492369 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:44.496181 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 312113us
I20260709 13:12:44.491472 29835 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:12:44.499356 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:44.491539 29837 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:44.499620 29803 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:44.503351 29894 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:44.505499 23414 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:44.509063 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.509949 29894 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:44.483089 29925 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.514818 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.515620 29863 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:12:44.516883 29925 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.518280 22598 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:44.519200 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.527426 22600 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.537647 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.548117 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.549764 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.551707 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.552551 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:44.540077 23420 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:44.551815 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.534824 29860 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:44.555557 22673 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:44.568100 22749 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.568771 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:44.569957 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:44.570664 22749 leader_election.cc:336] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.540055 29558 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:44.583166 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.582849 22601 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:44.581212 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:44.557215 29772 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:44.537985 29894 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 13:12:44.598582 22524 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38102 (request call id 287) took 4341 ms. Trace:
I20260709 13:12:44.542878 29781 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:12:44.584108 29832 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:12:44.611390 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 108633us
I20260709 13:12:44.581776 29925 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:44.599143 22524 rpcz_store.cc:276] 0709 13:12:40.250755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:40.250915 (+   160us) service_pool.cc:224] Handling call
0709 13:12:40.318351 (+ 67436us) spinlock_profiling.cc:243] Waited 63.3 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:12:44.592513 (+4274162us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:42.511147 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:42.511789 (+   642us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:42.511828 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:12:42.511847 (+    19us) tablet.cc:662] Decoding operations
0709 13:12:42.512523 (+   676us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:42.512582 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:42.512609 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:12:42.513000 (+   391us) tablet.cc:701] Row locks acquired
0709 13:12:42.513021 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:12:42.513181 (+   160us) write_op.cc:270] Start()
0709 13:12:42.513285 (+   104us) write_op.cc:276] Timestamp: P: 1783602762513127 usec, L: 0
0709 13:12:42.513306 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:12:42.514018 (+   712us) log.cc:844] Serialized 1694 byte log entry
0709 13:12:43.118523 (+604505us) op_driver.cc:464] REPLICATION: finished
0709 13:12:43.413885 (+295362us) write_op.cc:301] APPLY: starting
0709 13:12:43.421112 (+  7227us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:43.556289 (+135177us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:43.556422 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:43.597159 (+ 40737us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:43.636531 (+ 39372us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:12:43.647897 (+ 11366us) write_op.cc:312] APPLY: finished
0709 13:12:43.705968 (+ 58071us) log.cc:844] Serialized 49 byte log entry
0709 13:12:43.723073 (+ 17105us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:43.736807 (+ 13734us) write_op.cc:454] Released schema lock
0709 13:12:43.742962 (+  6155us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:43.808898 (+ 65936us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:43.808936 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":71868,"reported_tablets":16,"spinlock_wait_cycles":145699200,"child_traces":[["op",{"apply.queue_time_us":253117,"apply.run_cpu_time_us":9870,"apply.run_wall_time_us":396212,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20879,"num_ops":4,"prepare.queue_time_us":169880,"prepare.run_cpu_time_us":3213,"prepare.run_wall_time_us":3213,"replication_time_us":605018,"thread_start_us":252502,"threads_started":2}]]}
I20260709 13:12:44.615641 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.618387 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f4adfa257c34bb88d79bda9d39c5146" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.594245 29863 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:44.599918 29928 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.624162 22783 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:44.611857 29803 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:44.599506 29860 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:44.628132 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:44.613168 29935 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.613687 29715 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:44.649317 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.651022 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.651384 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.655709 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.615116 29835 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:44.618335 29870 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.711443 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.716572 22704 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 14: Already voted for candidate 1f99ad34cc414233929e0a92f8c3d8db in this term.
I20260709 13:12:44.626809 22526 catalog_manager.cc:5697] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 14 to 15, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:44.621850 29832 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:12:44.691582 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108820us
I20260709 13:12:44.727337 22601 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:44.692734 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 109660us
I20260709 13:12:44.622112 29896 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:44.717480 22705 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
W20260709 13:12:44.687309 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 116658us
W20260709 13:12:44.763206 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:44.713126 22703 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 10.
I20260709 13:12:44.741276 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.745599 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:44.748240 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 259324us
I20260709 13:12:44.635403 29803 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.765588 29719 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:44.766307 29719 consensus_queue.cc:1048] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:12:44.766474 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:44.767902 29719 raft_consensus.cc:2804] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:44.768258 29719 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.768642 29719 raft_consensus.cc:3060] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:44.768464 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.770166 29563 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:44.765597 29803 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:44.771502 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:44.772336 29563 raft_consensus.cc:2804] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:44.772722 29563 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:44.773066 29563 raft_consensus.cc:3060] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:44.779378 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.657276 29837 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:44.761319 22526 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 14, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:44.783618 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.680197 29830 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:44.804723 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:44.795773 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.804936 22526 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 17 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:44.795595 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:44.695873 29925 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:44.802069 22778 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:12:44.803052 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:44.806890 22749 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.803882 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:44.761366 29766 consensus_meta.cc:287] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.870s	user 0.015s	sys 0.004s
I20260709 13:12:44.796049 22675 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.812175 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:44.812825 22780 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 22.
I20260709 13:12:44.811990 22782 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 20.
W20260709 13:12:44.826700 22749 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.782028 29935 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:44.830477 22526 catalog_manager.cc:5697] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 20 to 21, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:44.835469 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 108279us
W20260709 13:12:44.861934 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 95956.8us
I20260709 13:12:44.851382 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:44.876564 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 49032.1us
W20260709 13:12:44.877383 22675 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.882658 22775 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 4578) took 1077 ms. Trace:
I20260709 13:12:44.848572 29850 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:12:44.848315 29783 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:44.829501 29960 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:44.893255 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:44.886107 22600 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:44.816737 29830 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.878958 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.893981 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:44.899535 22781 raft_consensus.cc:2393] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:44.884552 22775 rpcz_store.cc:276] 0709 13:12:43.805205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.173405 (+368200us) service_pool.cc:224] Handling call
0709 13:12:44.568583 (+395178us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b5800043598. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:12:44.568818 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:12:44.569062 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:12:44.593103 (+ 24041us) log.cc:844] Serialized 14 byte log entry
0709 13:12:44.882190 (+289087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:12:44.882255 (+    65us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:44.882318 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:44.882401 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:44.882559 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51160,"prepare.queue_time_us":18669,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":3780,"raft.queue_time_us":86271,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":15387,"spinlock_wait_cycles":27581184,"thread_start_us":249307,"threads_started":2,"wal-append.queue_time_us":278382}
I20260709 13:12:44.828397 29766 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.831434 29938 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.881364 29957 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:44.905606 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:44.908818 22675 leader_election.cc:336] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:44.912302 22675 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.897240 29715 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:12:44.851143 29935 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:44.910548 22782 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:44.914016 22776 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:44.860354 29860 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.919216 29870 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:12:44.975760 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 91554.3us
I20260709 13:12:44.908614 29783 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260709 13:12:44.972314 29949 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.156s	user 0.001s	sys 0.001s
I20260709 13:12:44.907240 29837 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.900583 29832 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.913713 29957 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:44.980775 29870 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:45.000226 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 91439.7us
W20260709 13:12:45.007383 22675 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.007936 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:45.009159 22675 leader_election.cc:336] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.009603 22675 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:44.993829 29960 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.005005 29870 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:12:45.039563 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 422406us
W20260709 13:12:45.043720 29961 log.cc:927] Time spent T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:12:45.043939 29969 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:45.014776 29971 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:45.044466 29766 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.045578 29963 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.047646 29963 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.048839 29830 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.061175 29963 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:44.995011 29719 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.035748 29967 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:12:45.066545 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:45.069651 29956 log.cc:927] Time spent T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:12:45.055604 29966 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:45.076326 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:45.097882 29943 consensus_meta.cc:287] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.705s	user 0.007s	sys 0.007s
I20260709 13:12:45.099373 29803 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.101918 29803 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.037642 29960 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.105373 29938 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.111932 29719 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.098529 29943 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.086524 29935 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.114682 29943 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:45.112022 29760 consensus_meta.cc:287] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.881s	user 0.032s	sys 0.047s
I20260709 13:12:45.115898 29760 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.117642 29928 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.118691 29760 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.123281 29943 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:45.124147 29943 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.118045 29563 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.128793 29563 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.115887 29957 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.136413 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:45.139261 29896 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.144954 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:45.145720 22704 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:45.148907 22675 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.149564 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.151243 29967 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:45.118294 29928 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:45.169203 22675 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.169929 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.181540 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:45.186327 29986 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.187160 29986 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.189754 29986 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:45.190300 29986 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.191188 29986 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.193468 29986 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:45.194411 29986 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.195017 29986 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.197374 29986 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:45.201702 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 58659.1us
I20260709 13:12:45.205443 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.139362 29954 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.139942 29896 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.146668 29943 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.210654 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:45.223958 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.177806 29967 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.205130 29935 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:45.252280 29954 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.253391 22704 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:12:45.242419 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:45.258381 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.251657 29960 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.305876 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:45.306571 22706 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:12:45.293095 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:45.208014 29870 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.325587 29935 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.326263 29935 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.329079 29935 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:45.334019 22675 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.334692 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.337708 29935 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:45.337815 29759 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.338480 29759 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.334712 22673 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.340656 29935 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:45.341071 29759 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.346143 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:45.347446 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:45.225606 29986 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.365924 29986 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.353111 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.353914 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:45.368350 29986 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.371378 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.371790 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.372220 23419 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:12:45.371475 23414 raft_consensus.cc:2368] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:12:45.359539 29968 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:45.360919 29954 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:45.363041 29870 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.364650 29896 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.380127 22706 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 25.
I20260709 13:12:45.379892 23385 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:45.386181 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:45.389088 22709 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:12:45.396472 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:45.397465 22703 raft_consensus.cc:1240] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 23. Current term is 25. Ops: []
I20260709 13:12:45.397403 22706 raft_consensus.cc:2376] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18. Re-sending same reply.
I20260709 13:12:45.302695 29967 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.407723 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:12:45.416651 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 174368us
I20260709 13:12:45.427850 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:45.417527 29930 consensus_meta.cc:287] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 1.022s	user 0.009s	sys 0.007s
I20260709 13:12:45.421123 22673 leader_election.cc:400] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:12:45.447135 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.454752 22780 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:45.455206 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:45.455904 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.451675 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.451983 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.403087 29971 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:45.453908 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 55146.7us
I20260709 13:12:45.451331 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.458585 22779 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 17: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:45.456710 22673 leader_election.cc:403] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:12:45.460492 22778 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 20: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:45.413620 29954 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.411563 29980 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.436721 29925 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:45.433810 29870 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:45.466240 22747 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:45.464736 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:45.485590 22749 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:45.491761 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78224.8us
I20260709 13:12:45.492744 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:45.477643 22673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 56806.5us
I20260709 13:12:45.475106 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:45.485719 22779 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:45.486439 29986 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:45.498431 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.500715 22709 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
W20260709 13:12:45.500728 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 73050.6us
I20260709 13:12:45.501300 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:45.501889 22778 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:12:45.444502 22675 leader_election.cc:336] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.503484 22675 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:45.494179 22776 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:12:45.505160 21526 tablet_replica.cc:333] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:45.505363 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60944.4us
I20260709 13:12:45.501749 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:45.508788 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 297313us
W20260709 13:12:45.510771 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 44623.8us
I20260709 13:12:45.482379 29935 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877) with higher term. Message: Invalid argument: T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 for earlier term 17. Current term is 18.
I20260709 13:12:45.515271 21526 raft_consensus.cc:2243] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.521806 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:45.524140 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:45.514065 22601 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:45.527446 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:45.418192 29968 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.530190 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:45.567950 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 43859.7us
I20260709 13:12:45.571794 22778 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 21.
I20260709 13:12:45.572297 22780 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:45.462018 29930 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:45.577780 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 63814.1us
I20260709 13:12:45.601701 21526 pending_rounds.cc:70] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Trying to abort 1 pending ops.
W20260709 13:12:45.600288 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 70024.1us
I20260709 13:12:45.498588 29928 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:45.601799 22601 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:45.511912 29900 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:45.486907 29954 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.510035 29941 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:45.512195 29989 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.602288 21526 pending_rounds.cc:77] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db: Aborting op as it isn't in flight: id { term: 16 index: 9 } timestamp: 7305636919234084864 op_type: NO_OP noop_request { }
I20260709 13:12:45.617590 21526 raft_consensus.cc:2889] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:45.618162 21526 raft_consensus.cc:2272] T a9a28904b6a34dbf81e5d1d4bb08a08c P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.527150 29924 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:45.461362 29971 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.491256 29870 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.615115 22601 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:45.577371 29829 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:12:45.504281 29986 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:45.659780 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81961.2us
W20260709 13:12:45.674026 22675 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:45.674878 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.615505 29974 raft_consensus.cc:2804] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:12:45.737985 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 71879.5us
I20260709 13:12:45.613831 29715 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:45.488286 29980 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:45.752290 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 76349us
I20260709 13:12:45.625293 29924 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:45.759590 29957 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:12:45.767022 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.767295 29957 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 25 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:45.768612 22600 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.768466 29957 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.769404 29765 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:45.770185 29765 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:45.770093 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.771293 29835 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:12:45.772194 29996 log.cc:927] Time spent T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.273s	user 0.000s	sys 0.000s
I20260709 13:12:45.772979 29981 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:45.773500 29981 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.773927 29981 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:45.774827 29991 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:45.775360 29991 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.775758 29991 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:45.781358 29974 raft_consensus.cc:697] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:45.760095 29941 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:45.783495 22600 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:45.785346 29772 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:45.787559 29995 log.cc:927] Time spent T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:12:45.787727 22600 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:45.789013 29772 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:12:45.789383 29984 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 13:12:45.789755 29772 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:45.791100 29772 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.782713 29964 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.793100 29964 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.795675 29964 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.652601 29829 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:45.668238 29850 raft_consensus.cc:493] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:45.661365 29994 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:45.737799 29759 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.750739 29928 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:45.872155 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 257122us
I20260709 13:12:45.825412 29924 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:45.831832 29829 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:45.651561 29989 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:45.940344 29973 log.cc:927] Time spent T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:12:45.940637 30000 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:12:45.941205 29783 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.872597 29759 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.942018 29783 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:45.943835 29803 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:45.944204 29783 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.944427 29803 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:45.945698 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72667.2us
I20260709 13:12:45.949342 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:45.944250 29759 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:45.950142 22776 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 14.
I20260709 13:12:45.875491 29928 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.844216 29850 raft_consensus.cc:515] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.782779 29974 consensus_queue.cc:237] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:45.941293 30000 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:45.952981 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.058744 22709 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:12:46.219841 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 124964us
I20260709 13:12:46.083679 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:46.212978 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 117350us
I20260709 13:12:46.113299 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.222649 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:45.979598 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.068866 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:45.980711 23420 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:12:46.092060 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.104441 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:45.961973 29931 consensus_meta.cc:287] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 1.657s	user 0.007s	sys 0.011s
W20260709 13:12:45.955862 29998 log.cc:927] Time spent T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
W20260709 13:12:46.277861 22749 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.042s	sys 0.089s
W20260709 13:12:46.258700 22675 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.000s	sys 0.012s
W20260709 13:12:46.278339 22749 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.042s	sys 0.089s
W20260709 13:12:46.273910 22750 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.037s	sys 0.101s
I20260709 13:12:45.988464 29830 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.212085 29935 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:46.278556 22675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.000s	sys 0.013s
I20260709 13:12:46.213655 29803 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.212358 29908 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:46.289276 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 615275us
I20260709 13:12:46.224335 29969 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:46.256596 29679 raft_consensus.cc:493] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.273535 29759 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.041947 29829 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.294689 29759 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.296770 29829 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.296921 29759 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.297529 29982 raft_consensus.cc:493] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:46.298154 29982 raft_consensus.cc:515] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.299171 29772 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:46.067050 29870 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.300498 29982 leader_election.cc:290] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:45.965103 29989 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.301951 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22e83136ddd24d71ad143a6f5fc28dd9" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.302723 29928 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.304468 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.306856 22704 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:46.089674 29967 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.291060 29830 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:46.278997 22750 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.037s	sys 0.101s
W20260709 13:12:46.315157 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 784999us
W20260709 13:12:46.049342 29966 consensus_meta.cc:287] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.001s
I20260709 13:12:46.316104 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:45.966455 29930 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.320066 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:46.311962 30002 log.cc:927] Time spent T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:12:46.331714 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:46.333958 22675 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.322244 21526 tablet_replica.cc:333] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:46.300534 29850 leader_election.cc:290] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.334669 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:46.336683 22750 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.334255 29930 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.311057 30000 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.321313 29966 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:12:46.362241 29991 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.004s
I20260709 13:12:46.362649 29930 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.348398 29928 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:46.363039 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.335485 29982 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.334729 29938 consensus_queue.cc:1059] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 13:12:46.364678 29989 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:46.365057 29924 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.367177 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:46.368099 30016 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:46.368127 30010 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.326277 29988 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:46.372161 29830 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.328275 29829 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.381899 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.279892 29931 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.315817 29679 raft_consensus.cc:515] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.312518 29935 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.318060 29908 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.388549 21526 raft_consensus.cc:2243] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:12:46.390151 22707 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:46.334012 30005 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:46.396648 23414 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:12:46.315258 29969 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:46.408299 22601 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:46.412554 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.430083 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.413998 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:46.426661 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 171332us
I20260709 13:12:46.431703 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:46.432300 22702 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:46.403610 29783 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:46.452538 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.454128 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.422046 30011 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:46.436167 29783 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.455392 21526 raft_consensus.cc:2272] T f09c3bbe948b4c7a9f50df8ec4277e9f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.456049 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.427003 29980 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.388059 29991 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.400094 29982 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.462769 22703 raft_consensus.cc:2393] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 18: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:46.466802 22702 raft_consensus.cc:2468] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:46.433835 30016 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:46.436770 30006 raft_consensus.cc:493] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.434523 29860 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.485451 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:46.487131 22750 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:46.427374 29829 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.493741 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.494027 22701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47418 (request call id 3674) took 1845 ms. Trace:
I20260709 13:12:46.495289 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.479681 29783 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:46.499091 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.500571 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.501319 29982 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:46.484265 29981 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.002s
I20260709 13:12:46.502524 29860 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.503122 29981 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.505017 29860 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:46.511205 30014 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:12:46.515376 29926 raft_consensus.cc:2804] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:46.515650 29931 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.515974 29679 leader_election.cc:290] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.516177 29926 raft_consensus.cc:697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:46.517585 29935 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.517320 29926 consensus_queue.cc:237] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.519388 29930 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.520524 29930 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.521827 29962 raft_consensus.cc:3055] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:12:46.523993 22781 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50372 (request call id 4596) took 1104 ms. Trace:
I20260709 13:12:46.524497 22781 rpcz_store.cc:276] 0709 13:12:45.425372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.444181 (+ 18809us) service_pool.cc:224] Handling call
0709 13:12:45.444827 (+   646us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:12:45.502830 (+ 58003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.502876 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:12:45.576948 (+ 74072us) log.cc:844] Serialized 26 byte log entry
0709 13:12:45.873239 (+296291us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:12:45.873283 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.873347 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:46.523667 (+650320us) raft_consensus.cc:1685] finished
0709 13:12:46.523703 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.523917 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124946,"raft.queue_time_us":64709,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":51549,"thread_start_us":222120,"threads_started":1,"wal-append.queue_time_us":450845}
I20260709 13:12:46.528354 29908 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.420295 30010 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.504884 22747 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.521330 29930 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.494786 22701 rpcz_store.cc:276] 0709 13:12:44.643960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:44.645083 (+  1123us) service_pool.cc:224] Handling call
0709 13:12:44.645572 (+   489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:12:45.106536 (+460964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:12:45.106570 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:12:45.144462 (+ 37892us) log.cc:844] Serialized 52 byte log entry
0709 13:12:45.429732 (+285270us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:12:45.493799 (+ 64067us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.496731 (+  2932us) log.cc:844] Serialized 14 byte log entry
0709 13:12:45.503634 (+  6903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:12:45.503714 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:12:45.777864 (+274150us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18002b07e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:12:45.795499 (+ 17635us) spinlock_profiling.cc:243] Waited 6.66 ms on lock 0x7b18002b07e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:12:46.489025 (+693526us) raft_consensus.cc:1685] finished
0709 13:12:46.489052 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:12:46.489232 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41107,"prepare.queue_time_us":742714,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":4496,"raft.queue_time_us":75115,"raft.run_cpu_time_us":535,"raft.run_wall_time_us":29416,"spinlock_wait_cycles":40858240,"thread_start_us":674476,"threads_started":3,"wal-append.queue_time_us":433331}
I20260709 13:12:46.493836 30016 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.543883 29991 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.522413 29962 raft_consensus.cc:740] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:46.545140 30016 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.545544 29962 consensus_queue.cc:260] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.547442 29962 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:12:46.548043 29930 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.552436 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:46.559007 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 956068us
I20260709 13:12:46.560730 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed3a2e4658ea473ba5c2b68313aeaf3b" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:46.561762 23421 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:46.535187 29925 raft_consensus.cc:3060] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:46.563975 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 276308us
I20260709 13:12:46.571830 30012 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.572628 30012 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.558974 29924 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.575167 29924 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.575508 30012 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.575852 29924 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.530807 30006 raft_consensus.cc:515] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.578205 29924 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.579383 30006 leader_election.cc:290] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.588202 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.588904 29986 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.589637 22707 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:46.592275 29975 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.609093 29975 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.596514 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
W20260709 13:12:46.598112 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 202049us
I20260709 13:12:46.604674 30016 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.612332 30016 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:46.613904 29975 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.614835 30016 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:46.606585 22675 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.595934 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:46.614975 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 177581us
I20260709 13:12:46.623386 30016 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:46.625474 29962 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.626514 23416 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 13:12:46.627553 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.628551 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.594328 21526 tablet_replica.cc:333] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:46.639881 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.640717 21526 raft_consensus.cc:2243] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:12:46.643093 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:46.644856 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.645737 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51e41e0d49c846d0ae7c7d737a15be9e" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.654130 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.659922 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.661160 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17042e7fd6cb40cfb21cfdfb5121a4bc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.663138 22776 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: []
W20260709 13:12:46.673864 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 113822us
I20260709 13:12:46.678452 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.652727 29930 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.656701 29896 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.686038 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.686879 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.687589 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.696940 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:46.697399 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.691051 30006 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
W20260709 13:12:46.713721 22675 leader_election.cc:336] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.653259 29962 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.712630 22750 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.683312 29930 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.725466 22675 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:46.689669 29896 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.732115 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.739770 29783 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.749552 29930 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.752866 29962 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.753579 23411 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:46.730121 29896 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.755301 29783 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.755599 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:46.763664 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:46.765048 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52534.6us
W20260709 13:12:46.769073 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 72600us
W20260709 13:12:46.774061 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 60370.5us
W20260709 13:12:46.775872 22675 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.778798 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.781239 22708 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:46.783720 22601 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:46.784708 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:46.786520 22702 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 17
W20260709 13:12:46.776753 22675 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.794394 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.795747 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.796448 22701 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:46.783005 29930 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:12:46.784053 30016 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:46.792301 30004 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:12:46.806638 29972 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:46.797909 30010 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.812105 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:46.814347 22601 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.817631 22701 raft_consensus.cc:2410] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1f99ad34cc414233929e0a92f8c3d8db for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:12:46.783298 29991 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:46.822871 22601 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.889990 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295bf86c5a444ebc9285e902fd5e9ae0" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:46.858680 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 136203us
W20260709 13:12:46.954901 22707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47414 (request call id 5019) took 1608 ms (client timeout 1999 ms). Trace:
I20260709 13:12:46.870150 22702 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:46.840463 22779 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:12:46.957669 22708 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:12:46.851231 23421 raft_consensus.cc:2468] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:46.846915 22782 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:12:46.955896 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.026857 23384 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 110 similar messages]
W20260709 13:12:47.027854 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 72034.3us
I20260709 13:12:46.884608 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:46.857203 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 188068us
I20260709 13:12:46.919152 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.855389 21526 raft_consensus.cc:2272] T f33fee9d64384b499e55f8bc58339f69 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:46.986332 22705 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:12:46.828716 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:47.031774 22675 leader_election.cc:336] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:46.917048 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb33c4b77a2451499c9c5e49b83aa12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.034473 23420 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:12:47.028731 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 205994us
I20260709 13:12:46.847998 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:46.924352 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:47.035771 22776 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 16
W20260709 13:12:47.038388 22749 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.039136 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:47.039485 22675 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.039997 22675 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.036278 22747 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:46.817732 29972 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:47.042824 22675 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.035687 22775 raft_consensus.cc:2468] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 23.
I20260709 13:12:46.818883 30004 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:47.022666 22707 rpcz_store.cc:269] 0709 13:12:45.345754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:45.345881 (+   127us) service_pool.cc:224] Handling call
0709 13:12:46.953628 (+1607747us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000a82d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:12:46.953847 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2399001472}
I20260709 13:12:47.001998 29954 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:46.841179 29971 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.937171 29925 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:46.852974 30010 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:46.966578 29981 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.985183 29968 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:46.982559 29969 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:47.053675 22598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.036504 29719 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:47.069801 22675 leader_election.cc:336] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.022652 29991 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:46.823192 30000 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.244s
I20260709 13:12:47.051623 30012 raft_consensus.cc:2804] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:46.857209 29941 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:47.044509 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:46.852296 29829 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:46.943306 29759 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:47.082149 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 78401.2us
I20260709 13:12:47.078750 22675 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:47.088111 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.093255 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.045029 29943 raft_consensus.cc:2749] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:12:47.096313 22675 leader_election.cc:336] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.100871 22749 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.102075 22749 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:47.103214 22675 leader_election.cc:336] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.073734 30010 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:47.108856 22675 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.107705 22749 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.113090 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 74395.4us
W20260709 13:12:47.113755 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.113855 22749 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.095808 29829 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.097671 30012 raft_consensus.cc:697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
W20260709 13:12:47.117338 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.117993 22749 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.102721 29986 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:12:47.123322 22749 leader_election.cc:336] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.125097 22749 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.108528 29898 raft_consensus.cc:2749] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.095150 29941 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.130250 23416 raft_consensus.cc:3055] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:12:47.130816 23416 raft_consensus.cc:740] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:47.131700 23416 consensus_queue.cc:260] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.132902 23416 raft_consensus.cc:3060] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:47.129558 22749 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.134116 22749 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.134759 22600 leader_election.cc:304] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.126019 30011 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:47.121421 29930 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:47.134914 22601 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.099457 29759 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.133436 30032 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:12:47.132824 30010 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:47.151361 22776 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:47.116677 29972 consensus_queue.cc:1059] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:12:47.130225 29925 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:12:47.158182 22675 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.154901 30024 raft_consensus.cc:2749] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:47.160512 22675 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.160588 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.161512 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.158969 22600 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.164103 22675 leader_election.cc:304] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.138574 29986 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.165666 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.169709 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.171489 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.170224 21526 tablet_replica.cc:333] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.173964 22675 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.172672 29981 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:47.182523 29978 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.155339 29896 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:47.180449 29715 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.187302 29759 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:47.212843 21526 raft_consensus.cc:2243] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:47.170734 29943 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:12:47.166263 29968 raft_consensus.cc:2749] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:47.166431 30023 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:12:47.227031 21526 pending_rounds.cc:70] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Trying to abort 1 pending ops.
I20260709 13:12:47.205372 29971 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.227460 21526 pending_rounds.cc:77] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting op as it isn't in flight: id { term: 18 index: 11 } timestamp: 7305636882504437760 op_type: NO_OP noop_request { }
I20260709 13:12:47.227943 21526 raft_consensus.cc:2889] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:47.228963 21526 raft_consensus.cc:2272] T 14efc068101d4b50a525206c447e1ae8 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:47.230736 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.232386 29991 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.233793 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:47.197074 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 57067.2us
I20260709 13:12:47.163535 30010 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:12:47.162349 30006 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:47.172992 29766 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:12:47.185886 29839 raft_consensus.cc:2804] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:47.209481 29715 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.233654 21526 tablet_replica.cc:333] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.199800 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:47.254451 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54744.6us
I20260709 13:12:47.261133 21526 raft_consensus.cc:2243] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:47.261494 22776 raft_consensus.cc:1240] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 14. Current term is 16. Ops: []
I20260709 13:12:47.262833 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:47.264235 22782 raft_consensus.cc:1240] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 10. Current term is 11. Ops: []
I20260709 13:12:47.265798 29715 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.266819 30010 consensus_queue.cc:1059] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:12:47.253178 29783 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 13:12:47.271303 21526 raft_consensus.cc:2272] T fbb33c4b77a2451499c9c5e49b83aa12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.260299 30006 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:47.276691 30006 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.260239 29981 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.280207 22675 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.282544 29829 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:47.253157 29839 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:47.287492 29839 raft_consensus.cc:3060] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:47.183691 30012 consensus_queue.cc:237] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.297564 22776 raft_consensus.cc:2468] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:12:47.299604 22675 leader_election.cc:304] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.300935 22781 raft_consensus.cc:1240] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 17. Current term is 18. Ops: []
I20260709 13:12:47.309501 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.315616 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.316704 22781 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
W20260709 13:12:47.338289 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 166917us
I20260709 13:12:47.311847 29981 raft_consensus.cc:2749] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.336498 30041 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.345808 30041 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.347241 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:47.348170 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.339254 22781 raft_consensus.cc:1275] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:12:47.342295 21526 tablet_replica.cc:333] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.343305 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.341919 22600 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:47.344213 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 186956us
I20260709 13:12:47.353642 22704 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:12:47.356398 22705 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:12:47.364459 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:47.366878 22701 raft_consensus.cc:1240] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 17. Ops: []
I20260709 13:12:47.370436 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.370960 22780 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:12:47.371824 22778 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:47.372287 22781 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.373780 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.374490 22777 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:47.370499 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.371934 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.385887 21526 raft_consensus.cc:2243] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:47.345825 29715 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.373608 30000 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:47.395324 22777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:47.397954 22675 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.415338 22675 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:47.422089 22749 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.423514 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.425014 21526 raft_consensus.cc:2272] T 51e41e0d49c846d0ae7c7d737a15be9e P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.441988 23416 raft_consensus.cc:1275] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 13:12:47.442931 23416 pending_rounds.cc:85] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Aborting all ops after (but not including) 12
I20260709 13:12:47.443372 23416 pending_rounds.cc:107] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 13:12:47.443783 23416 raft_consensus.cc:2889] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:47.416338 29941 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.451428 29759 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.452160 29759 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.421294 29981 raft_consensus.cc:2749] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.454557 30041 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:47.410140 30033 consensus_queue.cc:1059] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:12:47.456774 30033 raft_consensus.cc:3055] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:47.457159 30033 raft_consensus.cc:740] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:47.426333 29954 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 13:12:47.459669 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 45605.4us
I20260709 13:12:47.420692 30000 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:47.463331 30000 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.465047 21526 tablet_replica.cc:333] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.466547 30042 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:12:47.470173 29925 consensus_queue.cc:1048] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:47.458079 30033 consensus_queue.cc:260] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.472071 30033 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:12:47.476064 29759 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.476260 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:47.430960 30039 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:47.477559 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55521.5us
I20260709 13:12:47.481600 21526 raft_consensus.cc:2243] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
W20260709 13:12:47.482599 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.483147 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.484426 30039 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:47.487689 29960 raft_consensus.cc:3055] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:12:47.488210 29960 raft_consensus.cc:740] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:12:47.488605 22675 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:47.489238 22675 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.489148 29960 consensus_queue.cc:260] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.491437 29839 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.479471 29715 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.493010 29715 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.494635 29839 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.495245 29715 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:47.495673 22749 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.496250 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:47.496178 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:47.499104 22749 leader_election.cc:336] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.499686 22749 leader_election.cc:304] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.500609 30037 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.507680 30037 raft_consensus.cc:2749] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:47.508931 22749 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.509694 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:47.511858 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 344638us
I20260709 13:12:47.516022 29850 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:47.511317 22749 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.519629 22749 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.521677 30024 raft_consensus.cc:3055] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:47.522182 30024 raft_consensus.cc:740] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:12:47.514482 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.529850 21526 raft_consensus.cc:2272] T bfb716d0bf184679a89d2dde7563865d P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.523147 30024 consensus_queue.cc:260] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.499730 29960 raft_consensus.cc:3060] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:47.531128 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:47.534562 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 73142.4us
W20260709 13:12:47.536288 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 113361us
I20260709 13:12:47.546445 22777 raft_consensus.cc:1240] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Rejecting Update request from peer 91ebb78bf12f41baae84669c1effc423 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:12:47.546831 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.548631 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.532226 30024 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:47.529927 30052 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:47.556625 22601 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.562086 21526 tablet_replica.cc:333] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.551092 29981 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:47.562793 29977 consensus_queue.cc:1059] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:47.563242 30046 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:12:47.550822 29943 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.552328 30032 consensus_queue.cc:1059] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:12:47.565328 30032 raft_consensus.cc:3055] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:12:47.565688 30032 raft_consensus.cc:740] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:47.566665 30032 consensus_queue.cc:260] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.567205 22600 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.569275 30053 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.569960 30053 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:47.572216 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.571414 29972 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Committing config change with OpId 12.9: config changed from index 8 to 9, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:47.573830 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:47.550379 29920 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:12:47.575562 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:47.578760 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.585158 22703 raft_consensus.cc:1240] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 15. Current term is 16. Ops: []
I20260709 13:12:47.586534 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:47.587615 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:47.588658 21526 raft_consensus.cc:2243] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:47.588989 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:47.594542 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:47.552485 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:47.605540 22749 leader_election.cc:336] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:47.611563 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.611326 22749 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:47.614244 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.614982 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:47.553838 29968 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:12:47.620246 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 64319.5us
I20260709 13:12:47.605662 22701 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 8 to 9, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:47.627070 22526 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 287) took 8803 ms. Trace:
I20260709 13:12:47.632637 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.586915 29768 raft_consensus.cc:3055] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:12:47.639137 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 86723.7us
I20260709 13:12:47.652941 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:47.641788 22749 leader_election.cc:336] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.654126 22749 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.612963 29943 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:47.657644 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.657534 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.654451 22528 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 19, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.677211 22528 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.660485 21526 raft_consensus.cc:2272] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.621624 30011 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:47.681492 22528 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 19 to 21, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 21 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:47.617753 29938 raft_consensus.cc:3055] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:47.634193 29829 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:47.683669 22528 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 19, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:47.631697 22526 rpcz_store.cc:276] 0709 13:12:38.817070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:38.819604 (+  2534us) service_pool.cc:224] Handling call
0709 13:12:47.616437 (+8796833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:45.147290 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:45.253525 (+106235us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:45.253572 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:12:45.253586 (+    14us) tablet.cc:662] Decoding operations
0709 13:12:45.254370 (+   784us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:45.254430 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:45.254464 (+    34us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:12:45.308289 (+ 53825us) tablet.cc:701] Row locks acquired
0709 13:12:45.308307 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:12:45.351708 (+ 43401us) write_op.cc:270] Start()
0709 13:12:45.384743 (+ 33035us) write_op.cc:276] Timestamp: P: 1783602765322544 usec, L: 0
0709 13:12:45.384777 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:12:45.495237 (+110460us) log.cc:844] Serialized 1689 byte log entry
0709 13:12:46.574476 (+1079239us) spinlock_profiling.cc:243] Waited 82.3 ms on lock 0x7b18002f7940. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:12:46.797552 (+223076us) op_driver.cc:464] REPLICATION: finished
0709 13:12:46.948993 (+151441us) spinlock_profiling.cc:243] Waited 2.75 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f180a913ef0
0709 13:12:47.184921 (+235928us) write_op.cc:301] APPLY: starting
0709 13:12:47.193877 (+  8956us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:47.278340 (+ 84463us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:47.278381 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:47.289585 (+ 11204us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:47.289992 (+   407us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:12:47.290045 (+    53us) write_op.cc:312] APPLY: finished
0709 13:12:47.290313 (+   268us) log.cc:844] Serialized 49 byte log entry
0709 13:12:47.313468 (+ 23155us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:47.313591 (+   123us) write_op.cc:454] Released schema lock
0709 13:12:47.313648 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:47.314156 (+   508us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:47.314194 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":473637,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":371185,"apply.run_cpu_time_us":9416,"apply.run_wall_time_us":140933,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20879,"num_ops":4,"prepare.queue_time_us":222171,"prepare.run_cpu_time_us":35458,"prepare.run_wall_time_us":822890,"raft.queue_time_us":155326,"raft.run_cpu_time_us":62176,"raft.run_wall_time_us":383251,"replication_time_us":1395257,"spinlock_wait_cycles":195587840,"thread_start_us":856801,"threads_started":4,"wal-append.queue_time_us":469033}]]}
I20260709 13:12:47.677363 22779 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 8 to 9, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:47.655869 29900 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:47.662341 29941 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:47.650142 29768 raft_consensus.cc:740] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
W20260709 13:12:47.707571 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 50134.6us
I20260709 13:12:47.748569 22710 raft_consensus.cc:1240] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 18. Current term is 19. Ops: []
I20260709 13:12:47.745445 30042 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:12:47.754338 21526 tablet_replica.cc:333] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.689430 29938 raft_consensus.cc:740] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:47.704551 29829 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:47.772413 29829 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:47.663120 29969 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:47.681121 30000 raft_consensus.cc:493] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.685441 30010 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.766773 21526 raft_consensus.cc:2243] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:47.775331 29938 consensus_queue.cc:260] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.776615 21526 pending_rounds.cc:70] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Trying to abort 1 pending ops.
I20260709 13:12:47.776953 21526 pending_rounds.cc:77] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db: Aborting op as it isn't in flight: id { term: 17 index: 14 } timestamp: 7305636935022092288 op_type: NO_OP noop_request { }
I20260709 13:12:47.777000 29938 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:12:47.777436 21526 raft_consensus.cc:2889] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:47.777904 21526 raft_consensus.cc:2272] T 26e6130713d74c0ea9492f5151053c42 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.705912 22528 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:47.780006 22528 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 16 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 18 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:47.781642 22528 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:47.683996 29943 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.775362 30000 raft_consensus.cc:515] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.739429 30012 raft_consensus.cc:2749] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:47.802829 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 174450us
I20260709 13:12:47.806047 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.809242 23384 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:47.811383 22528 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
W20260709 13:12:47.708884 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 67160.7us
W20260709 13:12:47.820915 22749 leader_election.cc:336] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.821445 22749 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.807309 30021 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:12:47.827109 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 99491.5us
I20260709 13:12:47.829813 22600 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:47.838496 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 160603us
I20260709 13:12:47.841786 30066 consensus_queue.cc:1048] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:12:47.832015 22749 leader_election.cc:336] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:47.847647 22749 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:47.848527 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.854127 22600 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:47.826717 29920 raft_consensus.cc:493] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:47.826406 30033 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:47.824802 29829 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.849934 29772 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:12:47.892457 23384 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:47.829005 29898 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.862251 30042 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:47.862056 29900 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:47.891821 30049 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:12:47.931317 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 90842.8us
I20260709 13:12:47.882308 29759 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:47.871677 30033 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:47.887409 29954 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:47.950788 29954 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:47.945257 21526 tablet_replica.cc:333] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:47.950209 30033 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:47.953321 29954 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:47.882727 29920 raft_consensus.cc:515] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:47.965804 29829 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.966625 29920 leader_election.cc:290] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:47.971557 21526 raft_consensus.cc:2243] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:47.972811 21526 raft_consensus.cc:2272] T 22e83136ddd24d71ad143a6f5fc28dd9 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:47.851075 29768 consensus_queue.cc:260] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:47.950253 30054 log.cc:927] Time spent T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 13:12:48.024204 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 130766us
W20260709 13:12:48.025688 23382 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:12:48.025063 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:48.037931 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 173650us
I20260709 13:12:48.007089 30040 consensus_queue.cc:1059] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:12:48.032586 29943 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:48.060061 30055 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.276s	user 0.004s	sys 0.000s
I20260709 13:12:48.068275 22703 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Rejecting Update request from peer 6a31ac6451ff4f73bee634487d84d001 for earlier term 15. Current term is 16. Ops: []
W20260709 13:12:48.078552 30009 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.104s	user 0.001s	sys 0.001s
I20260709 13:12:48.046828 30000 leader_election.cc:290] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:48.083439 30033 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.087107 21526 tablet_replica.cc:333] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:48.108995 21526 raft_consensus.cc:2243] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:48.109227 30050 consensus_queue.cc:1175] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" responder_term: 18 status { last_received { term: 18 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 18 index: 10 } }
I20260709 13:12:48.091604 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:48.090904 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:48.109759 21526 pending_rounds.cc:70] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Trying to abort 1 pending ops.
I20260709 13:12:48.112100 21526 pending_rounds.cc:77] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: Aborting op as it isn't in flight: id { term: 14 index: 11 } timestamp: 7305636884235214848 op_type: NO_OP noop_request { }
I20260709 13:12:48.112545 21526 raft_consensus.cc:2889] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:48.112926 21526 raft_consensus.cc:2272] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:48.097002 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 47720.3us
I20260709 13:12:48.115945 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:48.122196 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.124775 23420 raft_consensus.cc:1240] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Rejecting Update request from peer 1f99ad34cc414233929e0a92f8c3d8db for earlier term 14. Current term is 15. Ops: []
I20260709 13:12:48.111842 30032 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:48.146469 23416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:48.164685 23419 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:12:48.169088 30033 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:48.168920 29896 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:12:48.171895 30076 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.118s	user 0.002s	sys 0.000s
I20260709 13:12:48.174551 29768 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:12:48.201680 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 347669us
W20260709 13:12:48.202819 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45579.3us
I20260709 13:12:48.205672 23421 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:12:48.206841 22749 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:48.225521 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99c51f61e4f74c1593ebf98255a83b4c" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.240348 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7235e142d8ec45adb46a8c389a2962fa" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.246647 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:48.249063 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:48.249989 22701 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:48.251845 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:48.254341 22675 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:48.278306 22706 raft_consensus.cc:1275] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:12:48.278878 22706 pending_rounds.cc:85] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Aborting all ops after (but not including) 9
I20260709 13:12:48.279124 22706 pending_rounds.cc:107] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:12:48.279366 22706 raft_consensus.cc:2889] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:48.295184 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:48.271507 29943 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:12:48.332391 29766 raft_consensus.cc:493] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.333071 29766 raft_consensus.cc:515] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:48.242156 29977 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 13:12:48.336134 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46431.3us
I20260709 13:12:48.339357 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:12:48.346778 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.329627 30040 raft_consensus.cc:493] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.286052 30042 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 13:12:48.394973 30051 log.cc:927] Time spent T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:12:48.413107 30055 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:12:48.403350 30012 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:48.448953 30012 raft_consensus.cc:493] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.449440 30012 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:48.443993 29924 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.337638 29715 consensus_queue.cc:1059] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:12:48.450294 29924 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:48.444859 29766 leader_election.cc:290] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:48.452692 29924 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:48.412123 30040 raft_consensus.cc:515] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.458410 30069 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:12:48.459000 30069 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:48.459981 30040 leader_election.cc:290] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:48.459930 30069 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.461688 30069 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:48.399286 30074 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:48.489125 30062 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:12:48.486521 30074 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.503645 30074 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:48.538062 21526 tablet_replica.cc:333] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:48.541106 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 206861us
W20260709 13:12:48.542321 22598 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 189164us
W20260709 13:12:48.563212 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 133732us
I20260709 13:12:48.563402 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:48.571260 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 213931us
W20260709 13:12:48.578243 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 99841us
I20260709 13:12:48.581019 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:48.593835 29943 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.601572 29943 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:48.608004 29943 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:48.499137 30024 consensus_meta.cc:287] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Time spent flushing consensus metadata: real 0.940s	user 0.012s	sys 0.004s
I20260709 13:12:48.614775 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6544eab6fd54f95baf389a8b15729cd" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.577173 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:48.415946 29900 consensus_queue.cc:1048] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:12:48.637935 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 286943us
I20260709 13:12:48.640166 23384 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:48.673017 30080 log.cc:927] Time spent T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:12:48.675324 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 126961us
W20260709 13:12:48.688483 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 131963us
I20260709 13:12:48.688359 29971 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:48.698268 30056 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:12:48.698827 29972 raft_consensus.cc:3055] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:12:48.699550 29972 raft_consensus.cc:740] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:48.673112 30094 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.703441 30094 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.699005 29971 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.707108 21526 raft_consensus.cc:2243] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:48.708710 21526 raft_consensus.cc:2272] T 2f4adfa257c34bb88d79bda9d39c5146 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:48.738880 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 47857.5us
I20260709 13:12:48.741591 22600 leader_election.cc:304] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:48.742239 30094 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:48.743752 21526 tablet_replica.cc:333] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:48.743927 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ac0029002284e64a5fde523ffaf77fb" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.745448 29971 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:48.745909 21526 raft_consensus.cc:2243] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:48.756254 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:48.768121 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:48.701778 29972 consensus_queue.cc:260] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.792865 29765 raft_consensus.cc:2749] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:48.795089 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:48.796197 23420 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 24.
I20260709 13:12:48.796705 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.797937 22775 raft_consensus.cc:2393] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 11: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:48.798511 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:48.799666 22776 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:12:48.803115 21526 raft_consensus.cc:2272] T 0ebec7fc99cb448ea382f8b3e769e7ca P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:48.804352 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:48.810752 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49920.4us
I20260709 13:12:48.822507 29930 raft_consensus.cc:493] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:48.823472 29930 raft_consensus.cc:515] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.828107 29930 leader_election.cc:290] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:48.828003 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53c258bc80ce4d88bac804cb01a34b32" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:48.830312 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 89075.7us
I20260709 13:12:48.831454 22600 leader_election.cc:304] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:48.834785 23383 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:12:48.835049 30102 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:12:48.845157 30102 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.845777 30102 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:12:48.849669 30064 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.850875 30064 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:48.848898 22600 raft_consensus.cc:2686] T 7235e142d8ec45adb46a8c389a2962fa P 1f99ad34cc414233929e0a92f8c3d8db: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:12:48.871591 21526 tablet_replica.cc:333] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:48.873639 21526 raft_consensus.cc:2243] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:12:48.827942 30012 raft_consensus.cc:515] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:48.881040 30064 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:48.882841 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:48.885054 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:48.886154 22710 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:48.887125 29943 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:48.891980 29943 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:48.892544 29943 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:48.881352 30012 leader_election.cc:290] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:48.910463 21526 raft_consensus.cc:2272] T ca809087a6724923ab06ace347425d31 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:48.915763 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 49758.2us
I20260709 13:12:48.916126 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:48.920925 30064 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:48.921550 30064 raft_consensus.cc:493] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:48.922029 30064 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:48.925966 29972 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:48.929543 21526 tablet_replica.cc:333] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:48.944450 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:48.951273 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 120057us
W20260709 13:12:48.939973 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 78935.3us
I20260709 13:12:48.940016 21526 raft_consensus.cc:2243] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:12:48.934057 22530 catalog_manager.cc:5697] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:48.959687 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9a28904b6a34dbf81e5d1d4bb08a08c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:48.972246 29765 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.973150 29765 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:48.991959 22704 raft_consensus.cc:3060] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:48.994719 29925 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:48.995419 29925 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:48.996603 29765 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:48.985317 30032 consensus_meta.cc:287] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.012s
I20260709 13:12:48.997807 23419 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:49.002065 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.002764 29925 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.003228 23412 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
I20260709 13:12:49.004560 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.005980 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.006156 22673 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:49.014310 22600 leader_election.cc:304] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:49.016664 29759 raft_consensus.cc:2749] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.020783 22600 leader_election.cc:304] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:49.021791 29759 raft_consensus.cc:2749] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:12:49.025596 22784 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:12:49.035982 29971 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:49.036574 29971 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:49.036973 29971 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:49.041649 29991 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.042559 29991 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.039637 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 308) took 6535 ms. Trace:
I20260709 13:12:49.047430 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.047850 29943 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.050021 29765 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.050793 29765 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.053392 29765 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:49.044202 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.055011 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:49.010927 29925 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:12:49.061623 29925 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.062206 29925 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.054358 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.046855 22528 rpcz_store.cc:276] 0709 13:12:42.504237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:42.504431 (+   194us) service_pool.cc:224] Handling call
0709 13:12:49.039375 (+6534944us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:47.846317 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:47.865349 (+ 19032us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:47.865392 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:12:47.865413 (+    21us) tablet.cc:662] Decoding operations
0709 13:12:47.866384 (+   971us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:47.866474 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:47.866507 (+    33us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:12:47.938781 (+ 72274us) tablet.cc:701] Row locks acquired
0709 13:12:47.941777 (+  2996us) write_op.cc:260] PREPARE: finished
0709 13:12:47.949380 (+  7603us) write_op.cc:270] Start()
0709 13:12:47.949550 (+   170us) write_op.cc:276] Timestamp: P: 1783602767949310 usec, L: 0
0709 13:12:47.949575 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:12:47.950400 (+   825us) log.cc:844] Serialized 3229 byte log entry
0709 13:12:48.244745 (+294345us) op_driver.cc:464] REPLICATION: finished
0709 13:12:48.496095 (+251350us) write_op.cc:301] APPLY: starting
0709 13:12:48.496682 (+   587us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:48.507484 (+ 10802us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:48.507566 (+    82us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:48.813106 (+305540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:48.835019 (+ 21913us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 13:12:48.837394 (+  2375us) write_op.cc:312] APPLY: finished
0709 13:12:48.839651 (+  2257us) log.cc:844] Serialized 81 byte log entry
0709 13:12:48.842777 (+  3126us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:48.844525 (+  1748us) write_op.cc:454] Released schema lock
0709 13:12:48.845248 (+   723us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:48.882108 (+ 36860us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:48.882152 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19439,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":232268,"apply.run_cpu_time_us":19231,"apply.run_wall_time_us":386651,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37631,"num_ops":8,"prepare.queue_time_us":13366,"prepare.run_cpu_time_us":5133,"prepare.run_wall_time_us":104481,"replication_time_us":267250,"thread_start_us":123083,"threads_started":2}]]}
I20260709 13:12:49.067128 22702 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:49.067848 30012 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.045245 29991 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:49.071659 29898 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:49.079872 29896 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.092345 29896 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.080772 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.050439 29943 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:49.094252 22776 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:49.096938 29896 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:49.097247 22601 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423; no voters: 
W20260709 13:12:49.008244 22675 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.072492 30102 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.090235 30012 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.115916 22704 raft_consensus.cc:2468] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:49.098757 29972 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:49.114845 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:49.132659 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.125818 29972 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:49.157823 30104 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:49.158633 30097 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 13:12:49.127764 29943 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.202795 29943 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.205184 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.207237 29943 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:49.217321 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 88170.9us
I20260709 13:12:49.231328 30039 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.231703 29719 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:49.232299 30039 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.183284 30012 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.232452 29719 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:49.145807 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:49.238880 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 95653.7us
W20260709 13:12:49.240094 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:49.240900 29719 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.241223 30064 raft_consensus.cc:515] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.244654 30064 leader_election.cc:290] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:49.252027 29972 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:49.253137 29925 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.257529 30102 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.262054 21526 raft_consensus.cc:2272] T 160d972ad1454f9db922aa360a0e7436 P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:49.263125 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 85772.5us
I20260709 13:12:49.272495 22600 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:49.272347 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:49.240981 22749 leader_election.cc:336] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.283236 22749 leader_election.cc:304] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:49.284113 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec9764b0b7dd4b67a5b50a652203c60a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:49.296703 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 55733.6us
W20260709 13:12:49.303704 22749 leader_election.cc:336] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.277652 29719 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:12:49.316025 23384 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.321908 23384 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:49.302121 29991 raft_consensus.cc:2749] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:12:49.331955 23384 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:49.357244 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 374753us
I20260709 13:12:49.384665 30072 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:49.378137 22750 leader_election.cc:304] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:49.413968 29943 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:12:49.430982 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 52972.1us
I20260709 13:12:49.432050 29898 raft_consensus.cc:2804] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:49.432869 29898 raft_consensus.cc:697] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:49.363746 29971 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.460593 22704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.461642 22704 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 11.
W20260709 13:12:49.485374 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 129896us
I20260709 13:12:49.487922 29971 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.489619 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:49.493908 23412 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.415060 29943 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:49.521893 22749 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.465716 29925 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:49.526731 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.541016 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f33fee9d64384b499e55f8bc58339f69" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:49.546186 23419 raft_consensus.cc:3060] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:12:49.434222 29898 consensus_queue.cc:237] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.529057 29991 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:49.540517 29971 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:49.559724 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:49.568254 22749 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.568711 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:49.575275 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:49.574335 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.584899 22775 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:49.591531 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.559345 29991 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.601317 22707 raft_consensus.cc:2393] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 19: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:49.599263 23421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:49.596567 22598 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:49.603230 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 285485us
W20260709 13:12:49.597265 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.600590 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f09c3bbe948b4c7a9f50df8ec4277e9f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:49.605757 22783 raft_consensus.cc:3060] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:49.582186 29971 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:49.614430 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:49.595058 29972 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.642215 23412 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:49.644199 22783 raft_consensus.cc:2468] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:49.618981 29943 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:49.628720 21526 tablet_replica.cc:333] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:49.647845 30107 log.cc:927] Time spent T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.219s	user 0.001s	sys 0.001s
I20260709 13:12:49.662854 22747 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.668164 29850 raft_consensus.cc:2804] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:12:49.668943 29850 raft_consensus.cc:697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:49.671267 30100 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:49.673775 29971 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:49.685272 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.686578 22710 raft_consensus.cc:2393] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:49.670256 29850 consensus_queue.cc:237] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.699435 29991 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.699934 21526 raft_consensus.cc:2243] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:49.700896 21526 raft_consensus.cc:2272] T b907163bc2174a2aa11df858ccae00ac P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:49.701560 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.702029 29972 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:49.702584 23414 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:49.704800 22747 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.707290 23419 raft_consensus.cc:2468] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 25.
I20260709 13:12:49.711169 29991 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:49.711709 29991 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:49.712107 29991 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:49.720472 30039 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.722072 22673 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:49.723867 29943 raft_consensus.cc:2804] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:12:49.724644 29943 raft_consensus.cc:697] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 25 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:49.726007 29943 consensus_queue.cc:237] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.729400 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.730352 23419 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:49.731985 21526 tablet_replica.cc:333] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:49.732041 22747 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:49.734972 30012 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:49.740378 22600 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 119788us
I20260709 13:12:49.797971 21526 raft_consensus.cc:2243] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:12:49.801011 23384 leader_election.cc:336] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.814776 30011 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:49.817559 30006 raft_consensus.cc:493] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:49.821347 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:49.822139 22702 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:49.797844 29971 raft_consensus.cc:493] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:49.818512 30006 raft_consensus.cc:515] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.840346 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:49.846565 22780 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:12:49.842865 29971 raft_consensus.cc:515] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.851497 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.851752 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e6130713d74c0ea9492f5151053c42" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:49.852473 22707 raft_consensus.cc:3060] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:49.852952 29991 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.855482 30110 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.856281 30110 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.850167 29971 leader_election.cc:290] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.864243 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:49.866850 30064 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:49.867586 30064 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.870800 29991 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.876792 30110 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:49.872247 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:49.877024 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.877949 23419 raft_consensus.cc:2393] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 15: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
W20260709 13:12:49.877867 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:49.882896 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:49.889776 30006 leader_election.cc:290] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:49.892383 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:49.896509 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11f8cc3feaf345e397b685036c912e13" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.900246 22707 raft_consensus.cc:2468] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
I20260709 13:12:49.915030 23414 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:49.921200 22747 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:49.924497 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:49.925381 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:49.926211 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:49.927170 29991 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:49.927754 29991 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:49.927906 22600 leader_election.cc:304] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:49.931412 29759 raft_consensus.cc:2749] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:12:49.928755 29991 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.935885 30064 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:49.937966 22702 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
W20260709 13:12:49.938359 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 87514.3us
I20260709 13:12:49.940359 22780 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 17.
I20260709 13:12:49.946946 22600 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.947965 30006 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:49.948567 30006 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:49.949587 30006 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:49.937379 30012 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:49.958142 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:49.963896 23385 leader_election.cc:304] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:49.955294 21526 raft_consensus.cc:2272] T af344e0ef1b1489a9bc301ba3b375d4c P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:49.956873 22750 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:49.965018 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.965791 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:49.966116 22784 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
I20260709 13:12:49.956029 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.972908 30011 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:12:49.979457 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:49.966814 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.980410 22784 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:49.980944 22675 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:49.981950 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:49.987180 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:49.987977 22710 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 14.
I20260709 13:12:49.988637 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fee10603c0c042fd8e28683daeb86a99" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:49.989240 22707 raft_consensus.cc:3060] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:12:49.988233 30110 raft_consensus.cc:2804] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:49.992710 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:49.991637 29930 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:50.001125 30110 raft_consensus.cc:697] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:50.002506 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.001260 22710 raft_consensus.cc:2393] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:49.997018 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e3855678a8148a09ffe99d7624c9b4f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.008001 30110 consensus_queue.cc:237] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.011425 22707 raft_consensus.cc:2468] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 12.
I20260709 13:12:49.988538 30102 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:50.026963 30102 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:50.027392 29971 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.028035 29971 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:49.990491 30064 raft_consensus.cc:2804] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:50.030261 30064 raft_consensus.cc:697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:50.030447 29971 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.027446 30102 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:50.032246 30064 consensus_queue.cc:237] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.031888 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.033455 23414 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
W20260709 13:12:50.033522 22675 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.034807 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:50.035785 29971 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:50.036199 29971 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.036569 29971 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:50.034830 22675 consensus_peers.cc:597] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:50.037771 22675 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.038852 22675 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.040443 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:50.040977 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48293.5us
W20260709 13:12:50.043105 22675 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.044241 22675 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.044420 21526 tablet_replica.cc:333] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:50.045202 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.045873 21526 raft_consensus.cc:2243] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:12:50.047294 21526 raft_consensus.cc:2272] T 9a4549b98fd04604afe4d21f9bb0acfe P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:50.045863 22675 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.051848 22675 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.052338 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:50.053316 29943 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:50.056974 30102 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.063298 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:50.063549 22675 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.064184 22780 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:50.065055 30102 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.067328 29943 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:50.077858 21526 tablet_replica.cc:333] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.079219 21526 raft_consensus.cc:2243] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:12:50.080523 21526 raft_consensus.cc:2272] T 05be2cb1839f447296223033d364a7cc P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.102771 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:50.104971 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.106333 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:50.108201 22675 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.109575 30102 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:50.109990 30102 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:50.110385 30102 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:50.110833 30006 raft_consensus.cc:493] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.115006 30006 raft_consensus.cc:515] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.116585 29991 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:50.117206 29991 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.118547 30006 leader_election.cc:290] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.111289 29943 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:50.118813 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.119314 29943 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.119753 29943 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:50.120002 21526 tablet_replica.cc:333] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.121342 21526 raft_consensus.cc:2243] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.122104 21526 raft_consensus.cc:2272] T 295bf86c5a444ebc9285e902fd5e9ae0 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.123809 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a79a6f618f4944b1a11b9c1df766ae7f" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 16 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.125399 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.125576 22601 leader_election.cc:304] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:50.126121 23414 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 19.
I20260709 13:12:50.127583 30011 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:50.127739 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:50.128254 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.128580 30012 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:50.129175 30012 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:50.129824 30012 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:50.131991 21526 tablet_replica.cc:333] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.133170 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a4b81cf2b9c48b59c92dd3ace99c8db" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:50.133576 21526 raft_consensus.cc:2243] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.133893 22780 raft_consensus.cc:3060] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:50.128670 29991 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.141189 29765 raft_consensus.cc:2749] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:50.148533 21526 raft_consensus.cc:2272] T 53c258bc80ce4d88bac804cb01a34b32 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.146533 30032 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:50.152954 29971 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.157226 22780 raft_consensus.cc:2468] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 18.
W20260709 13:12:50.159323 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:50.157729 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:50.163482 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:50.164172 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:50.166217 30110 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:12:50.178380 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:50.188822 29971 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.189627 29971 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:50.191840 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 83731.9us
I20260709 13:12:50.192718 22675 leader_election.cc:304] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.193176 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:50.193846 23414 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:50.195459 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:50.196787 22749 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.201154 30012 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.197896 29943 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.242743 30012 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.213263 23414 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
W20260709 13:12:50.246037 22749 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.246035 23417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "160d972ad1454f9db922aa360a0e7436" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:50.248405 23417 raft_consensus.cc:3060] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:50.254742 22673 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:50.190162 29971 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:50.259184 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 66538.1us
I20260709 13:12:50.267427 22675 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:50.273469 30102 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.275868 30102 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.277928 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 182) took 6816 ms. Trace:
I20260709 13:12:50.279299 21526 tablet_replica.cc:333] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.280474 21526 raft_consensus.cc:2243] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.281164 21526 raft_consensus.cc:2272] T ec9764b0b7dd4b67a5b50a652203c60a P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.269351 30118 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.283087 30118 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.264671 30110 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:50.278422 22530 rpcz_store.cc:276] 0709 13:12:43.461279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:43.468461 (+  7182us) service_pool.cc:224] Handling call
0709 13:12:50.277852 (+6809391us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:49.343809 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:49.471464 (+127655us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:49.471547 (+    83us) write_op.cc:435] Acquired schema lock
0709 13:12:49.471580 (+    33us) tablet.cc:662] Decoding operations
0709 13:12:49.472203 (+   623us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:49.472337 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:49.472383 (+    46us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:12:49.472794 (+   411us) tablet.cc:701] Row locks acquired
0709 13:12:49.472855 (+    61us) write_op.cc:260] PREPARE: finished
0709 13:12:49.473160 (+   305us) write_op.cc:270] Start()
0709 13:12:49.473368 (+   208us) write_op.cc:276] Timestamp: P: 1783602769473059 usec, L: 0
0709 13:12:49.473402 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:12:49.474536 (+  1134us) log.cc:844] Serialized 533 byte log entry
0709 13:12:49.792312 (+317776us) op_driver.cc:464] REPLICATION: finished
0709 13:12:49.854453 (+ 62141us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f180a913ef0
0709 13:12:49.991407 (+136954us) write_op.cc:301] APPLY: starting
0709 13:12:50.051050 (+ 59643us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:50.199524 (+148474us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:50.199578 (+    54us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:50.200908 (+  1330us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:50.201172 (+   264us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 13:12:50.201239 (+    67us) write_op.cc:312] APPLY: finished
0709 13:12:50.201503 (+   264us) log.cc:844] Serialized 25 byte log entry
0709 13:12:50.202535 (+  1032us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:50.207011 (+  4476us) write_op.cc:454] Released schema lock
0709 13:12:50.207065 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:50.245072 (+ 38007us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:50.245108 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8192,"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":174370,"apply.run_cpu_time_us":5590,"apply.run_wall_time_us":260661,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8315,"num_ops":1,"prepare.queue_time_us":227878,"prepare.run_cpu_time_us":5484,"prepare.run_wall_time_us":211651,"replication_time_us":318660,"spinlock_wait_cycles":5267328,"thread_start_us":150818,"threads_started":2}]]}
W20260709 13:12:50.291754 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:50.293809 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.294699 23414 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:50.296018 30131 raft_consensus.cc:2804] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:50.296648 30131 raft_consensus.cc:697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:50.297683 30131 consensus_queue.cc:237] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.298866 30110 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:50.303208 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.307052 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.314316 22702 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:50.294773 30130 raft_consensus.cc:2749] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:50.315721 22601 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:50.313585 22779 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 15.
I20260709 13:12:50.324429 30110 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.313091 29943 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:50.333849 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:50.323226 22528 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: config changed from index 8 to 9, term changed from 12 to 14, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New cstate: current_term: 14 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:50.286391 30118 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
W20260709 13:12:50.348613 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 81282.2us
I20260709 13:12:50.349725 23412 raft_consensus.cc:1275] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:12:50.354010 29759 raft_consensus.cc:493] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.354411 21526 tablet_replica.cc:333] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.343575 30117 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:50.377686 30117 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.366885 23419 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:50.378198 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:50.379097 29759 raft_consensus.cc:515] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.379606 29943 consensus_queue.cc:1048] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:12:50.381824 29759 leader_election.cc:290] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.378208 30117 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.378226 21526 raft_consensus.cc:2243] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:12:50.386756 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.389063 23417 raft_consensus.cc:2468] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:50.392230 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d580d080ca644b2af67c9e48cbd3b78" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.392654 22747 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:50.397383 30039 raft_consensus.cc:2804] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:50.397992 30039 raft_consensus.cc:697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:50.398918 30039 consensus_queue.cc:237] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.409509 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfb716d0bf184679a89d2dde7563865d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:50.410753 22779 raft_consensus.cc:3060] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:50.406704 22598 leader_election.cc:304] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.382412 22528 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 18 to 19, leader changed from a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 19 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:50.416450 21526 raft_consensus.cc:2272] T b8291ea7e33a40b5afaebc02d6ce7e9d P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.447638 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ebec7fc99cb448ea382f8b3e769e7ca" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:50.448735 22780 raft_consensus.cc:3060] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:50.394135 30012 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:12:50.457224 29971 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.476614 30117 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.487241 29971 raft_consensus.cc:493] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:50.488562 22702 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:50.492043 22780 raft_consensus.cc:2468] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:50.507278 22710 raft_consensus.cc:1275] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:12:50.507742 22704 raft_consensus.cc:1275] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:12:50.494421 22528 catalog_manager.cc:5697] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 19 to 20, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 20 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:50.455020 29719 raft_consensus.cc:2749] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:50.502475 22779 raft_consensus.cc:2468] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:50.533963 22528 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:50.559481 30118 raft_consensus.cc:493] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.568269 22707 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:50.571403 23412 raft_consensus.cc:1240] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Rejecting Update request from peer a0e9f7dc2a2e4bdb9287d910e4b491f7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:12:50.567022 29765 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.573446 29765 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.576257 29765 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.577637 21526 tablet_replica.cc:333] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.578536 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.578991 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.581125 21526 raft_consensus.cc:2243] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:50.581868 22600 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.569602 22528 catalog_manager.cc:5697] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 10 to 12, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.588495 30006 raft_consensus.cc:493] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.589268 30006 raft_consensus.cc:515] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.591904 30006 leader_election.cc:290] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.586194 22528 catalog_manager.cc:5697] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 16 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.594311 22528 catalog_manager.cc:5697] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 10 to 12, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.602216 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.605374 30117 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:50.602797 30118 raft_consensus.cc:515] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.608208 30118 leader_election.cc:290] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.609503 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:50.609645 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:50.497776 29971 raft_consensus.cc:515] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.610677 22707 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.614442 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.631135 22776 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:50.618980 30011 consensus_queue.cc:1059] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:50.622900 22528 catalog_manager.cc:5697] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:50.637257 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:50.637657 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "594d7db14b30467aac30a69dc1570d43" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.497579 29759 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:12:50.638599 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:50.640182 23412 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.638917 22528 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:50.638075 22779 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:50.644542 22600 leader_election.cc:304] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:50.638383 29719 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:50.646945 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6d9cbf659974b358ba6d364ca9ca431" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:50.657639 22779 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:50.664398 30139 consensus_queue.cc:1048] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:50.665030 29925 consensus_queue.cc:1048] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:12:50.665951 21526 raft_consensus.cc:2272] T a79a6f618f4944b1a11b9c1df766ae7f P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.667635 22598 leader_election.cc:304] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:50.652194 30011 raft_consensus.cc:3055] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:12:50.668789 30011 raft_consensus.cc:740] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:50.669724 30011 consensus_queue.cc:260] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:50.670912 30100 raft_consensus.cc:2749] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:50.672205 22707 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:50.680821 23412 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f99ad34cc414233929e0a92f8c3d8db in term 16.
I20260709 13:12:50.677098 22600 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:50.685516 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 56926.8us
I20260709 13:12:50.688122 30118 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:50.697350 22776 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:50.700837 30118 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Becoming Leader. State: Replica: 1f99ad34cc414233929e0a92f8c3d8db, State: Running, Role: LEADER
I20260709 13:12:50.702209 30118 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:50.703011 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:50.685384 30006 raft_consensus.cc:2749] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:50.717561 21526 tablet_replica.cc:333] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.740970 30011 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:50.753505 21526 raft_consensus.cc:2243] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:12:50.755162 21526 raft_consensus.cc:2272] T 308af34f5c0144e185e0af82342c7f92 P 1f99ad34cc414233929e0a92f8c3d8db [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.760103 29719 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:50.788440 29971 leader_election.cc:290] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:50.800920 23420 raft_consensus.cc:1275] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
W20260709 13:12:50.805478 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 54376.9us
W20260709 13:12:50.817744 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 131154us
I20260709 13:12:50.820340 21526 tablet_replica.cc:333] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:50.842528 22675 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:50.811810 29896 consensus_queue.cc:1048] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 13:12:50.847677 30110 raft_consensus.cc:2804] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:50.848392 30110 raft_consensus.cc:697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:50.849368 30110 consensus_queue.cc:237] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.845351 22675 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:50.855512 21526 raft_consensus.cc:2243] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:12:50.855007 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:50.856582 30102 raft_consensus.cc:2804] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:50.859414 30102 raft_consensus.cc:697] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:50.860507 30102 consensus_queue.cc:237] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.869916 30134 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:50.874104 30134 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:50.874564 30134 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:50.878038 22702 raft_consensus.cc:1275] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:12:50.880298 23419 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:50.886013 29943 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:12:50.911194 21526 raft_consensus.cc:2272] T bf401fd06552416592129b334cdf8167 P 1f99ad34cc414233929e0a92f8c3d8db [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:50.924237 22776 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:12:50.924971 22776 pending_rounds.cc:85] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting all ops after (but not including) 11
I20260709 13:12:50.927800 22779 raft_consensus.cc:1275] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:50.938220 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14efc068101d4b50a525206c447e1ae8" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:50.925277 22776 pending_rounds.cc:107] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: Aborting uncommitted NO_OP operation due to leader change: 16.12
W20260709 13:12:50.939502 22675 consensus_peers.cc:597] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:50.939571 22776 raft_consensus.cc:2889] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:12:50.946523 22675 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.951622 22675 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:50.949043 29943 consensus_queue.cc:1048] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
W20260709 13:12:50.956671 22675 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:50.958074 22675 consensus_peers.cc:597] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:50.968078 29971 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:12:50.929754 30032 consensus_queue.cc:1048] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:12:50.975606 30134 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:50.977656 22776 raft_consensus.cc:1275] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:12:50.965306 30146 log.cc:927] Time spent T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.106s	user 0.003s	sys 0.001s
I20260709 13:12:50.980293 21526 tablet_replica.cc:333] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:51.013345 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.047046 21526 raft_consensus.cc:2243] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:12:51.009801 30110 consensus_queue.cc:1048] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:12:51.103139 21526 raft_consensus.cc:2272] T 34f27615de694e7faaba89794bf9b5c6 P 1f99ad34cc414233929e0a92f8c3d8db [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.104534 29986 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.105089 29930 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.105240 29986 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.105839 29930 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.107909 29986 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:51.108387 29930 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:51.125176 30009 log.cc:927] Time spent T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:12:51.138424 30134 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.159024 22707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.159452 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.159825 22707 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
W20260709 13:12:51.184437 22749 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:51.191193 21526 tablet_replica.cc:333] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.211658 23420 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:51.219484 30118 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:12:51.227085 23417 raft_consensus.cc:2955] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 8 to 9, 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:51.230100 21526 raft_consensus.cc:2243] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:51.231022 21526 pending_rounds.cc:70] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Trying to abort 1 pending ops.
I20260709 13:12:51.231336 21526 pending_rounds.cc:77] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db: Aborting op as it isn't in flight: id { term: 16 index: 13 } timestamp: 7305636891032293376 op_type: NO_OP noop_request { }
I20260709 13:12:51.231760 21526 raft_consensus.cc:2889] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:51.232162 21526 raft_consensus.cc:2272] T 99c51f61e4f74c1593ebf98255a83b4c P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:51.270450 30167 log.cc:927] Time spent T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: Append to log took a long time: real 0.067s	user 0.003s	sys 0.001s
I20260709 13:12:51.272686 21526 tablet_replica.cc:333] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.273856 21526 raft_consensus.cc:2243] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:51.285207 22675 leader_election.cc:336] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:51.274492 21526 raft_consensus.cc:2272] T d6544eab6fd54f95baf389a8b15729cd P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.285797 22675 leader_election.cc:304] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:51.292008 30110 raft_consensus.cc:2749] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:12:51.282475 22749 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:51.296888 22675 consensus_peers.cc:597] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:51.299719 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca809087a6724923ab06ace347425d31" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:51.300331 22780 raft_consensus.cc:3060] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:51.303207 22784 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:51.311002 30110 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.318914 21526 tablet_replica.cc:333] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:51.328460 22675 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:51.298074 29925 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.332059 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:51.351812 22780 raft_consensus.cc:2468] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:51.311694 30110 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.355631 22675 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:51.310487 30117 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
W20260709 13:12:51.362367 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 70948.1us
I20260709 13:12:51.363569 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:51.375438 21526 raft_consensus.cc:2243] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:51.353528 29925 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.372301 29896 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:51.373734 30102 raft_consensus.cc:2804] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:12:51.377975 30141 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:12:51.475420 22601 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 119089us
I20260709 13:12:51.490811 30175 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.491660 30175 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:51.492941 30169 log.cc:927] Time spent T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:12:51.424413 29896 raft_consensus.cc:493] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:51.475886 30163 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:12:51.535741 21526 raft_consensus.cc:2272] T 594d7db14b30467aac30a69dc1570d43 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.473541 30102 raft_consensus.cc:697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
W20260709 13:12:51.552369 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 166767us
W20260709 13:12:51.554618 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 191188us
W20260709 13:12:51.561643 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 67679.4us
I20260709 13:12:51.535657 29896 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:51.574450 30175 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:51.574805 30117 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.587018 30102 consensus_queue.cc:237] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.611800 30161 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.612699 30161 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.601075 30117 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.619328 29925 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:51.620324 30170 log.cc:927] Time spent T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 13:12:51.621784 30165 log.cc:927] Time spent T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:12:51.623651 30110 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.627949 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.645085 21526 tablet_replica.cc:333] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.646368 21526 raft_consensus.cc:2243] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:51.642022 30117 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.650918 21526 raft_consensus.cc:2272] T 91be8db7516b4bca939f439f045b40eb P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.657162 21526 tablet_replica.cc:333] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.658150 21526 raft_consensus.cc:2243] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:51.658919 21526 raft_consensus.cc:2272] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.661134 29896 raft_consensus.cc:515] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.663904 29896 leader_election.cc:290] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:51.678740 22703 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Refusing update from remote peer 1f99ad34cc414233929e0a92f8c3d8db: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:12:51.684041 29765 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:12:51.674932 22749 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:51.692111 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:51.710717 21526 tablet_replica.cc:333] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:51.723306 22749 leader_election.cc:336] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:51.708559 30161 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.730476 30187 raft_consensus.cc:493] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.739148 30187 raft_consensus.cc:515] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:51.741950 30187 leader_election.cc:290] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:51.756808 22701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.761021 29986 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.761732 29986 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.764101 29986 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:51.773188 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:51.792770 21526 raft_consensus.cc:2243] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:12:51.805853 22600 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:51.807832 30117 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:51.810909 30131 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.811772 30131 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.814409 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc8b169704cc4157ad021d774f83419e" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:51.736227 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:51.819280 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 86251.4us
I20260709 13:12:51.827450 21526 raft_consensus.cc:2272] T 4a5cf575974c43e79448d00fab49d7ab P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.830960 30006 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.833067 22598 leader_election.cc:304] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:51.835292 30187 raft_consensus.cc:2749] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:51.835330 30006 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:51.842855 30131 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.873875 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:51.865437 21526 tablet_replica.cc:333] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.865396 22530 catalog_manager.cc:5697] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:51.875530 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.875903 21526 raft_consensus.cc:2243] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:51.878038 21526 raft_consensus.cc:2272] T cc8b169704cc4157ad021d774f83419e P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:51.885030 22600 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:51.877027 22530 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 16, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:51.888980 30187 raft_consensus.cc:2749] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:12:51.858585 30006 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:51.890913 22530 catalog_manager.cc:5697] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 22 to 25, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 25 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:51.907471 22530 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 23 to 25, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129). New cstate: current_term: 25 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:51.913527 21526 tablet_replica.cc:333] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:51.929198 21526 raft_consensus.cc:2243] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:51.932262 22528 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38114 (request call id 288) took 2728 ms. Trace:
I20260709 13:12:51.935616 22784 raft_consensus.cc:1275] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:12:51.935628 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.936725 22710 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 19.
I20260709 13:12:51.938395 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:51.939149 22779 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 19.
I20260709 13:12:51.940598 22784 raft_consensus.cc:1275] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:12:51.939100 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:51.932844 22528 rpcz_store.cc:276] 0709 13:12:49.194993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.195754 (+   761us) service_pool.cc:224] Handling call
0709 13:12:51.923170 (+2727416us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:50.901303 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:50.981570 (+ 80267us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:50.981630 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:12:50.981652 (+    22us) tablet.cc:662] Decoding operations
0709 13:12:50.982892 (+  1240us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:50.982964 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:50.983001 (+    37us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:12:50.983790 (+   789us) tablet.cc:701] Row locks acquired
0709 13:12:50.983812 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:12:50.983993 (+   181us) write_op.cc:270] Start()
0709 13:12:50.984149 (+   156us) write_op.cc:276] Timestamp: P: 1783602770983932 usec, L: 0
0709 13:12:50.984178 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:12:51.043474 (+ 59296us) log.cc:844] Serialized 3657 byte log entry
0709 13:12:51.488309 (+444835us) spinlock_profiling.cc:243] Waited 983 us on lock 0x7b5400357ba8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180d2999e1 00007f180d29864c 00007f180a9143bb 00007f180a91671a 00007f180a9166a8 00007f180a916628 00007f180a9165e3 00007f180a9157fc 00007f180cd48626 00007f180cd42dd8 00007f1806ee9310
0709 13:12:51.488470 (+   161us) op_driver.cc:464] REPLICATION: finished
0709 13:12:51.616362 (+127892us) write_op.cc:301] APPLY: starting
0709 13:12:51.616465 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:51.670038 (+ 53573us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:51.670065 (+    27us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:51.789846 (+119781us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:51.790145 (+   299us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 13:12:51.790193 (+    48us) write_op.cc:312] APPLY: finished
0709 13:12:51.790524 (+   331us) log.cc:844] Serialized 89 byte log entry
0709 13:12:51.827199 (+ 36675us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.830857 (+  3658us) write_op.cc:454] Released schema lock
0709 13:12:51.830928 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:51.843117 (+ 12189us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:51.844345 (+  1228us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":127518,"apply.run_cpu_time_us":17108,"apply.run_wall_time_us":220879,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41819,"num_ops":9,"prepare.queue_time_us":137548,"prepare.run_cpu_time_us":5755,"prepare.run_wall_time_us":163448,"replication_time_us":309969,"spinlock_wait_cycles":2260224,"thread_start_us":154255,"threads_started":2}]]}
I20260709 13:12:51.927412 30102 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:51.949137 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b907163bc2174a2aa11df858ccae00ac" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:51.949893 22708 raft_consensus.cc:3060] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:51.950997 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:51.951772 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:51.951936 22779 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
W20260709 13:12:51.953249 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:51.954069 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:51.952395 22784 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:12:51.961603 22710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:51.960358 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:51.968906 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:51.966198 22710 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:51.980880 21526 raft_consensus.cc:2272] T 11f8cc3feaf345e397b685036c912e13 P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:51.969764 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:51.949244 30102 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.012300 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.016480 29896 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:52.017001 29896 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.017431 29896 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:52.019212 21526 tablet_replica.cc:333] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.016536 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:12:52.035379 22749 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 229 similar messages]
I20260709 13:12:52.037473 30175 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:52.038010 30175 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.038307 22702 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:52.038416 30175 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:52.039911 30102 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.029307 30161 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.047573 30161 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.050137 30161 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:52.054659 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:52.032553 21526 raft_consensus.cc:2243] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 LEADER]: Raft consensus shutting down.
W20260709 13:12:52.077642 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:52.077733 22708 raft_consensus.cc:2468] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:52.097518 21526 raft_consensus.cc:2272] T a840ecd041fa46a198e4fb917ef79b2a P 1f99ad34cc414233929e0a92f8c3d8db [term 10 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.084439 30032 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:52.099888 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 76599.6us
I20260709 13:12:52.101747 30032 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.135548 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:52.137280 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.136373 22779 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 21.
I20260709 13:12:52.140704 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.143750 22750 leader_election.cc:304] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.150125 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.120988 30175 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.146445 30191 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:12:52.153980 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.147466 30178 raft_consensus.cc:2804] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:52.147090 30140 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:52.156684 30186 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:52.163868 30134 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:52.165143 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.142916 30161 consensus_queue.cc:1048] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:12:52.145280 30131 consensus_queue.cc:1048] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 13:12:52.174484 30134 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.179746 30134 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.181063 30102 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:52.181571 30102 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.181980 30102 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:52.159302 30191 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.166363 30140 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.166160 30186 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.177546 30178 raft_consensus.cc:697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:52.209175 30178 consensus_queue.cc:237] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.207604 30140 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:52.208072 30186 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:52.216722 30191 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:12:52.262701 22530 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:43414 (request call id 183) took 1710 ms. Trace:
I20260709 13:12:52.272781 21526 tablet_replica.cc:333] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.273953 21526 raft_consensus.cc:2243] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:12:52.275344 30032 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.263376 22530 rpcz_store.cc:276] 0709 13:12:50.539014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:50.539538 (+   524us) service_pool.cc:224] Handling call
0709 13:12:52.249716 (+1710178us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:12:51.953503 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:12:51.954120 (+   617us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:12:51.957706 (+  3586us) write_op.cc:435] Acquired schema lock
0709 13:12:51.957736 (+    30us) tablet.cc:662] Decoding operations
0709 13:12:52.014479 (+ 56743us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:12:52.014573 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 13:12:52.014601 (+    28us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:12:52.015029 (+   428us) tablet.cc:701] Row locks acquired
0709 13:12:52.015055 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:12:52.015240 (+   185us) write_op.cc:270] Start()
0709 13:12:52.015376 (+   136us) write_op.cc:276] Timestamp: P: 1783602772015188 usec, L: 0
0709 13:12:52.015405 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:12:52.051278 (+ 35873us) log.cc:844] Serialized 1674 byte log entry
0709 13:12:52.057248 (+  5970us) op_driver.cc:464] REPLICATION: finished
0709 13:12:52.066812 (+  9564us) write_op.cc:301] APPLY: starting
0709 13:12:52.068849 (+  2037us) spinlock_profiling.cc:243] Waited 9.49 ms on lock 0x7b5400357d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed8e28 00007f180a913581 00007f180a912d58 00007f180a9179f1 00007f180a9179a9 00007f180a917939 00007f180a917901 00007f180a916b2d 000000000038f354 000000000038f199
0709 13:12:52.069194 (+   345us) tablet.cc:1367] starting BulkCheckPresence
0709 13:12:52.071948 (+  2754us) tablet.cc:1370] finished BulkCheckPresence
0709 13:12:52.071976 (+    28us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:12:52.108877 (+ 36901us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:12:52.109073 (+   196us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=0,mrs_lookups=0
0709 13:12:52.109126 (+    53us) write_op.cc:312] APPLY: finished
0709 13:12:52.109414 (+   288us) log.cc:844] Serialized 49 byte log entry
0709 13:12:52.114774 (+  5360us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:52.114908 (+   134us) write_op.cc:454] Released schema lock
0709 13:12:52.116027 (+  1119us) write_op.cc:341] FINISH: Updating metrics
0709 13:12:52.145637 (+ 29610us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:12:52.146990 (+  1353us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":22868,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":7141,"apply.run_cpu_time_us":8783,"apply.run_wall_time_us":67312,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16752,"num_ops":4,"prepare.queue_time_us":6416,"prepare.run_cpu_time_us":6846,"prepare.run_wall_time_us":115547,"replication_time_us":41647,"spinlock_wait_cycles":21819648,"thread_start_us":6007,"threads_started":1}]]}
I20260709 13:12:52.281256 22528 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
I20260709 13:12:52.287870 22526 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:38108 (request call id 309) took 2390 ms. Trace:
I20260709 13:12:52.288308 22526 rpcz_store.cc:276] 0709 13:12:49.887061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:12:49.887300 (+   239us) service_pool.cc:224] Handling call
0709 13:12:52.276937 (+2389637us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1027,"reported_tablets":19}
I20260709 13:12:52.291501 29943 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.285458 22528 catalog_manager.cc:5697] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 10 to 12, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 12 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:52.292098 29943 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.284278 30176 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.300441 22528 catalog_manager.cc:5697] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 19 to 20, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 20 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:52.303184 23384 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.310144 21526 raft_consensus.cc:2272] T c015ab1d14c443258576403d9d39c485 P 1f99ad34cc414233929e0a92f8c3d8db [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:52.286540 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 121492us
I20260709 13:12:52.322980 22528 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 15 to 17, leader changed from 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } } }
W20260709 13:12:52.324501 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:52.331151 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:52.338181 21526 tablet_replica.cc:333] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.348285 30099 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:52.369681 21526 raft_consensus.cc:2243] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:52.370110 30099 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.338281 30186 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.373528 30186 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.373654 30099 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.375946 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:52.379478 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "1f99ad34cc414233929e0a92f8c3d8db" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:52.323283 30176 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.381836 22601 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1f99ad34cc414233929e0a92f8c3d8db; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:52.324281 30012 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.344125 29943 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.381970 30176 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.383105 30012 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.385058 30006 raft_consensus.cc:2749] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:52.385632 30012 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.330847 30039 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.395186 30039 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.397554 30039 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.391687 30175 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.308929 29896 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.398856 30191 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.400180 29896 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.401298 30191 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:52.402163 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.402068 30102 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.404656 30102 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.416623 30134 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:52.421700 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.422928 22528 catalog_manager.cc:5697] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 17 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 18 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:52.424892 22528 catalog_manager.cc:5697] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:52.431448 22528 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "1f99ad34cc414233929e0a92f8c3d8db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } }
I20260709 13:12:52.437059 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:52.441156 22675 consensus_peers.cc:597] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:52.455331 22705 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 25.
I20260709 13:12:52.442925 30140 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:52.447378 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.451694 21526 raft_consensus.cc:2272] T 60c24d6125434f4caf1190c8bdba2f12 P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:52.458274 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 94852.7us
I20260709 13:12:52.461148 22528 catalog_manager.cc:5697] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 13 to 17, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:52.466123 21526 tablet_replica.cc:333] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.467239 30134 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.467866 21526 raft_consensus.cc:2243] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.468813 21526 raft_consensus.cc:2272] T e6d9cbf659974b358ba6d364ca9ca431 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.472522 21526 tablet_replica.cc:333] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.473460 21526 raft_consensus.cc:2243] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.474313 21526 raft_consensus.cc:2272] T 1d580d080ca644b2af67c9e48cbd3b78 P 1f99ad34cc414233929e0a92f8c3d8db [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.476363 30140 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.477609 21526 tablet_replica.cc:333] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:52.478199 22675 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.482822 21526 raft_consensus.cc:2243] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:12:52.484601 22675 consensus_peers.cc:597] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:52.485420 22675 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:52.494727 22675 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.496156 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.501292 21526 raft_consensus.cc:2272] T 3e474e7307e645259a4f6d201b451bef P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.505504 21526 tablet_replica.cc:333] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:52.518489 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:52.523452 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:52.524931 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.525672 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.526289 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.526902 22749 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.527120 22526 catalog_manager.cc:5697] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.532513 22526 catalog_manager.cc:5697] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 19 to 20, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.536453 22526 catalog_manager.cc:5697] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 18 to 20, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.506454 21526 raft_consensus.cc:2243] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:52.540457 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:52.541437 22705 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 23.
I20260709 13:12:52.541929 21526 raft_consensus.cc:2272] T 79ed72c313bf419bb01d45a59391a262 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.544096 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:52.545015 22702 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:52.546350 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:52.547016 22702 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:52.548518 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:52.549366 22702 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 20: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:52.560704 22702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:52.561800 22702 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 22: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:52.562582 21526 tablet_replica.cc:333] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.577610 21526 raft_consensus.cc:2243] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.578308 21526 raft_consensus.cc:2272] T 92329501c20843f68ffa5b3fdfc8f83d P 1f99ad34cc414233929e0a92f8c3d8db [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.590035 21526 tablet_replica.cc:333] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.591168 21526 raft_consensus.cc:2243] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:52.594079 21526 raft_consensus.cc:2272] T 71723544b43c4cb99dbeaf0321d14da6 P 1f99ad34cc414233929e0a92f8c3d8db [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.599804 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:52.600699 22784 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 20.
I20260709 13:12:52.603407 22776 raft_consensus.cc:1275] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:12:52.602461 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:52.605085 22780 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:52.607179 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:52.608013 22779 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 23.
I20260709 13:12:52.610517 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.612087 30180 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:12:52.612593 30180 raft_consensus.cc:493] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.613008 30180 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:12:52.613561 21526 tablet_replica.cc:333] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.615554 21526 raft_consensus.cc:2243] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.616983 22528 catalog_manager.cc:5697] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:52.623106 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:52.624049 22779 raft_consensus.cc:2393] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 20: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:52.627676 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:52.628250 21526 raft_consensus.cc:2272] T 9ac0029002284e64a5fde523ffaf77fb P 1f99ad34cc414233929e0a92f8c3d8db [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.628505 22779 raft_consensus.cc:2393] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 22: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
W20260709 13:12:52.629810 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:52.636215 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.639046 30180 raft_consensus.cc:515] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.644595 21526 tablet_replica.cc:333] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
W20260709 13:12:52.650141 23384 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.651194 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.652216 30140 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:52.652665 30140 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.652818 21526 raft_consensus.cc:2243] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.653074 30140 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:52.656289 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:52.657330 29930 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:12:52.637652 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:52.661233 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.662191 29930 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:52.662678 29930 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.663066 29930 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:52.665726 30140 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.668499 30140 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.669638 21526 raft_consensus.cc:2272] T 227f18995af4452ab6e5db39ef780c8a P 1f99ad34cc414233929e0a92f8c3d8db [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.673401 21526 tablet_replica.cc:333] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.674358 21526 raft_consensus.cc:2243] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.675071 21526 raft_consensus.cc:2272] T ef7d2e730e694ade8b424e4ff007a48a P 1f99ad34cc414233929e0a92f8c3d8db [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.677964 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.678881 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:52.687933 30207 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:12:52.683629 30180 leader_election.cc:290] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.689765 29930 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.703188 29930 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:52.703878 21526 tablet_replica.cc:333] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.704411 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:52.705305 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a4549b98fd04604afe4d21f9bb0acfe" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:52.707881 30102 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:52.725766 30102 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.726248 30102 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:52.713413 30191 consensus_queue.cc:1048] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:12:52.714640 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.728603 30191 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:12:52.723157 22779 raft_consensus.cc:3060] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:52.728973 30191 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.729387 30191 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:12:52.731750 22706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "308af34f5c0144e185e0af82342c7f92" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:52.732419 22706 raft_consensus.cc:3060] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:12:52.724090 21526 raft_consensus.cc:2243] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:52.724817 30039 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:12:52.742659 21526 raft_consensus.cc:2272] T 56c28a1350eb4e148e6e49cab764b300 P 1f99ad34cc414233929e0a92f8c3d8db [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:52.742686 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.751250 22702 raft_consensus.cc:1275] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:52.753338 22706 raft_consensus.cc:2468] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 26.
I20260709 13:12:52.753578 22779 raft_consensus.cc:2468] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
I20260709 13:12:52.754467 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:52.759410 21526 tablet_replica.cc:333] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.760445 21526 raft_consensus.cc:2243] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 LEADER]: Raft consensus shutting down.
W20260709 13:12:52.765700 22675 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.773229 22675 leader_election.cc:336] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.773852 22675 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:52.776065 29943 raft_consensus.cc:2749] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:12:52.780843 23385 leader_election.cc:304] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:52.782244 30205 raft_consensus.cc:2804] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:12:52.782881 30205 raft_consensus.cc:697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 26 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:52.783934 30205 consensus_queue.cc:237] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.773275 21526 raft_consensus.cc:2272] T 4f6afadc577c4fd08e4345a82ae50e5f P 1f99ad34cc414233929e0a92f8c3d8db [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.794206 22675 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.801059 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
W20260709 13:12:52.801786 22675 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.802263 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:52.802961 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.803508 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:52.804764 22675 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.805915 22675 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.806416 22675 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:52.806770 30131 raft_consensus.cc:2804] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:52.807374 30131 raft_consensus.cc:697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:52.808379 30131 consensus_queue.cc:237] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.800417 30039 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
W20260709 13:12:52.809085 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.809396 29930 consensus_queue.cc:1048] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:12:52.809515 30039 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.807396 29943 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:12:52.809892 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:52.825387 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:52.829996 22675 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.832310 22675 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:52.833702 22675 consensus_peers.cc:597] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:52.834329 22675 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.834908 30203 raft_consensus.cc:2749] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:12:52.836995 30119 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.837596 30119 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:52.831480 30206 raft_consensus.cc:2749] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:12:52.865487 22528 catalog_manager.cc:5697] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.876554 30119 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:52.879365 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.882073 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:52.882841 22784 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
W20260709 13:12:52.884724 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:52.896179 21526 tablet_replica.cc:333] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db: stopping tablet replica
I20260709 13:12:52.894729 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:52.870987 30102 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.913661 21526 raft_consensus.cc:2243] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:12:52.914307 30180 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:52.926254 22709 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 21: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
I20260709 13:12:52.941329 30039 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:52.928030 30180 raft_consensus.cc:493] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:52.943246 30180 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:52.978996 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:52.980347 30214 log.cc:927] Time spent T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:12:52.955174 30191 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:52.982693 30191 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:52.988808 21526 raft_consensus.cc:2272] T 1918c6e39d7d46139a210c432e0f4d84 P 1f99ad34cc414233929e0a92f8c3d8db [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:52.990236 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:52.993507 22705 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 24: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:12:52.995894 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 62006.5us
W20260709 13:12:53.004716 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:53.007010 23384 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:52.996909 30039 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:12:53.017119 30102 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:53.064517 22675 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.065518 22675 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.072687 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:53.081578 30180 raft_consensus.cc:515] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:53.084837 30223 log.cc:927] Time spent T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:12:53.089123 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.091019 22776 raft_consensus.cc:2393] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 24: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:53.094915 23419 raft_consensus.cc:2393] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 17: Already voted for candidate 6a31ac6451ff4f73bee634487d84d001 in this term.
I20260709 13:12:53.098062 30180 leader_election.cc:290] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.107025 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.109123 22776 raft_consensus.cc:2393] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 21: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:53.094780 30224 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:12:53.136137 22675 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:53.140137 22675 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:53.140681 30161 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.146811 30161 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.149346 30161 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.148962 22526 catalog_manager.cc:5697] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 25 to 26, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 26 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:53.142376 30131 raft_consensus.cc:2749] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:12:53.142874 29943 raft_consensus.cc:2749] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:12:53.209081 22675 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.211680 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:53.212980 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:53.213593 23384 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.226153 22675 consensus_peers.cc:597] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:53.226760 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.227550 22776 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 13.
I20260709 13:12:53.244176 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91be8db7516b4bca939f439f045b40eb" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.245055 22776 raft_consensus.cc:3060] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:53.257222 22703 raft_consensus.cc:1275] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 26 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 13:12:53.261025 30180 consensus_queue.cc:1048] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 13:12:53.261574 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.263653 30186 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:53.264106 30186 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.264506 30186 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:53.273892 30207 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.274760 30207 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.279280 30207 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:53.279487 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.289062 29943 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.289675 29943 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.290877 30186 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.292145 29943 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.295212 22776 raft_consensus.cc:2468] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:53.296834 30186 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:53.300588 22675 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.305284 22675 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.307731 22749 leader_election.cc:336] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.308248 22749 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:53.309401 30216 raft_consensus.cc:2749] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:53.337051 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:53.342648 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.342833 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a5cf575974c43e79448d00fab49d7ab" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.343657 22776 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 10.
I20260709 13:12:53.343708 22784 raft_consensus.cc:3060] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:12:53.348871 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:53.349843 23384 leader_election.cc:304] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.356493 30180 raft_consensus.cc:2804] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:53.357990 30180 raft_consensus.cc:697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:53.362317 30180 consensus_queue.cc:237] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.362716 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.378803 30186 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:12:53.379390 30186 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.379850 30186 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.378938 22784 raft_consensus.cc:2468] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 14.
W20260709 13:12:53.395831 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.396433 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:53.397384 30207 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:12:53.399542 22784 raft_consensus.cc:1275] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:53.410900 22530 catalog_manager.cc:5697] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 17 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:53.414760 22675 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.415781 30131 raft_consensus.cc:2804] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:53.416345 30131 raft_consensus.cc:697] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:53.416908 30186 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.417353 30131 consensus_queue.cc:237] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.419358 30186 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.439334 30186 consensus_queue.cc:1048] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:12:53.448329 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a840ecd041fa46a198e4fb917ef79b2a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.449106 22784 raft_consensus.cc:3060] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:12:53.466843 22784 raft_consensus.cc:2468] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 11.
I20260709 13:12:53.469355 22530 catalog_manager.cc:5697] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 14 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:53.526124 22675 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:53.527151 30131 raft_consensus.cc:2804] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:12:53.527388 30180 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.527843 30131 raft_consensus.cc:697] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:53.528342 30119 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.529670 30119 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.528103 30180 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.534673 30131 consensus_queue.cc:237] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.535180 30186 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.538702 30186 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.540573 30119 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.543176 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:53.536799 22675 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.545507 23419 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 15.
I20260709 13:12:53.548237 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 
I20260709 13:12:53.549273 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.549889 22780 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.551246 29943 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:12:53.551764 29943 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.552151 29943 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:53.548187 30206 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.553143 30206 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.556216 30206 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:53.557682 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:53.558511 23419 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
I20260709 13:12:53.541363 30186 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:53.560362 22709 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 16.
W20260709 13:12:53.567749 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.562413 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.568321 23384 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:53.569811 23384 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:53.563107 22675 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.571573 30119 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:53.571601 30207 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.572491 30207 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.573002 30180 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:53.575032 30207 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:53.581812 22749 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.590917 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.592067 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.594434 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:53.596410 30175 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:53.626245 22675 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:53.626876 22675 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:53.627192 29943 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.631564 30131 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:12:53.635983 22530 catalog_manager.cc:5697] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 10 to 11, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:53.656917 22675 consensus_peers.cc:597] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:53.661564 22705 raft_consensus.cc:2393] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6a31ac6451ff4f73bee634487d84d001 in current term 16: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:12:53.661736 22675 consensus_peers.cc:597] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:53.674064 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:53.678059 30119 raft_consensus.cc:3060] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:12:53.690086 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c015ab1d14c443258576403d9d39c485" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:53.695988 29943 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:53.702914 30131 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.705493 30131 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.704720 30206 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.706851 30206 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.711717 30131 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.716732 30206 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.721305 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:53.724309 22598 connection.cc:466] server connection from 127.0.0.1:45674 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 3335) could send its response
I20260709 13:12:53.726233 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.727234 22784 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
W20260709 13:12:53.728605 22598 connection.cc:466] server connection from 127.0.0.1:45674 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 3336) could send its response
W20260709 13:12:53.731820 22598 connection.cc:466] server connection from 127.0.0.1:45674 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45674 (request call id 3337) could send its response
W20260709 13:12:53.732731 22598 connection.cc:466] server connection from 127.0.0.1:45674 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 3338) could send its response
I20260709 13:12:53.730325 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:53.734969 30131 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:53.735606 30131 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.735991 30131 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:12:53.744449 22675 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: recv got EOF from 127.21.5.129:37283 (error 108)
I20260709 13:12:53.742601 30119 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:53.761076 30131 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.763720 23419 raft_consensus.cc:2468] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 16.
W20260709 13:12:53.765507 21526 connection.cc:466] server connection from 127.0.0.1:45674 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45674 (request call id 3334) could send its response
W20260709 13:12:53.772955 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:53.774925 22675 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.775588 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:53.776808 22675 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.777562 29943 raft_consensus.cc:2749] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:53.781510 30131 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:53.787401 22673 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.788421 30131 raft_consensus.cc:2804] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:12:53.788981 30131 raft_consensus.cc:697] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:53.789981 30131 consensus_queue.cc:237] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.809641 30140 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.810379 30140 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:53.812441 22675 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.813206 30140 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:53.814399 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.828619 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71723544b43c4cb99dbeaf0321d14da6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:53.829344 22784 raft_consensus.cc:3060] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:53.842201 22784 raft_consensus.cc:2468] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 17.
I20260709 13:12:53.844023 22675 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.845268 30206 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.845841 30206 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:53.846849 29943 raft_consensus.cc:2804] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:53.847436 29943 raft_consensus.cc:697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:53.848284 30206 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:53.848377 29943 consensus_queue.cc:237] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:53.855945 22675 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.862403 30191 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.863209 30191 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.865506 30191 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:53.872195 22675 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:53.878911 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.879789 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:53.880932 30140 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:12:53.882696 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:53.883526 22784 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 22.
I20260709 13:12:53.889875 30140 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.890551 30140 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.893107 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:53.895987 30140 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:53.903419 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:53.904847 22709 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:53.906916 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:53.907948 30140 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:53.908401 30140 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.908813 30140 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:53.915122 22673 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
I20260709 13:12:53.916455 30206 raft_consensus.cc:2749] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:12:53.922251 22530 catalog_manager.cc:5697] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 15 to 16, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:53.924655 22675 consensus_peers.cc:597] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:53.931962 30140 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.936061 30140 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:53.936306 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:53.937166 22709 raft_consensus.cc:3060] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:53.947770 22709 raft_consensus.cc:2468] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 18.
I20260709 13:12:53.949136 22750 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:53.952083 30140 raft_consensus.cc:2804] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:53.952975 30140 raft_consensus.cc:697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:53.953965 30140 consensus_queue.cc:237] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.954355 22530 catalog_manager.cc:5697] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:53.958014 22675 consensus_peers.cc:597] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:53.969933 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:53.970070 22784 raft_consensus.cc:1275] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:12:53.972434 29943 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:53.972872 29943 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:53.973272 29943 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:12:53.977695 22749 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:53.982651 30119 consensus_queue.cc:1048] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:53.990306 29943 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:53.994578 29943 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:53.996826 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:54.001555 30175 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.002285 30175 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:54.002709 22675 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.005491 30175 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:54.007964 23419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:54.011162 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.011963 22705 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:54.013394 23419 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 16.
I20260709 13:12:54.015218 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:54.015720 22749 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.019172 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e474e7307e645259a4f6d201b451bef" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:54.020018 22784 raft_consensus.cc:3060] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:54.019172 30207 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:12:54.021807 30207 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.022213 30207 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:54.029310 22784 raft_consensus.cc:2468] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 23.
I20260709 13:12:54.037750 22776 raft_consensus.cc:1275] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:12:54.046898 22675 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.048424 29943 raft_consensus.cc:2804] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:12:54.048983 29943 raft_consensus.cc:697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 23 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:54.049935 29943 consensus_queue.cc:237] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.051126 22780 raft_consensus.cc:1275] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:12:54.053735 30134 consensus_queue.cc:1048] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
W20260709 13:12:54.063069 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.063628 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.037177 30207 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:54.071077 30134 consensus_queue.cc:1048] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:12:54.100172 22675 consensus_peers.cc:597] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:54.109476 22530 catalog_manager.cc:5697] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 17 to 18, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:54.147553 22528 catalog_manager.cc:5697] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 22 to 23, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 23 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:54.210510 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 51890us
I20260709 13:12:54.230369 30240 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.238731 30240 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.239777 30207 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:54.241434 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001"
I20260709 13:12:54.242166 23420 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:54.248288 30140 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.249020 30140 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.252375 30240 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.255965 30140 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.273432 23420 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:54.310269 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79ed72c313bf419bb01d45a59391a262" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.311334 22709 raft_consensus.cc:3060] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:12:54.326109 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.332839 30216 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.334136 30216 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.337859 30216 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.338949 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.339792 22703 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 22.
W20260709 13:12:54.361250 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:39075, user_credentials={real_user=slave}} blocked reactor thread for 149659us
I20260709 13:12:54.361975 22784 raft_consensus.cc:1275] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:12:54.374770 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.375939 22703 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 20.
I20260709 13:12:54.389046 22709 raft_consensus.cc:2468] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 17.
I20260709 13:12:54.407968 30246 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.408974 30246 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:54.392356 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:54.426754 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.427552 22749 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.433085 23420 raft_consensus.cc:1275] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:12:54.426188 30216 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:54.440438 30253 consensus_queue.cc:1048] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:12:54.443066 30246 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:54.452747 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.440086 30216 raft_consensus.cc:493] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.455361 30216 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:54.453625 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:54.457233 30140 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:12:54.458420 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.463754 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.486299 23384 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.489549 22705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.490109 30206 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.490371 22705 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 13.
I20260709 13:12:54.490854 30206 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.493377 30206 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:54.505745 22747 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:36877, user_credentials={real_user=slave}} blocked reactor thread for 60611us
I20260709 13:12:54.507632 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.510017 30207 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:12:54.510449 30207 raft_consensus.cc:493] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.508309 30206 consensus_queue.cc:1048] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:54.513054 22750 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.516175 30207 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:54.518085 30240 raft_consensus.cc:2804] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:12:54.523034 30240 raft_consensus.cc:697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:54.524200 30240 consensus_queue.cc:237] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:54.535367 30216 raft_consensus.cc:515] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.538264 30216 leader_election.cc:290] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.538945 30207 raft_consensus.cc:515] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.552243 30207 leader_election.cc:290] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.545114 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.557536 30180 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:12:54.558048 30180 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.558420 30180 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:54.573264 30180 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:54.591647 22703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05be2cb1839f447296223033d364a7cc" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.617049 22703 raft_consensus.cc:3060] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:54.592512 22784 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:54.620431 22784 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 21.
I20260709 13:12:54.592723 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af344e0ef1b1489a9bc301ba3b375d4c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.621671 22709 raft_consensus.cc:3060] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:12:54.633975 22709 raft_consensus.cc:2468] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
W20260709 13:12:54.643759 22675 consensus_peers.cc:597] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:54.637660 22703 raft_consensus.cc:2468] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 23.
I20260709 13:12:54.660522 30180 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:54.655447 23384 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.674453 22750 leader_election.cc:304] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.675653 30259 raft_consensus.cc:2804] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:12:54.676242 30259 raft_consensus.cc:697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:54.677160 30259 consensus_queue.cc:237] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.689656 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.690372 22708 raft_consensus.cc:3060] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:12:54.694526 22702 raft_consensus.cc:1275] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:12:54.696731 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.698514 22749 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.701797 22675 consensus_peers.cc:597] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:54.702708 22675 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.703239 22675 consensus_peers.cc:597] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:54.709002 30254 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.710258 30254 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.712611 30254 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:54.714780 22750 leader_election.cc:304] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.716959 30240 raft_consensus.cc:2804] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:12:54.718272 30140 consensus_queue.cc:1048] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:54.720845 30240 raft_consensus.cc:697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:54.721881 30240 consensus_queue.cc:237] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.726964 30140 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.727679 30140 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.732967 22708 raft_consensus.cc:2468] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
I20260709 13:12:54.709303 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.736416 30206 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:54.736928 30206 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 1f99ad34cc414233929e0a92f8c3d8db)
I20260709 13:12:54.737339 30206 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:54.738343 30140 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.732268 30207 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.741231 30207 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:54.746412 22675 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:54.748605 22675 consensus_peers.cc:597] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:54.735225 22528 catalog_manager.cc:5697] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 16 to 17, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:54.755633 30207 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.762182 30206 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.764623 30206 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:54.775925 22749 leader_election.cc:336] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.783974 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.784509 22749 leader_election.cc:336] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.786286 23385 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.789461 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.789475 22776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:54.790325 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 24.
I20260709 13:12:54.790333 22776 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 21.
I20260709 13:12:54.791512 22709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
I20260709 13:12:54.792179 22709 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
I20260709 13:12:54.793370 22780 raft_consensus.cc:1275] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:12:54.804013 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1918c6e39d7d46139a210c432e0f4d84" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:54.804766 22783 raft_consensus.cc:3060] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:54.801365 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.814724 30191 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:54.815209 30191 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.815610 30191 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:54.815339 30258 consensus_queue.cc:1048] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:54.816021 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.817004 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:54.819893 30140 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:12:54.820374 30140 raft_consensus.cc:493] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.820776 30140 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:12:54.822844 30216 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:54.823318 30216 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.823740 30216 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:54.801272 30257 raft_consensus.cc:2804] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:12:54.826200 30257 raft_consensus.cc:697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:54.828202 22783 raft_consensus.cc:2468] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 22.
W20260709 13:12:54.831635 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.833490 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:54.834439 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.836638 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:54.841730 22749 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:54.848196 30257 consensus_queue.cc:237] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:54.851532 30216 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.859313 22675 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
W20260709 13:12:54.863278 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:54.864429 22675 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.870157 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.866278 29943 raft_consensus.cc:2804] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:12:54.858495 30140 raft_consensus.cc:515] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.862782 30216 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:54.907368 30207 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:54.915015 30207 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.909948 29943 raft_consensus.cc:697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 22 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:54.922120 29943 consensus_queue.cc:237] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.924595 30140 leader_election.cc:290] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:54.928444 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:54.930593 30207 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:54.939921 22675 consensus_peers.cc:597] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:54.919771 30191 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:54.942792 22708 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in current term 22: Already voted for candidate 91ebb78bf12f41baae84669c1effc423 in this term.
W20260709 13:12:54.947474 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:54.952059 30191 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:54.957262 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:54.962013 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423" is_pre_election: true
W20260709 13:12:54.963587 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.963564 22708 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 21.
W20260709 13:12:54.964581 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.965122 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.965691 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:54.966122 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.976085 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf401fd06552416592129b334cdf8167" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:54.977519 22708 raft_consensus.cc:3060] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:12:54.987280 22750 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:54.988847 30140 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:12:54.994170 22675 consensus_peers.cc:597] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:54.997018 22749 leader_election.cc:336] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:54.997895 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:54.998988 30216 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:12:54.999440 30216 raft_consensus.cc:493] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:54.999827 30216 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:55.023329 30216 raft_consensus.cc:515] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:55.029922 30216 leader_election.cc:290] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:55.033000 22779 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
W20260709 13:12:55.034862 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:55.035849 22779 raft_consensus.cc:2393] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 91ebb78bf12f41baae84669c1effc423 in current term 22: Already voted for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in this term.
I20260709 13:12:55.036428 22708 raft_consensus.cc:2468] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 25.
I20260709 13:12:55.043509 22708 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34f27615de694e7faaba89794bf9b5c6" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "91ebb78bf12f41baae84669c1effc423"
I20260709 13:12:55.044498 22708 raft_consensus.cc:3060] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:12:55.045018 22750 leader_election.cc:304] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:55.045329 22526 catalog_manager.cc:5697] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 13 to 14, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.046332 30140 raft_consensus.cc:2804] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:12:55.049479 22705 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:12:55.056156 22749 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:55.056064 30140 raft_consensus.cc:697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 25 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:55.057605 30140 consensus_queue.cc:237] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:55.059439 30244 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:12:55.071002 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:55.052263 22530 catalog_manager.cc:5697] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 reported cstate change: term changed from 21 to 22, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to 91ebb78bf12f41baae84669c1effc423 (127.21.5.130). New cstate: current_term: 22 leader_uuid: "91ebb78bf12f41baae84669c1effc423" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:55.071852 22749 leader_election.cc:336] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:55.073596 22708 raft_consensus.cc:2468] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0e9f7dc2a2e4bdb9287d910e4b491f7 in term 22.
I20260709 13:12:55.071094 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:55.078327 29943 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:12:55.099882 22750 leader_election.cc:304] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
I20260709 13:12:55.101029 30216 raft_consensus.cc:2804] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:12:55.101825 30216 raft_consensus.cc:697] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Becoming Leader. State: Replica: a0e9f7dc2a2e4bdb9287d910e4b491f7, State: Running, Role: LEADER
I20260709 13:12:55.102900 30216 consensus_queue.cc:237] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:55.111143 22528 catalog_manager.cc:5697] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 19 to 21, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.113636 22528 catalog_manager.cc:5697] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 21 to 23, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 23 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
W20260709 13:12:55.127982 22749 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:55.149441 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:55.150678 22708 raft_consensus.cc:1275] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:12:55.155675 23420 raft_consensus.cc:1275] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:12:55.158812 30140 consensus_queue.cc:1048] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:55.167585 30207 consensus_queue.cc:1048] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:12:55.179720 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:55.230681 22702 raft_consensus.cc:1275] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:12:55.245652 22526 catalog_manager.cc:5697] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 23 to 25, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 25 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.250398 22526 catalog_manager.cc:5697] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 reported cstate change: term changed from 20 to 22, leader changed from 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129) to a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.261543 30244 consensus_queue.cc:1048] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:12:55.293715 22675 consensus_peers.cc:597] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:55.308131 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:55.357427 22675 consensus_peers.cc:597] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:55.423476 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:55.475517 22749 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:55.482366 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:55.493666 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:55.503885 22675 consensus_peers.cc:597] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:55.517156 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:55.522866 22702 raft_consensus.cc:1275] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:12:55.529647 22749 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:55.534521 22779 raft_consensus.cc:1275] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Refusing update from remote peer 91ebb78bf12f41baae84669c1effc423: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:12:55.536123 22702 raft_consensus.cc:1275] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:12:55.539994 30140 consensus_queue.cc:1048] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:12:55.553314 30180 consensus_queue.cc:1048] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:12:55.557298 30206 consensus_queue.cc:1048] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:12:55.562688 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:55.564064 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:55.605108 22705 raft_consensus.cc:1275] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Refusing update from remote peer a0e9f7dc2a2e4bdb9287d910e4b491f7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:12:55.638864 30244 consensus_queue.cc:1048] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:12:55.656347 22749 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:55.709682 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:55.846242 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:55.901233 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:56.267325 22675 consensus_peers.cc:597] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:56.305742 22675 consensus_peers.cc:597] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:56.407378 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
W20260709 13:12:56.456244 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:56.519001 22675 consensus_peers.cc:597] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:56.577680 22675 consensus_peers.cc:597] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:56.652384 22675 consensus_peers.cc:597] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:56.736756 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:56.739805 22675 consensus_peers.cc:597] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:56.830446 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:12:56.852622 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
W20260709 13:12:56.885362 22675 consensus_peers.cc:597] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:56.887539 22750 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:56.889825 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 46395.5us
W20260709 13:12:56.893011 22750 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:56.896909 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:56.919266 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:56.922247 22750 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:56.952924 23385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 60283.2us
W20260709 13:12:57.028728 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:57.029531 22750 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.030711 22750 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.036928 22675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111) [suppressed 476 similar messages]
W20260709 13:12:57.037801 22675 consensus_peers.cc:597] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:57.096832 22675 consensus_peers.cc:597] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:57.097774 23385 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.098415 23385 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.099684 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:57.101226 23385 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.157227 22750 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.158008 22750 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.162202 22750 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.167606 22675 consensus_peers.cc:597] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:57.205168 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:57.221608 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:57.232102 22749 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:57.261235 22750 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.262460 22750 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.263557 22750 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.271545 22750 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.272270 22750 consensus_peers.cc:597] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.272920 22750 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.274019 22750 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.274955 22750 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.277160 22750 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.277427 23385 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.277958 22750 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.278199 23385 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.279408 22675 consensus_peers.cc:597] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:57.298321 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:57.299925 21526 tablet_replica.cc:333] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.301205 21526 raft_consensus.cc:2243] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.308408 21526 raft_consensus.cc:2272] T a9a28904b6a34dbf81e5d1d4bb08a08c P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.309839 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:57.313355 22750 consensus_peers.cc:597] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.315580 23385 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.316129 21526 tablet_replica.cc:333] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.317564 21526 raft_consensus.cc:2243] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:57.317658 22750 consensus_peers.cc:597] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.320865 21526 raft_consensus.cc:2272] T 9e3855678a8148a09ffe99d7624c9b4f P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.324201 21526 tablet_replica.cc:333] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.325259 21526 raft_consensus.cc:2243] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.326392 30293 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:57.327049 21526 raft_consensus.cc:2272] T f33fee9d64384b499e55f8bc58339f69 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.327062 30293 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.329643 30293 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:57.330196 22750 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.330461 21526 tablet_replica.cc:333] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.331875 21526 raft_consensus.cc:2243] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.333671 21526 raft_consensus.cc:2272] T f09c3bbe948b4c7a9f50df8ec4277e9f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.336479 22750 consensus_peers.cc:597] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.337534 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:57.338285 22783 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 22.
I20260709 13:12:57.336695 21526 tablet_replica.cc:333] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.340971 21526 raft_consensus.cc:2243] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.341688 21526 raft_consensus.cc:2272] T 14efc068101d4b50a525206c447e1ae8 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.345894 21526 tablet_replica.cc:333] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.346937 21526 raft_consensus.cc:2243] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.347784 21526 raft_consensus.cc:2272] T fbb33c4b77a2451499c9c5e49b83aa12 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.350948 21526 tablet_replica.cc:333] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.352159 21526 raft_consensus.cc:2243] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.353133 21526 raft_consensus.cc:2272] T 51e41e0d49c846d0ae7c7d737a15be9e P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.353427 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:57.354377 30293 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:12:57.354810 30293 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:57.355191 30293 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:57.356561 21526 tablet_replica.cc:333] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.357641 21526 raft_consensus.cc:2243] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.358780 21526 raft_consensus.cc:2272] T ed3a2e4658ea473ba5c2b68313aeaf3b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.359974 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:57.361896 21526 tablet_replica.cc:333] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.366164 30293 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.363808 21526 raft_consensus.cc:2243] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.376369 21526 raft_consensus.cc:2272] T bfb716d0bf184679a89d2dde7563865d P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.378687 23385 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.387820 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:57.388110 22675 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:57.394090 30293 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Requested vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:57.405967 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:57.406826 22783 raft_consensus.cc:3060] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:12:57.407229 21526 tablet_replica.cc:333] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.408294 21526 raft_consensus.cc:2243] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:12:57.417946 22750 consensus_peers.cc:597] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:57.418792 22750 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.419075 21526 raft_consensus.cc:2272] T 8a4b81cf2b9c48b59c92dd3ace99c8db P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.422757 21526 tablet_replica.cc:333] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.423811 21526 raft_consensus.cc:2243] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.424489 21526 raft_consensus.cc:2272] T 26e6130713d74c0ea9492f5151053c42 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.426469 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:57.427755 21526 tablet_replica.cc:333] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.428901 21526 raft_consensus.cc:2243] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Raft consensus shutting down.
W20260709 13:12:57.434962 22750 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:57.437763 22783 raft_consensus.cc:2468] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ebb78bf12f41baae84669c1effc423 in term 23.
I20260709 13:12:57.441036 22675 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db
W20260709 13:12:57.441916 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:57.442256 30293 raft_consensus.cc:2804] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:12:57.443467 30293 raft_consensus.cc:697] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 LEADER]: Becoming Leader. State: Replica: 91ebb78bf12f41baae84669c1effc423, State: Running, Role: LEADER
I20260709 13:12:57.438266 21526 raft_consensus.cc:2272] T 9a4549b98fd04604afe4d21f9bb0acfe P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.444485 30293 consensus_queue.cc:237] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.455178 21526 tablet_replica.cc:333] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.456343 21526 raft_consensus.cc:2243] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:57.458760 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:57.472800 21526 raft_consensus.cc:2272] T 05be2cb1839f447296223033d364a7cc P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.479858 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:12:57.483444 21526 tablet_replica.cc:333] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.484421 21526 raft_consensus.cc:2243] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.485670 21526 raft_consensus.cc:2272] T 0ebec7fc99cb448ea382f8b3e769e7ca P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.490983 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:57.495522 21526 tablet_replica.cc:333] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.496615 21526 raft_consensus.cc:2243] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.498021 21526 raft_consensus.cc:2272] T ca809087a6724923ab06ace347425d31 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.501252 21526 tablet_replica.cc:333] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.519053 21526 raft_consensus.cc:2243] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.520285 21526 raft_consensus.cc:2272] T 2f4adfa257c34bb88d79bda9d39c5146 P 91ebb78bf12f41baae84669c1effc423 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.521849 22749 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.523542 21526 tablet_replica.cc:333] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.524521 21526 raft_consensus.cc:2243] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.525682 21526 raft_consensus.cc:2272] T d92b65b2cbd546e18ddac5ce049ea91c P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.539449 21526 tablet_replica.cc:333] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.540401 21526 raft_consensus.cc:2243] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.541086 21526 raft_consensus.cc:2272] T b907163bc2174a2aa11df858ccae00ac P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.550899 21526 tablet_replica.cc:333] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.552060 21526 raft_consensus.cc:2243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:57.550967 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:57.554169 21526 raft_consensus.cc:2272] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 91ebb78bf12f41baae84669c1effc423 [term 13 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.596936 22749 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.603061 21526 tablet_replica.cc:333] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.604385 21526 raft_consensus.cc:2243] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.605083 21526 raft_consensus.cc:2272] T af344e0ef1b1489a9bc301ba3b375d4c P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.612026 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:12:57.617929 21526 tablet_replica.cc:333] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.620505 21526 raft_consensus.cc:2243] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.626735 21526 raft_consensus.cc:2272] T 295bf86c5a444ebc9285e902fd5e9ae0 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.633281 21526 tablet_replica.cc:333] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:57.648018 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:12:57.649382 21526 raft_consensus.cc:2243] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.650537 21526 raft_consensus.cc:2272] T 308af34f5c0144e185e0af82342c7f92 P 91ebb78bf12f41baae84669c1effc423 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.663941 21526 tablet_replica.cc:333] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.665148 21526 raft_consensus.cc:2243] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.665990 21526 raft_consensus.cc:2272] T 887cfff73fab4d52b2e39f5b6089133a P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.735637 21526 tablet_replica.cc:333] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.743907 21526 raft_consensus.cc:2243] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.754513 21526 raft_consensus.cc:2272] T bf401fd06552416592129b334cdf8167 P 91ebb78bf12f41baae84669c1effc423 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.765082 21526 tablet_replica.cc:333] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:57.768654 22749 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.774495 21526 raft_consensus.cc:2243] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.775271 21526 raft_consensus.cc:2272] T 34f27615de694e7faaba89794bf9b5c6 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.796346 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:57.800406 21526 tablet_replica.cc:333] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.806820 30131 raft_consensus.cc:493] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:57.807647 30131 raft_consensus.cc:515] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.811439 30131 leader_election.cc:290] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:57.823349 22675 leader_election.cc:336] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:57.823894 21526 raft_consensus.cc:2243] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.824688 21526 raft_consensus.cc:2272] T d6544eab6fd54f95baf389a8b15729cd P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.825819 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12b8ec59bdc04cd8b5dae5bcc35466f6" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:57.832512 22675 leader_election.cc:304] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:57.834249 30131 raft_consensus.cc:2749] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:12:57.840216 21526 tablet_replica.cc:333] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.841413 21526 raft_consensus.cc:2243] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.842247 21526 raft_consensus.cc:2272] T a79a6f618f4944b1a11b9c1df766ae7f P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.845583 21526 tablet_replica.cc:333] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.846712 21526 raft_consensus.cc:2243] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.847831 21526 pending_rounds.cc:70] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Trying to abort 1 pending ops.
I20260709 13:12:57.848171 21526 pending_rounds.cc:77] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423: Aborting op as it isn't in flight: id { term: 23 index: 14 } timestamp: 7305636976423698432 op_type: NO_OP noop_request { }
I20260709 13:12:57.848605 21526 raft_consensus.cc:2889] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:57.849031 21526 raft_consensus.cc:2272] T b8291ea7e33a40b5afaebc02d6ce7e9d P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.854873 21526 tablet_replica.cc:333] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.868686 21526 raft_consensus.cc:2243] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.878938 21526 raft_consensus.cc:2272] T a937bd6b17c04d5199f368b1d595cf8b P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.882400 21526 tablet_replica.cc:333] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.883502 21526 raft_consensus.cc:2243] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.891613 21526 raft_consensus.cc:2272] T 4a5cf575974c43e79448d00fab49d7ab P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.901624 21526 tablet_replica.cc:333] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.902827 21526 raft_consensus.cc:2243] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.903636 21526 raft_consensus.cc:2272] T cc8b169704cc4157ad021d774f83419e P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:57.886881 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:57.896238 30293 raft_consensus.cc:493] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:57.911092 30293 raft_consensus.cc:515] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.913412 30293 leader_election.cc:290] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:57.929669 21526 tablet_replica.cc:333] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:57.931831 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.933043 21526 raft_consensus.cc:2243] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.934124 21526 raft_consensus.cc:2272] T 594d7db14b30467aac30a69dc1570d43 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.944027 21526 tablet_replica.cc:333] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.945624 21526 raft_consensus.cc:2243] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.946578 21526 raft_consensus.cc:2272] T fee10603c0c042fd8e28683daeb86a99 P 91ebb78bf12f41baae84669c1effc423 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.955369 21526 tablet_replica.cc:333] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:57.966073 21526 raft_consensus.cc:2243] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:57.971055 30293 raft_consensus.cc:493] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:57.974992 30293 raft_consensus.cc:515] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.975582 21526 raft_consensus.cc:2272] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P 91ebb78bf12f41baae84669c1effc423 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.977589 30293 leader_election.cc:290] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:57.979094 21526 tablet_replica.cc:333] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:57.983062 22749 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.987524 21526 raft_consensus.cc:2243] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:57.988970 21526 raft_consensus.cc:2272] T ec9764b0b7dd4b67a5b50a652203c60a P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:57.989562 30131 raft_consensus.cc:493] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
W20260709 13:12:57.974941 22675 consensus_peers.cc:597] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:57.990077 30131 raft_consensus.cc:515] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:57.992545 21526 tablet_replica.cc:333] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:57.999804 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:12:58.045591 23384 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45858.1us
I20260709 13:12:58.014053 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c24d6125434f4caf1190c8bdba2f12" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.046074 21526 raft_consensus.cc:2243] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.046815 30131 leader_election.cc:290] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:58.039256 22675 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 64366.5us
I20260709 13:12:57.990675 30293 raft_consensus.cc:493] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:58.052299 30293 raft_consensus.cc:515] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.055035 30293 leader_election.cc:290] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:58.060340 21526 raft_consensus.cc:2272] T 66c163cd962d4a568a04fc251b913b43 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.113234 21526 tablet_replica.cc:333] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:58.113435 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:58.114391 21526 raft_consensus.cc:2243] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 11 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.116423 21526 raft_consensus.cc:2272] T a840ecd041fa46a198e4fb917ef79b2a P 91ebb78bf12f41baae84669c1effc423 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.119999 21526 tablet_replica.cc:333] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.137279 21526 raft_consensus.cc:2243] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.171980 21526 raft_consensus.cc:2272] T c015ab1d14c443258576403d9d39c485 P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.175338 21526 tablet_replica.cc:333] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.179557 21526 raft_consensus.cc:2243] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.180684 21526 raft_consensus.cc:2272] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.193886 21526 tablet_replica.cc:333] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.199496 21526 raft_consensus.cc:2243] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.200380 21526 raft_consensus.cc:2272] T e6d9cbf659974b358ba6d364ca9ca431 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.213788 21526 tablet_replica.cc:333] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:58.186518 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:58.216089 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef7d2e730e694ade8b424e4ff007a48a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.217077 21526 raft_consensus.cc:2243] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.222091 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "227f18995af4452ab6e5db39ef780c8a" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.224982 21526 raft_consensus.cc:2272] T 1918c6e39d7d46139a210c432e0f4d84 P 91ebb78bf12f41baae84669c1effc423 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.226514 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56c28a1350eb4e148e6e49cab764b300" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.228828 21526 tablet_replica.cc:333] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.234803 21526 raft_consensus.cc:2243] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.237859 21526 raft_consensus.cc:2272] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.252091 30293 raft_consensus.cc:493] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:58.252774 30293 raft_consensus.cc:515] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.254745 21526 tablet_replica.cc:333] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.255154 30293 leader_election.cc:290] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:58.255890 21526 raft_consensus.cc:2243] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.257191 21526 raft_consensus.cc:2272] T 79ed72c313bf419bb01d45a59391a262 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:58.261960 22675 leader_election.cc:336] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.262548 22675 leader_election.cc:304] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:58.263391 22675 raft_consensus.cc:2686] T 60c24d6125434f4caf1190c8bdba2f12 P 91ebb78bf12f41baae84669c1effc423: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:12:58.275127 21526 tablet_replica.cc:333] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.276316 21526 raft_consensus.cc:2243] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.278785 30295 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.287019 30295 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.289430 30295 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:58.290444 21526 raft_consensus.cc:2272] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.293824 21526 tablet_replica.cc:333] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.294857 21526 raft_consensus.cc:2243] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.296428 21526 raft_consensus.cc:2272] T 71723544b43c4cb99dbeaf0321d14da6 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:58.304569 22675 leader_election.cc:336] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.305253 22675 leader_election.cc:304] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:58.306355 22675 leader_election.cc:336] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.306126 30131 raft_consensus.cc:2749] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:12:58.307009 22675 leader_election.cc:304] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:58.308099 22675 raft_consensus.cc:2686] T 227f18995af4452ab6e5db39ef780c8a P 91ebb78bf12f41baae84669c1effc423: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:12:58.309058 22675 leader_election.cc:336] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.309685 22675 leader_election.cc:304] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:12:58.310791 30131 raft_consensus.cc:2749] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:58.317803 22675 leader_election.cc:336] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.304838 30293 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:12:58.318919 30293 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.323582 30293 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:58.324513 30293 raft_consensus.cc:493] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6a31ac6451ff4f73bee634487d84d001)
I20260709 13:12:58.327059 30288 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.330405 23420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:58.330907 30288 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.326857 30293 raft_consensus.cc:515] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.336391 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92329501c20843f68ffa5b3fdfc8f83d" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.336997 30293 leader_election.cc:290] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:58.338651 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4f6afadc577c4fd08e4345a82ae50e5f" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:58.343238 30296 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.344319 30296 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.342841 30288 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:58.346505 30296 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:58.349870 23384 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.351337 22783 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ffde8dbdd2d54723860d280e3e916e60" candidate_uuid: "91ebb78bf12f41baae84669c1effc423" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
W20260709 13:12:58.359596 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:12:58.360271 21526 tablet_replica.cc:333] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.361241 21526 raft_consensus.cc:2243] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.361876 21526 raft_consensus.cc:2272] T ef7d2e730e694ade8b424e4ff007a48a P 91ebb78bf12f41baae84669c1effc423 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.370162 21526 tablet_replica.cc:333] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:58.379253 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.386201 22675 leader_election.cc:304] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:58.387756 22675 raft_consensus.cc:2686] T 92329501c20843f68ffa5b3fdfc8f83d P 91ebb78bf12f41baae84669c1effc423: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:12:58.388643 21526 raft_consensus.cc:2243] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.399375 21526 raft_consensus.cc:2272] T 56c28a1350eb4e148e6e49cab764b300 P 91ebb78bf12f41baae84669c1effc423 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.404199 21526 tablet_replica.cc:333] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.405190 21526 raft_consensus.cc:2243] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.406057 21526 raft_consensus.cc:2272] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.409438 21526 tablet_replica.cc:333] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
W20260709 13:12:58.388758 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.411545 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.412523 30296 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:58.413096 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.419782 21526 raft_consensus.cc:2243] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:12:58.421238 21526 raft_consensus.cc:2272] T 3e474e7307e645259a4f6d201b451bef P 91ebb78bf12f41baae84669c1effc423 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:58.424701 21526 tablet_replica.cc:333] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: stopping tablet replica
I20260709 13:12:58.425530 21526 raft_consensus.cc:2243] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:58.426226 21526 raft_consensus.cc:2272] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:58.427712 22749 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:58.410737 22675 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:12:58.428877 22675 raft_consensus.cc:2686] T ffde8dbdd2d54723860d280e3e916e60 P 91ebb78bf12f41baae84669c1effc423: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:12:58.460505 22675 leader_election.cc:336] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.461149 22675 leader_election.cc:304] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91ebb78bf12f41baae84669c1effc423; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 6a31ac6451ff4f73bee634487d84d001
W20260709 13:12:58.466243 22675 raft_consensus.cc:2686] T 4f6afadc577c4fd08e4345a82ae50e5f P 91ebb78bf12f41baae84669c1effc423: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:12:58.467543 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.468006 22750 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.469226 30295 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:12:58.485776 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.486321 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.488325 30288 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:12:58.505952 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:58.612582 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:12:58.687397 30290 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.688455 30290 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.692930 23412 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
I20260709 13:12:58.693253 30295 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.693959 30295 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:58.694893 22750 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.696668 30295 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:58.697183 30296 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.700800 30296 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.703238 30296 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:58.700008 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.708751 22747 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
W20260709 13:12:58.711684 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.712242 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.713784 30290 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:58.715911 30290 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:12:58.714905 30296 raft_consensus.cc:2749] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:58.718859 30288 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.719820 30288 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.723609 30288 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:58.729135 30288 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.729754 30288 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:58.730906 23384 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.732664 30288 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:58.739188 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:58.740114 22775 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 14.
W20260709 13:12:58.742730 22749 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.756735 30301 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.758935 30301 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:58.762212 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.766942 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.768307 30288 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:12:58.762967 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.769340 22750 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.770252 30290 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:12:58.774602 30301 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:58.776070 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.780541 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:58.781556 30288 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:12:58.781987 30288 raft_consensus.cc:493] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.782378 30288 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:12:58.784094 22749 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:58.792335 30288 raft_consensus.cc:515] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:58.795151 30288 leader_election.cc:290] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:58.881505 23385 leader_election.cc:336] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:58.880995 30296 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:58.883462 30296 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:58.922554 30296 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:12:59.039794 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d92b65b2cbd546e18ddac5ce049ea91c" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:59.041031 22775 raft_consensus.cc:3060] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:12:59.088811 22775 raft_consensus.cc:2468] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 15.
W20260709 13:12:59.094308 22749 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.100100 30307 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.121464 30307 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:59.106033 30309 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.124189 30309 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:59.132993 30310 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.133876 30310 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:59.136786 30310 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.137293 23384 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.139103 30309 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.141131 23384 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.144973 23384 leader_election.cc:304] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.148252 30309 raft_consensus.cc:2804] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:12:59.151180 30309 raft_consensus.cc:697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:59.152705 30309 consensus_queue.cc:237] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:59.154524 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:59.157094 22749 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.157696 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.162099 30308 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:12:59.165210 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:59.165825 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.166783 30308 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:59.157413 30307 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.190681 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:59.191351 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.220266 22526 catalog_manager.cc:5697] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 14 to 15, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:12:59.223027 30311 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:12:59.235121 22750 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:59.235673 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.245736 30312 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:12:59.270254 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:12:59.293004 22750 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.298751 22750 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.299901 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:59.300526 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.301565 30312 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:12:59.324980 30311 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.325712 30311 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:59.337275 30311 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132:36877)
I20260709 13:12:59.340430 23414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6a31ac6451ff4f73bee634487d84d001" is_pre_election: true
W20260709 13:12:59.349534 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:12:59.356158 22750 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.359578 22750 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:12:59.360240 22750 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 6a31ac6451ff4f73bee634487d84d001, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.361203 30311 raft_consensus.cc:2749] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:59.364578 30310 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.365375 30310 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:59.372279 30310 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.392923 22750 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:12:59.420454 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.421037 22749 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.421852 30310 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:12:59.442014 22750 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.438297 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:12:59.450348 30311 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.451368 30311 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:59.453738 30311 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.457643 22749 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:59.466387 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:12:59.466969 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.468186 30310 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:12:59.486011 30309 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:12:59.487043 23385 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.489935 30309 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:12:59.493158 30309 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:12:59.496935 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" is_pre_election: true
I20260709 13:12:59.497666 22775 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 17.
I20260709 13:12:59.498739 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 
I20260709 13:12:59.499703 30307 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:12:59.500154 30307 raft_consensus.cc:493] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.500648 30307 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:12:59.503659 22750 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.500967 23385 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:12:59.535799 23385 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.539053 30307 raft_consensus.cc:515] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:59.546576 23385 leader_election.cc:336] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:12:59.554350 22775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66c163cd962d4a568a04fc251b913b43" candidate_uuid: "6a31ac6451ff4f73bee634487d84d001" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7"
I20260709 13:12:59.555029 22775 raft_consensus.cc:3060] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:12:59.568473 30307 leader_election.cc:290] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Requested vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:12:59.580135 23385 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.583904 22775 raft_consensus.cc:2468] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a31ac6451ff4f73bee634487d84d001 in term 18.
W20260709 13:12:59.584182 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:12:59.588027 23384 leader_election.cc:304] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001, a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.589461 30307 raft_consensus.cc:2804] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:12:59.590458 30307 raft_consensus.cc:697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Becoming Leader. State: Replica: 6a31ac6451ff4f73bee634487d84d001, State: Running, Role: LEADER
I20260709 13:12:59.593240 30307 consensus_queue.cc:237] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:12:59.597847 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:12:59.624244 22750 consensus_peers.cc:597] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.631209 22750 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.642805 22775 raft_consensus.cc:1275] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:12:59.646735 23385 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.650316 30307 consensus_queue.cc:1048] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:12:59.662508 22526 catalog_manager.cc:5697] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 reported cstate change: term changed from 17 to 18, leader changed from 91ebb78bf12f41baae84669c1effc423 (127.21.5.130) to 6a31ac6451ff4f73bee634487d84d001 (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6a31ac6451ff4f73bee634487d84d001" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:12:59.667737 23385 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:59.678640 22750 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.679281 22750 consensus_peers.cc:597] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.686015 22750 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.707469 23385 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.709668 22750 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.736737 22750 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.749804 30312 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.750494 30312 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:12:59.752858 30312 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.768317 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:12:59.777797 23385 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.781298 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:59.782963 22750 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.783524 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:12:59.783948 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.787385 30312 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:12:59.797010 22749 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:59.811435 22750 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.815516 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:59.830412 30312 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:12:59.831188 30312 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:12:59.854161 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:12:59.855389 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:12:59.856340 30312 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:12:59.856506 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:12:59.856026 22749 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:12:59.863118 30311 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:12:59.867417 23385 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:12:59.877611 22522 maintenance_manager.cc:419] P 71605ba8587a4b5193d8d69d9967c1af: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033393
W20260709 13:12:59.879449 22750 consensus_peers.cc:597] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.880127 22750 consensus_peers.cc:597] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.902655 22750 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.903283 22750 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.908530 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:12:59.924365 22750 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.946456 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:12:59.959514 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:12:59.968926 22750 consensus_peers.cc:597] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.969661 22750 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:59.978802 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:12:59.999140 22521 maintenance_manager.cc:643] P 71605ba8587a4b5193d8d69d9967c1af: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.120s	user 0.049s	sys 0.000s Metrics: {"bytes_written":7745,"delete_count":0,"lbm_write_time_us":1249,"lbm_writes_lt_1ms":7,"reinsert_count":0,"update_count":311}
I20260709 13:13:00.004982 22522 maintenance_manager.cc:419] P 71605ba8587a4b5193d8d69d9967c1af: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.339098
W20260709 13:13:00.021584 22750 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:00.022702 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:00.039247 23385 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:00.039590 22750 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:00.045958 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.053345 22749 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:00.054076 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:13:00.113195 30321 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:00.115352 30321 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:00.121955 30321 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.147773 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:00.156529 22750 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:00.175201 22779 raft_consensus.cc:1275] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Refusing update from remote peer 6a31ac6451ff4f73bee634487d84d001: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:13:00.184737 30309 consensus_queue.cc:1048] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:13:00.197924 22749 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:00.198743 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.199260 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.200186 30326 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:13:00.224756 30321 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.225461 30321 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:00.245808 30321 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.248879 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.273232 22750 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:00.280768 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.282510 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:13:00.286151 30309 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.287163 30309 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:00.298698 23384 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:00.310773 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:00.311375 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.312973 30308 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:13:00.315842 30309 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.346658 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.366410 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:00.369726 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.370317 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:13:00.373878 22750 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:00.374569 30321 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:00.388252 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22731)
I20260709 13:13:00.395212 22521 maintenance_manager.cc:643] P 71605ba8587a4b5193d8d69d9967c1af: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.389s	user 0.107s	sys 0.007s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":28027,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":10432,"dirs.run_cpu_time_us":346,"dirs.run_wall_time_us":345,"lbm_read_time_us":2237,"lbm_reads_lt_1ms":24,"lbm_write_time_us":26182,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":11,"peak_mem_usage":310727,"reinsert_count":0,"thread_start_us":13457,"threads_started":1,"update_count":311}
I20260709 13:13:00.464316 30321 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.465118 30321 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:00.480337 30321 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.498031 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.531558 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.538203 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.541360 30326 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:00.566676 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.595835 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.664273 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 48751.6us
W20260709 13:13:00.668118 22749 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:00.677011 30326 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.677811 30326 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:00.681048 30326 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.685249 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:00.713479 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:13:00.728340 30309 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.728979 30309 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:00.731755 30309 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:00.736703 30309 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.738016 30309 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:00.740844 30309 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.744273 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.763979 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.755702 22749 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:00.776013 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.786957 23384 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.788826 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.791518 30308 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:00.794699 23384 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.797655 23384 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.801434 30308 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:00.802565 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:00.815649 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:00.817714 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.827382 30326 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:13:00.846988 30321 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:00.847702 30321 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:00.850342 30321 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:00.865780 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:00.880918 22749 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:00.887017 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:00.881520 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:00.894681 30321 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:00.908797 22749 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:00.999807 30331 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.000692 30331 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:01.013255 23384 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:01.013949 30321 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.014704 30321 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.015511 30331 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:01.016675 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.016925 30321 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:01.017194 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.018675 30331 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:01.044219 22749 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:01.077632 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.079342 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.080422 30326 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:01.175736 30326 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.176484 30326 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.179639 30326 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:01.185782 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:01.195168 22749 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:01.195716 22749 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.196782 30326 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:01.269297 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:01.356904 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:01.380492 30326 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.381558 30326 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.384197 30326 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:01.388844 21526 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22731)
W20260709 13:13:01.423868 22749 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:01.470660 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 45980.3us
W20260709 13:13:01.484449 22750 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.485010 22750 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.486425 30326 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:01.505750 30332 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.506481 30332 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.509364 30332 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:01.548906 30326 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.549561 30326 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:13:01.559991 22750 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:01.575816 30326 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:01.587649 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:01.603936 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.604568 22750 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:13:01.606308 22749 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:01.607594 30326 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:13:01.627283 30326 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.628000 30326 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.639564 30326 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:01.647921 30332 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:01.648576 30332 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:01.673491 30332 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:01.696290 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 49896.3us
W20260709 13:13:01.702232 22750 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:01.702986 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.703435 22750 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.715544 30326 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:01.731506 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:01.732275 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:01.786746 22749 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:37283, user_credentials={real_user=slave}} blocked reactor thread for 53946.9us
W20260709 13:13:01.789980 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.790599 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.816630 30332 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:13:01.825510 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:01.832158 22750 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:01.832778 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:01.835850 30321 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:01.854467 23385 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:01.891095 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:01.892380 22750 consensus_peers.cc:597] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:01.937914 22750 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.010251 22750 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.014281 22750 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.031752 23385 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.044418 22750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111) [suppressed 744 similar messages]
W20260709 13:13:02.055647 22749 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:02.091065 23385 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.098358 23385 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.104103 23385 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.112442 22750 consensus_peers.cc:597] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.165486 22750 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.183547 22750 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.213663 22750 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.242264 23385 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:02.244413 22750 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.265678 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.276654 22750 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:02.280812 30335 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:02.281656 30335 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:02.286016 30335 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.289369 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:02.306074 22750 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.313752 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.323875 23384 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:02.324615 23384 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:02.325610 30334 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:13:02.334199 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:02.335126 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:02.358286 23385 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.375941 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:02.376724 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:02.389425 21526 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22731)
W20260709 13:13:02.435282 22750 consensus_peers.cc:597] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.438216 22750 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.450273 22750 consensus_peers.cc:597] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.465497 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.470669 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.477798 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.488135 22750 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.492628 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.506883 22750 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.519919 22750 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.526448 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:02.526820 22750 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.540904 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.577626 22750 consensus_peers.cc:597] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.625062 23385 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.626844 22750 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:02.645932 30335 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:02.646705 30335 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:02.649124 30335 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.662647 23384 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:02.664158 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:02.665372 23385 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:02.666280 30335 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:02.673642 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:02.678009 30335 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:02.678850 30335 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:02.688231 23384 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:02.691785 30335 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.699592 22750 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.702014 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:02.702656 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:02.703979 30335 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:02.747871 30333 raft_consensus.cc:493] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:02.748764 30333 raft_consensus.cc:515] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:13:02.758409 22750 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:02.760681 22749 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:02.761333 30333 leader_election.cc:290] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.770263 22749 leader_election.cc:336] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:02.770730 22750 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.777998 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:13:02.789613 22749 leader_election.cc:304] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:13:02.790923 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:13:02.791489 30333 raft_consensus.cc:2749] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:02.836634 23385 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:02.855820 22749 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:02.859220 22750 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:02.879626 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:02.880365 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:13:02.896780 30333 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:02.897377 30333 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:02.899816 30333 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.905346 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:02.932089 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:02.933992 22750 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:02.938838 30333 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:02.945215 22749 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:02.983951 30333 raft_consensus.cc:493] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:02.984577 30333 raft_consensus.cc:515] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:02.988371 30333 leader_election.cc:290] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:02.992619 22750 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:02.996397 22749 leader_election.cc:336] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:02.996922 22749 leader_election.cc:304] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:03.006688 30332 raft_consensus.cc:2749] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:13:03.079355 30334 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:03.080044 30334 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:03.085372 30334 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:03.092888 23384 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:03.103230 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:03.103893 23385 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:03.119585 30334 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:03.155776 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:13:03.205916 30333 raft_consensus.cc:493] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:03.206732 30333 raft_consensus.cc:515] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:03.212513 30333 leader_election.cc:290] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:03.240697 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:03.244645 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:03.261327 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:03.279590 22750 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:03.290342 22749 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:03.293112 22749 leader_election.cc:336] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:03.293706 22749 leader_election.cc:304] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:03.294596 30333 raft_consensus.cc:2749] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:03.325562 22749 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:03.390190 21526 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22731)
I20260709 13:13:03.557046 30341 raft_consensus.cc:493] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:03.557976 30341 raft_consensus.cc:515] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:03.562587 30341 leader_election.cc:290] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:03.582255 22750 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34681, user_credentials={real_user=slave}} blocked reactor thread for 171392us
W20260709 13:13:03.613418 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:03.630111 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:13:03.640321 30333 raft_consensus.cc:493] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:03.641043 30333 raft_consensus.cc:515] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:03.644930 30333 leader_election.cc:290] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:03.688975 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:03.742678 22749 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:03.749469 22750 leader_election.cc:336] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:03.762751 22750 leader_election.cc:304] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:03.772977 30339 raft_consensus.cc:2749] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:13:03.776377 22750 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:03.783033 22749 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:03.803453 22749 leader_election.cc:336] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:03.806908 22749 leader_election.cc:304] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:03.807809 30332 raft_consensus.cc:2749] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:03.840641 22750 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.119691 22750 consensus_peers.cc:597] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:04.120795 30332 raft_consensus.cc:493] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.121425 30332 raft_consensus.cc:515] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.123900 30332 leader_election.cc:290] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.131093 22750 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:04.139819 22749 leader_election.cc:336] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:04.140597 22749 leader_election.cc:304] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:04.141700 30332 raft_consensus.cc:2749] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:04.167342 22750 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:04.204912 30341 raft_consensus.cc:493] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.205554 30341 raft_consensus.cc:515] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.207942 30341 leader_election.cc:290] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.218330 22749 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:04.222748 22750 leader_election.cc:336] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:04.224054 22750 leader_election.cc:304] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:04.225253 30341 raft_consensus.cc:2749] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:13:04.250018 22750 consensus_peers.cc:597] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.269531 22749 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:04.271844 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:04.284034 30341 raft_consensus.cc:493] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.284797 30341 raft_consensus.cc:515] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.287516 30341 leader_election.cc:290] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.294952 22750 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:04.300230 22749 leader_election.cc:336] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:04.300768 22749 leader_election.cc:304] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:13:04.302060 22749 consensus_peers.cc:597] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:13:04.302937 30341 raft_consensus.cc:2749] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:13:04.398761 30332 raft_consensus.cc:493] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.399588 30332 raft_consensus.cc:515] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.428499 30332 leader_election.cc:290] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.490751 22749 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:04.516440 22750 leader_election.cc:336] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:04.517046 22750 leader_election.cc:304] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:04.517894 30332 raft_consensus.cc:493] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.518561 30332 raft_consensus.cc:515] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.521193 30332 leader_election.cc:290] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:04.534551 30332 raft_consensus.cc:2749] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:04.540277 23385 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.550661 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080036fba0 after lost signal to thread 21529
W20260709 13:13:04.552299 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800392ae0 after lost signal to thread 22567
W20260709 13:13:04.554257 22564 debug-util.cc:398] Leaking SignalData structure 0x7b080014ffc0 after lost signal to thread 22808
W20260709 13:13:04.556882 22749 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:04.573071 22564 debug-util.cc:398] Leaking SignalData structure 0x7b0800340160 after lost signal to thread 23445
W20260709 13:13:04.576508 22750 leader_election.cc:336] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:04.577194 22750 leader_election.cc:304] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:04.579459 30346 raft_consensus.cc:2749] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:04.626312 30345 raft_consensus.cc:493] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:04.627050 30345 raft_consensus.cc:515] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
W20260709 13:13:04.641439 22750 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:04.641984 30345 leader_election.cc:290] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.642333 22750 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.642976 22750 consensus_peers.cc:597] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.643474 22750 consensus_peers.cc:597] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.651486 22750 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:04.648039 22749 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:04.675629 22749 leader_election.cc:336] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:04.676304 22749 leader_election.cc:304] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
W20260709 13:13:04.677624 22749 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:13:04.678439 30332 raft_consensus.cc:2749] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:13:04.679404 23385 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:04.707542 30345 raft_consensus.cc:493] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
W20260709 13:13:04.708027 22750 consensus_peers.cc:597] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:04.708491 30345 raft_consensus.cc:515] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } }
I20260709 13:13:04.711006 30345 leader_election.cc:290] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:04.717458 22750 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:04.743058 23385 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.746709 23385 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.754228 23385 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.780936 22749 leader_election.cc:336] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:04.781561 22749 leader_election.cc:304] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0e9f7dc2a2e4bdb9287d910e4b491f7; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:04.788636 30332 raft_consensus.cc:2749] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:13:04.798557 22749 consensus_peers.cc:597] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:04.808220 22750 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.812791 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:04.814680 22749 consensus_peers.cc:597] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:04.890036 22750 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.904903 22750 consensus_peers.cc:597] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.912907 23385 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:13:04.935426 22750 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.973452 23385 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:04.991099 22750 consensus_peers.cc:597] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.002633 23385 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.010008 22750 consensus_peers.cc:597] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.043192 22749 consensus_peers.cc:597] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:13:05.051065 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
W20260709 13:13:05.079396 22749 consensus_peers.cc:597] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:05.104502 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:05.113155 22750 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.143597 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:05.172614 22749 consensus_peers.cc:597] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:05.185250 22750 consensus_peers.cc:597] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.221787 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:05.230844 23385 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:13:05.248097 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
W20260709 13:13:05.250563 22750 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.255553 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:05.258658 22749 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:05.271499 22750 consensus_peers.cc:597] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.280579 22750 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.313259 22750 consensus_peers.cc:597] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.321544 22750 consensus_peers.cc:597] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.322402 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:05.323621 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:05.326498 22749 consensus_peers.cc:597] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:05.327224 22749 consensus_peers.cc:597] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:05.327726 22749 consensus_peers.cc:597] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:05.332446 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:05.332171 22749 consensus_peers.cc:597] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:05.333849 22749 consensus_peers.cc:597] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:05.355026 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:05.361196 22750 consensus_peers.cc:597] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:05.399964 23384 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.403379 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:05.404841 21526 tablet_replica.cc:333] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.406078 21526 raft_consensus.cc:2243] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.408044 21526 raft_consensus.cc:2272] T 9e3855678a8148a09ffe99d7624c9b4f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.413807 21526 tablet_replica.cc:333] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.415102 21526 raft_consensus.cc:2243] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.416882 21526 raft_consensus.cc:2272] T a9a28904b6a34dbf81e5d1d4bb08a08c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.420631 21526 tablet_replica.cc:333] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.421844 21526 raft_consensus.cc:2243] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.422984 21526 raft_consensus.cc:2272] T f09c3bbe948b4c7a9f50df8ec4277e9f P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.427822 21526 tablet_replica.cc:333] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.429003 21526 raft_consensus.cc:2243] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.430222 21526 raft_consensus.cc:2272] T fbb33c4b77a2451499c9c5e49b83aa12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.438745 22749 consensus_peers.cc:597] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:05.440588 21526 tablet_replica.cc:333] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.441900 21526 raft_consensus.cc:2243] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.443648 21526 raft_consensus.cc:2272] T 14efc068101d4b50a525206c447e1ae8 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.447395 21526 tablet_replica.cc:333] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.448657 21526 raft_consensus.cc:2243] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.450915 21526 raft_consensus.cc:2272] T ef7d2e730e694ade8b424e4ff007a48a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.454761 21526 tablet_replica.cc:333] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.456113 21526 raft_consensus.cc:2243] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.457835 21526 raft_consensus.cc:2272] T 56c28a1350eb4e148e6e49cab764b300 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.461336 21526 tablet_replica.cc:333] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.462386 21526 raft_consensus.cc:2243] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.463864 21526 raft_consensus.cc:2272] T 51e41e0d49c846d0ae7c7d737a15be9e P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.468302 21526 tablet_replica.cc:333] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.469755 21526 raft_consensus.cc:2243] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Raft consensus shutting down.
W20260709 13:13:05.470276 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:05.471803 21526 raft_consensus.cc:2272] T ed3a2e4658ea473ba5c2b68313aeaf3b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.475152 21526 tablet_replica.cc:333] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
W20260709 13:13:05.476562 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:05.479100 21526 raft_consensus.cc:2243] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.480072 21526 raft_consensus.cc:2272] T bfb716d0bf184679a89d2dde7563865d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.487717 21526 tablet_replica.cc:333] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.488940 21526 raft_consensus.cc:2243] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.490296 21526 raft_consensus.cc:2272] T 8a4b81cf2b9c48b59c92dd3ace99c8db P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.494026 21526 tablet_replica.cc:333] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.495158 21526 raft_consensus.cc:2243] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.496034 21526 raft_consensus.cc:2272] T 22e83136ddd24d71ad143a6f5fc28dd9 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.499197 21526 tablet_replica.cc:333] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.501567 21526 raft_consensus.cc:2243] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.502514 21526 raft_consensus.cc:2272] T 7235e142d8ec45adb46a8c389a2962fa P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.505992 21526 tablet_replica.cc:333] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.507074 21526 raft_consensus.cc:2243] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.508145 21526 raft_consensus.cc:2272] T 0ebec7fc99cb448ea382f8b3e769e7ca P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.511687 21526 tablet_replica.cc:333] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
W20260709 13:13:05.518833 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:05.522575 21526 raft_consensus.cc:2243] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:13:05.524638 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.525851 21526 raft_consensus.cc:2272] T ca809087a6724923ab06ace347425d31 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.529273 21526 tablet_replica.cc:333] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.530440 21526 raft_consensus.cc:2243] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.531993 21526 raft_consensus.cc:2272] T 160d972ad1454f9db922aa360a0e7436 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.535437 21526 tablet_replica.cc:333] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.536495 21526 raft_consensus.cc:2243] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.537308 21526 raft_consensus.cc:2272] T d92b65b2cbd546e18ddac5ce049ea91c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.540539 21526 tablet_replica.cc:333] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.541625 21526 raft_consensus.cc:2243] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.543618 21526 raft_consensus.cc:2272] T b907163bc2174a2aa11df858ccae00ac P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.547194 21526 tablet_replica.cc:333] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.548249 21526 raft_consensus.cc:2243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.549877 21526 raft_consensus.cc:2272] T 17042e7fd6cb40cfb21cfdfb5121a4bc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.553272 21526 tablet_replica.cc:333] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.554839 21526 raft_consensus.cc:2243] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.556988 21526 raft_consensus.cc:2272] T af344e0ef1b1489a9bc301ba3b375d4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.561496 21526 tablet_replica.cc:333] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.562685 21526 raft_consensus.cc:2243] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.563987 21526 raft_consensus.cc:2272] T 9a4549b98fd04604afe4d21f9bb0acfe P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.567975 21526 tablet_replica.cc:333] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
W20260709 13:13:05.568568 22805 debug-util.cc:398] Leaking SignalData structure 0x7b08002a33c0 after lost signal to thread 21529
I20260709 13:13:05.569545 21526 raft_consensus.cc:2243] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:13:05.569940 22805 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc5c0 after lost signal to thread 22567
I20260709 13:13:05.571785 21526 raft_consensus.cc:2272] T 05be2cb1839f447296223033d364a7cc P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.573387 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080009e8a0 after lost signal to thread 22808
W20260709 13:13:05.575079 22805 debug-util.cc:398] Leaking SignalData structure 0x7b080039f5e0 after lost signal to thread 23445
I20260709 13:13:05.583146 21526 tablet_replica.cc:333] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.584517 21526 raft_consensus.cc:2243] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.585572 21526 raft_consensus.cc:2272] T 53c258bc80ce4d88bac804cb01a34b32 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.589763 21526 tablet_replica.cc:333] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.590826 21526 raft_consensus.cc:2243] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.592284 21526 raft_consensus.cc:2272] T 295bf86c5a444ebc9285e902fd5e9ae0 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.595511 21526 tablet_replica.cc:333] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.596508 21526 raft_consensus.cc:2243] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.597291 21526 raft_consensus.cc:2272] T b8291ea7e33a40b5afaebc02d6ce7e9d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.597553 23384 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.600857 21526 tablet_replica.cc:333] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.601939 21526 raft_consensus.cc:2243] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.603482 21526 raft_consensus.cc:2272] T a937bd6b17c04d5199f368b1d595cf8b P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.605427 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:05.606866 21526 tablet_replica.cc:333] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.608062 21526 raft_consensus.cc:2243] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.609617 21526 raft_consensus.cc:2272] T 887cfff73fab4d52b2e39f5b6089133a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.613130 21526 tablet_replica.cc:333] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
W20260709 13:13:05.617444 22750 consensus_peers.cc:597] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:13:05.625398 21526 raft_consensus.cc:2243] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.627350 21526 raft_consensus.cc:2272] T bf401fd06552416592129b334cdf8167 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.630645 21526 tablet_replica.cc:333] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.631507 21526 raft_consensus.cc:2243] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.633292 21526 raft_consensus.cc:2272] T 34f27615de694e7faaba89794bf9b5c6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.636364 21526 tablet_replica.cc:333] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.637236 21526 raft_consensus.cc:2243] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.638164 21526 raft_consensus.cc:2272] T 99c51f61e4f74c1593ebf98255a83b4c P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.646587 21526 tablet_replica.cc:333] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.647624 21526 raft_consensus.cc:2243] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.649113 21526 raft_consensus.cc:2272] T d6544eab6fd54f95baf389a8b15729cd P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.652720 21526 tablet_replica.cc:333] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.653592 21526 raft_consensus.cc:2243] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.654986 21526 raft_consensus.cc:2272] T 594d7db14b30467aac30a69dc1570d43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.658066 21526 tablet_replica.cc:333] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
W20260709 13:13:05.656418 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:05.659060 21526 raft_consensus.cc:2243] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.660144 21526 raft_consensus.cc:2272] T 91be8db7516b4bca939f439f045b40eb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.661942 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.663419 21526 tablet_replica.cc:333] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.664392 21526 raft_consensus.cc:2243] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.690501 21526 raft_consensus.cc:2272] T fee10603c0c042fd8e28683daeb86a99 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.694281 21526 tablet_replica.cc:333] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.695125 21526 raft_consensus.cc:2243] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.696995 21526 raft_consensus.cc:2272] T 12b8ec59bdc04cd8b5dae5bcc35466f6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.701455 21526 tablet_replica.cc:333] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.702283 21526 raft_consensus.cc:2243] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.703210 21526 raft_consensus.cc:2272] T 4a5cf575974c43e79448d00fab49d7ab P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.706015 21526 tablet_replica.cc:333] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.710817 21526 raft_consensus.cc:2243] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.717672 21526 raft_consensus.cc:2272] T 11f8cc3feaf345e397b685036c912e13 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.721021 21526 tablet_replica.cc:333] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.722005 21526 raft_consensus.cc:2243] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.722976 21526 raft_consensus.cc:2272] T a840ecd041fa46a198e4fb917ef79b2a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 11 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.726042 21526 tablet_replica.cc:333] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.726912 21526 raft_consensus.cc:2243] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Raft consensus shutting down.
W20260709 13:13:05.718111 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.728410 21526 raft_consensus.cc:2272] T 60c24d6125434f4caf1190c8bdba2f12 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.731674 21526 tablet_replica.cc:333] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.732527 21526 raft_consensus.cc:2243] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.733765 21526 raft_consensus.cc:2272] T 66c163cd962d4a568a04fc251b913b43 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.736888 21526 tablet_replica.cc:333] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.737728 21526 raft_consensus.cc:2243] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.738405 21526 raft_consensus.cc:2272] T 1d580d080ca644b2af67c9e48cbd3b78 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.741518 21526 tablet_replica.cc:333] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.742362 21526 raft_consensus.cc:2243] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.743207 21526 raft_consensus.cc:2272] T 3e474e7307e645259a4f6d201b451bef P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.744051 23384 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:05.744802 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:05.746240 21526 tablet_replica.cc:333] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.747105 21526 raft_consensus.cc:2243] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.749266 21526 raft_consensus.cc:2272] T 79ed72c313bf419bb01d45a59391a262 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.752214 21526 tablet_replica.cc:333] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.752975 21526 raft_consensus.cc:2243] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.754256 21526 raft_consensus.cc:2272] T 92329501c20843f68ffa5b3fdfc8f83d P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.757997 21526 tablet_replica.cc:333] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.758803 21526 raft_consensus.cc:2243] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.759487 21526 raft_consensus.cc:2272] T 71723544b43c4cb99dbeaf0321d14da6 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.762590 21526 tablet_replica.cc:333] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.763475 21526 raft_consensus.cc:2243] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.764595 21526 raft_consensus.cc:2272] T 9ac0029002284e64a5fde523ffaf77fb P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.777184 21526 tablet_replica.cc:333] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.777992 21526 raft_consensus.cc:2243] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.779423 21526 raft_consensus.cc:2272] T 227f18995af4452ab6e5db39ef780c8a P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.782564 21526 tablet_replica.cc:333] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.783368 21526 raft_consensus.cc:2243] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:05.784099 21526 raft_consensus.cc:2272] T 1918c6e39d7d46139a210c432e0f4d84 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:05.787250 21526 tablet_replica.cc:333] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7: stopping tablet replica
I20260709 13:13:05.787918 21526 raft_consensus.cc:2243] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:05.789299 21526 raft_consensus.cc:2272] T ffde8dbdd2d54723860d280e3e916e60 P a0e9f7dc2a2e4bdb9287d910e4b491f7 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:05.833182 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:13:06.003834 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:06.216136 30351 raft_consensus.cc:493] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:06.216907 30351 raft_consensus.cc:515] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:06.240211 23385 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:06.242349 30351 leader_election.cc:290] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:06.256459 23384 leader_election.cc:336] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:06.259155 23384 leader_election.cc:304] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:06.270808 30350 raft_consensus.cc:2749] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:06.570969 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:13:06.650506 30335 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.651291 30335 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.653640 30335 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:13:06.658933 30350 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.659584 30350 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.662096 30350 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:06.685348 23385 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:06.686148 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:06.693012 23384 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.702234 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:06.703392 30350 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:06.705065 23384 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.705652 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:06.711561 30350 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:06.827950 23384 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:13:06.871166 30335 raft_consensus.cc:493] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.871831 30335 raft_consensus.cc:515] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.878427 30350 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.879199 30350 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.883558 30350 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:06.891184 23385 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:06.895671 30335 leader_election.cc:290] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:06.905406 23385 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:06.908340 23384 leader_election.cc:336] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.913561 30350 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.914490 30350 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.917405 30350 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
I20260709 13:13:06.911116 23384 leader_election.cc:304] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
W20260709 13:13:06.919709 23384 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.927346 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:06.926569 30350 raft_consensus.cc:2749] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:06.929092 30335 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:06.931941 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22805)
W20260709 13:13:06.957669 23385 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:06.966925 30350 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:06.967698 30350 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:06.981473 30350 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:06.983239 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:06.982256 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:06.985241 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.985683 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:06.986508 30350 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:06.988824 23384 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:06.989370 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:06.990198 30350 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:13:06.997082 30335 raft_consensus.cc:493] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:06.997615 30335 raft_consensus.cc:515] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:07.005339 23384 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:06.999930 30335 leader_election.cc:290] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:07.011981 23385 leader_election.cc:336] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:07.012662 23385 leader_election.cc:304] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:07.013639 30335 raft_consensus.cc:2749] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:13:07.015599 30335 raft_consensus.cc:493] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:07.016258 30335 raft_consensus.cc:515] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:07.018847 30335 leader_election.cc:290] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:07.027998 23384 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
W20260709 13:13:07.041407 23385 leader_election.cc:336] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:07.042109 23385 leader_election.cc:304] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:07.045284 30335 raft_consensus.cc:2749] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:13:07.050434 23384 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111) [suppressed 627 similar messages]
I20260709 13:13:07.127280 30335 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:07.127988 30335 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:07.137580 23384 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
W20260709 13:13:07.143074 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:07.143781 23385 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:07.145426 30350 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:07.144333 30335 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
I20260709 13:13:07.160391 30335 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:07.161065 30335 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:07.170532 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:07.173298 30335 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:07.178793 23385 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:07.179334 23385 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:07.181033 30335 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:07.182412 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
W20260709 13:13:07.326289 23385 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:07.331485 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:07.350693 23385 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:07.383294 23385 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:07.440723 23385 consensus_peers.cc:597] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:07.455382 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:07.490082 23385 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:13:07.560873 23385 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:07.617897 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:07.669646 23384 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:07.701550 23385 consensus_peers.cc:597] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:13:07.742933 30335 raft_consensus.cc:493] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 91ebb78bf12f41baae84669c1effc423)
I20260709 13:13:07.743639 30335 raft_consensus.cc:515] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:07.752022 23385 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:07.753136 30335 leader_election.cc:290] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:07.755244 23384 consensus_peers.cc:597] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:07.775722 23384 leader_election.cc:336] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:07.776360 23384 leader_election.cc:304] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, 91ebb78bf12f41baae84669c1effc423
I20260709 13:13:07.777215 30335 raft_consensus.cc:2749] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:13:07.790915 23384 consensus_peers.cc:597] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:07.791860 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:07.856166 23384 consensus_peers.cc:597] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:07.875361 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:07.881538 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:07.897702 23385 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Couldn't send request to peer 91ebb78bf12f41baae84669c1effc423. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:13:07.904637 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:07.945073 23384 consensus_peers.cc:597] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:08.056519 23384 consensus_peers.cc:597] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:08.058315 23384 consensus_peers.cc:597] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:13:08.122591 23384 consensus_peers.cc:597] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:08.152716 23384 consensus_peers.cc:597] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:08.153710 23384 consensus_peers.cc:597] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:13:08.166977 23384 consensus_peers.cc:597] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:08.263437 23384 consensus_peers.cc:597] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:08.281373 30350 raft_consensus.cc:493] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.282119 30350 raft_consensus.cc:515] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.284626 30350 leader_election.cc:290] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.290565 23385 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:08.296804 23384 leader_election.cc:336] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:08.297437 23384 leader_election.cc:304] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:08.298290 30350 raft_consensus.cc:2749] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:13:08.331243 23384 consensus_peers.cc:597] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:13:08.413530 23384 consensus_peers.cc:597] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 -> Peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Couldn't send request to peer a0e9f7dc2a2e4bdb9287d910e4b491f7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:08.447283 23384 consensus_peers.cc:597] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:13:08.448374 30350 raft_consensus.cc:493] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.448921 30350 raft_consensus.cc:515] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.458420 30350 leader_election.cc:290] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.466255 23385 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:08.470149 30335 raft_consensus.cc:493] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.470770 30335 raft_consensus.cc:515] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.472775 30335 leader_election.cc:290] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.478385 23385 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:08.483628 23384 leader_election.cc:336] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:08.484277 23384 leader_election.cc:304] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:08.485113 30350 raft_consensus.cc:2749] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:13:08.493513 23384 leader_election.cc:336] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:08.494107 23384 leader_election.cc:304] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:08.495033 30350 raft_consensus.cc:2749] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:13:08.552779 23384 consensus_peers.cc:597] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 -> Peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Couldn't send request to peer 1f99ad34cc414233929e0a92f8c3d8db. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:13:08.687748 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:13:08.693055 30350 raft_consensus.cc:493] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.693755 30350 raft_consensus.cc:515] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.699326 30350 leader_election.cc:290] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.710192 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
W20260709 13:13:08.712011 23384 leader_election.cc:336] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:08.712522 23384 leader_election.cc:304] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 1f99ad34cc414233929e0a92f8c3d8db, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:08.713737 30350 raft_consensus.cc:2749] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:13:08.828779 30350 raft_consensus.cc:493] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.829495 30350 raft_consensus.cc:515] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } attrs { replace: true } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
W20260709 13:13:08.842111 23385 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:08.843433 30350 leader_election.cc:290] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.858453 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:08.859026 23384 leader_election.cc:304] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:08.859980 30350 raft_consensus.cc:2749] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:13:08.862773 23384 leader_election.cc:336] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 1f99ad34cc414233929e0a92f8c3d8db (127.21.5.129:37283): Network error: Client connection negotiation failed: client connection to 127.21.5.129:37283: connect: Connection refused (error 111)
I20260709 13:13:08.870784 21526 tablet_server.cc:179] TabletServer@127.21.5.132:0 shutting down...
I20260709 13:13:08.955879 30335 raft_consensus.cc:493] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.960740 30335 raft_consensus.cc:515] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.975692 30335 leader_election.cc:290] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681), a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075)
W20260709 13:13:08.976858 23385 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
I20260709 13:13:08.988530 30335 raft_consensus.cc:493] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a0e9f7dc2a2e4bdb9287d910e4b491f7)
I20260709 13:13:08.989760 30335 raft_consensus.cc:515] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: false } }
I20260709 13:13:08.997728 30335 leader_election.cc:290] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075), 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681)
W20260709 13:13:08.999094 23384 leader_election.cc:336] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:08.999744 23384 leader_election.cc:304] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:09.000720 30335 raft_consensus.cc:2749] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:13:09.001628 23385 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 91ebb78bf12f41baae84669c1effc423 (127.21.5.130:34681): Network error: Client connection negotiation failed: client connection to 127.21.5.130:34681: connect: Connection refused (error 111)
W20260709 13:13:09.003118 23384 leader_election.cc:336] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer a0e9f7dc2a2e4bdb9287d910e4b491f7 (127.21.5.131:39075): Network error: Client connection negotiation failed: client connection to 127.21.5.131:39075: connect: Connection refused (error 111)
I20260709 13:13:09.003618 23384 leader_election.cc:304] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a31ac6451ff4f73bee634487d84d001; no voters: 91ebb78bf12f41baae84669c1effc423, a0e9f7dc2a2e4bdb9287d910e4b491f7
I20260709 13:13:09.004496 30335 raft_consensus.cc:2749] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:13:09.017041 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:09.020481 21526 tablet_replica.cc:333] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.023881 21526 raft_consensus.cc:2243] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEARNER]: Raft consensus shutting down.
I20260709 13:13:09.033314 21526 pending_rounds.cc:70] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Trying to abort 1 pending ops.
I20260709 13:13:09.033826 21526 pending_rounds.cc:77] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001: Aborting op as it isn't in flight: id { term: 17 index: 12 } timestamp: 7305636864491016192 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "b8291ea7e33a40b5afaebc02d6ce7e9d" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } } new_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "1f99ad34cc414233929e0a92f8c3d8db" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 37283 } attrs { replace: true } } peers { permanent_uuid: "91ebb78bf12f41baae84669c1effc423" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34681 } } peers { permanent_uuid: "a0e9f7dc2a2e4bdb9287d910e4b491f7" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 39075 } } peers { permanent_uuid: "6a31ac6451ff4f73bee634487d84d001" member_type: NON_VOTER last_known_addr { host: "127.21.5.132" port: 36877 } attrs { promote: true } } } }
I20260709 13:13:09.036490 21526 raft_consensus.cc:2917] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEARNER]: Aborting config change with OpId 17.12: Aborted: Op aborted
I20260709 13:13:09.037442 21526 raft_consensus.cc:2889] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:13:09.037914 21526 raft_consensus.cc:2272] T b8291ea7e33a40b5afaebc02d6ce7e9d P 6a31ac6451ff4f73bee634487d84d001 [term 18 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 13:13:09.043998 21526 tablet_replica.cc:333] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.046213 21526 raft_consensus.cc:2243] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.047611 21526 raft_consensus.cc:2272] T 79ed72c313bf419bb01d45a59391a262 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.054402 21526 tablet_replica.cc:333] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.056519 21526 raft_consensus.cc:2243] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.061427 21526 raft_consensus.cc:2272] T 2f4adfa257c34bb88d79bda9d39c5146 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.066839 21526 tablet_replica.cc:333] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.068472 21526 raft_consensus.cc:2243] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.071583 21526 raft_consensus.cc:2272] T 53c258bc80ce4d88bac804cb01a34b32 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.076790 21526 tablet_replica.cc:333] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.080331 21526 raft_consensus.cc:2243] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.082360 21526 raft_consensus.cc:2272] T a937bd6b17c04d5199f368b1d595cf8b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.086978 21526 tablet_replica.cc:333] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.089073 21526 raft_consensus.cc:2243] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.090438 21526 raft_consensus.cc:2272] T ffde8dbdd2d54723860d280e3e916e60 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.094559 21526 tablet_replica.cc:333] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.100734 21526 raft_consensus.cc:2243] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.103196 21526 raft_consensus.cc:2272] T 9ac0029002284e64a5fde523ffaf77fb P 6a31ac6451ff4f73bee634487d84d001 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.107784 21526 tablet_replica.cc:333] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.108922 21526 raft_consensus.cc:2243] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.111764 21526 raft_consensus.cc:2272] T 308af34f5c0144e185e0af82342c7f92 P 6a31ac6451ff4f73bee634487d84d001 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.116392 21526 tablet_replica.cc:333] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.117673 21526 raft_consensus.cc:2243] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.120638 21526 raft_consensus.cc:2272] T 7235e142d8ec45adb46a8c389a2962fa P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.125703 21526 tablet_replica.cc:333] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.127111 21526 raft_consensus.cc:2243] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.129400 21526 raft_consensus.cc:2272] T 22e83136ddd24d71ad143a6f5fc28dd9 P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.133391 21526 tablet_replica.cc:333] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.134490 21526 raft_consensus.cc:2243] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.135623 21526 raft_consensus.cc:2272] T 887cfff73fab4d52b2e39f5b6089133a P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.140117 21526 tablet_replica.cc:333] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.141597 21526 raft_consensus.cc:2243] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.143983 21526 raft_consensus.cc:2272] T ec9764b0b7dd4b67a5b50a652203c60a P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.149024 21526 tablet_replica.cc:333] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.150277 21526 raft_consensus.cc:2243] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.153383 21526 raft_consensus.cc:2272] T 4f6afadc577c4fd08e4345a82ae50e5f P 6a31ac6451ff4f73bee634487d84d001 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.157966 21526 tablet_replica.cc:333] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.159126 21526 raft_consensus.cc:2243] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.159942 21526 raft_consensus.cc:2272] T fee10603c0c042fd8e28683daeb86a99 P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.164120 21526 tablet_replica.cc:333] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.165155 21526 raft_consensus.cc:2243] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.167301 21526 raft_consensus.cc:2272] T e6d9cbf659974b358ba6d364ca9ca431 P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.171639 21526 tablet_replica.cc:333] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.172735 21526 raft_consensus.cc:2243] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.174206 21526 raft_consensus.cc:2272] T 11f8cc3feaf345e397b685036c912e13 P 6a31ac6451ff4f73bee634487d84d001 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.178071 21526 tablet_replica.cc:333] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.179198 21526 raft_consensus.cc:2243] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.181833 21526 raft_consensus.cc:2272] T d92b65b2cbd546e18ddac5ce049ea91c P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.186022 21526 tablet_replica.cc:333] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.187103 21526 raft_consensus.cc:2243] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.189253 21526 raft_consensus.cc:2272] T cc8b169704cc4157ad021d774f83419e P 6a31ac6451ff4f73bee634487d84d001 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.193652 21526 tablet_replica.cc:333] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.194796 21526 raft_consensus.cc:2243] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.197484 21526 raft_consensus.cc:2272] T f33fee9d64384b499e55f8bc58339f69 P 6a31ac6451ff4f73bee634487d84d001 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.201686 21526 tablet_replica.cc:333] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.202586 21526 raft_consensus.cc:2243] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.203716 21526 raft_consensus.cc:2272] T ed3a2e4658ea473ba5c2b68313aeaf3b P 6a31ac6451ff4f73bee634487d84d001 [term 12 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.207850 21526 tablet_replica.cc:333] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.208788 21526 raft_consensus.cc:2243] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.211606 21526 raft_consensus.cc:2272] T 91be8db7516b4bca939f439f045b40eb P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.215735 21526 tablet_replica.cc:333] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.216775 21526 raft_consensus.cc:2243] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.218578 21526 raft_consensus.cc:2272] T a79a6f618f4944b1a11b9c1df766ae7f P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.222496 21526 tablet_replica.cc:333] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.223613 21526 raft_consensus.cc:2243] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.225478 21526 raft_consensus.cc:2272] T 26e6130713d74c0ea9492f5151053c42 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.229529 21526 tablet_replica.cc:333] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.230518 21526 raft_consensus.cc:2243] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.232419 21526 raft_consensus.cc:2272] T 66c163cd962d4a568a04fc251b913b43 P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.236872 21526 tablet_replica.cc:333] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.237814 21526 raft_consensus.cc:2243] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.238827 21526 raft_consensus.cc:2272] T 160d972ad1454f9db922aa360a0e7436 P 6a31ac6451ff4f73bee634487d84d001 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.242861 21526 tablet_replica.cc:333] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.243887 21526 raft_consensus.cc:2243] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.245095 21526 raft_consensus.cc:2272] T 17042e7fd6cb40cfb21cfdfb5121a4bc P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.249125 21526 tablet_replica.cc:333] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.250177 21526 raft_consensus.cc:2243] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.252372 21526 raft_consensus.cc:2272] T 9e3855678a8148a09ffe99d7624c9b4f P 6a31ac6451ff4f73bee634487d84d001 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.255764 21526 tablet_replica.cc:333] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.256611 21526 raft_consensus.cc:2243] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.258818 21526 raft_consensus.cc:2272] T 99c51f61e4f74c1593ebf98255a83b4c P 6a31ac6451ff4f73bee634487d84d001 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.263175 21526 tablet_replica.cc:333] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.264004 21526 raft_consensus.cc:2243] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 LEADER]: Raft consensus shutting down.
I20260709 13:13:09.265744 21526 raft_consensus.cc:2272] T 1d580d080ca644b2af67c9e48cbd3b78 P 6a31ac6451ff4f73bee634487d84d001 [term 13 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:09.269663 21526 tablet_replica.cc:333] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001: stopping tablet replica
I20260709 13:13:09.270529 21526 raft_consensus.cc:2243] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:09.271795 21526 raft_consensus.cc:2272] T c015ab1d14c443258576403d9d39c485 P 6a31ac6451ff4f73bee634487d84d001 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:10.382032 23442 debug-util.cc:398] Leaking SignalData structure 0x7b0800054a20 after lost signal to thread 21529
W20260709 13:13:10.383340 23442 debug-util.cc:398] Leaking SignalData structure 0x7b08002f1b60 after lost signal to thread 22567
W20260709 13:13:10.383996 23442 debug-util.cc:398] Leaking SignalData structure 0x7b080039f600 after lost signal to thread 23445
W20260709 13:13:10.425283 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 23442)
I20260709 13:13:11.238380 21526 tablet_server.cc:196] TabletServer@127.21.5.132:0 shutdown complete.
I20260709 13:13:11.312606 21526 master.cc:562] Master@127.21.5.190:44025 shutting down...
W20260709 13:13:11.360041 22581 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:13:11.360486 22581 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:13:11.360743 22581 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
I20260709 13:13:11.366890 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:11.367841 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:11.368245 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 71605ba8587a4b5193d8d69d9967c1af: stopping tablet replica
I20260709 13:13:11.533932 21526 master.cc:584] Master@127.21.5.190:44025 shutdown complete.
I20260709 13:13:11.592809 21526 test_util.cc:182] -----------------------------------------------
I20260709 13:13:11.593092 21526 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602506348444-21526-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (260714 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:13:11.597317 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:45121
I20260709 13:13:11.599005 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:11.606298 30357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:11.606494 30358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:11.607129 30360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:11.609485 21526 server_base.cc:1061] running on GCE node
I20260709 13:13:11.614136 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:11.614456 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:11.614598 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602791614586 us; error 0 us; skew 500 ppm
I20260709 13:13:11.615264 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:11.618268 21526 webserver.cc:533] Webserver started at http://127.21.5.190:42833/ using document root <none> and password file <none>
I20260709 13:13:11.618981 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:11.619189 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:11.619467 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:11.621101 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "9e2d0e882b2940d8b150869e4b2b2e79"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.627269 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:13:11.631369 30365 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:11.632249 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:13:11.632613 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "9e2d0e882b2940d8b150869e4b2b2e79"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.633004 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:11.654403 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:11.655949 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:11.703365 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:45121
I20260709 13:13:11.703500 30416 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:45121 every 8 connection(s)
I20260709 13:13:11.708166 30417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:11.721014 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Bootstrap starting.
I20260709 13:13:11.726936 30417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:11.732416 30417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: No bootstrap required, opened a new log
I20260709 13:13:11.735383 30417 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER }
I20260709 13:13:11.735860 30417 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:11.736156 30417 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e2d0e882b2940d8b150869e4b2b2e79, State: Initialized, Role: FOLLOWER
I20260709 13:13:11.736918 30417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER }
I20260709 13:13:11.737458 30417 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:13:11.737701 30417 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:13:11.737995 30417 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:11.744095 30417 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER }
I20260709 13:13:11.744804 30417 leader_election.cc:304] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e2d0e882b2940d8b150869e4b2b2e79; no voters: 
I20260709 13:13:11.746232 30417 leader_election.cc:290] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:13:11.746868 30420 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:11.748747 30420 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 1 LEADER]: Becoming Leader. State: Replica: 9e2d0e882b2940d8b150869e4b2b2e79, State: Running, Role: LEADER
I20260709 13:13:11.749788 30417 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:13:11.749712 30420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER }
I20260709 13:13:11.755491 30421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER } }
I20260709 13:13:11.757465 30421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:11.756651 30422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e2d0e882b2940d8b150869e4b2b2e79. Latest consensus state: current_term: 1 leader_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e2d0e882b2940d8b150869e4b2b2e79" member_type: VOTER } }
I20260709 13:13:11.758709 30422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [sys.catalog]: This master's current role is: LEADER
I20260709 13:13:11.762378 30430 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:13:11.771260 30430 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:13:11.772545 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:13:11.782032 30430 catalog_manager.cc:1383] Generated new cluster ID: 556ae0dc614a4b3998099d2daabaa57f
I20260709 13:13:11.782426 30430 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:13:11.817098 30430 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:13:11.818763 30430 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:13:11.833539 30430 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Generated new TSK 0
I20260709 13:13:11.834297 30430 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:13:11.839655 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:11.847005 30438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:11.848359 30439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:11.850314 30441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:11.851280 21526 server_base.cc:1061] running on GCE node
I20260709 13:13:11.854279 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:11.854529 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:11.854753 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602791854718 us; error 0 us; skew 500 ppm
I20260709 13:13:11.855320 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:11.859172 21526 webserver.cc:533] Webserver started at http://127.21.5.129:42141/ using document root <none> and password file <none>
I20260709 13:13:11.859927 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:11.860203 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:11.860581 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:11.862082 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "015ca80eba674552b34856a4f20271f1"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.867473 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.002s
I20260709 13:13:11.871364 30446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:11.872344 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:13:11.872696 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "015ca80eba674552b34856a4f20271f1"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.873082 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:11.894466 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:11.896055 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:11.897903 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:11.900933 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:11.901216 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:11.901577 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:11.901789 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:11.955808 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:41021
I20260709 13:13:11.955938 30508 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:41021 every 8 connection(s)
I20260709 13:13:11.961776 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:11.971928 30514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:11.972077 30513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:11.975755 21526 server_base.cc:1061] running on GCE node
W20260709 13:13:11.976019 30516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:11.979722 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:13:11.979902 30509 heartbeater.cc:344] Connected to a master server at 127.21.5.190:45121
W20260709 13:13:11.980089 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:11.980320 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602791980300 us; error 0 us; skew 500 ppm
I20260709 13:13:11.980406 30509 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:11.981210 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:11.981492 30509 heartbeater.cc:507] Master 127.21.5.190:45121 requested a full tablet report, sending...
I20260709 13:13:11.984318 21526 webserver.cc:533] Webserver started at http://127.21.5.130:36961/ using document root <none> and password file <none>
I20260709 13:13:11.984300 30382 ts_manager.cc:194] Registered new tserver with Master: 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:11.985019 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:11.985275 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:11.985569 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:11.986672 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58826
I20260709 13:13:11.987062 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "5a03a8214bba4eb1b0ff00895c565564"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.992949 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:13:11.997243 30521 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:11.998272 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:13:11.998759 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "5a03a8214bba4eb1b0ff00895c565564"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:11.999191 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:12.010604 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:12.012166 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:12.013898 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:12.016515 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:12.016781 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.017056 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:12.017241 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.066805 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:39473
I20260709 13:13:12.066942 30583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:39473 every 8 connection(s)
I20260709 13:13:12.072572 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:12.081786 30587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:12.084157 30588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:12.085980 30590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:12.088387 30584 heartbeater.cc:344] Connected to a master server at 127.21.5.190:45121
I20260709 13:13:12.088871 30584 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:12.089114 21526 server_base.cc:1061] running on GCE node
I20260709 13:13:12.089975 30584 heartbeater.cc:507] Master 127.21.5.190:45121 requested a full tablet report, sending...
I20260709 13:13:12.090374 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:12.090708 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:12.090936 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602792090914 us; error 0 us; skew 500 ppm
I20260709 13:13:12.091665 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:12.092723 30382 ts_manager.cc:194] Registered new tserver with Master: 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:12.094976 21526 webserver.cc:533] Webserver started at http://127.21.5.131:46795/ using document root <none> and password file <none>
I20260709 13:13:12.095054 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58828
I20260709 13:13:12.095844 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:12.096118 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:12.096436 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:12.098215 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "acf852f85f47483d99cbe4020baee627"
format_stamp: "Formatted at 2026-07-09 13:13:12 on dist-test-slave-h0rz"
I20260709 13:13:12.103955 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:13:12.108075 30595 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.108974 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:13:12.109357 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "acf852f85f47483d99cbe4020baee627"
format_stamp: "Formatted at 2026-07-09 13:13:12 on dist-test-slave-h0rz"
I20260709 13:13:12.109695 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:12.134699 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:12.136193 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:12.137694 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:12.140410 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:12.140699 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.140986 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:12.141182 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.189893 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:45277
I20260709 13:13:12.190013 30657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:45277 every 8 connection(s)
I20260709 13:13:12.196798 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:12.207427 30661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:12.208942 30662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:12.212702 30664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:12.212941 30658 heartbeater.cc:344] Connected to a master server at 127.21.5.190:45121
I20260709 13:13:12.213830 30658 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:12.214545 21526 server_base.cc:1061] running on GCE node
I20260709 13:13:12.215044 30658 heartbeater.cc:507] Master 127.21.5.190:45121 requested a full tablet report, sending...
I20260709 13:13:12.215988 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:12.216404 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:12.216643 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602792216619 us; error 0 us; skew 500 ppm
I20260709 13:13:12.217545 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:13:12.218200 30382 ts_manager.cc:194] Registered new tserver with Master: acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:12.220695 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58834
I20260709 13:13:12.221163 21526 webserver.cc:533] Webserver started at http://127.21.5.132:35711/ using document root <none> and password file <none>
I20260709 13:13:12.222007 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:12.222290 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:13:12.222725 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:13:12.224555 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-3-root/instance:
uuid: "6b393e79ded4461ea5da7b6b6063305f"
format_stamp: "Formatted at 2026-07-09 13:13:12 on dist-test-slave-h0rz"
I20260709 13:13:12.239778 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.015s	user 0.008s	sys 0.008s
I20260709 13:13:12.246090 30669 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.247206 21526 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.000s
I20260709 13:13:12.247629 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
uuid: "6b393e79ded4461ea5da7b6b6063305f"
format_stamp: "Formatted at 2026-07-09 13:13:12 on dist-test-slave-h0rz"
I20260709 13:13:12.247984 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:12.274925 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:12.276497 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:12.277959 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:12.280447 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:13:12.280704 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.281021 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:13:12.281198 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:12.330681 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.132:35425
I20260709 13:13:12.330780 30731 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.132:35425 every 8 connection(s)
I20260709 13:13:12.346199 30732 heartbeater.cc:344] Connected to a master server at 127.21.5.190:45121
I20260709 13:13:12.346743 30732 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:12.347604 30732 heartbeater.cc:507] Master 127.21.5.190:45121 requested a full tablet report, sending...
I20260709 13:13:12.349993 30382 ts_manager.cc:194] Registered new tserver with Master: 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:12.350255 21526 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016478973s
I20260709 13:13:12.351907 30382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58838
I20260709 13:13:12.381588 30382 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58846:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:13:12.640931 30549 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7844be2162490091882b0a0344414c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:12.643056 30549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7844be2162490091882b0a0344414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.654201 30548 tablet_service.cc:1511] Processing CreateTablet for tablet c4358103a0de4d8d88d2fa7c86875470 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:12.656227 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4358103a0de4d8d88d2fa7c86875470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.658074 30623 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7844be2162490091882b0a0344414c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:12.660040 30623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7844be2162490091882b0a0344414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.660373 30473 tablet_service.cc:1511] Processing CreateTablet for tablet c4358103a0de4d8d88d2fa7c86875470 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:12.661971 30473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4358103a0de4d8d88d2fa7c86875470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.658290 30474 tablet_service.cc:1511] Processing CreateTablet for tablet 6c7844be2162490091882b0a0344414c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:13:12.663802 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c7844be2162490091882b0a0344414c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.665546 30622 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc9ab09182445eaeceb5ea60a2ebfa (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:12.667297 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc9ab09182445eaeceb5ea60a2ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.667697 30472 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc9ab09182445eaeceb5ea60a2ebfa (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:12.669402 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc9ab09182445eaeceb5ea60a2ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.676069 30697 tablet_service.cc:1511] Processing CreateTablet for tablet c4358103a0de4d8d88d2fa7c86875470 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:13:12.678010 30697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4358103a0de4d8d88d2fa7c86875470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.679646 30547 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3e983dd2fc4a56857ff13e699d9517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:12.681519 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3e983dd2fc4a56857ff13e699d9517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.679937 30696 tablet_service.cc:1511] Processing CreateTablet for tablet d4bc9ab09182445eaeceb5ea60a2ebfa (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:13:12.687181 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4bc9ab09182445eaeceb5ea60a2ebfa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.699316 30621 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3e983dd2fc4a56857ff13e699d9517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:12.700433 30695 tablet_service.cc:1511] Processing CreateTablet for tablet 0e3e983dd2fc4a56857ff13e699d9517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:13:12.701159 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3e983dd2fc4a56857ff13e699d9517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.702070 30695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e3e983dd2fc4a56857ff13e699d9517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.705622 30471 tablet_service.cc:1511] Processing CreateTablet for tablet fd1749966f254b6a8c54ddfd73cb8d14 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:12.706708 30546 tablet_service.cc:1511] Processing CreateTablet for tablet fd1749966f254b6a8c54ddfd73cb8d14 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:12.707701 30471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1749966f254b6a8c54ddfd73cb8d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.708506 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1749966f254b6a8c54ddfd73cb8d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.724380 30620 tablet_service.cc:1511] Processing CreateTablet for tablet fd1749966f254b6a8c54ddfd73cb8d14 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:13:12.726183 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd1749966f254b6a8c54ddfd73cb8d14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.726404 30694 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf727a8c7b4599bb21be645db6983f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:12.728050 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf727a8c7b4599bb21be645db6983f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.733450 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf727a8c7b4599bb21be645db6983f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:12.735467 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf727a8c7b4599bb21be645db6983f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.758065 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf727a8c7b4599bb21be645db6983f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:13:12.759922 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf727a8c7b4599bb21be645db6983f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:12.763935 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:12.764264 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 471d4059785144ffa1ac2459a1f183fb (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:12.765858 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471d4059785144ffa1ac2459a1f183fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.767007 30544 tablet_service.cc:1511] Processing CreateTablet for tablet c05f1e9839ab444dbb4bcf0fa071cfc5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:12.768613 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05f1e9839ab444dbb4bcf0fa071cfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.771955 30751 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:12.775807 30693 tablet_service.cc:1511] Processing CreateTablet for tablet 471d4059785144ffa1ac2459a1f183fb (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:12.777354 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471d4059785144ffa1ac2459a1f183fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.782509 30692 tablet_service.cc:1511] Processing CreateTablet for tablet c05f1e9839ab444dbb4bcf0fa071cfc5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:12.783861 30752 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:12.784399 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05f1e9839ab444dbb4bcf0fa071cfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.788664 30468 tablet_service.cc:1511] Processing CreateTablet for tablet 907e974a334846208c70f0b813199df0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:12.790437 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907e974a334846208c70f0b813199df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.796073 30549 tablet_service.cc:1511] Processing CreateTablet for tablet 907e974a334846208c70f0b813199df0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:12.797742 30549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907e974a334846208c70f0b813199df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.797946 30623 tablet_service.cc:1511] Processing CreateTablet for tablet c05f1e9839ab444dbb4bcf0fa071cfc5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:13:12.799604 30623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c05f1e9839ab444dbb4bcf0fa071cfc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.803771 30751 tablet_bootstrap.cc:654] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:12.810199 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 907e974a334846208c70f0b813199df0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:13:12.811970 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 907e974a334846208c70f0b813199df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.813894 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 471d4059785144ffa1ac2459a1f183fb (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:13:12.819582 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 471d4059785144ffa1ac2459a1f183fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.826448 30754 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:12.828821 30691 tablet_service.cc:1511] Processing CreateTablet for tablet ef183e787ad14ee6bef37037a6ba59c0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:12.832005 30543 tablet_service.cc:1511] Processing CreateTablet for tablet ef183e787ad14ee6bef37037a6ba59c0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:12.833642 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef183e787ad14ee6bef37037a6ba59c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.837908 30752 tablet_bootstrap.cc:654] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:12.830543 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef183e787ad14ee6bef37037a6ba59c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.838563 30469 tablet_service.cc:1511] Processing CreateTablet for tablet ef183e787ad14ee6bef37037a6ba59c0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:13:12.844859 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef183e787ad14ee6bef37037a6ba59c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.854346 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 22a37ee3d67e45c394c7f5963e5767ed (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:12.856093 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a37ee3d67e45c394c7f5963e5767ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.855876 30690 tablet_service.cc:1511] Processing CreateTablet for tablet 22a37ee3d67e45c394c7f5963e5767ed (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:12.857350 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a37ee3d67e45c394c7f5963e5767ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.877777 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 22a37ee3d67e45c394c7f5963e5767ed (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:13:12.879500 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22a37ee3d67e45c394c7f5963e5767ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.945904 30751 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:12.946658 30751 ts_tablet_manager.cc:1403] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.175s	user 0.032s	sys 0.034s
I20260709 13:13:12.948701 30753 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:12.954655 30752 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:12.955417 30752 ts_tablet_manager.cc:1403] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.172s	user 0.030s	sys 0.031s
I20260709 13:13:12.958159 30751 raft_consensus.cc:359] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:12.959160 30751 raft_consensus.cc:385] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:12.958957 30752 raft_consensus.cc:359] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:12.959578 30751 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:12.959889 30752 raft_consensus.cc:385] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:12.960278 30752 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:12.960767 30751 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:12.965735 30752 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:12.942731 30754 tablet_bootstrap.cc:654] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:12.974082 30584 heartbeater.cc:499] Master 127.21.5.190:45121 was elected leader, sending a full tablet report...
I20260709 13:13:12.978693 30752 ts_tablet_manager.cc:1434] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.001s
I20260709 13:13:12.978258 30622 tablet_service.cc:1511] Processing CreateTablet for tablet a55a2b63593d4202b9b70bac4a5ef28b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:12.979615 30658 heartbeater.cc:499] Master 127.21.5.190:45121 was elected leader, sending a full tablet report...
I20260709 13:13:12.979988 30752 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:12.980516 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55a2b63593d4202b9b70bac4a5ef28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:12.981310 30545 tablet_service.cc:1511] Processing CreateTablet for tablet a55a2b63593d4202b9b70bac4a5ef28b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:12.988041 30753 tablet_bootstrap.cc:654] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:12.991520 30509 heartbeater.cc:499] Master 127.21.5.190:45121 was elected leader, sending a full tablet report...
I20260709 13:13:12.991763 30751 ts_tablet_manager.cc:1434] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.045s	user 0.014s	sys 0.005s
I20260709 13:13:12.996922 30693 tablet_service.cc:1511] Processing CreateTablet for tablet a55a2b63593d4202b9b70bac4a5ef28b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:13:12.999976 30752 tablet_bootstrap.cc:654] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.009652 30472 tablet_service.cc:1511] Processing CreateTablet for tablet d847606257cc4750be11b056651106a6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:13.011305 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d847606257cc4750be11b056651106a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.009652 30547 tablet_service.cc:1511] Processing CreateTablet for tablet d847606257cc4750be11b056651106a6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:13.023252 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55a2b63593d4202b9b70bac4a5ef28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.026082 30751 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:13.040084 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a55a2b63593d4202b9b70bac4a5ef28b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.027485 30620 tablet_service.cc:1511] Processing CreateTablet for tablet d847606257cc4750be11b056651106a6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:13:13.039649 30754 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:13.051775 30754 ts_tablet_manager.cc:1403] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.226s	user 0.014s	sys 0.016s
I20260709 13:13:13.051972 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d847606257cc4750be11b056651106a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.051028 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d847606257cc4750be11b056651106a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.054440 30694 tablet_service.cc:1511] Processing CreateTablet for tablet bf65c2e5cf5b41a2956231a0868f2a1f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:13.055365 30757 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.056116 30757 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.057471 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf65c2e5cf5b41a2956231a0868f2a1f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:13.072652 30585 tablet.cc:2406] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:13:13.073635 30757 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:13.083662 30470 tablet_service.cc:1511] Processing CreateTablet for tablet bf65c2e5cf5b41a2956231a0868f2a1f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:13.085358 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf65c2e5cf5b41a2956231a0868f2a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.072329 30754 raft_consensus.cc:359] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.094178 30751 tablet_bootstrap.cc:654] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.090754 30754 raft_consensus.cc:385] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.101401 30754 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.106539 30753 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:13.113123 30753 ts_tablet_manager.cc:1403] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.165s	user 0.033s	sys 0.027s
I20260709 13:13:13.117056 30753 raft_consensus.cc:359] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.118014 30753 raft_consensus.cc:385] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.124100 30752 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:13.124918 30752 ts_tablet_manager.cc:1403] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.145s	user 0.011s	sys 0.011s
I20260709 13:13:13.118359 30753 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.127627 30753 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.135352 30754 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.136294 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0922dea2d4414ca94fe5f85f65ca8b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:13.139144 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0922dea2d4414ca94fe5f85f65ca8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.145660 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:13.147931 30523 leader_election.cc:343] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.155256 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:13.155146 30547 tablet_service.cc:1511] Processing CreateTablet for tablet bf65c2e5cf5b41a2956231a0868f2a1f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:13:13.156950 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf65c2e5cf5b41a2956231a0868f2a1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.158607 30694 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0922dea2d4414ca94fe5f85f65ca8b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:13.160018 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 1d0922dea2d4414ca94fe5f85f65ca8b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:13:13.161265 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0922dea2d4414ca94fe5f85f65ca8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.161739 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d0922dea2d4414ca94fe5f85f65ca8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:13.157754 30522 leader_election.cc:343] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.169924 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:13.176183 30757 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:13.190402 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 490460505b2d424286323b79fb9fb2bc (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:13.191149 30691 tablet_service.cc:1511] Processing CreateTablet for tablet 490460505b2d424286323b79fb9fb2bc (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:13.192392 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490460505b2d424286323b79fb9fb2bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.192655 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490460505b2d424286323b79fb9fb2bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.193581 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 6f217397202e476a967c268887497cf5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:13.194991 30732 heartbeater.cc:499] Master 127.21.5.190:45121 was elected leader, sending a full tablet report...
I20260709 13:13:13.195819 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f217397202e476a967c268887497cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.208811 30544 tablet_service.cc:1511] Processing CreateTablet for tablet 490460505b2d424286323b79fb9fb2bc (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:13:13.211138 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 490460505b2d424286323b79fb9fb2bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.219671 30754 ts_tablet_manager.cc:1434] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.168s	user 0.012s	sys 0.012s
I20260709 13:13:13.221220 30754 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:13.224561 30752 raft_consensus.cc:359] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.225441 30751 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:13.225581 30752 raft_consensus.cc:385] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.226182 30752 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.226727 30751 ts_tablet_manager.cc:1403] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.201s	user 0.020s	sys 0.020s
I20260709 13:13:13.229364 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 6f217397202e476a967c268887497cf5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:13.230479 30751 raft_consensus.cc:359] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.231544 30751 raft_consensus.cc:385] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.231093 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f217397202e476a967c268887497cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.231920 30751 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.235033 30753 ts_tablet_manager.cc:1434] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.121s	user 0.011s	sys 0.001s
I20260709 13:13:13.236378 30753 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:13.243209 30754 tablet_bootstrap.cc:654] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.251325 30751 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.251797 30693 tablet_service.cc:1511] Processing CreateTablet for tablet 87824e2fdc9d4d33885d41c010f7bec4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:13.253293 30474 tablet_service.cc:1511] Processing CreateTablet for tablet 87824e2fdc9d4d33885d41c010f7bec4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:13.255353 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87824e2fdc9d4d33885d41c010f7bec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.260154 30751 ts_tablet_manager.cc:1434] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.001s
I20260709 13:13:13.261261 30546 tablet_service.cc:1511] Processing CreateTablet for tablet 87824e2fdc9d4d33885d41c010f7bec4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:13:13.262869 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87824e2fdc9d4d33885d41c010f7bec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.265130 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 9657733894c94f86ac3b047814fb8fd9 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:13.266759 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9657733894c94f86ac3b047814fb8fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.268285 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 9657733894c94f86ac3b047814fb8fd9 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:13.275667 30751 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:13.277195 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 6f217397202e476a967c268887497cf5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:13:13.281533 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 9657733894c94f86ac3b047814fb8fd9 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:13:13.254096 30752 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.283034 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9657733894c94f86ac3b047814fb8fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.283317 30758 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.283972 30758 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.284672 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f217397202e476a967c268887497cf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.294270 30751 tablet_bootstrap.cc:654] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.253304 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87824e2fdc9d4d33885d41c010f7bec4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.299870 30758 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:13.301716 30545 tablet_service.cc:1511] Processing CreateTablet for tablet e1452d68baa440e39ae0dad239d033c8 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:13.302404 30754 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:13.307286 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9657733894c94f86ac3b047814fb8fd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.306106 30696 tablet_service.cc:1511] Processing CreateTablet for tablet e1452d68baa440e39ae0dad239d033c8 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:13.308740 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1452d68baa440e39ae0dad239d033c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.307319 30754 ts_tablet_manager.cc:1403] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.086s	user 0.006s	sys 0.012s
I20260709 13:13:13.310880 30752 ts_tablet_manager.cc:1434] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.186s	user 0.009s	sys 0.004s
I20260709 13:13:13.315873 30754 raft_consensus.cc:359] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.316874 30754 raft_consensus.cc:385] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.317246 30754 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.318176 30754 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.320847 30754 ts_tablet_manager.cc:1434] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:13:13.333101 30754 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:13.304778 30751 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:13.331246 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 5de1dd97d9f342288201946ead3a9445 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:13:13.347837 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de1dd97d9f342288201946ead3a9445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.347308 30751 ts_tablet_manager.cc:1403] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.002s
I20260709 13:13:13.351563 30690 tablet_service.cc:1511] Processing CreateTablet for tablet 97d1629ffbcd48d1b72c843365f37329 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:13.353139 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d1629ffbcd48d1b72c843365f37329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.327457 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1452d68baa440e39ae0dad239d033c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.332903 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 5de1dd97d9f342288201946ead3a9445 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:13:13.354588 30751 raft_consensus.cc:359] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.356317 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de1dd97d9f342288201946ead3a9445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.372927 30751 raft_consensus.cc:385] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.373402 30751 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.372031 30474 tablet_service.cc:1511] Processing CreateTablet for tablet 97d1629ffbcd48d1b72c843365f37329 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:13.346884 30752 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:13.380985 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d1629ffbcd48d1b72c843365f37329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.381971 30547 tablet_service.cc:1511] Processing CreateTablet for tablet 97d1629ffbcd48d1b72c843365f37329 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:13:13.383605 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d1629ffbcd48d1b72c843365f37329. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.374431 30751 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.401388 30696 tablet_service.cc:1511] Processing CreateTablet for tablet 3511f2706d78410aa91eb654ab345803 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:13.406867 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511f2706d78410aa91eb654ab345803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.408984 30754 tablet_bootstrap.cc:654] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.434495 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:13.436735 30596 leader_election.cc:343] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.441962 30764 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.442737 30764 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.444095 30753 tablet_bootstrap.cc:654] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.446596 30474 tablet_service.cc:1511] Processing CreateTablet for tablet 3511f2706d78410aa91eb654ab345803 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:13.449373 30764 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:13.450028 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511f2706d78410aa91eb654ab345803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.452612 30696 tablet_service.cc:1511] Processing CreateTablet for tablet dc6f2eea5d3844fab97d019c3af6cf2f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:13.459196 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6f2eea5d3844fab97d019c3af6cf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.458966 30758 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.477316 30758 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.483480 30758 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:13.485145 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:13.492959 30622 tablet_service.cc:1511] Processing CreateTablet for tablet e1452d68baa440e39ae0dad239d033c8 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:13:13.493430 30623 tablet_service.cc:1511] Processing CreateTablet for tablet 3511f2706d78410aa91eb654ab345803 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:13:13.495679 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 5de1dd97d9f342288201946ead3a9445 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
W20260709 13:13:13.490959 30596 leader_election.cc:343] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.499226 30543 tablet_service.cc:1511] Processing CreateTablet for tablet dc6f2eea5d3844fab97d019c3af6cf2f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:13.499859 30621 tablet_service.cc:1511] Processing CreateTablet for tablet dc6f2eea5d3844fab97d019c3af6cf2f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:13:13.500780 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6f2eea5d3844fab97d019c3af6cf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.504480 30623 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3511f2706d78410aa91eb654ab345803. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.501662 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:13.503207 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 233dbd206c1f419697b1d1b21a74c633 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:13.496580 30751 ts_tablet_manager.cc:1434] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.145s	user 0.010s	sys 0.000s
I20260709 13:13:13.514511 30751 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:13.515754 30752 tablet_bootstrap.cc:654] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.515348 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 233dbd206c1f419697b1d1b21a74c633 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:13.521394 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dc6f2eea5d3844fab97d019c3af6cf2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.522275 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233dbd206c1f419697b1d1b21a74c633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.525307 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1452d68baa440e39ae0dad239d033c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.531088 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de1dd97d9f342288201946ead3a9445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.532228 30754 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:13.532902 30754 ts_tablet_manager.cc:1403] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.200s	user 0.016s	sys 0.023s
I20260709 13:13:13.534772 30765 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.535295 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233dbd206c1f419697b1d1b21a74c633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.537397 30765 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.546586 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:13.547344 30691 tablet_service.cc:1511] Processing CreateTablet for tablet a6b916ed19e94ddb8e8a2bca582ed0c6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:13.548302 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:13.512868 30559 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:13.548523 30765 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:13.548857 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b916ed19e94ddb8e8a2bca582ed0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.552417 30558 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:13.553107 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 233dbd206c1f419697b1d1b21a74c633 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:13:13.554781 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 233dbd206c1f419697b1d1b21a74c633. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.555583 30546 tablet_service.cc:1511] Processing CreateTablet for tablet a6b916ed19e94ddb8e8a2bca582ed0c6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:13.558601 30754 raft_consensus.cc:359] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.562044 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:13.567622 30754 raft_consensus.cc:385] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.568284 30764 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:13.575273 30764 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.575939 30764 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.575085 30754 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.587266 30754 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.565691 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:13.581030 30753 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:13.582996 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b916ed19e94ddb8e8a2bca582ed0c6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:13.569024 30670 leader_election.cc:343] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.598484 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.601246 30753 ts_tablet_manager.cc:1403] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.365s	user 0.012s	sys 0.009s
I20260709 13:13:13.611958 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:13.606251 30753 raft_consensus.cc:359] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.624245 30544 tablet_service.cc:1511] Processing CreateTablet for tablet f8c076453f774c4eb79096017a386e6c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:13:13.625967 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c076453f774c4eb79096017a386e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.626241 30691 tablet_service.cc:1511] Processing CreateTablet for tablet f8c076453f774c4eb79096017a386e6c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:13:13.627844 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c076453f774c4eb79096017a386e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.630470 30765 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.631215 30765 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.634857 30765 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:13.636454 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73461us
I20260709 13:13:13.644850 30757 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.645164 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:13.645670 30757 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:13.648214 30757 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:13.649271 30597 leader_election.cc:343] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.649885 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:13.650794 30758 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:13.620213 30753 raft_consensus.cc:385] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.654881 30753 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.656200 30754 ts_tablet_manager.cc:1434] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.123s	user 0.010s	sys 0.000s
I20260709 13:13:13.655972 30753 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.657459 30754 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:13.670073 30751 tablet_bootstrap.cc:654] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.672472 30470 tablet_service.cc:1511] Processing CreateTablet for tablet a6b916ed19e94ddb8e8a2bca582ed0c6 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:13:13.674659 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b916ed19e94ddb8e8a2bca582ed0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.682750 30547 tablet_service.cc:1511] Processing CreateTablet for tablet f029f5d5b1ee4391a2b10e12ec33b844 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:13.684481 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f029f5d5b1ee4391a2b10e12ec33b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.687270 30696 tablet_service.cc:1511] Processing CreateTablet for tablet b15e4d74180b49daad15cc459dde3181 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:13.688786 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e4d74180b49daad15cc459dde3181. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:13.694018 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 119912us
I20260709 13:13:13.695874 30764 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.697413 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:13.699019 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.699966 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:13.700716 30559 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.666308 30758 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:13.714723 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:13.715497 30484 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.727131 30617 tablet_service.cc:1511] Processing CreateTablet for tablet f029f5d5b1ee4391a2b10e12ec33b844 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:13.728880 30617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f029f5d5b1ee4391a2b10e12ec33b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.711982 30758 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.727651 30752 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:13.712199 30764 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:13.745955 30548 tablet_service.cc:1511] Processing CreateTablet for tablet b15e4d74180b49daad15cc459dde3181 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:13.747690 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e4d74180b49daad15cc459dde3181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.730839 30758 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.757282 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:13.757602 30783 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:13.758805 30618 tablet_service.cc:1511] Processing CreateTablet for tablet f8c076453f774c4eb79096017a386e6c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
W20260709 13:13:13.759481 30450 leader_election.cc:343] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.760510 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8c076453f774c4eb79096017a386e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.761368 30472 tablet_service.cc:1511] Processing CreateTablet for tablet b15e4d74180b49daad15cc459dde3181 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:13:13.762913 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b15e4d74180b49daad15cc459dde3181. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.767537 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:13:13.769181 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3d2c26e9174e5c942f6f2e1c84cee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.753798 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:13.772241 30523 leader_election.cc:343] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.772843 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:13.774556 30522 leader_election.cc:343] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.775084 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:13.776419 30757 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:13.776504 30474 tablet_service.cc:1511] Processing CreateTablet for tablet f029f5d5b1ee4391a2b10e12ec33b844 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:13:13.779670 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:13.781140 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:13.783376 30448 leader_election.cc:343] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.783712 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:13.783987 30448 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:13.786072 30751 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:13.786803 30751 ts_tablet_manager.cc:1403] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.273s	user 0.031s	sys 0.009s
I20260709 13:13:13.790877 30765 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:13.792057 30621 tablet_service.cc:1511] Processing CreateTablet for tablet 26f2e1d0fad840b5b8718f931fc92316 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
W20260709 13:13:13.792732 30448 leader_election.cc:343] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.793766 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f2e1d0fad840b5b8718f931fc92316. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:13.795325 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:13.758215 30783 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.740576 30752 ts_tablet_manager.cc:1403] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.394s	user 0.034s	sys 0.015s
I20260709 13:13:13.798945 30707 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:13.792851 30484 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:13.815441 30448 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:13.816501 30765 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:13.816409 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:13.817106 30765 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:13.817569 30765 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.817679 30764 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:13.818363 30764 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:13.818820 30752 raft_consensus.cc:359] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:13.819509 30764 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.820197 30783 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:13.810777 30559 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:13.823173 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:13.818866 30696 tablet_service.cc:1511] Processing CreateTablet for tablet 26f2e1d0fad840b5b8718f931fc92316 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:13:13.824968 30753 ts_tablet_manager.cc:1434] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.223s	user 0.020s	sys 0.016s
W20260709 13:13:13.824898 30671 leader_election.cc:343] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.825431 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f2e1d0fad840b5b8718f931fc92316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.828505 30468 tablet_service.cc:1511] Processing CreateTablet for tablet 26f2e1d0fad840b5b8718f931fc92316 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:13:13.825582 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f029f5d5b1ee4391a2b10e12ec33b844. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.829914 30753 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:13.830034 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26f2e1d0fad840b5b8718f931fc92316. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.846575 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 29980abbc08249a490fcf7b22cf63bb5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:13.848223 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29980abbc08249a490fcf7b22cf63bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.833400 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:13.804270 30754 tablet_bootstrap.cc:654] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.852478 30691 tablet_service.cc:1511] Processing CreateTablet for tablet 29980abbc08249a490fcf7b22cf63bb5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:13.854038 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29980abbc08249a490fcf7b22cf63bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.798942 30758 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.857801 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:13.858455 30559 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.860499 30758 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:13.861827 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:13.863430 30596 leader_election.cc:343] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.800494 30751 raft_consensus.cc:359] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.834950 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 29980abbc08249a490fcf7b22cf63bb5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:13:13.838765 30752 raft_consensus.cc:385] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.867122 30543 tablet_service.cc:1511] Processing CreateTablet for tablet f9117b9966b14416ba3ee00bd7424f65 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:13.868151 30752 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.868801 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29980abbc08249a490fcf7b22cf63bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.869370 30752 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:13.871217 30622 tablet_service.cc:1511] Processing CreateTablet for tablet f9117b9966b14416ba3ee00bd7424f65 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:13.872184 30752 ts_tablet_manager.cc:1434] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.002s
I20260709 13:13:13.860738 30753 tablet_bootstrap.cc:654] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.872314 30765 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.873430 30752 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:13.874018 30751 raft_consensus.cc:385] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:13.874398 30751 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:13.875794 30751 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:13.873481 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9117b9966b14416ba3ee00bd7424f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.880637 30690 tablet_service.cc:1511] Processing CreateTablet for tablet 3410292026774f0ca95a2c77eab3a8b4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:13.882104 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410292026774f0ca95a2c77eab3a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.883188 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:13.889128 30559 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:13.889534 30752 tablet_bootstrap.cc:654] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:13.890581 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:13.891002 30484 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:13.904513 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:13.905835 30785 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:13.906352 30758 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:13.906975 30758 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:13.907912 30758 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:13.911576 30544 tablet_service.cc:1511] Processing CreateTablet for tablet 3410292026774f0ca95a2c77eab3a8b4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:13.911664 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9117b9966b14416ba3ee00bd7424f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.916612 30765 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:13.895685 30448 leader_election.cc:343] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:13.935161 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410292026774f0ca95a2c77eab3a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.937244 30692 tablet_service.cc:1511] Processing CreateTablet for tablet e734f70ec47c410d8a7468e2bf91608d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 13:13:13.938791 30448 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47171.3us
I20260709 13:13:13.939745 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 3410292026774f0ca95a2c77eab3a8b4 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:13:13.940901 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:13.941713 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3410292026774f0ca95a2c77eab3a8b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.942387 30473 tablet_service.cc:1511] Processing CreateTablet for tablet e734f70ec47c410d8a7468e2bf91608d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:13:13.944398 30473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f70ec47c410d8a7468e2bf91608d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.953729 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f70ec47c410d8a7468e2bf91608d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:13.968698 30751 ts_tablet_manager.cc:1434] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.182s	user 0.004s	sys 0.004s
I20260709 13:13:13.918210 30472 tablet_service.cc:1511] Processing CreateTablet for tablet f9117b9966b14416ba3ee00bd7424f65 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:13:13.978685 30785 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:14.009491 30751 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
W20260709 13:13:14.010048 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 119589us
I20260709 13:13:14.017385 30694 tablet_service.cc:1511] Processing CreateTablet for tablet d5048dabcc194abca9a02315086e0eee (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:14.019938 30754 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:14.028012 30754 ts_tablet_manager.cc:1403] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.371s	user 0.025s	sys 0.022s
I20260709 13:13:14.029325 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5048dabcc194abca9a02315086e0eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.030761 30471 tablet_service.cc:1511] Processing CreateTablet for tablet bdaa99b885784af3b7e73563f113185a (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:14.031723 30754 raft_consensus.cc:359] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.032605 30754 raft_consensus.cc:385] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.032909 30754 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.033735 30754 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.035792 30548 tablet_service.cc:1511] Processing CreateTablet for tablet bdaa99b885784af3b7e73563f113185a (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:14.037303 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdaa99b885784af3b7e73563f113185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.037362 30753 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:14.038057 30753 ts_tablet_manager.cc:1403] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.208s	user 0.013s	sys 0.006s
I20260709 13:13:14.039340 30543 tablet_service.cc:1511] Processing CreateTablet for tablet d5048dabcc194abca9a02315086e0eee (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:14.043712 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5048dabcc194abca9a02315086e0eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.044061 30754 ts_tablet_manager.cc:1434] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:13:14.045526 30754 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:14.047235 30757 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.047850 30757 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:14.034826 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9117b9966b14416ba3ee00bd7424f65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.055059 30757 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:14.056849 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:14.063686 30522 leader_election.cc:343] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.069801 30471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdaa99b885784af3b7e73563f113185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.048799 30753 raft_consensus.cc:359] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.075665 30789 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.078743 30789 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.081422 30789 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:14.082283 30753 raft_consensus.cc:385] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.086189 30751 tablet_bootstrap.cc:654] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.086650 30753 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.095170 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:14.096694 30690 tablet_service.cc:1511] Processing CreateTablet for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:14.098269 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.104606 30468 tablet_service.cc:1511] Processing CreateTablet for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:14.106280 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.106114 30707 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:14.108759 30752 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:14.109218 30547 tablet_service.cc:1511] Processing CreateTablet for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:13:14.114701 30618 tablet_service.cc:1511] Processing CreateTablet for tablet bdaa99b885784af3b7e73563f113185a (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:13:14.118955 30617 tablet_service.cc:1511] Processing CreateTablet for tablet d5048dabcc194abca9a02315086e0eee (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:13:14.120987 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:14.128567 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:14.131490 30622 tablet_service.cc:1511] Processing CreateTablet for tablet a1265e789b7c41d7ba82825a0c126517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:14.133324 30693 tablet_service.cc:1511] Processing CreateTablet for tablet a1265e789b7c41d7ba82825a0c126517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:14.144912 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1265e789b7c41d7ba82825a0c126517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.145673 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1265e789b7c41d7ba82825a0c126517. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.148685 30765 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.149362 30765 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.151902 30765 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:14.152153 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdaa99b885784af3b7e73563f113185a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.154765 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:14.155485 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb3b5ffa2c364cacbd6f5fd9a69f1a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.156158 30559 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:14.159873 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:14.159818 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 6bae0ee744c54691b48d52b137264f92 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:14.161820 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bae0ee744c54691b48d52b137264f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.163126 30617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5048dabcc194abca9a02315086e0eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.166545 30448 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:14.167686 30765 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.137341 30619 tablet_service.cc:1511] Processing CreateTablet for tablet e734f70ec47c410d8a7468e2bf91608d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:13:14.138306 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:14.138772 30753 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.169970 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e734f70ec47c410d8a7468e2bf91608d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.139640 30754 tablet_bootstrap.cc:654] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.173638 30706 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:14.180617 30752 ts_tablet_manager.cc:1403] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.307s	user 0.018s	sys 0.027s
I20260709 13:13:14.181962 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:14.187939 30448 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:14.188503 30789 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:14.185146 30753 ts_tablet_manager.cc:1434] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.147s	user 0.008s	sys 0.003s
I20260709 13:13:14.188974 30789 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.189415 30789 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.190169 30753 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
W20260709 13:13:14.186064 30523 leader_election.cc:343] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.191664 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:14.164813 30381 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:14.185096 30696 tablet_service.cc:1511] Processing CreateTablet for tablet 6bae0ee744c54691b48d52b137264f92 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:14.192974 30757 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:14.204114 30751 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:14.204875 30751 ts_tablet_manager.cc:1403] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.208s	user 0.013s	sys 0.004s
I20260709 13:13:14.207697 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bae0ee744c54691b48d52b137264f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.212888 30751 raft_consensus.cc:359] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.213933 30751 raft_consensus.cc:385] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.214332 30751 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.206416 30753 tablet_bootstrap.cc:654] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.221948 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd45f50776e4101a900943b8df9353c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:14.234018 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd45f50776e4101a900943b8df9353c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.234244 30785 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:14.236279 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 37a97221d4ba4f889ce430d3dadc5754 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:14.237892 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a97221d4ba4f889ce430d3dadc5754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.224383 30752 raft_consensus.cc:359] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.271078 30752 raft_consensus.cc:385] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.271616 30752 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.271821 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 6bae0ee744c54691b48d52b137264f92 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:13:14.273912 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bae0ee744c54691b48d52b137264f92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.167212 30474 tablet_service.cc:1511] Processing CreateTablet for tablet a1265e789b7c41d7ba82825a0c126517 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:13:14.284780 30474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1265e789b7c41d7ba82825a0c126517. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:14.284950 30599 leader_election.cc:343] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.288163 30621 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd45f50776e4101a900943b8df9353c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:14.290321 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd45f50776e4101a900943b8df9353c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.293128 30754 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:14.293864 30754 ts_tablet_manager.cc:1403] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.249s	user 0.018s	sys 0.020s
I20260709 13:13:14.297490 30754 raft_consensus.cc:359] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.312019 30789 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.315814 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:14.317322 30752 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:14.318059 30599 leader_election.cc:343] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.320147 30752 ts_tablet_manager.cc:1434] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.127s	user 0.009s	sys 0.000s
I20260709 13:13:14.321413 30752 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:14.298444 30754 raft_consensus.cc:385] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.336736 30754 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.337865 30754 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.270426 30751 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.330054 30789 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:14.343890 30448 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 155518us
I20260709 13:13:14.357556 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:14.367602 30796 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:14.377274 30765 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:14.398070 30753 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:14.397372 30468 tablet_service.cc:1511] Processing CreateTablet for tablet 37a97221d4ba4f889ce430d3dadc5754 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:14.398793 30753 ts_tablet_manager.cc:1403] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.209s	user 0.038s	sys 0.012s
I20260709 13:13:14.399353 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 9fd45f50776e4101a900943b8df9353c (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:13:14.400954 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fd45f50776e4101a900943b8df9353c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:14.402873 30367 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75481.2us
I20260709 13:13:14.402678 30753 raft_consensus.cc:359] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.403544 30753 raft_consensus.cc:385] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.403861 30753 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.404006 30785 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:14.404707 30753 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.406687 30753 ts_tablet_manager.cc:1434] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:13:14.407668 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 72181014679a4c538f96f2d305a5474f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:14.409446 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72181014679a4c538f96f2d305a5474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.407752 30691 tablet_service.cc:1511] Processing CreateTablet for tablet 72181014679a4c538f96f2d305a5474f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:14.412508 30751 ts_tablet_manager.cc:1434] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.207s	user 0.010s	sys 0.000s
I20260709 13:13:14.413280 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72181014679a4c538f96f2d305a5474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.413792 30751 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:14.385568 30707 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.416592 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a97221d4ba4f889ce430d3dadc5754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.429381 30548 tablet_service.cc:1511] Processing CreateTablet for tablet 37a97221d4ba4f889ce430d3dadc5754 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:13:14.431447 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 37a97221d4ba4f889ce430d3dadc5754. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.432099 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 72181014679a4c538f96f2d305a5474f (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:13:14.439299 30765 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.439975 30544 tablet_service.cc:1511] Processing CreateTablet for tablet 177c3663558a41cba8e75193425ead61 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:14.441563 30544 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177c3663558a41cba8e75193425ead61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.434461 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 72181014679a4c538f96f2d305a5474f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.458329 30753 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:14.459240 30757 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.459896 30757 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.460431 30618 tablet_service.cc:1511] Processing CreateTablet for tablet 177c3663558a41cba8e75193425ead61 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:14.462113 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177c3663558a41cba8e75193425ead61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.464433 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:14.465255 30483 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:14.466289 30757 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:14.467636 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:14.474045 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:14.484341 30694 tablet_service.cc:1511] Processing CreateTablet for tablet 177c3663558a41cba8e75193425ead61 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:13:14.486142 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 177c3663558a41cba8e75193425ead61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.440012 30765 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.476501 30757 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:14.487505 30633 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:14.492097 30757 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.493741 30757 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.513038 30471 tablet_service.cc:1511] Processing CreateTablet for tablet 125a8386fffd41c0b18a3e6e0690283b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:14.516219 30752 tablet_bootstrap.cc:654] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.486917 30796 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:14.521399 30471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125a8386fffd41c0b18a3e6e0690283b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.502524 30707 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:14.528071 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:14.530071 30789 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:14.530648 30789 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:14.538676 30765 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:14.545698 30546 tablet_service.cc:1511] Processing CreateTablet for tablet 125a8386fffd41c0b18a3e6e0690283b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:14.545919 30789 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.547236 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:14.548069 30633 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:14.543197 30758 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:13:14.550098 30448 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:14.550360 30753 tablet_bootstrap.cc:654] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.551277 30765 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:14.551723 30765 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:14.552171 30765 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.547551 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125a8386fffd41c0b18a3e6e0690283b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.578526 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:14.580664 30448 leader_election.cc:343] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.612506 30751 tablet_bootstrap.cc:654] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.606297 30754 ts_tablet_manager.cc:1434] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.311s	user 0.029s	sys 0.043s
W20260709 13:13:14.600927 30367 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 54126.1us
I20260709 13:13:14.636906 30758 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:14.620420 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89268.6us
I20260709 13:13:14.635658 30757 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.662773 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 125a8386fffd41c0b18a3e6e0690283b (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:13:14.663036 30468 tablet_service.cc:1511] Processing CreateTablet for tablet 731ce9bb8d6143d4860cfae0d68c8024 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:14.664410 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 125a8386fffd41c0b18a3e6e0690283b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.664346 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 731ce9bb8d6143d4860cfae0d68c8024 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:14.664595 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731ce9bb8d6143d4860cfae0d68c8024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.667747 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731ce9bb8d6143d4860cfae0d68c8024. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.668673 30754 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:14.673017 30621 tablet_service.cc:1511] Processing CreateTablet for tablet 536e99f894744c35a0490416a32a875e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:14.674647 30621 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536e99f894744c35a0490416a32a875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.606256 30796 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.691736 30696 tablet_service.cc:1511] Processing CreateTablet for tablet 536e99f894744c35a0490416a32a875e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:14.693493 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536e99f894744c35a0490416a32a875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.725590 30754 tablet_bootstrap.cc:654] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:14.725821 30369 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55406.4us
I20260709 13:13:14.727486 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 536e99f894744c35a0490416a32a875e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:13:14.729050 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536e99f894744c35a0490416a32a875e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.731467 30785 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:14.732242 30785 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.783387 30622 tablet_service.cc:1511] Processing CreateTablet for tablet 8d07c4d131b54934aaedbe589e95d746 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:14.785583 30622 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d07c4d131b54934aaedbe589e95d746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.809711 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:14.810460 30633 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.811224 30751 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:14.811846 30751 ts_tablet_manager.cc:1403] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.398s	user 0.023s	sys 0.020s
I20260709 13:13:14.815327 30690 tablet_service.cc:1511] Processing CreateTablet for tablet 8d07c4d131b54934aaedbe589e95d746 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:14.816969 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d07c4d131b54934aaedbe589e95d746. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:14.818364 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:14.819643 30751 raft_consensus.cc:359] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:14.820499 30751 raft_consensus.cc:385] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.820822 30751 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.822268 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 07fae2111d284c78bf054bd74aee6b94 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:14.823884 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fae2111d284c78bf054bd74aee6b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.825732 30547 tablet_service.cc:1511] Processing CreateTablet for tablet 07fae2111d284c78bf054bd74aee6b94 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:14.827342 30547 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fae2111d284c78bf054bd74aee6b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.827893 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:14.828598 30483 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.831933 30549 tablet_service.cc:1511] Processing CreateTablet for tablet 496aec2d89db4dbe892cb6d96b075225 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:13:14.833681 30549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496aec2d89db4dbe892cb6d96b075225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.836174 30785 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:14.837441 30696 tablet_service.cc:1511] Processing CreateTablet for tablet 496aec2d89db4dbe892cb6d96b075225 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:13:14.838330 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:14.838868 30696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496aec2d89db4dbe892cb6d96b075225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.839260 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:14.841001 30671 leader_election.cc:343] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.821597 30751 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:14.844089 30751 ts_tablet_manager.cc:1434] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 13:13:14.844967 30753 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:14.845199 30751 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:14.845628 30753 ts_tablet_manager.cc:1403] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.388s	user 0.015s	sys 0.014s
I20260709 13:13:14.807744 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 731ce9bb8d6143d4860cfae0d68c8024 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:13:14.847951 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 731ce9bb8d6143d4860cfae0d68c8024. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:14.848428 30670 leader_election.cc:343] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:14.848995 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:14.851980 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 07fae2111d284c78bf054bd74aee6b94 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:13:14.853564 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07fae2111d284c78bf054bd74aee6b94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.806366 30757 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:14.856000 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 5315ab4cdcff4579984ba8de8f9d4d82 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:14.857452 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5315ab4cdcff4579984ba8de8f9d4d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.861069 30472 tablet_service.cc:1511] Processing CreateTablet for tablet b85a1f7a06b642de885b159b7f7bba1e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:14.862977 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 496aec2d89db4dbe892cb6d96b075225 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:13:14.866111 30751 tablet_bootstrap.cc:654] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:14.868803 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a1f7a06b642de885b159b7f7bba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.875723 30618 tablet_service.cc:1511] Processing CreateTablet for tablet b85a1f7a06b642de885b159b7f7bba1e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:14.856333 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 8d07c4d131b54934aaedbe589e95d746 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:13:14.876833 30379 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:14.878532 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8d07c4d131b54934aaedbe589e95d746. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.881345 30765 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.884655 30753 raft_consensus.cc:359] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.889777 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 496aec2d89db4dbe892cb6d96b075225. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.847868 30693 tablet_service.cc:1511] Processing CreateTablet for tablet b85a1f7a06b642de885b159b7f7bba1e (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:13:14.891520 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a1f7a06b642de885b159b7f7bba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.893139 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b85a1f7a06b642de885b159b7f7bba1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.885653 30785 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:14.909480 30752 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:14.910244 30752 ts_tablet_manager.cc:1403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.589s	user 0.045s	sys 0.071s
I20260709 13:13:14.887733 30753 raft_consensus.cc:385] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.912472 30753 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.914407 30546 tablet_service.cc:1511] Processing CreateTablet for tablet 5315ab4cdcff4579984ba8de8f9d4d82 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:14.920042 30765 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:14.914143 30752 raft_consensus.cc:359] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.920864 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:14.921372 30752 raft_consensus.cc:385] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:14.921758 30632 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.921005 30753 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:14.921938 30752 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:14.929826 30691 tablet_service.cc:1511] Processing CreateTablet for tablet 5315ab4cdcff4579984ba8de8f9d4d82 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:13:14.945464 30483 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:14.948043 30468 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc8bff1e9b745c7a95ecd1b6306719d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:14.950421 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc8bff1e9b745c7a95ecd1b6306719d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.956792 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5315ab4cdcff4579984ba8de8f9d4d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.972949 30754 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:14.974306 30691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5315ab4cdcff4579984ba8de8f9d4d82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.952215 30633 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:14.970454 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:14.983565 30757 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:14.984253 30757 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:14.985209 30548 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc8bff1e9b745c7a95ecd1b6306719d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:14.986929 30548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc8bff1e9b745c7a95ecd1b6306719d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:14.985191 30757 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:14.995106 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:14.995954 30707 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:14.998729 30382 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:14.967696 30752 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:14.979138 30754 ts_tablet_manager.cc:1403] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.316s	user 0.014s	sys 0.007s
I20260709 13:13:15.003216 30617 tablet_service.cc:1511] Processing CreateTablet for tablet 0dc8bff1e9b745c7a95ecd1b6306719d (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:13:15.044492 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 2b685fd119d941538e6fee8dc8f8c930 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:15.046164 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b685fd119d941538e6fee8dc8f8c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.027485 30632 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.052932 30448 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:15.054080 30796 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.054659 30796 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:15.055598 30796 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.058561 30617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dc8bff1e9b745c7a95ecd1b6306719d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.066304 30754 raft_consensus.cc:359] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.067317 30754 raft_consensus.cc:385] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.067649 30754 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.068672 30754 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.071754 30752 ts_tablet_manager.cc:1434] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.161s	user 0.006s	sys 0.006s
I20260709 13:13:15.073098 30752 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.074441 30789 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.077617 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 2b685fd119d941538e6fee8dc8f8c930 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:15.079216 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b685fd119d941538e6fee8dc8f8c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.088634 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 2b685fd119d941538e6fee8dc8f8c930 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:13:15.096014 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2b685fd119d941538e6fee8dc8f8c930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.104178 30753 ts_tablet_manager.cc:1434] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.258s	user 0.010s	sys 0.001s
I20260709 13:13:15.105829 30753 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:15.082520 30789 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.114703 30789 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.117076 30751 tablet_bootstrap.cc:492] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.117473 30789 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.117784 30751 ts_tablet_manager.cc:1403] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.273s	user 0.014s	sys 0.010s
I20260709 13:13:15.119086 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:15.121273 30597 leader_election.cc:343] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.121524 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:15.123353 30599 leader_election.cc:343] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.124166 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.125612 30789 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:15.129209 30754 ts_tablet_manager.cc:1434] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.104s	user 0.008s	sys 0.001s
I20260709 13:13:15.132457 30752 tablet_bootstrap.cc:654] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.132668 30707 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.134567 30754 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.142217 30754 tablet_bootstrap.cc:654] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.144104 30381 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.132004 30751 raft_consensus.cc:359] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.150866 30751 raft_consensus.cc:385] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.151283 30751 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.175187 30694 tablet_service.cc:1511] Processing CreateTablet for tablet 4b674451b4d5465d82d8ef4f14fb52f5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:15.176892 30694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b674451b4d5465d82d8ef4f14fb52f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.191634 30472 tablet_service.cc:1511] Processing CreateTablet for tablet 4b674451b4d5465d82d8ef4f14fb52f5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:15.193686 30472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b674451b4d5465d82d8ef4f14fb52f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.194828 30690 tablet_service.cc:1511] Processing CreateTablet for tablet 01dd2254fe30409b8a08179436dd1e10 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:15.196329 30690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dd2254fe30409b8a08179436dd1e10. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:13:15.192525 30599 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.202209 30796 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.203241 30469 tablet_service.cc:1511] Processing CreateTablet for tablet 742a437d7cdc48f58f38b6c6a39ce5ae (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:15.207070 30620 tablet_service.cc:1511] Processing CreateTablet for tablet 01dd2254fe30409b8a08179436dd1e10 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:15.207465 30796 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.153605 30751 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.208364 30469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 742a437d7cdc48f58f38b6c6a39ce5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.208458 30620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dd2254fe30409b8a08179436dd1e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.209897 30757 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.210294 30751 ts_tablet_manager.cc:1434] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.092s	user 0.009s	sys 0.000s
I20260709 13:13:15.217882 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:15.204881 30618 tablet_service.cc:1511] Processing CreateTablet for tablet 4b674451b4d5465d82d8ef4f14fb52f5 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:13:15.221765 30751 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
W20260709 13:13:15.222317 30448 leader_election.cc:343] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.200732 30545 tablet_service.cc:1511] Processing CreateTablet for tablet 742a437d7cdc48f58f38b6c6a39ce5ae (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:15.223232 30796 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:15.224262 30545 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 742a437d7cdc48f58f38b6c6a39ce5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.211544 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.222422 30618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b674451b4d5465d82d8ef4f14fb52f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.224905 30559 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:15.213343 30619 tablet_service.cc:1511] Processing CreateTablet for tablet 742a437d7cdc48f58f38b6c6a39ce5ae (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:13:15.199693 30753 tablet_bootstrap.cc:654] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.226701 30450 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:15.226723 30619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 742a437d7cdc48f58f38b6c6a39ce5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.231462 30796 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.210438 30757 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.231950 30796 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.232374 30796 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.235735 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:15.237461 30522 leader_election.cc:343] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.234277 30757 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:15.242164 30543 tablet_service.cc:1511] Processing CreateTablet for tablet 01dd2254fe30409b8a08179436dd1e10 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:13:15.243628 30543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01dd2254fe30409b8a08179436dd1e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.253180 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.255260 30381 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:15.259835 30523 leader_election.cc:343] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.265566 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.272195 30757 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:15.265719 30693 tablet_service.cc:1511] Processing CreateTablet for tablet c351eef49bfd49568fd63877d05eab98 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:13:15.275925 30693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351eef49bfd49568fd63877d05eab98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.287755 30796 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.288830 30546 tablet_service.cc:1511] Processing CreateTablet for tablet c351eef49bfd49568fd63877d05eab98 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:13:15.290364 30546 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351eef49bfd49568fd63877d05eab98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.294775 30470 tablet_service.cc:1511] Processing CreateTablet for tablet 54da309ba151423ab2739966b9caebd0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:15.310812 30470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54da309ba151423ab2739966b9caebd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.319239 30468 tablet_service.cc:1511] Processing CreateTablet for tablet c351eef49bfd49568fd63877d05eab98 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:13:15.320964 30468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c351eef49bfd49568fd63877d05eab98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.321520 30752 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.322921 30754 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:15.323503 30754 ts_tablet_manager.cc:1403] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.189s	user 0.016s	sys 0.004s
I20260709 13:13:15.327069 30754 raft_consensus.cc:359] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.328248 30754 raft_consensus.cc:385] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.328603 30754 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.329689 30754 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.314690 30617 tablet_service.cc:1511] Processing CreateTablet for tablet 54da309ba151423ab2739966b9caebd0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:15.322351 30752 ts_tablet_manager.cc:1403] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.250s	user 0.017s	sys 0.029s
I20260709 13:13:15.342033 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:15.342979 30558 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.348333 30796 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:15.351949 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:15.352661 30752 raft_consensus.cc:359] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.353236 30692 tablet_service.cc:1511] Processing CreateTablet for tablet 54da309ba151423ab2739966b9caebd0 (DEFAULT_TABLE table=test-workload [id=779e1f3b9cf14437906ac4a4af1fcf9f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:13:15.358022 30751 tablet_bootstrap.cc:654] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.362989 30617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54da309ba151423ab2739966b9caebd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.372316 30692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54da309ba151423ab2739966b9caebd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:13:15.374812 30796 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.379415 30752 raft_consensus.cc:385] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.380012 30752 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.401276 30753 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:15.401925 30753 ts_tablet_manager.cc:1403] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.296s	user 0.041s	sys 0.023s
I20260709 13:13:15.401648 30752 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.404618 30752 ts_tablet_manager.cc:1434] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:13:15.400728 30754 ts_tablet_manager.cc:1434] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.000s
I20260709 13:13:15.405210 30632 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:15.411983 30448 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:15.418403 30752 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.423233 30754 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.428826 30753 raft_consensus.cc:359] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.429710 30753 raft_consensus.cc:385] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.430542 30558 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.432060 30450 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:15.432967 30796 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.433492 30796 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:15.433745 30632 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.434305 30751 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.435034 30751 ts_tablet_manager.cc:1403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.214s	user 0.021s	sys 0.013s
I20260709 13:13:15.434424 30796 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.438107 30751 raft_consensus.cc:359] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.439234 30751 raft_consensus.cc:385] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.439610 30751 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.439704 30754 tablet_bootstrap.cc:654] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.440421 30751 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.442909 30751 ts_tablet_manager.cc:1434] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:13:15.430083 30753 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.444543 30751 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:15.446797 30757 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.444432 30753 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.447391 30757 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.452164 30751 tablet_bootstrap.cc:654] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.453539 30752 tablet_bootstrap.cc:654] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.452107 30753 ts_tablet_manager.cc:1434] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.050s	user 0.001s	sys 0.007s
I20260709 13:13:15.455910 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:15.456648 30753 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:15.463032 30812 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.463562 30757 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.466760 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.469834 30381 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.470849 30753 tablet_bootstrap.cc:654] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.478658 30752 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.479732 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:15.480717 30752 ts_tablet_manager.cc:1403] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.063s	user 0.013s	sys 0.006s
I20260709 13:13:15.483455 30812 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:15.484750 30752 raft_consensus.cc:359] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.485620 30752 raft_consensus.cc:385] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.485966 30752 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.486968 30752 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:15.445601 30833 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:15.493185 30752 ts_tablet_manager.cc:1434] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:13:15.494498 30752 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.495121 30751 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.495747 30751 ts_tablet_manager.cc:1403] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.008s
I20260709 13:13:15.499405 30751 raft_consensus.cc:359] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.500247 30751 raft_consensus.cc:385] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.500555 30751 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.502043 30752 tablet_bootstrap.cc:654] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.502154 30751 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.507150 30751 ts_tablet_manager.cc:1434] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:13:15.507220 30753 tablet_bootstrap.cc:492] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:15.512050 30753 ts_tablet_manager.cc:1403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.004s
I20260709 13:13:15.512166 30751 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:15.515846 30753 raft_consensus.cc:359] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.517330 30753 raft_consensus.cc:385] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.517199 30812 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.517961 30753 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.519047 30753 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.519760 30751 tablet_bootstrap.cc:654] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.523015 30764 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.523684 30764 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.525820 30765 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.527899 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:15.530012 30764 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:15.531519 30670 leader_election.cc:343] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.534086 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.535115 30558 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:15.537501 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:15.539336 30764 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.539999 30764 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.540584 30764 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.555749 30753 ts_tablet_manager.cc:1434] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 13:13:15.566401 30753 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:15.558398 30846 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.579210 30846 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.569845 30752 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.586254 30752 ts_tablet_manager.cc:1403] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.000s
I20260709 13:13:15.597302 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.598522 30558 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:15.600698 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:15.605604 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:15.606514 30631 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:15.608476 30848 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.606443 30846 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:15.612059 30752 raft_consensus.cc:359] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.613415 30844 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.617326 30844 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.617909 30844 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.619465 30751 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.624379 30751 ts_tablet_manager.cc:1403] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.113s	user 0.033s	sys 0.060s
I20260709 13:13:15.626523 30558 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.617465 30752 raft_consensus.cc:385] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.635835 30752 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.622709 30764 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.638270 30752 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.650357 30754 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:15.651043 30754 ts_tablet_manager.cc:1403] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.228s	user 0.048s	sys 0.111s
I20260709 13:13:15.652209 30752 ts_tablet_manager.cc:1434] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.065s	user 0.006s	sys 0.002s
I20260709 13:13:15.653434 30752 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.654506 30754 raft_consensus.cc:359] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.655354 30754 raft_consensus.cc:385] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.655629 30754 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.656551 30754 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.658012 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:15.650942 30764 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:15.659338 30754 ts_tablet_manager.cc:1434] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 13:13:15.660116 30670 leader_election.cc:343] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.663043 30752 tablet_bootstrap.cc:654] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.669476 30752 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.672271 30752 ts_tablet_manager.cc:1403] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.000s
I20260709 13:13:15.675555 30754 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.675465 30752 raft_consensus.cc:359] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.676287 30752 raft_consensus.cc:385] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.676604 30752 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.677512 30752 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.679826 30752 ts_tablet_manager.cc:1434] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:13:15.680519 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:15.680917 30752 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.681238 30558 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.681712 30754 tablet_bootstrap.cc:654] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.688220 30754 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:15.688488 30752 tablet_bootstrap.cc:654] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.689041 30754 ts_tablet_manager.cc:1403] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 13:13:15.692375 30754 raft_consensus.cc:359] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.693215 30754 raft_consensus.cc:385] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.693579 30754 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.694509 30754 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.641923 30751 raft_consensus.cc:359] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.695518 30751 raft_consensus.cc:385] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.695839 30751 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.696171 30848 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.696628 30751 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.698253 30754 ts_tablet_manager.cc:1434] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:13:15.699373 30751 ts_tablet_manager.cc:1434] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.002s
I20260709 13:13:15.646844 30753 tablet_bootstrap.cc:654] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.699505 30754 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.694741 30758 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.696728 30848 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.702930 30758 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.705873 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:15.706820 30633 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:15.707252 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.707746 30754 tablet_bootstrap.cc:654] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.707988 30706 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:15.708636 30448 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:15.709241 30848 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.709895 30848 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.710330 30848 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.705931 30758 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:15.707269 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.710879 30848 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.712010 30751 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:15.640017 30844 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.726390 30751 tablet_bootstrap.cc:654] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.734869 30558 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:15.747195 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:15.747826 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:15.748075 30633 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.750279 30764 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.750934 30764 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:15.751021 30758 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.751551 30758 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.751924 30764 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.753383 30848 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.754112 30758 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.760656 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.738492 30753 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:15.762678 30796 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.763623 30753 ts_tablet_manager.cc:1403] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.198s	user 0.012s	sys 0.011s
I20260709 13:13:15.764668 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.763561 30796 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.765450 30558 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:15.765983 30597 leader_election.cc:343] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.741390 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:15.743821 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:15.766865 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.767251 30559 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.768209 30758 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.768380 30796 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:15.768682 30758 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.769101 30758 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.769534 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:15.771250 30448 leader_election.cc:343] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.773339 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.779547 30753 raft_consensus.cc:359] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.745707 30844 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:15.780100 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.774170 30558 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:15.788324 30789 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:15.788887 30752 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.789109 30450 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.789513 30752 ts_tablet_manager.cc:1403] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.011s
I20260709 13:13:15.790500 30796 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.790946 30796 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.791339 30796 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.792223 30382 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.794819 30754 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:15.795415 30754 ts_tablet_manager.cc:1403] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.096s	user 0.041s	sys 0.020s
I20260709 13:13:15.780493 30758 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.781272 30559 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:15.798861 30754 raft_consensus.cc:359] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.802866 30754 raft_consensus.cc:385] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.784085 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:15.785980 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:15.783946 30633 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:15.803690 30632 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.804005 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:15.804673 30559 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.803727 30707 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.786262 30848 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.806898 30754 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.800127 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:15.807766 30754 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.802640 30796 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.808555 30844 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.809160 30844 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:15.810369 30796 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:15.811792 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:15.812508 30558 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.802802 30631 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.816345 30707 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.803428 30753 raft_consensus.cc:385] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.800499 30758 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:15.801995 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:15.817402 30632 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.817673 30753 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.818944 30448 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:15.819171 30597 leader_election.cc:343] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:15.820003 30753 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.822350 30753 ts_tablet_manager.cc:1434] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.000s
I20260709 13:13:15.822877 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:15.823441 30753 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:15.823482 30706 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.825171 30850 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.825739 30850 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:15.826731 30850 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.744395 30751 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.831130 30559 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:15.831781 30751 ts_tablet_manager.cc:1403] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.120s	user 0.015s	sys 0.000s
W20260709 13:13:15.832440 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:15.832640 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.833427 30558 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.833916 30758 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.834474 30758 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:15.810105 30844 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.834882 30450 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:15.835377 30758 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:15.835923 30796 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:15.836457 30796 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:15.837361 30796 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.838608 30751 raft_consensus.cc:359] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.839591 30754 ts_tablet_manager.cc:1434] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.001s
I20260709 13:13:15.848341 30843 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.854377 30751 raft_consensus.cc:385] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.854839 30751 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.855213 30754 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.853143 30382 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:15.855846 30751 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.858820 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:15.805840 30752 raft_consensus.cc:359] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.860118 30752 raft_consensus.cc:385] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.860467 30752 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.861436 30752 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.862733 30754 tablet_bootstrap.cc:654] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.863142 30751 ts_tablet_manager.cc:1434] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 13:13:15.864289 30751 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:15.866541 30752 ts_tablet_manager.cc:1434] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.077s	user 0.008s	sys 0.001s
I20260709 13:13:15.867760 30752 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.872746 30757 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.873566 30757 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.876156 30751 tablet_bootstrap.cc:654] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.883992 30753 tablet_bootstrap.cc:654] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.890432 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:15.892982 30483 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:15.899539 30757 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:15.899693 30381 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:15.902832 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:15.906550 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:15.907191 30631 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:15.910553 30757 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:15.911316 30757 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:15.912027 30757 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.932118 30754 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:15.932822 30754 ts_tablet_manager.cc:1403] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.078s	user 0.019s	sys 0.032s
I20260709 13:13:15.933333 30752 tablet_bootstrap.cc:654] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.940981 30751 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:15.942930 30751 ts_tablet_manager.cc:1403] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.079s	user 0.018s	sys 0.006s
I20260709 13:13:15.944931 30754 raft_consensus.cc:359] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.945747 30754 raft_consensus.cc:385] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.946070 30754 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.944144 30850 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:15.947028 30754 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.947827 30850 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.950335 30843 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:15.952464 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:15.954456 30753 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:15.955093 30753 ts_tablet_manager.cc:1403] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.005s
I20260709 13:13:15.958429 30850 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:15.958581 30753 raft_consensus.cc:359] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.964854 30753 raft_consensus.cc:385] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.965243 30753 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.966483 30754 ts_tablet_manager.cc:1434] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.005s
I20260709 13:13:15.961913 30379 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:15.966660 30753 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:15.967701 30754 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:15.960767 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:15.972018 30751 raft_consensus.cc:359] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.972873 30751 raft_consensus.cc:385] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.973291 30751 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.974123 30752 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:15.974752 30752 ts_tablet_manager.cc:1403] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.107s	user 0.021s	sys 0.006s
I20260709 13:13:15.974830 30751 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:15.975976 30559 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:15.979172 30754 tablet_bootstrap.cc:654] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.983415 30753 ts_tablet_manager.cc:1434] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.003s
I20260709 13:13:15.984346 30753 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:15.982386 30752 raft_consensus.cc:359] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.985977 30752 raft_consensus.cc:385] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:15.986271 30752 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:15.987018 30752 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:15.989274 30752 ts_tablet_manager.cc:1434] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.000s
I20260709 13:13:15.990309 30752 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:15.990078 30450 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:15.990680 30753 tablet_bootstrap.cc:654] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:15.991163 30850 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:15.997871 30752 tablet_bootstrap.cc:654] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.005542 30850 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.010902 30757 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.015957 30757 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.016022 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:16.017349 30483 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.022563 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:16.025290 30632 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.033171 30753 tablet_bootstrap.cc:492] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.033797 30753 ts_tablet_manager.cc:1403] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.024s
I20260709 13:13:16.037084 30753 raft_consensus.cc:359] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.037827 30753 raft_consensus.cc:385] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.038115 30753 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.038909 30753 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.041807 30753 ts_tablet_manager.cc:1434] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 13:13:16.039422 30381 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:16.042969 30753 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.047981 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.049191 30754 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:16.049230 30751 ts_tablet_manager.cc:1434] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.000s
I20260709 13:13:16.052829 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.051556 30752 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.052934 30483 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.053972 30752 ts_tablet_manager.cc:1403] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.064s	user 0.017s	sys 0.011s
I20260709 13:13:16.054199 30757 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.055354 30757 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:16.056196 30757 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.050256 30753 tablet_bootstrap.cc:654] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.057561 30752 raft_consensus.cc:359] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.058396 30752 raft_consensus.cc:385] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.058740 30752 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.060400 30754 ts_tablet_manager.cc:1403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.093s	user 0.013s	sys 0.001s
I20260709 13:13:16.067539 30751 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.072770 30753 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.073487 30753 ts_tablet_manager.cc:1403] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.031s	user 0.008s	sys 0.009s
I20260709 13:13:16.076406 30753 raft_consensus.cc:359] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.077302 30753 raft_consensus.cc:385] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.077597 30753 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.078408 30753 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.080392 30754 raft_consensus.cc:359] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.081243 30754 raft_consensus.cc:385] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.081538 30754 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.082376 30754 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.059500 30752 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.084420 30753 ts_tablet_manager.cc:1434] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:13:16.085340 30752 ts_tablet_manager.cc:1434] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:13:16.085619 30753 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.086835 30757 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.087402 30757 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.089430 30752 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.090219 30757 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.090452 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.091697 30380 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:16.092903 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.093298 30753 tablet_bootstrap.cc:654] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:16.094735 30522 leader_election.cc:343] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:16.096050 30523 leader_election.cc:343] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.096458 30751 tablet_bootstrap.cc:654] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.096508 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.100812 30757 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.103037 30754 ts_tablet_manager.cc:1434] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.001s
I20260709 13:13:16.104372 30754 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:16.113888 30752 tablet_bootstrap.cc:654] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.120535 30751 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.121307 30751 ts_tablet_manager.cc:1403] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.001s
I20260709 13:13:16.124744 30758 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.125393 30758 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.125571 30751 raft_consensus.cc:359] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.127703 30758 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.128590 30754 tablet_bootstrap.cc:654] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.128772 30757 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.129436 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.131075 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:16.131685 30597 leader_election.cc:343] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.131798 30483 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:16.133034 30753 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.133567 30753 ts_tablet_manager.cc:1403] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.015s
I20260709 13:13:16.129308 30757 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.134290 30751 raft_consensus.cc:385] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.134683 30751 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.133953 30752 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.135622 30752 ts_tablet_manager.cc:1403] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.005s
I20260709 13:13:16.136049 30757 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.136765 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.133906 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:16.137492 30706 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:16.138108 30751 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.138537 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.139007 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:16.139264 30483 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:16.140301 30753 raft_consensus.cc:359] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.141933 30812 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.142251 30753 raft_consensus.cc:385] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.141410 30758 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.142382 30812 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.142905 30753 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.143085 30812 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.143467 30758 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.143579 30751 ts_tablet_manager.cc:1434] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.022s	user 0.011s	sys 0.000s
I20260709 13:13:16.140977 30752 raft_consensus.cc:359] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.143887 30758 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.144217 30752 raft_consensus.cc:385] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.144547 30752 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.145431 30752 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.146602 30751 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.148106 30752 ts_tablet_manager.cc:1434] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:13:16.149284 30752 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.154551 30844 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.154543 30751 tablet_bootstrap.cc:654] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.158974 30753 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.160356 30754 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:16.162380 30754 ts_tablet_manager.cc:1403] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.004s
I20260709 13:13:16.163995 30753 ts_tablet_manager.cc:1434] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.003s
I20260709 13:13:16.165789 30753 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.165585 30754 raft_consensus.cc:359] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.166421 30754 raft_consensus.cc:385] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.166754 30754 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.167558 30754 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.170363 30754 ts_tablet_manager.cc:1434] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:13:16.171681 30754 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:16.173261 30844 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.173915 30844 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.174939 30753 tablet_bootstrap.cc:654] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.176965 30752 tablet_bootstrap.cc:654] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.177933 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.161098 30812 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:16.180078 30671 leader_election.cc:343] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.180825 30844 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.183089 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.183089 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.183775 30812 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.183954 30706 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:16.185564 30673 leader_election.cc:343] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.186131 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.187199 30844 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.182896 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:16.188647 30484 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.189447 30753 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.190102 30753 ts_tablet_manager.cc:1403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:13:16.193338 30758 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.193406 30753 raft_consensus.cc:359] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.194442 30753 raft_consensus.cc:385] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.194876 30753 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.201635 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.203632 30753 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.208628 30758 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.209502 30707 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.211079 30851 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.211668 30851 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.214135 30851 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:16.216246 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.216667 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.221418 30705 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:16.207370 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:16.222649 30482 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.223026 30448 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:13:16.225783 30450 leader_election.cc:343] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.226109 30851 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.226688 30851 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.227140 30851 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.232167 30844 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.236371 30482 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:16.247380 30706 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.248023 30844 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.248629 30844 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.248864 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:16.249873 30757 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.250487 30757 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:16.251564 30757 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.254657 30758 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.255326 30758 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.260264 30754 tablet_bootstrap.cc:654] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.254854 30484 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.257630 30758 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.274296 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.276356 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:16.279371 30596 leader_election.cc:343] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.279724 30705 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.282361 30844 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.282778 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.284669 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.286016 30558 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:16.287537 30707 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:16.288941 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.295866 30753 ts_tablet_manager.cc:1434] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.004s
I20260709 13:13:16.297044 30751 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.297115 30753 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.297752 30751 ts_tablet_manager.cc:1403] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.151s	user 0.030s	sys 0.012s
I20260709 13:13:16.276356 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.277472 30758 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.299211 30758 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:16.300261 30758 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.302480 30851 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.308811 30753 tablet_bootstrap.cc:654] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:16.312304 30597 leader_election.cc:343] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.312942 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:16.313416 30848 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:16.315045 30789 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.323601 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:16.323767 30885 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.325138 30851 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:16.314112 30751 raft_consensus.cc:359] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.327044 30751 raft_consensus.cc:385] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.327491 30751 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.328560 30751 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.330060 30559 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.339876 30846 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.343250 30851 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:16.347500 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.348625 30706 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.349852 30559 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:16.352432 30885 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.356655 30752 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.360256 30846 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.363538 30844 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.387540 30752 ts_tablet_manager.cc:1403] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.238s	user 0.012s	sys 0.017s
I20260709 13:13:16.368001 30754 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:16.390132 30754 ts_tablet_manager.cc:1403] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.219s	user 0.024s	sys 0.008s
I20260709 13:13:16.391182 30752 raft_consensus.cc:359] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.376991 30382 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:16.395478 30848 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.406456 30450 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:16.407773 30848 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.408883 30848 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:16.410135 30848 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.414516 30751 ts_tablet_manager.cc:1434] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.116s	user 0.009s	sys 0.003s
I20260709 13:13:16.419832 30752 raft_consensus.cc:385] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.421591 30843 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:16.425053 30752 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.426362 30752 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.420326 30751 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.442939 30754 raft_consensus.cc:359] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.443938 30754 raft_consensus.cc:385] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.444377 30754 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.449775 30752 ts_tablet_manager.cc:1434] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.062s	user 0.008s	sys 0.000s
I20260709 13:13:16.454401 30706 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:16.455849 30752 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.460639 30757 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.468488 30753 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.469204 30753 ts_tablet_manager.cc:1403] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.172s	user 0.055s	sys 0.091s
I20260709 13:13:16.451105 30754 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.472694 30754 ts_tablet_manager.cc:1434] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.001s
I20260709 13:13:16.479641 30754 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:16.479382 30753 raft_consensus.cc:359] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.480254 30753 raft_consensus.cc:385] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.480459 30757 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.469367 30751 tablet_bootstrap.cc:654] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.481210 30757 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.478156 30752 tablet_bootstrap.cc:654] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.480618 30753 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.486047 30753 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.488817 30753 ts_tablet_manager.cc:1434] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:13:16.490036 30753 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.491730 30754 tablet_bootstrap.cc:654] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.493489 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.494000 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.494343 30632 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:16.495625 30757 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:16.495869 30523 leader_election.cc:343] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.497218 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:16.498652 30757 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.499141 30757 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.499610 30757 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.502682 30753 tablet_bootstrap.cc:654] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.503629 30751 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.503647 30752 tablet_bootstrap.cc:492] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.504380 30751 ts_tablet_manager.cc:1403] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.009s
I20260709 13:13:16.504521 30752 ts_tablet_manager.cc:1403] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.007s
I20260709 13:13:16.496441 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:16.507894 30751 raft_consensus.cc:359] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.508502 30752 raft_consensus.cc:359] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.513249 30751 raft_consensus.cc:385] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.513455 30752 raft_consensus.cc:385] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.513998 30751 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.514117 30752 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.518147 30751 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.518272 30752 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.516218 30757 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.520998 30751 ts_tablet_manager.cc:1434] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 13:13:16.521468 30753 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.522126 30753 ts_tablet_manager.cc:1403] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.011s
I20260709 13:13:16.522258 30751 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.525056 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:16.525856 30631 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.527842 30753 raft_consensus.cc:359] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.528664 30753 raft_consensus.cc:385] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.528963 30753 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.529739 30753 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.531359 30751 tablet_bootstrap.cc:654] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.534190 30752 ts_tablet_manager.cc:1434] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:13:16.536186 30757 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:16.537611 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:16.539650 30523 leader_election.cc:343] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.540118 30753 ts_tablet_manager.cc:1434] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 13:13:16.541657 30753 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.544827 30752 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.548605 30751 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.549415 30751 ts_tablet_manager.cc:1403] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.004s
I20260709 13:13:16.549527 30753 tablet_bootstrap.cc:654] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.553089 30751 raft_consensus.cc:359] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.553992 30751 raft_consensus.cc:385] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.554313 30751 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.555284 30751 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.558671 30751 ts_tablet_manager.cc:1434] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:13:16.559894 30751 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.560252 30754 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:16.561178 30754 ts_tablet_manager.cc:1403] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.082s	user 0.008s	sys 0.019s
I20260709 13:13:16.548977 30907 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 13:13:16.575369 30631 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.577808 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:16.579429 30757 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.579645 30752 tablet_bootstrap.cc:654] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.579896 30758 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.580090 30757 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:16.580533 30758 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.581192 30757 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.583503 30844 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.584080 30844 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.588510 30753 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.588836 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.589205 30753 ts_tablet_manager.cc:1403] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.021s
I20260709 13:13:16.590397 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.590744 30843 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.591215 30844 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.591355 30843 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.593052 30751 tablet_bootstrap.cc:654] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.592991 30753 raft_consensus.cc:359] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.594065 30843 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.593931 30753 raft_consensus.cc:385] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.595683 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.595953 30753 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
W20260709 13:13:16.597568 30671 leader_election.cc:343] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.598820 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.596896 30753 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.599745 30558 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:16.597762 30482 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:16.601370 30670 leader_election.cc:343] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.601938 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.603915 30753 ts_tablet_manager.cc:1434] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 13:13:16.604235 30450 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:16.605356 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.606426 30885 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.606887 30885 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.607221 30885 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.612260 30758 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:16.612645 30753 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.613384 30751 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.613713 30844 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.614276 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.596124 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.615109 30558 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:16.613992 30751 ts_tablet_manager.cc:1403] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.000s
I20260709 13:13:16.618542 30751 raft_consensus.cc:359] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.619395 30751 raft_consensus.cc:385] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.619434 30848 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.619719 30751 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.620033 30848 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.620512 30848 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.621165 30752 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.620602 30751 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.622635 30752 ts_tablet_manager.cc:1403] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.078s	user 0.025s	sys 0.001s
I20260709 13:13:16.623320 30751 ts_tablet_manager.cc:1434] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:13:16.624434 30751 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.625764 30752 raft_consensus.cc:359] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.626546 30752 raft_consensus.cc:385] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.626878 30752 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.627766 30752 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.630486 30752 ts_tablet_manager.cc:1434] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:13:16.631453 30753 tablet_bootstrap.cc:654] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.631644 30752 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.632121 30751 tablet_bootstrap.cc:654] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.648358 30758 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.565832 30754 raft_consensus.cc:359] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.649132 30758 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.650516 30751 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.650898 30754 raft_consensus.cc:385] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.651491 30751 ts_tablet_manager.cc:1403] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.000s
I20260709 13:13:16.652105 30758 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:16.651624 30754 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.653582 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.655100 30707 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:16.655436 30751 raft_consensus.cc:359] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.656167 30751 raft_consensus.cc:385] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.656421 30751 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.656678 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.657647 30758 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.658051 30758 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.658560 30758 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.659973 30751 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.661408 30911 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.661908 30911 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.657974 30379 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:16.663918 30751 ts_tablet_manager.cc:1434] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 13:13:16.664242 30911 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.654222 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:16.665099 30751 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.648463 30752 tablet_bootstrap.cc:654] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.665468 30753 tablet_bootstrap.cc:492] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:16.671329 30753 ts_tablet_manager.cc:1403] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.066s	user 0.008s	sys 0.014s
I20260709 13:13:16.669862 30885 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.675282 30632 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:16.675508 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.676504 30885 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:16.676657 30558 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:16.679415 30754 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:16.682191 30522 leader_election.cc:343] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.690580 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:16.691152 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:16.691670 30558 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.695523 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.696305 30707 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:16.698005 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:16.699944 30914 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.700448 30914 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.700822 30914 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.702821 30911 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.714311 30753 raft_consensus.cc:359] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.715165 30753 raft_consensus.cc:385] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.715498 30753 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.715773 30914 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.716281 30753 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.718657 30753 ts_tablet_manager.cc:1434] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.000s
I20260709 13:13:16.719821 30753 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:16.719975 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.720646 30707 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.724056 30754 ts_tablet_manager.cc:1434] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.162s	user 0.012s	sys 0.001s
I20260709 13:13:16.694115 30848 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.728088 30558 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:16.729172 30707 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:16.729581 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.730526 30885 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.730649 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:16.731120 30751 tablet_bootstrap.cc:654] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.731186 30885 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:16.731782 30911 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.732681 30911 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:16.732292 30885 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.733657 30911 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.727432 30482 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 1: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:16.741870 30753 tablet_bootstrap.cc:654] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.725906 30754 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:16.748001 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:16.745095 30914 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:16.759897 30752 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:16.760747 30752 ts_tablet_manager.cc:1403] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.129s	user 0.024s	sys 0.017s
I20260709 13:13:16.761803 30751 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.762600 30751 ts_tablet_manager.cc:1403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.001s
W20260709 13:13:16.767668 30522 leader_election.cc:343] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.772063 30848 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.775309 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:16.780040 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:16.781343 30559 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:16.781759 30789 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:16.783432 30450 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.784741 30843 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.774767 30848 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.787050 30848 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.789235 30848 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:16.795919 30751 raft_consensus.cc:359] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.798576 30751 raft_consensus.cc:385] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.799039 30751 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.801721 30758 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.803066 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:16.804060 30706 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:16.805979 30448 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:16.807209 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.807716 30848 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:16.808353 30848 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:16.811126 30848 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.812887 30754 tablet_bootstrap.cc:654] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.812536 30751 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:16.813396 30911 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:16.808332 30632 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:16.818591 30758 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:16.820066 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.848502 30706 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.822814 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:16.851003 30559 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.857491 30379 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:16.846486 30751 ts_tablet_manager.cc:1434] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.083s	user 0.002s	sys 0.006s
I20260709 13:13:16.858642 30911 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:16.829267 30848 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:16.860546 30751 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:16.860850 30911 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.865463 30706 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:16.866282 30911 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:16.870744 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:16.872915 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:16.874248 30758 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:16.874868 30753 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
W20260709 13:13:16.874766 30523 leader_election.cc:343] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.875141 30758 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:16.875553 30753 ts_tablet_manager.cc:1403] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.156s	user 0.033s	sys 0.048s
I20260709 13:13:16.876529 30758 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:16.878939 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:16.881152 30753 raft_consensus.cc:359] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.882699 30753 raft_consensus.cc:385] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.883036 30753 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.872918 30379 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:16.889014 30753 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.889840 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:16.891736 30753 ts_tablet_manager.cc:1434] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:13:16.892935 30753 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
W20260709 13:13:16.892776 30522 leader_election.cc:343] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:16.893426 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:16.894761 30911 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:16.897334 30751 tablet_bootstrap.cc:654] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.901988 30843 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:13:16.898882 30885 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:13:16.912842 30796 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:16.918092 30753 tablet_bootstrap.cc:654] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:16.925741 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:16.926848 30632 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.879712 30559 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:16.935768 30752 raft_consensus.cc:359] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.936903 30752 raft_consensus.cc:385] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.937242 30752 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.940985 30752 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:16.950179 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:16.952055 30705 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:16.957052 30751 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:16.957748 30751 ts_tablet_manager.cc:1403] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.097s	user 0.015s	sys 0.009s
I20260709 13:13:16.958675 30752 ts_tablet_manager.cc:1434] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.198s	user 0.008s	sys 0.002s
I20260709 13:13:16.960577 30752 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:16.963984 30751 raft_consensus.cc:359] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:16.964869 30751 raft_consensus.cc:385] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:16.965166 30751 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:16.934266 30848 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:16.966291 30751 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.010145 30751 ts_tablet_manager.cc:1434] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.001s
I20260709 13:13:17.017796 30751 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:17.032991 30632 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:17.033497 30885 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.034021 30885 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.037525 30448 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:17.038601 30796 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.039191 30796 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:17.039482 30757 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.040159 30796 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.040247 30757 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.042455 30885 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:17.043644 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.044731 30559 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:17.069473 30753 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:17.070358 30753 ts_tablet_manager.cc:1403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.178s	user 0.014s	sys 0.037s
I20260709 13:13:17.071497 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:17.076745 30707 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.073909 30753 raft_consensus.cc:359] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.079195 30753 raft_consensus.cc:385] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.079547 30753 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.085695 30753 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.077469 30757 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.092787 30754 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:17.094033 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:17.093426 30754 ts_tablet_manager.cc:1403] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.368s	user 0.027s	sys 0.026s
W20260709 13:13:17.095821 30596 leader_election.cc:343] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.098577 30754 raft_consensus.cc:359] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.099391 30754 raft_consensus.cc:385] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.099659 30754 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.100438 30754 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.087268 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.102843 30754 ts_tablet_manager.cc:1434] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:13:17.103905 30754 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:17.083982 30885 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.115887 30844 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:17.093185 30448 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 52725.9us
I20260709 13:13:17.110129 30885 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.120348 30753 ts_tablet_manager.cc:1434] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.049s	user 0.005s	sys 0.004s
I20260709 13:13:17.120949 30751 tablet_bootstrap.cc:654] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.128116 30705 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:17.129151 30752 tablet_bootstrap.cc:654] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.132117 30753 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
W20260709 13:13:17.118428 30523 leader_election.cc:343] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.145016 30757 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.120308 30844 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.147082 30936 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.148142 30844 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.147779 30936 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.134167 30758 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.151151 30758 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.153685 30758 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:17.121814 30885 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.154757 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.155078 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:17.155820 30706 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:17.156602 30599 leader_election.cc:343] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.157007 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.157671 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:17.158967 30673 leader_election.cc:343] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.158428 30633 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:17.167227 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:17.167054 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:17.168612 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.170223 30930 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.170728 30930 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.171085 30930 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:17.171222 30522 leader_election.cc:343] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.171859 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.173144 30914 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:17.175747 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:17.177158 30936 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:17.178184 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:17.179136 30482 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.179184 30706 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:17.152864 30379 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.183980 30844 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.184477 30448 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:17.184497 30379 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.184475 30844 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.186342 30844 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.203544 30914 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.204466 30914 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.205998 30754 tablet_bootstrap.cc:654] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.208834 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:17.209255 30846 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.211100 30706 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:17.210036 30846 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.215423 30885 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.215834 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:17.217010 30757 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.218971 30812 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.219619 30812 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.220119 30812 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.224186 30930 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.225924 30751 tablet_bootstrap.cc:492] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:17.226976 30751 ts_tablet_manager.cc:1403] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.215s	user 0.015s	sys 0.000s
I20260709 13:13:17.228407 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:17.230304 30751 raft_consensus.cc:359] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.232822 30751 raft_consensus.cc:385] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.233302 30751 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.230563 30753 tablet_bootstrap.cc:654] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.230356 30914 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:17.230315 30599 leader_election.cc:343] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.245682 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:17.251706 30522 leader_election.cc:343] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.252103 30758 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.252646 30846 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.254644 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.255189 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.260663 30751 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.265337 30936 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:17.269011 30930 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:17.242112 30844 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.273360 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.276328 30752 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:17.278752 30752 ts_tablet_manager.cc:1403] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.319s	user 0.020s	sys 0.007s
I20260709 13:13:17.284763 30751 ts_tablet_manager.cc:1434] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.002s
I20260709 13:13:17.287045 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:17.287782 30557 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.288091 30754 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:17.288719 30754 ts_tablet_manager.cc:1403] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.185s	user 0.016s	sys 0.030s
I20260709 13:13:17.293450 30844 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.294505 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:17.295368 30559 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.297284 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:17.297948 30757 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:17.301986 30707 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:17.304073 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:17.276050 30885 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:17.305613 30885 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:17.310992 30930 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.313243 30751 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:17.315678 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:17.320406 30557 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:17.325376 30812 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.326571 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:17.328760 30930 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.329605 30930 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:17.332614 30559 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:17.331594 30930 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:17.350222 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83272us
W20260709 13:13:17.342907 30596 leader_election.cc:343] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:17.337356 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64866us
I20260709 13:13:17.363706 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:17.364715 30631 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:17.370644 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:17.371887 30707 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.374054 30812 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:17.378932 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:17.380497 30752 raft_consensus.cc:359] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:17.380836 30522 leader_election.cc:343] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.377583 30751 tablet_bootstrap.cc:654] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.384915 30752 raft_consensus.cc:385] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.385489 30752 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.386356 30752 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.386982 30754 raft_consensus.cc:359] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.388032 30754 raft_consensus.cc:385] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.388386 30754 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.390997 30754 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.393935 30752 ts_tablet_manager.cc:1434] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 13:13:17.395176 30752 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:17.383999 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.400269 30380 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.403264 30945 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.403992 30945 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:17.405182 30945 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.406402 30812 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:17.405714 30948 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 13:13:17.413164 30754 ts_tablet_manager.cc:1434] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.124s	user 0.006s	sys 0.004s
I20260709 13:13:17.414645 30754 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:17.426187 30753 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:17.427233 30753 ts_tablet_manager.cc:1403] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.305s	user 0.012s	sys 0.020s
I20260709 13:13:17.428893 30707 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:17.431644 30753 raft_consensus.cc:359] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.432680 30753 raft_consensus.cc:385] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.433022 30753 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.434047 30753 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.445998 30753 ts_tablet_manager.cc:1434] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 13:13:17.446491 30752 tablet_bootstrap.cc:654] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.448352 30753 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:17.455197 30848 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.456105 30848 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.445781 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:17.474843 30812 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.475731 30812 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:17.477006 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.478605 30848 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:17.479517 30448 leader_election.cc:343] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.479329 30885 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.477082 30812 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.483054 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.483973 30558 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:17.485482 30450 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.493294 30848 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.496062 30846 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:17.502285 30848 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.507972 30848 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.513473 30751 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:17.514199 30751 ts_tablet_manager.cc:1403] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.201s	user 0.028s	sys 0.045s
I20260709 13:13:17.522545 30930 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.524282 30753 tablet_bootstrap.cc:654] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.528501 30754 tablet_bootstrap.cc:654] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.581184 30751 raft_consensus.cc:359] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.582542 30751 raft_consensus.cc:385] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.582942 30751 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.590286 30751 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.594452 30789 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.595381 30789 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.598737 30943 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.600075 30789 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:17.601984 30957 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:13:17.605222 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:17.607757 30848 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.614118 30705 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:17.615842 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:17.616045 30848 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.617153 30930 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.617612 30930 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.609053 30752 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:17.617961 30930 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.618392 30752 ts_tablet_manager.cc:1403] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.223s	user 0.025s	sys 0.026s
I20260709 13:13:17.619618 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:17.610924 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:17.621564 30448 leader_election.cc:343] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:17.622587 30596 leader_election.cc:343] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.623485 30752 raft_consensus.cc:359] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.624533 30752 raft_consensus.cc:385] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.624890 30752 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.625871 30752 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.628666 30752 ts_tablet_manager.cc:1434] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:13:17.630141 30752 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:17.631863 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:17.640306 30558 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.612577 30751 ts_tablet_manager.cc:1434] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.098s	user 0.014s	sys 0.003s
I20260709 13:13:17.641197 30754 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:17.641775 30754 ts_tablet_manager.cc:1403] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.227s	user 0.026s	sys 0.012s
I20260709 13:13:17.642481 30751 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:17.639971 30930 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.646106 30753 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:17.646731 30753 ts_tablet_manager.cc:1403] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.200s	user 0.023s	sys 0.013s
I20260709 13:13:17.649976 30753 raft_consensus.cc:359] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.651144 30753 raft_consensus.cc:385] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.651415 30753 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.656032 30753 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.659433 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:17.660383 30705 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.663938 30752 tablet_bootstrap.cc:654] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.680711 30930 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:17.682766 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:17.687305 30753 ts_tablet_manager.cc:1434] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.040s	user 0.005s	sys 0.003s
I20260709 13:13:17.688840 30753 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
W20260709 13:13:17.689651 30596 leader_election.cc:343] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.697553 30753 tablet_bootstrap.cc:654] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.699646 30752 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:17.700354 30752 ts_tablet_manager.cc:1403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.071s	user 0.020s	sys 0.001s
I20260709 13:13:17.687479 30751 tablet_bootstrap.cc:654] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.703248 30943 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:13:17.696626 30705 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:17.712954 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:17.700507 30379 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.714553 30930 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.715037 30558 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:17.715152 30930 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:17.716531 30930 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.718585 30450 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.719287 30753 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:17.719937 30753 ts_tablet_manager.cc:1403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.008s
I20260709 13:13:17.720111 30943 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.720722 30943 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:17.721633 30943 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.723670 30753 raft_consensus.cc:359] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.724578 30753 raft_consensus.cc:385] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.724915 30753 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.725780 30753 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.728161 30752 raft_consensus.cc:359] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.728790 30753 ts_tablet_manager.cc:1434] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:13:17.728996 30752 raft_consensus.cc:385] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.729034 30751 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:17.729358 30752 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.730134 30751 ts_tablet_manager.cc:1403] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.000s
I20260709 13:13:17.730525 30753 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:17.735031 30844 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.735783 30844 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.712921 30754 raft_consensus.cc:359] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.737192 30754 raft_consensus.cc:385] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.737535 30754 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.738312 30754 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.730268 30752 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.742214 30751 raft_consensus.cc:359] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.743287 30751 raft_consensus.cc:385] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.743702 30751 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.744581 30751 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.745298 30844 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.746163 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:17.747359 30558 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:17.747650 30751 ts_tablet_manager.cc:1434] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:13:17.748306 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.749082 30751 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:17.748783 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:17.750059 30844 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.750749 30844 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.750124 30381 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.751176 30844 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.754228 30753 tablet_bootstrap.cc:654] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:17.755724 30671 leader_election.cc:343] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.760571 30758 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.762751 30752 ts_tablet_manager.cc:1434] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:13:17.766374 30752 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:17.773308 30754 ts_tablet_manager.cc:1434] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.131s	user 0.008s	sys 0.000s
I20260709 13:13:17.774641 30754 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:17.785962 30482 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.787109 30752 tablet_bootstrap.cc:654] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.790357 30945 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.791106 30945 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.794003 30945 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:17.795440 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.796283 30633 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:17.798563 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:17.800082 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:17.801837 30933 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:17.802255 30933 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:17.802594 30933 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.800549 30751 tablet_bootstrap.cc:654] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:17.807492 30670 leader_election.cc:343] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.813653 30844 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.817960 30751 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:17.818653 30751 ts_tablet_manager.cc:1403] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.070s	user 0.014s	sys 0.000s
I20260709 13:13:17.819629 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:17.820402 30558 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.822118 30751 raft_consensus.cc:359] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.822974 30751 raft_consensus.cc:385] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.823268 30751 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.831756 30751 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.834843 30380 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.837004 30751 ts_tablet_manager.cc:1434] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:13:17.838272 30751 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:17.839895 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:13:17.841547 30671 leader_election.cc:343] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.844908 30753 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:17.845336 30844 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:17.845530 30753 ts_tablet_manager.cc:1403] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.116s	user 0.027s	sys 0.028s
I20260709 13:13:17.846909 30558 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:17.848403 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.848801 30754 tablet_bootstrap.cc:654] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.848897 30753 raft_consensus.cc:359] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.849905 30753 raft_consensus.cc:385] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:17.850014 30844 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.850346 30753 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:17.850768 30844 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:17.851347 30753 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.852250 30844 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:17.854208 30753 ts_tablet_manager.cc:1434] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:13:17.855500 30962 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.856262 30962 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.846925 30933 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:17.860091 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:17.860167 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:17.861913 30522 leader_election.cc:343] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.862082 30962 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:17.862718 30523 leader_election.cc:343] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.863240 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:17.864511 30962 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:17.865563 30751 tablet_bootstrap.cc:654] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.869621 30753 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:17.870044 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:17.870744 30633 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:17.877161 30753 tablet_bootstrap.cc:654] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:17.887810 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:17.890142 30670 leader_election.cc:343] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.890983 30945 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.892966 30758 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.898780 30945 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:17.886430 30933 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:17.935585 30758 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.936321 30758 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:17.940604 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:17.914526 30633 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:17.942482 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:17.952993 30382 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:17.958194 30754 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:17.966873 30754 ts_tablet_manager.cc:1403] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.193s	user 0.011s	sys 0.011s
I20260709 13:13:17.956890 30844 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:17.960762 30933 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:17.972473 30381 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:17.967303 30758 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:17.980413 30933 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:17.995555 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:17.997453 30673 leader_election.cc:343] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:17.998301 30933 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.001303 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.002514 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.003639 30754 raft_consensus.cc:359] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.004691 30754 raft_consensus.cc:385] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.004806 30557 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 1: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:18.005023 30754 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.005951 30754 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:18.007050 30670 leader_election.cc:343] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.007651 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:18.008702 30754 ts_tablet_manager.cc:1434] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:13:18.010041 30754 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:18.014487 30753 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:18.015210 30753 ts_tablet_manager.cc:1403] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.146s	user 0.031s	sys 0.018s
I20260709 13:13:17.975155 30844 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:18.016394 30844 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.018589 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:18.020123 30758 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.018499 30753 raft_consensus.cc:359] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.024698 30753 raft_consensus.cc:385] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.025112 30753 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.025736 30752 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.026552 30752 ts_tablet_manager.cc:1403] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.261s	user 0.021s	sys 0.038s
I20260709 13:13:18.014681 30933 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:18.031188 30754 tablet_bootstrap.cc:654] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.033751 30751 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:18.034384 30751 ts_tablet_manager.cc:1403] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.196s	user 0.025s	sys 0.045s
I20260709 13:13:18.025265 30914 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.040488 30753 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.038247 30914 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.037705 30751 raft_consensus.cc:359] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.045145 30751 raft_consensus.cc:385] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.045559 30751 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.048880 30751 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.053866 30983 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.055600 30753 ts_tablet_manager.cc:1434] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.003s
I20260709 13:13:18.061353 30914 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.063213 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:18.065441 30522 leader_election.cc:343] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.070134 30753 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:18.054738 30983 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.078353 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:18.080152 30670 leader_election.cc:343] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.083890 30751 ts_tablet_manager.cc:1434] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.002s
I20260709 13:13:18.084964 30751 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
W20260709 13:13:18.094863 30522 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:18.100551 30962 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.101102 30752 raft_consensus.cc:359] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:18.103420 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:18.104132 30983 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.107139 30752 raft_consensus.cc:385] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.107674 30752 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.109336 30752 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.112416 30752 ts_tablet_manager.cc:1434] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.085s	user 0.006s	sys 0.002s
I20260709 13:13:18.113557 30752 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.119796 30382 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:18.132714 30753 tablet_bootstrap.cc:654] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.135315 30752 tablet_bootstrap.cc:654] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.120082 30758 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:18.121187 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.147619 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.148491 30558 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:18.150920 30705 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:18.152563 30484 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.157931 30751 tablet_bootstrap.cc:654] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.157550 30754 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:18.159538 30754 ts_tablet_manager.cc:1403] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.013s
I20260709 13:13:18.164314 30848 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.164928 30848 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.168352 30758 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.170928 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.175220 30933 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.175601 30933 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.175555 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.176137 30933 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.177078 30962 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.177515 30962 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.177856 30962 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.182155 30753 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:18.182890 30753 ts_tablet_manager.cc:1403] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.125s	user 0.009s	sys 0.010s
I20260709 13:13:18.189700 30933 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.192988 30380 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:18.193857 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.202344 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.198500 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:18.203444 30707 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:18.163038 30754 raft_consensus.cc:359] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.204934 30754 raft_consensus.cc:385] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.205368 30754 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.207621 30754 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.207695 30448 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:18.208715 30483 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.214810 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:18.198920 30753 raft_consensus.cc:359] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.201256 30848 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.222409 30753 raft_consensus.cc:385] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.194936 30933 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.232024 30751 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:18.233515 30943 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:18.235680 30751 ts_tablet_manager.cc:1403] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.151s	user 0.014s	sys 0.019s
I20260709 13:13:18.236383 30943 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:18.236930 30943 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:18.234880 30483 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:18.222849 30753 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.239647 30752 tablet_bootstrap.cc:492] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.246039 30752 ts_tablet_manager.cc:1403] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.133s	user 0.022s	sys 0.016s
I20260709 13:13:18.231776 30962 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.247243 30557 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:18.249349 30796 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.252135 30962 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.240702 30753 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.254390 30753 ts_tablet_manager.cc:1434] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.001s
I20260709 13:13:18.254993 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:18.255594 30753 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:18.255971 30482 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:18.241359 30754 ts_tablet_manager.cc:1434] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.081s	user 0.009s	sys 0.000s
I20260709 13:13:18.261925 30754 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:18.262868 30752 raft_consensus.cc:359] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.263734 30752 raft_consensus.cc:385] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.264137 30752 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.264883 30752 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.243772 30751 raft_consensus.cc:359] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.267778 30751 raft_consensus.cc:385] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.268038 30751 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.268769 30751 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.271071 30751 ts_tablet_manager.cc:1434] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:13:18.270965 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:18.272238 30933 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:18.272387 30751 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:18.272830 30933 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:18.273741 30933 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.282102 30754 tablet_bootstrap.cc:654] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.286885 30753 tablet_bootstrap.cc:654] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.287480 30751 tablet_bootstrap.cc:654] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.289085 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:18.293931 30758 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:18.301627 30752 ts_tablet_manager.cc:1434] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.000s
I20260709 13:13:18.302909 30752 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.291795 30943 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.307504 30484 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:18.308732 30943 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.308905 30848 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.310891 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:18.312929 30751 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:18.313467 30751 ts_tablet_manager.cc:1403] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.004s
I20260709 13:13:18.314426 30933 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.315008 30933 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.316069 30752 tablet_bootstrap.cc:654] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.316845 30751 raft_consensus.cc:359] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.317430 30933 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.317658 30751 raft_consensus.cc:385] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.317956 30751 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.318981 30751 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.320676 30753 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:18.321302 30753 ts_tablet_manager.cc:1403] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.066s	user 0.007s	sys 0.005s
I20260709 13:13:18.321043 30930 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:18.323196 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.321322 30751 ts_tablet_manager.cc:1434] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:13:18.325477 30751 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:18.325301 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:18.327409 30758 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:18.327946 30758 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:18.329135 30758 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.329752 30753 raft_consensus.cc:359] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.330478 30753 raft_consensus.cc:385] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.330799 30758 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:18.333686 30751 tablet_bootstrap.cc:654] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.333190 30380 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:13:18.327066 30671 leader_election.cc:343] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.327062 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.340107 30752 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.340807 30752 ts_tablet_manager.cc:1403] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.003s
W20260709 13:13:18.345005 30670 leader_election.cc:343] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.344923 30752 raft_consensus.cc:359] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.345758 30752 raft_consensus.cc:385] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.345613 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:18.346086 30752 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.347060 30752 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.349041 30933 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:18.350298 30752 ts_tablet_manager.cc:1434] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.002s
I20260709 13:13:18.351984 30752 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.358794 30448 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:18.359828 30848 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:18.361549 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:18.362871 30962 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:18.377960 30751 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:18.378643 30751 ts_tablet_manager.cc:1403] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.053s	user 0.007s	sys 0.004s
I20260709 13:13:18.387756 30752 tablet_bootstrap.cc:654] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.330824 30753 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.399237 30753 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.382004 30751 raft_consensus.cc:359] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.400290 30751 raft_consensus.cc:385] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.401103 30754 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:18.401458 30752 tablet_bootstrap.cc:492] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.401700 30754 ts_tablet_manager.cc:1403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.140s	user 0.037s	sys 0.025s
I20260709 13:13:18.401971 30753 ts_tablet_manager.cc:1434] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.080s	user 0.009s	sys 0.000s
I20260709 13:13:18.401999 30752 ts_tablet_manager.cc:1403] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.003s
I20260709 13:13:18.403223 30753 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:18.400599 30751 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.404899 30754 raft_consensus.cc:359] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.406940 30754 raft_consensus.cc:385] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.407264 30754 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.405126 30752 raft_consensus.cc:359] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.408325 30752 raft_consensus.cc:385] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.408087 30754 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.408628 30752 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.409744 30751 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.411345 30754 ts_tablet_manager.cc:1434] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:13:18.412616 30754 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:18.413169 30752 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.428622 30752 ts_tablet_manager.cc:1434] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.026s	user 0.005s	sys 0.001s
I20260709 13:13:18.419309 30631 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.435786 30933 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.439478 30752 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.443740 30930 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.444543 30930 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.447400 30752 tablet_bootstrap.cc:654] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.447989 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.448807 30557 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:18.419514 30754 tablet_bootstrap.cc:654] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.449667 30933 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.450641 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:18.452905 30789 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:18.453267 30789 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.453675 30789 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:18.422338 30753 tablet_bootstrap.cc:654] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:18.462370 30670 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:18.416824 30751 ts_tablet_manager.cc:1434] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 13:13:18.468658 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.466499 30930 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.479146 30751 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:18.486142 30789 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.487664 30633 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:13:18.489638 30933 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.490257 30933 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.495949 30962 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.495124 30983 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.496551 30962 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.496869 30983 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:18.497422 30983 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:18.498129 30844 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:18.503547 30962 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.507759 30933 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:18.509140 30752 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.509750 30752 ts_tablet_manager.cc:1403] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.071s	user 0.017s	sys 0.006s
I20260709 13:13:18.510341 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.511152 30707 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:18.509253 30846 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.512858 30846 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.513283 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.513365 30752 raft_consensus.cc:359] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.514407 30752 raft_consensus.cc:385] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.514827 30752 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.515753 30846 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.515849 30752 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:18.517472 30522 leader_election.cc:343] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.518225 30983 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.518846 30752 ts_tablet_manager.cc:1434] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:13:18.519083 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.520639 30752 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.507036 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.522014 30558 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
W20260709 13:13:18.522733 30670 leader_election.cc:343] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.523582 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.524653 30914 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.525401 30914 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.525972 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:18.528622 30752 tablet_bootstrap.cc:654] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.528756 30914 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.529783 30914 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.536624 30844 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.537195 30844 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.537593 30844 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.541168 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:18.541908 30558 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:18.542028 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.542843 30632 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:18.541000 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.543967 30631 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:18.545617 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.545642 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.546929 30633 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:18.548261 30983 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:18.551059 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.552114 30984 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.552552 30984 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.552933 30984 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.557242 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:18.558306 30789 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.560595 30789 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.530519 30914 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.562161 30914 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.548290 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:18.561255 30789 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.566596 30558 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:18.567940 30751 tablet_bootstrap.cc:654] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.576972 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:18.578121 30930 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:18.578866 30930 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:18.583141 30930 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.592430 30753 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:18.593025 30753 ts_tablet_manager.cc:1403] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.190s	user 0.016s	sys 0.004s
I20260709 13:13:18.587975 30992 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.600548 30992 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.600996 30992 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.590305 30962 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.607064 30962 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.615324 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.616051 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.619011 30962 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:18.620114 30596 leader_election.cc:343] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:18.621708 30522 leader_election.cc:343] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.623656 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.624801 30911 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.624935 30984 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.625459 30911 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.628786 30911 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:18.631263 30523 leader_election.cc:343] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.648083 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:18.653831 30911 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:18.636924 30984 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.637291 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:18.639274 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.640170 30914 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.655735 30558 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:18.657053 30752 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:18.632797 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:18.635643 30789 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.657400 30914 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:18.642462 30753 raft_consensus.cc:359] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:18.658669 30670 leader_election.cc:343] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.640861 30844 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:18.659232 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43699.2us
I20260709 13:13:18.661388 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.667665 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:18.669880 30753 raft_consensus.cc:385] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.670779 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:18.675040 30789 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.681308 30754 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:18.679121 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:18.682161 30754 ts_tablet_manager.cc:1403] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.270s	user 0.067s	sys 0.050s
I20260709 13:13:18.672338 30707 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:18.670025 30752 ts_tablet_manager.cc:1403] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.013s
I20260709 13:13:18.676981 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:18.678078 30753 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.684295 30753 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:18.685649 30523 leader_election.cc:343] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.685887 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:18.686537 30704 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.686870 30753 ts_tablet_manager.cc:1434] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.094s	user 0.011s	sys 0.000s
I20260709 13:13:18.671962 30844 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:18.672741 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:18.692313 30753 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:18.681416 30789 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.688395 30631 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.694139 30705 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 13:13:18.691752 30706 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:18.696593 30754 raft_consensus.cc:359] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.697350 30754 raft_consensus.cc:385] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.697608 30754 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.697855 30789 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.698381 30754 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.700910 30754 ts_tablet_manager.cc:1434] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:13:18.702750 30633 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.708998 30753 tablet_bootstrap.cc:654] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:18.707885 30522 leader_election.cc:343] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.701985 30992 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.711584 30558 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:18.715371 30844 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.716789 30484 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.726404 30754 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:18.724539 30704 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:18.746320 30751 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:18.746995 30751 ts_tablet_manager.cc:1403] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.268s	user 0.058s	sys 0.054s
I20260709 13:13:18.749512 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:18.757609 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.760924 30911 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:18.765614 30752 raft_consensus.cc:359] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.813515 30752 raft_consensus.cc:385] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.759122 30378 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:18.813912 30633 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:18.810527 30753 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:18.815151 30631 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:18.815958 30753 ts_tablet_manager.cc:1403] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.059s
I20260709 13:13:18.813942 30752 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.817399 30752 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.815519 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:18.821499 30753 raft_consensus.cc:359] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.822464 30753 raft_consensus.cc:385] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.822868 30753 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.823992 30753 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.825289 30984 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:18.825822 30984 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:18.826355 30752 ts_tablet_manager.cc:1434] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.143s	user 0.011s	sys 0.004s
I20260709 13:13:18.827112 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:18.826772 30984 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.827576 30752 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:18.827773 30704 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:18.832897 30911 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:18.833427 30911 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:18.835322 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:18.835407 30753 ts_tablet_manager.cc:1434] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:13:18.778156 30754 tablet_bootstrap.cc:654] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.750249 30751 raft_consensus.cc:359] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.764592 30992 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:18.810432 30983 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:18.787994 30885 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.841689 30885 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.844228 30885 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.734688 30789 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:18.836997 30846 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:18.840827 30751 raft_consensus.cc:385] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:18.846094 30751 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:18.846158 30846 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:18.847038 30751 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.847321 30846 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.848845 30751 ts_tablet_manager.cc:1434] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.102s	user 0.001s	sys 0.015s
I20260709 13:13:18.849942 30751 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:18.858299 30751 tablet_bootstrap.cc:654] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.860635 30789 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.861093 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:18.870492 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.761880 30962 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:18.871411 30559 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:18.871609 30596 leader_election.cc:343] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.872320 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:18.912969 30848 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:18.914077 30631 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:18.913921 30848 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.915345 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:18.916749 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:18.917936 30848 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:18.919549 30557 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:18.920609 30596 leader_election.cc:343] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:18.921092 30754 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:18.921679 30754 ts_tablet_manager.cc:1403] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.208s	user 0.018s	sys 0.006s
I20260709 13:13:18.859426 30753 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:18.928737 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:18.913193 30962 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:18.939608 30962 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:18.944706 30846 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.931075 30911 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:18.955521 30754 raft_consensus.cc:359] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.945456 30846 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:18.976578 30752 tablet_bootstrap.cc:654] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.978333 30753 tablet_bootstrap.cc:654] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:18.989682 30848 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:18.985536 30789 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.005429 30754 raft_consensus.cc:385] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.009469 30754 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.012703 30846 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.006059 30751 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:19.015450 30751 ts_tablet_manager.cc:1403] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.166s	user 0.006s	sys 0.019s
I20260709 13:13:19.010555 30754 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.023948 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.014183 30848 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.032090 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.032717 30911 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:19.032886 30848 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:19.017194 30789 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.034921 30706 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 13:13:19.040439 30706 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 0
I20260709 13:13:19.040728 30706 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:13:19.040994 30706 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:19.013599 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.042584 30758 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:19.044875 30758 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.045401 30758 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.056811 30911 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:13:19.058336 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63429.1us
I20260709 13:13:19.059602 30559 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:19.050125 30631 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:19.062454 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:19.064371 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.067272 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.068104 30706 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:19.068994 30885 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:19.046896 30789 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.085126 30758 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.087726 30930 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 13:13:19.104334 30751 raft_consensus.cc:359] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.105266 30751 raft_consensus.cc:385] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.105554 30751 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.097235 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:19.108078 30704 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.106371 30751 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.118603 30758 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:19.121838 30754 ts_tablet_manager.cc:1434] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.200s	user 0.020s	sys 0.034s
I20260709 13:13:19.123225 30754 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:19.123340 30752 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:19.124132 30752 ts_tablet_manager.cc:1403] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.297s	user 0.016s	sys 0.016s
W20260709 13:13:19.181506 30523 leader_election.cc:343] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.191987 30751 ts_tablet_manager.cc:1434] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.176s	user 0.009s	sys 0.001s
I20260709 13:13:19.200119 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.204548 30911 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.214079 30911 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:19.201048 30751 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:19.226778 30481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:19.095846 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:19.230576 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 141689us
I20260709 13:13:19.246940 30911 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.230973 31031 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:19.248698 31031 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.249964 31031 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.235863 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.236173 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:19.272007 30670 leader_election.cc:343] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:19.274165 30596 leader_election.cc:343] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.276556 30751 tablet_bootstrap.cc:654] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:19.281075 30596 leader_election.cc:343] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.281409 31021 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.282388 31021 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.285848 31021 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:19.287264 30523 leader_election.cc:343] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.292403 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.292982 30911 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.293843 30632 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:19.294005 30911 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.302125 30448 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:19.303870 31021 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:19.319531 30378 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:19.327550 30844 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:19.342224 30382 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:19.352088 30753 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:19.352890 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.353093 30753 ts_tablet_manager.cc:1403] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.494s	user 0.024s	sys 0.006s
I20260709 13:13:19.356271 30754 tablet_bootstrap.cc:654] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.353601 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.363122 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.365236 30751 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:19.368217 30751 ts_tablet_manager.cc:1403] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.168s	user 0.013s	sys 0.017s
I20260709 13:13:19.353981 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.399072 30751 raft_consensus.cc:359] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:19.400331 30751 raft_consensus.cc:385] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.400774 30751 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.400400 31031 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.399298 30752 raft_consensus.cc:359] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.402864 30752 raft_consensus.cc:385] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.403337 30752 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.404349 30752 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.401911 30751 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:19.414081 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 343274us
I20260709 13:13:19.412284 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:19.415498 30751 ts_tablet_manager.cc:1434] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:13:19.416165 30707 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.417101 30751 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:19.426950 31031 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.427942 30911 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:19.439286 30448 leader_election.cc:343] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.442153 30844 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:19.449828 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.449577 30752 ts_tablet_manager.cc:1434] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.325s	user 0.013s	sys 0.002s
I20260709 13:13:19.461284 30751 tablet_bootstrap.cc:654] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:19.447606 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90513us
I20260709 13:13:19.475423 30752 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:19.483407 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.484720 30703 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 0.
I20260709 13:13:19.486858 30484 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:19.497604 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82208.8us
I20260709 13:13:19.501034 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:19.516431 30448 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.518656 30752 tablet_bootstrap.cc:654] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.521697 30984 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 13:13:19.522992 30754 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:19.531265 30448 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.523800 30754 ts_tablet_manager.cc:1403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.401s	user 0.036s	sys 0.042s
W20260709 13:13:19.523576 30522 leader_election.cc:343] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.535832 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:19.511086 30753 raft_consensus.cc:359] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:19.537621 30753 raft_consensus.cc:385] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.538106 30753 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.542512 30704 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:19.542917 31030 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:19.544190 31030 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.544643 31030 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:19.544798 30596 leader_election.cc:343] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.547914 30754 raft_consensus.cc:359] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:19.548758 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:19.548853 30754 raft_consensus.cc:385] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.550514 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.551435 30754 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
W20260709 13:13:19.529855 30557 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.602s	user 0.008s	sys 0.018s
I20260709 13:13:19.553993 31036 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:19.553758 30557 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:19.553148 30992 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:19.553227 30754 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.568568 30911 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:19.569380 30911 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:19.570907 30911 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:19.573151 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:19.574280 30559 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.568590 30751 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:19.588999 30751 ts_tablet_manager.cc:1403] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.001s
W20260709 13:13:19.591655 30799 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:13:19.593729 30522 leader_election.cc:343] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.602312 30751 raft_consensus.cc:359] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.603134 30751 raft_consensus.cc:385] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.603416 30751 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.606410 31021 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:19.632309 31021 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.633126 31021 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.636345 31021 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:19.638504 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.639025 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.643777 30751 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.646274 30751 ts_tablet_manager.cc:1434] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.057s	user 0.007s	sys 0.000s
I20260709 13:13:19.628592 30753 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:19.649185 30753 ts_tablet_manager.cc:1434] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.291s	user 0.009s	sys 0.000s
I20260709 13:13:19.652527 31036 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.653342 31036 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:19.657286 30448 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.659108 31021 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:19.662954 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.661973 30707 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:19.671882 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.672585 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.674073 30754 ts_tablet_manager.cc:1434] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.141s	user 0.007s	sys 0.008s
I20260709 13:13:19.673956 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.674769 30945 raft_consensus.cc:2764] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:13:19.675781 30754 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:19.674825 30930 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:19.677337 30930 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:19.678443 30930 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.679709 30945 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:19.686996 30753 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:19.664520 31036 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:19.654487 30559 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:19.698566 31030 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.703435 30789 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.703665 31030 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:19.704499 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:19.704309 30789 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:19.709906 30448 leader_election.cc:343] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:19.713305 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:19.720856 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:19.721295 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.723645 31031 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:19.724012 30789 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.724241 31031 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:19.725472 31031 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:19.726791 30523 leader_election.cc:343] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.721762 30706 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.732546 30752 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:19.733225 30752 ts_tablet_manager.cc:1403] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.258s	user 0.026s	sys 0.064s
I20260709 13:13:19.738306 30752 raft_consensus.cc:359] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.741544 30752 raft_consensus.cc:385] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.741955 30752 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.751183 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.752323 30482 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:19.753013 30752 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:19.755076 30523 leader_election.cc:343] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.755702 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.756760 30962 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:13:19.758291 30523 leader_election.cc:343] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.756834 30752 ts_tablet_manager.cc:1434] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.001s
I20260709 13:13:19.755504 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.761127 30885 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:19.761703 30885 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:19.762923 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:19.764158 31041 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:19.764719 31041 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.765169 31041 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:19.766062 30751 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:19.768749 30758 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.769340 30758 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.772689 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:19.774567 30596 leader_election.cc:343] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:19.774957 31044 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:13:19.775813 30758 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:19.777308 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:19.762952 30885 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.758858 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.796491 30992 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:19.813905 30945 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.814594 30945 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:19.822783 30945 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:19.793746 30706 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:19.827795 30754 tablet_bootstrap.cc:654] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.832064 30752 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:19.833323 30448 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.841265 30753 tablet_bootstrap.cc:654] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.833784 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:19.835173 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:19.826841 30911 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:19.848033 30751 tablet_bootstrap.cc:654] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.822474 31041 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.856298 30752 tablet_bootstrap.cc:654] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:19.859287 30557 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:19.859333 30984 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:19.859411 31047 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:19.861343 31047 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:19.862433 31047 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.865785 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:19.866931 30789 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:19.867380 30789 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.867755 30789 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.852264 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:19.887384 30480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.860657 30984 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:19.900203 30984 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.903635 30752 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:19.904587 30752 ts_tablet_manager.cc:1403] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.079s	user 0.015s	sys 0.008s
I20260709 13:13:19.906586 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:19.907913 30631 raft_consensus.cc:2368] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:19.911206 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:19.912352 30671 leader_election.cc:400] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:19.921681 30671 leader_election.cc:403] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:19.923791 30983 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:19.924431 30983 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:19.924878 30983 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.928404 31041 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:19.913326 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:19.930146 30984 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:19.854096 30911 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:19.945021 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:19.947988 30522 leader_election.cc:343] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.945194 30752 raft_consensus.cc:359] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:19.952917 30752 raft_consensus.cc:385] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:19.958976 30752 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:19.951784 30380 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:19.967430 30670 leader_election.cc:343] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.971678 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:19.972467 30484 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:19.975723 30523 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:19.976075 30480 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:19.979225 30911 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:19.976281 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:19.980520 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:19.986123 30523 leader_election.cc:343] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:19.986717 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:19.994560 30911 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.006511 30752 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.009514 30752 ts_tablet_manager.cc:1434] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.000s
I20260709 13:13:20.006309 30984 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:20.029839 30751 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:20.033386 30484 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:20.030539 30751 ts_tablet_manager.cc:1403] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.265s	user 0.029s	sys 0.009s
I20260709 13:13:20.037678 30752 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:20.039034 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:20.040578 30885 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:20.041179 30885 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:20.042347 30885 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.043473 30378 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.046171 30930 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.046816 30930 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.047885 30381 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.020176 30789 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.052955 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:20.053936 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.054368 30930 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:20.054766 30556 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:20.056282 30597 leader_election.cc:343] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.026418 31031 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.058521 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:20.060910 31041 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:20.061357 31041 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.061828 31041 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.062033 30754 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:20.062659 30754 ts_tablet_manager.cc:1403] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.387s	user 0.036s	sys 0.063s
I20260709 13:13:20.063515 30911 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.034391 30983 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.058549 31031 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.067991 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:20.068418 31031 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.068516 30983 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.068760 30556 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.065871 30754 raft_consensus.cc:359] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.069878 30754 raft_consensus.cc:385] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.070166 30754 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.070976 30754 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.052883 30789 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.078986 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:20.079869 30557 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.087702 30753 tablet_bootstrap.cc:492] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:20.094481 31029 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.095237 31029 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.100170 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:20.104000 30754 ts_tablet_manager.cc:1434] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.041s	user 0.003s	sys 0.004s
I20260709 13:13:20.105444 30754 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:20.108417 31029 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:20.111865 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.112673 30558 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:20.113804 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:20.114660 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:20.110368 30751 raft_consensus.cc:359] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.115510 30751 raft_consensus.cc:385] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.115310 30481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:20.116446 30751 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
W20260709 13:13:20.117213 30596 leader_election.cc:343] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.117111 30930 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:20.118796 30596 leader_election.cc:343] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.119110 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.119968 30558 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:13:20.123492 30670 leader_election.cc:343] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.124634 30752 tablet_bootstrap.cc:654] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.125474 30557 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:20.117242 30751 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.088941 30753 ts_tablet_manager.cc:1403] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.438s	user 0.015s	sys 0.017s
I20260709 13:13:20.130250 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:20.135414 30930 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:20.135571 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:20.136032 30930 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.136610 30930 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.137302 30846 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:20.137884 30846 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:20.139370 30846 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.142598 30754 tablet_bootstrap.cc:654] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.147784 30705 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:20.165653 30751 ts_tablet_manager.cc:1434] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.130s	user 0.009s	sys 0.000s
I20260709 13:13:20.166424 30556 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:20.164204 30382 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.173465 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.174918 30751 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:20.176422 31041 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.184202 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:20.193111 30382 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.200040 30885 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:20.200749 30885 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:20.201655 30885 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.218904 30450 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:13:20.220330 30450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67197us
I20260709 13:13:20.223742 31047 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.225553 31041 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:20.226150 30753 raft_consensus.cc:359] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.227051 30753 raft_consensus.cc:385] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.227381 30753 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.231052 30753 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.236315 30751 tablet_bootstrap.cc:654] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.242044 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:20.242955 30702 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.250473 30930 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:20.256732 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.263320 30930 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.265372 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:20.266650 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:20.267403 30558 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:20.267032 30596 leader_election.cc:343] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.256798 30753 ts_tablet_manager.cc:1434] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.128s	user 0.006s	sys 0.001s
I20260709 13:13:20.268497 30789 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:13:20.294289 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:20.295207 30557 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:20.305529 30596 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.312336 31072 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.319813 30557 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:20.331455 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:20.339792 30753 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:20.342481 30558 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:20.344259 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:20.348489 30754 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:20.349162 30754 ts_tablet_manager.cc:1403] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.244s	user 0.021s	sys 0.013s
I20260709 13:13:20.337188 30930 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:20.351083 30930 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:20.351807 31031 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:20.352460 31031 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:20.353360 31031 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.363142 30754 raft_consensus.cc:359] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.363986 30754 raft_consensus.cc:385] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.364296 30754 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.365017 30754 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.371973 30754 ts_tablet_manager.cc:1434] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:13:20.375344 30752 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:20.375960 30752 ts_tablet_manager.cc:1403] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.358s	user 0.014s	sys 0.008s
I20260709 13:13:20.365985 30702 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:20.370275 31047 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.381743 30752 raft_consensus.cc:359] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.384191 30751 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:20.353364 30930 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.389488 30752 raft_consensus.cc:385] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.390084 30751 ts_tablet_manager.cc:1403] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.218s	user 0.016s	sys 0.024s
I20260709 13:13:20.390257 30752 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.392038 30752 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.401499 30754 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:20.402405 30381 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.419284 30754 tablet_bootstrap.cc:654] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.421347 31030 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.422547 31029 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.423319 31029 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.424086 31073 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:20.428722 30448 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.428520 30381 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.437321 30752 ts_tablet_manager.cc:1434] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.001s
I20260709 13:13:20.453193 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:20.455713 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.467494 30553 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:20.468178 30752 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:20.467760 30751 raft_consensus.cc:359] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.469489 30450 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:20.484090 30751 raft_consensus.cc:385] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.484486 30751 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.447791 31029 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:20.473546 30911 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.480460 30753 tablet_bootstrap.cc:654] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.485497 30382 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.489809 30751 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.488946 31047 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.505125 30911 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.511929 30911 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:20.513269 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:20.512528 31047 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.514015 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:20.515031 30523 leader_election.cc:343] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:20.516036 30523 leader_election.cc:343] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.516574 30752 tablet_bootstrap.cc:654] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.516609 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:20.522078 30751 ts_tablet_manager.cc:1434] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.131s	user 0.007s	sys 0.002s
I20260709 13:13:20.524681 31017 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.528802 30751 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:20.534723 30754 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:20.535434 30754 ts_tablet_manager.cc:1403] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.134s	user 0.019s	sys 0.012s
I20260709 13:13:20.548259 30381 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:20.550405 30381 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:20.561162 30753 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:20.561836 30753 ts_tablet_manager.cc:1403] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.226s	user 0.020s	sys 0.027s
I20260709 13:13:20.549921 30754 raft_consensus.cc:359] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.565907 30754 raft_consensus.cc:385] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.566164 30754 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.568688 30754 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.570577 30754 ts_tablet_manager.cc:1434] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 13:13:20.571741 30754 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:20.581153 30752 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:20.582000 30752 ts_tablet_manager.cc:1403] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.114s	user 0.013s	sys 0.007s
I20260709 13:13:20.585423 30752 raft_consensus.cc:359] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.586350 30752 raft_consensus.cc:385] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.586714 30752 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.587913 30752 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.588531 30753 raft_consensus.cc:359] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.589455 30753 raft_consensus.cc:385] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.598084 31029 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.598672 30753 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.598973 31029 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.599839 30753 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.602283 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.603350 31029 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:20.598001 30752 ts_tablet_manager.cc:1434] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:13:20.604076 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:20.604971 30752 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:20.605932 30448 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:20.608282 30754 tablet_bootstrap.cc:654] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.609822 30707 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:20.611166 30753 ts_tablet_manager.cc:1434] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.001s
I20260709 13:13:20.612404 30753 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:20.613917 30930 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:20.618157 31073 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.625723 31073 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.626343 30752 tablet_bootstrap.cc:654] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.623544 30484 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.628477 31073 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:20.631861 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:20.636430 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:20.638798 30597 leader_election.cc:343] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.639246 30753 tablet_bootstrap.cc:654] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:20.640837 30596 leader_election.cc:343] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.641408 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:20.643546 31073 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.651605 30482 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:20.638270 30751 tablet_bootstrap.cc:654] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.655638 30930 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:20.656932 30753 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:20.707304 30753 ts_tablet_manager.cc:1403] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.095s	user 0.010s	sys 0.006s
I20260709 13:13:20.710868 30753 raft_consensus.cc:359] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.711766 30753 raft_consensus.cc:385] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.712080 30753 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.674708 31017 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.712989 31017 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.715449 31017 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:20.716759 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:20.678279 30983 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.717796 30983 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.719133 30702 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:20.720434 30983 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.685077 30754 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:20.691840 30751 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:20.721817 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.722261 30751 ts_tablet_manager.cc:1403] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.196s	user 0.015s	sys 0.001s
I20260709 13:13:20.722709 30553 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:20.723209 30753 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.724489 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:20.725781 30844 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:20.726326 30844 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:20.684099 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:20.722162 30754 ts_tablet_manager.cc:1403] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.151s	user 0.027s	sys 0.018s
I20260709 13:13:20.726830 30844 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.700824 31047 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.729044 30752 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:20.726070 30751 raft_consensus.cc:359] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.733058 30752 ts_tablet_manager.cc:1403] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.128s	user 0.035s	sys 0.019s
I20260709 13:13:20.732800 30480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:20.669988 30846 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.735044 30754 raft_consensus.cc:359] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.737146 30754 raft_consensus.cc:385] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.737486 30754 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
W20260709 13:13:20.736049 30522 leader_election.cc:343] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.738344 30754 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.732915 30751 raft_consensus.cc:385] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.740310 30751 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.741273 30751 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.743616 30754 ts_tablet_manager.cc:1434] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 13:13:20.745272 30754 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:20.743700 30752 raft_consensus.cc:359] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.747658 30752 raft_consensus.cc:385] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.748029 30752 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.725829 30753 ts_tablet_manager.cc:1434] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:13:20.756150 30753 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:20.759799 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:20.760226 30480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:20.748960 30752 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:20.761904 30482 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.765193 30911 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:20.765990 30911 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:20.767148 30911 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:20.768455 30751 ts_tablet_manager.cc:1434] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
W20260709 13:13:20.769430 30670 leader_election.cc:343] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.769972 30751 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:20.772248 31072 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.793771 30753 tablet_bootstrap.cc:654] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.804352 30752 ts_tablet_manager.cc:1434] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.071s	user 0.008s	sys 0.000s
I20260709 13:13:20.805827 30752 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:20.848162 30844 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.862236 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:20.864945 30670 leader_election.cc:343] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.863162 30751 tablet_bootstrap.cc:654] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.872973 31073 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.886231 30911 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.888933 30911 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:20.890419 30844 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.891609 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:20.892127 30752 tablet_bootstrap.cc:654] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.892539 30553 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:20.892933 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:20.893618 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:20.893846 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:20.894496 30707 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:13:20.894559 30522 leader_election.cc:343] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.898099 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:20.900760 30754 tablet_bootstrap.cc:654] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:20.907971 31017 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:13:20.908139 30596 consensus_peers.cc:597] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:20.909626 30844 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:20.918927 30846 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:20.922479 31031 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.923499 31031 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:20.926687 30753 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:20.927367 30753 ts_tablet_manager.cc:1403] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.172s	user 0.026s	sys 0.011s
I20260709 13:13:20.927706 31047 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:20.928313 31047 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:20.932531 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:20.930748 31047 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:20.957048 30753 raft_consensus.cc:359] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.957945 30753 raft_consensus.cc:385] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:20.958223 30753 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:20.958891 31072 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:20.958985 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:20.964411 30596 leader_election.cc:343] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:20.967749 30753 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:20.970418 30753 ts_tablet_manager.cc:1434] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 13:13:20.991540 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.994580 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:20.996978 30558 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:20.999532 30450 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.001135 31030 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:21.008270 30753 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:21.010425 31072 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:21.013218 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:21.014153 31073 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:21.014568 31073 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.014941 31073 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.026176 31073 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.027218 31031 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.028789 31073 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.030820 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:21.032517 30596 leader_election.cc:343] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.034837 31073 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:21.042088 30751 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:21.043108 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:21.044446 30557 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.046837 30751 ts_tablet_manager.cc:1403] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.277s	user 0.018s	sys 0.004s
I20260709 13:13:21.063900 30751 raft_consensus.cc:359] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.067709 30751 raft_consensus.cc:385] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.068182 30751 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.069702 30751 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.073499 30844 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.091040 30752 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:21.091769 30752 ts_tablet_manager.cc:1403] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.286s	user 0.018s	sys 0.017s
I20260709 13:13:21.093080 30751 ts_tablet_manager.cc:1434] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.001s
I20260709 13:13:21.094391 30751 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:21.090545 30844 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.103368 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.101227 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.105950 30633 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
W20260709 13:13:21.107365 30670 leader_election.cc:343] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.108996 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:21.109793 30846 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:21.110194 30846 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.110536 30846 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.111666 30751 tablet_bootstrap.cc:654] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.111430 30752 raft_consensus.cc:359] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.112771 30752 raft_consensus.cc:385] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.113679 30752 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.114550 30752 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.117715 30752 ts_tablet_manager.cc:1434] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:13:21.101244 30844 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:21.125119 30930 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.125734 30930 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.127993 30754 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:21.128661 30754 ts_tablet_manager.cc:1403] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.384s	user 0.021s	sys 0.009s
W20260709 13:13:21.134796 31102 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.005s
I20260709 13:13:21.135761 30754 raft_consensus.cc:359] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.142817 30754 raft_consensus.cc:385] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.136691 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.146027 30754 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.136433 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.146273 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:21.141875 30557 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:21.146920 30930 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:21.154088 30752 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
W20260709 13:13:21.159242 30597 leader_election.cc:343] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.161578 30754 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.172287 30846 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.174813 30846 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:21.176268 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:21.179425 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:13:21.179996 30670 leader_election.cc:343] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.180346 30631 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.187732 30753 tablet_bootstrap.cc:654] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.188220 30754 ts_tablet_manager.cc:1434] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.059s	user 0.008s	sys 0.000s
I20260709 13:13:21.190485 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.191998 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:21.192867 31072 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:21.193436 31072 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:21.194356 31072 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.197281 30751 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:21.197992 30751 ts_tablet_manager.cc:1403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.104s	user 0.041s	sys 0.018s
I20260709 13:13:21.198487 30754 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:21.207610 30930 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:21.208246 30930 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.208598 30930 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.217723 30631 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:21.219290 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:21.220645 30844 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:21.223073 30911 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.223770 30911 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.225580 30844 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:21.213241 30846 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.236542 30846 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.240701 30754 tablet_bootstrap.cc:654] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.241670 31017 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.241369 30844 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.242193 31017 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.243697 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.244531 31017 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:21.245393 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.246930 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.250439 30751 raft_consensus.cc:359] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.251224 30751 raft_consensus.cc:385] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.251514 30751 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.246147 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.253701 30846 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:21.253765 30523 leader_election.cc:343] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.254751 30751 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:21.255110 30523 leader_election.cc:343] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.229272 30911 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:21.252081 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.230062 30752 tablet_bootstrap.cc:654] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:21.256595 30522 leader_election.cc:343] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:21.265941 30671 leader_election.cc:343] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.266745 30751 ts_tablet_manager.cc:1434] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.000s
I20260709 13:13:21.267590 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:21.268039 30751 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
W20260709 13:13:21.283051 30522 leader_election.cc:343] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.283810 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.286008 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.287729 30911 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.288082 30992 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.310215 30754 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:21.309715 30930 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.310899 30754 ts_tablet_manager.cc:1403] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.002s
I20260709 13:13:21.314158 30382 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:21.317058 30754 raft_consensus.cc:359] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.317778 30754 raft_consensus.cc:385] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.318045 30754 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.321766 30754 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.319013 30930 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:21.319267 30670 leader_election.cc:343] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.332780 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:21.333256 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:21.333948 30557 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.336548 30753 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
W20260709 13:13:21.337028 30597 leader_election.cc:343] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.340289 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:21.337237 30753 ts_tablet_manager.cc:1403] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.348s	user 0.016s	sys 0.006s
I20260709 13:13:21.344208 30752 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:21.344363 30844 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.344884 30752 ts_tablet_manager.cc:1403] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.194s	user 0.019s	sys 0.005s
I20260709 13:13:21.357764 30752 raft_consensus.cc:359] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.358565 30752 raft_consensus.cc:385] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.358857 30752 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.359534 30752 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.361109 30752 ts_tablet_manager.cc:1434] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:13:21.362205 30752 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
W20260709 13:13:21.365541 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48986.5us
I20260709 13:13:21.371321 30751 tablet_bootstrap.cc:654] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.375895 30753 raft_consensus.cc:359] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.376770 30753 raft_consensus.cc:385] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.377074 30753 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.384524 30753 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.396060 30378 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:21.406443 30754 ts_tablet_manager.cc:1434] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.095s	user 0.007s	sys 0.001s
I20260709 13:13:21.409304 30753 ts_tablet_manager.cc:1434] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 13:13:21.409668 30752 tablet_bootstrap.cc:654] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.411703 30557 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:21.412693 30754 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:21.415419 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.417588 30753 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:21.418311 30930 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:21.419195 30930 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:21.420579 30930 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.408061 30789 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.428447 30789 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.429855 30844 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.430444 30844 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.432570 30844 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.433736 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.444657 30753 tablet_bootstrap.cc:654] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:21.445783 30670 leader_election.cc:343] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.446410 30631 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.435849 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:21.452679 30673 leader_election.cc:343] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.453325 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:21.454902 30844 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.460824 30754 tablet_bootstrap.cc:654] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.465339 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.474424 30752 tablet_bootstrap.cc:492] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:21.475432 30752 ts_tablet_manager.cc:1403] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.000s
I20260709 13:13:21.477466 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.480257 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.484575 30789 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:21.485721 31073 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.492548 30752 raft_consensus.cc:359] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.494311 30753 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:21.494912 30752 raft_consensus.cc:385] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.495126 30753 ts_tablet_manager.cc:1403] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.000s
I20260709 13:13:21.495666 30752 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.499483 30753 raft_consensus.cc:359] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.500360 30753 raft_consensus.cc:385] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.500689 30753 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.501847 30753 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.504345 30754 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:21.505236 30754 ts_tablet_manager.cc:1403] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.093s	user 0.017s	sys 0.000s
I20260709 13:13:21.508608 30752 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.511394 30754 raft_consensus.cc:359] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.515025 30754 raft_consensus.cc:385] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.515457 30754 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.513820 30752 ts_tablet_manager.cc:1434] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 13:13:21.513768 30753 ts_tablet_manager.cc:1434] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:13:21.516500 30754 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.519567 30753 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:21.521416 30754 ts_tablet_manager.cc:1434] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:13:21.529807 30753 tablet_bootstrap.cc:654] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.516376 30484 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.547987 30752 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
W20260709 13:13:21.551074 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50653.2us
I20260709 13:13:21.552335 30753 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:21.552934 30753 ts_tablet_manager.cc:1403] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.034s	user 0.022s	sys 0.006s
I20260709 13:13:21.558145 30753 raft_consensus.cc:359] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.558954 30754 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:21.558925 30753 raft_consensus.cc:385] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.565124 30753 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.566460 30753 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.569108 31052 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.570101 30753 ts_tablet_manager.cc:1434] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:13:21.569766 31052 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.572511 31052 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.577715 30753 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:21.577863 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.581640 30707 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:21.586051 30752 tablet_bootstrap.cc:654] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.576498 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.596568 30753 tablet_bootstrap.cc:654] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.601902 31072 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:21.598662 30553 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:21.609146 30450 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.614748 31052 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.620731 31029 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:21.621738 31029 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.633219 30983 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.633968 30983 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.635689 30631 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.636600 30983 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.638422 31029 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.640089 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.641292 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.639642 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.642119 30556 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:21.638501 30478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.644572 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:21.648620 30450 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:21.661638 30753 tablet_bootstrap.cc:492] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:21.650373 30983 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:21.666455 30753 ts_tablet_manager.cc:1403] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.089s	user 0.014s	sys 0.014s
I20260709 13:13:21.650220 31047 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:21.663801 30670 leader_election.cc:343] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.669900 30983 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.677760 30983 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.684002 30754 tablet_bootstrap.cc:654] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.688972 31052 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.696568 30983 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.694509 30930 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.702674 30930 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.704921 30983 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:21.705163 30930 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:21.698793 30844 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:13:21.713826 30753 raft_consensus.cc:359] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.714413 30752 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:21.714715 30753 raft_consensus.cc:385] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.716053 30752 ts_tablet_manager.cc:1403] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.000s
I20260709 13:13:21.712332 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:21.716244 30753 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.719848 30751 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:21.720460 30751 ts_tablet_manager.cc:1403] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.453s	user 0.055s	sys 0.081s
I20260709 13:13:21.711011 31052 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.714381 30911 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.720033 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:13:21.731022 30670 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.725771 31052 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:21.733989 30753 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.739116 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:21.740267 30707 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:21.745748 31052 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:21.746594 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.747560 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.751231 30753 ts_tablet_manager.cc:1434] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.082s	user 0.009s	sys 0.000s
I20260709 13:13:21.753001 30753 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:21.754024 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:21.756705 30670 leader_election.cc:343] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.757510 30983 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:21.768841 30983 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.771457 30983 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:21.772447 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:21.731298 30911 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:21.773391 30599 leader_election.cc:343] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.761652 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.762253 30751 raft_consensus.cc:359] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.776917 30751 raft_consensus.cc:385] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.777400 30751 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:21.742334 30553 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.787644 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:21.789915 30558 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:21.783495 31072 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:21.794867 30911 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:21.794927 31072 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:21.796129 31072 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.801123 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:21.807408 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.811097 30752 raft_consensus.cc:359] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.813099 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:21.841084 30844 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.811187 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:21.816424 30482 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.834161 30450 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:21.820602 30751 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.872221 30751 ts_tablet_manager.cc:1434] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.151s	user 0.009s	sys 0.000s
I20260709 13:13:21.873519 30751 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:21.879071 30844 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.834589 30752 raft_consensus.cc:385] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:13:21.815142 30522 leader_election.cc:343] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:21.882541 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:21.882712 30752 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
W20260709 13:13:21.882875 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67781.5us
I20260709 13:13:21.883747 30752 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.864903 30553 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:21.818238 30753 tablet_bootstrap.cc:654] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:21.884958 30523 leader_election.cc:343] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.890728 30751 tablet_bootstrap.cc:654] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.882014 30754 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:21.829352 30378 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:21.892189 30754 ts_tablet_manager.cc:1403] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.337s	user 0.061s	sys 0.036s
I20260709 13:13:21.887251 30752 ts_tablet_manager.cc:1434] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.170s	user 0.003s	sys 0.005s
I20260709 13:13:21.891399 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:21.894354 30752 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:21.894785 30911 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.895702 30754 raft_consensus.cc:359] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.896445 30754 raft_consensus.cc:385] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:21.896734 30754 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
W20260709 13:13:21.887539 30450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53822.1us
I20260709 13:13:21.897387 30754 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:21.889698 31052 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:21.903741 30754 ts_tablet_manager.cc:1434] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:13:21.908429 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:21.910133 30844 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:21.910800 30844 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:21.911834 30844 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:21.920998 30789 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:21.888198 31072 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:21.954494 30930 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:21.962301 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:21.963165 30705 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:21.957875 30752 tablet_bootstrap.cc:654] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:21.975065 31072 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:21.968473 31132 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:13:21.978336 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:13:21.982436 30599 leader_election.cc:343] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:21.996273 30705 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.000861 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:22.005077 31031 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:22.005829 31031 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:22.006932 31031 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.007601 30380 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.013140 30752 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.014174 30752 ts_tablet_manager.cc:1403] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.120s	user 0.010s	sys 0.005s
I20260709 13:13:22.017987 30752 raft_consensus.cc:359] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.020035 30752 raft_consensus.cc:385] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.020398 30752 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.021276 30752 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.025542 30752 ts_tablet_manager.cc:1434] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:13:22.030084 30751 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:22.026839 30752 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:22.033149 30751 ts_tablet_manager.cc:1403] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.160s	user 0.013s	sys 0.001s
I20260709 13:13:22.043224 30751 raft_consensus.cc:359] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.045367 30751 raft_consensus.cc:385] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.045792 30751 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.046550 30751 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.047348 30752 tablet_bootstrap.cc:654] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.048686 30751 ts_tablet_manager.cc:1434] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 13:13:22.066166 30754 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:22.071564 30753 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:22.072410 30753 ts_tablet_manager.cc:1403] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.320s	user 0.048s	sys 0.077s
I20260709 13:13:22.081077 30753 raft_consensus.cc:359] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.081887 30753 raft_consensus.cc:385] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.082170 30753 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.082980 30753 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.085462 30753 ts_tablet_manager.cc:1434] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:13:22.092482 30751 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:22.097290 30753 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:22.098245 30844 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.098863 30844 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.101029 30844 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.104303 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:22.106200 30671 leader_election.cc:343] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.108521 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.125252 30754 tablet_bootstrap.cc:654] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:22.126535 30670 leader_election.cc:343] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.127162 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:22.129555 30753 tablet_bootstrap.cc:654] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.128760 30380 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.132362 30844 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.133270 30751 tablet_bootstrap.cc:654] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.140982 30752 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.141597 30752 ts_tablet_manager.cc:1403] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.115s	user 0.032s	sys 0.007s
I20260709 13:13:22.144212 30751 tablet_bootstrap.cc:492] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:22.144893 30751 ts_tablet_manager.cc:1403] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.000s
I20260709 13:13:22.145054 30752 raft_consensus.cc:359] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.146404 30752 raft_consensus.cc:385] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.146776 30752 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.147549 30752 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.151301 30752 ts_tablet_manager.cc:1434] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.007s
I20260709 13:13:22.151376 30751 raft_consensus.cc:359] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.152683 30751 raft_consensus.cc:385] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.153525 30751 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.152706 30752 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:22.156338 30753 tablet_bootstrap.cc:492] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:22.156976 30753 ts_tablet_manager.cc:1403] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.001s
I20260709 13:13:22.157003 30751 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.163488 31072 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.164167 31072 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.167804 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.168107 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.168627 30557 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
W20260709 13:13:22.169741 30597 leader_election.cc:343] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.170387 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:22.172312 30752 tablet_bootstrap.cc:654] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.172492 31031 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.173017 30751 ts_tablet_manager.cc:1434] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 13:13:22.173547 31031 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.173859 31072 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:22.176602 31031 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.177354 30754 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:22.177975 30754 ts_tablet_manager.cc:1403] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.112s	user 0.010s	sys 0.018s
I20260709 13:13:22.182425 30752 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.184264 30752 ts_tablet_manager.cc:1403] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.004s
I20260709 13:13:22.188242 30754 raft_consensus.cc:359] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.188535 30752 raft_consensus.cc:359] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.189132 30754 raft_consensus.cc:385] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.189316 30752 raft_consensus.cc:385] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.189532 30754 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.189745 30752 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.189644 31031 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.190495 30754 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.190783 30752 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.192672 31031 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:22.192929 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:22.193292 30754 ts_tablet_manager.cc:1434] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.007s
I20260709 13:13:22.193645 30557 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.193852 30751 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:22.194499 30753 raft_consensus.cc:359] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.195290 30753 raft_consensus.cc:385] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.195562 30753 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.195999 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:22.198894 30752 ts_tablet_manager.cc:1434] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:13:22.200210 30754 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:22.203045 30752 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:22.203651 31135 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.204209 31135 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.208010 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.209542 30754 tablet_bootstrap.cc:654] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.211341 31135 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.211261 30752 tablet_bootstrap.cc:654] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.212663 31072 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.212886 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.213378 31072 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.213754 30705 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
W20260709 13:13:22.215253 30522 leader_election.cc:343] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.216874 30751 tablet_bootstrap.cc:654] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.217023 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:22.218152 31135 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.218600 31135 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.217705 31072 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.220250 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.221060 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:22.222743 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.221451 31135 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.226163 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:22.227993 30596 leader_election.cc:343] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.225862 31073 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.235410 31073 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.235932 31073 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.240675 31134 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.241304 31134 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.243599 31134 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:22.246402 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:22.247921 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.249454 30448 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:22.250825 31030 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.260466 30482 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.263721 31135 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.271060 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:22.271797 30754 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:22.272483 30754 ts_tablet_manager.cc:1403] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.073s	user 0.014s	sys 0.013s
I20260709 13:13:22.271823 30702 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.276381 30754 raft_consensus.cc:359] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.278403 31135 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.277160 30754 raft_consensus.cc:385] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.279474 30754 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.280380 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.280298 30754 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:22.284502 30522 leader_election.cc:343] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.285017 30754 ts_tablet_manager.cc:1434] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:13:22.262009 30557 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.288012 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.286361 30754 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:22.291198 30789 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:22.294606 30484 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.296567 30789 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:22.297622 30789 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.196456 30753 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.308203 30707 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.316882 30754 tablet_bootstrap.cc:654] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.322633 30751 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:22.323624 30751 ts_tablet_manager.cc:1403] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.142s	user 0.035s	sys 0.041s
I20260709 13:13:22.324007 30752 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.333887 30752 ts_tablet_manager.cc:1403] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.131s	user 0.014s	sys 0.015s
I20260709 13:13:22.325726 31073 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.336359 30751 raft_consensus.cc:359] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.337222 30751 raft_consensus.cc:385] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.318332 30707 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.329267 30702 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:22.337071 30752 raft_consensus.cc:359] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.342433 30752 raft_consensus.cc:385] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.342916 30752 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.337566 30751 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.338295 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:22.352394 30705 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.343665 30752 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.354737 30752 ts_tablet_manager.cc:1434] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 13:13:22.356230 30752 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:22.363260 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:22.364162 31135 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:22.364703 31135 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:22.365180 31073 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.366732 30751 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.368527 30751 ts_tablet_manager.cc:1434] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.036s	user 0.003s	sys 0.003s
I20260709 13:13:22.369656 30751 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:22.375181 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.376119 30479 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.376714 30751 tablet_bootstrap.cc:654] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.365594 31135 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.390021 31136 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.400347 31136 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:22.401517 30670 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:22.392740 30752 tablet_bootstrap.cc:654] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.407723 31136 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.388253 30753 ts_tablet_manager.cc:1434] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.231s	user 0.012s	sys 0.000s
I20260709 13:13:22.407999 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.445050 30753 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:22.456588 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.457851 30930 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:22.458465 30930 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:22.446198 30478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.459218 30930 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.467271 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:22.467604 30522 leader_election.cc:343] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.474148 30844 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:22.476915 30523 leader_election.cc:343] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.477552 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:22.482048 30789 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.482663 30789 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.484932 30789 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.481052 31136 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.486115 30751 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:22.487979 30751 ts_tablet_manager.cc:1403] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.119s	user 0.030s	sys 0.029s
I20260709 13:13:22.496660 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.500349 30754 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:22.506130 30754 ts_tablet_manager.cc:1403] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.220s	user 0.031s	sys 0.038s
I20260709 13:13:22.515995 30751 raft_consensus.cc:359] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.517277 30751 raft_consensus.cc:385] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.517555 30751 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.512670 30479 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.529557 30753 tablet_bootstrap.cc:654] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.524567 30381 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.530943 30789 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:22.525990 30751 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:22.527475 30596 leader_election.cc:343] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.544651 30751 ts_tablet_manager.cc:1434] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.000s
I20260709 13:13:22.565868 30751 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:22.566577 30752 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.567945 30752 ts_tablet_manager.cc:1403] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.212s	user 0.028s	sys 0.009s
I20260709 13:13:22.575825 30789 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.576776 30789 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.560436 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.592032 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:22.593447 30754 raft_consensus.cc:359] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.594336 30754 raft_consensus.cc:385] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.594660 30754 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.595368 30754 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.596797 30789 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.601675 30751 tablet_bootstrap.cc:654] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.597823 30754 ts_tablet_manager.cc:1434] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.091s	user 0.007s	sys 0.000s
I20260709 13:13:22.604369 30754 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:22.586968 31136 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.607743 31136 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.610242 31136 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:22.609679 30752 raft_consensus.cc:359] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.616290 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.631961 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:22.636384 30752 raft_consensus.cc:385] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.636871 30752 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.637740 30752 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.638989 30930 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.639839 30930 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.642263 30930 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.644696 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.640290 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:22.647042 30752 ts_tablet_manager.cc:1434] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.079s	user 0.008s	sys 0.001s
I20260709 13:13:22.653035 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:22.653517 30752 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:22.643594 31031 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.654657 31031 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.655687 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.656452 30556 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:22.657176 31031 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.641335 30753 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:22.659363 30753 ts_tablet_manager.cc:1403] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.226s	user 0.013s	sys 0.021s
I20260709 13:13:22.659762 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.660499 30707 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:22.661949 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.662352 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.663223 31072 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.642755 30754 tablet_bootstrap.cc:654] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.663655 31072 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.663930 31073 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.664033 31072 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.664463 31073 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.664875 31073 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.673038 30751 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:22.673713 30751 ts_tablet_manager.cc:1403] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.118s	user 0.028s	sys 0.015s
I20260709 13:13:22.653959 31163 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.657254 30789 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.662894 30753 raft_consensus.cc:359] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.681041 30753 raft_consensus.cc:385] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.684291 30753 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.679631 30789 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.681680 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.681716 31134 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:22.710832 30596 leader_election.cc:343] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.713249 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.715667 30789 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.689731 30380 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:22.723506 30930 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.725924 30930 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.686918 31134 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.744172 30753 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.762897 30751 raft_consensus.cc:359] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.765121 30751 raft_consensus.cc:385] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.765504 30751 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.762578 30381 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:22.751549 31166 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.773751 30751 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.759138 30553 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.780357 30754 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:22.781013 30754 ts_tablet_manager.cc:1403] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.177s	user 0.012s	sys 0.021s
I20260709 13:13:22.772505 31163 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.789506 30751 ts_tablet_manager.cc:1434] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.115s	user 0.010s	sys 0.003s
W20260709 13:13:22.781477 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68456.2us
I20260709 13:13:22.801236 30751 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Bootstrap starting.
I20260709 13:13:22.801961 31135 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.802582 31135 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:22.804939 31135 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:22.782284 31166 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.808259 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.808970 31166 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.813313 30752 tablet_bootstrap.cc:654] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.813627 31073 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.814090 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.813720 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:22.817538 30706 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:22.819367 31134 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:22.819952 30523 leader_election.cc:343] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.791778 31163 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.820801 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:22.823328 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:22.824445 30558 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.823873 31136 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:22.826818 31136 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.823916 30557 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:22.827940 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.828732 31175 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:22.827214 31136 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:22.831112 31175 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.827020 31073 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.835467 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.837903 30558 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.844236 31163 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:22.844521 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:22.847052 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:22.849906 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:22.850754 30558 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:22.854768 31072 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:22.857307 31072 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:22.859779 30523 leader_election.cc:343] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.859934 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:22.860471 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:22.868296 31163 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.871193 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:22.872398 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:22.878846 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:22.880041 30557 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.840709 31175 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:22.882656 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.894215 31176 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.900034 31176 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.900496 31176 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:22.901263 30596 leader_election.cc:343] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:22.902235 30596 leader_election.cc:343] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.902582 31143 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:22.905043 31143 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:22.905893 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:22.906831 30628 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
I20260709 13:13:22.905997 31143 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.907912 31073 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:22.903337 30596 leader_election.cc:343] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.916378 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:22.877792 30702 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:22.917100 30522 leader_election.cc:343] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:22.919552 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:22.917704 30789 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.920909 30789 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.921317 30789 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.924852 30448 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:22.944691 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:22.945815 30482 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:22.952728 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.952821 30930 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:22.955677 30754 raft_consensus.cc:359] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.956513 30754 raft_consensus.cc:385] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:22.956804 30754 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:22.933069 31030 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:22.926868 31166 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:22.958424 30557 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:22.959384 30754 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:22.960422 30752 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:22.964476 30753 ts_tablet_manager.cc:1434] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.305s	user 0.025s	sys 0.008s
I20260709 13:13:22.964944 30752 ts_tablet_manager.cc:1403] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.314s	user 0.016s	sys 0.042s
I20260709 13:13:22.969394 30754 ts_tablet_manager.cc:1434] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.188s	user 0.005s	sys 0.004s
I20260709 13:13:22.966883 31166 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:22.988811 31166 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:22.988983 31136 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:22.983196 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:22.984205 30753 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:22.994211 31136 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:22.996030 30751 tablet_bootstrap.cc:654] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:22.997280 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.997665 30789 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:22.998013 30484 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:23.005146 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:23.006568 31073 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:23.008316 31073 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:23.009264 31073 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:23.012362 31072 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.016182 30477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:22.971653 30478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:23.048787 30752 raft_consensus.cc:359] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.049685 30752 raft_consensus.cc:385] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.050002 30752 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
W20260709 13:13:23.050171 30670 leader_election.cc:343] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.050024 31072 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.059610 30789 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:23.062606 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:23.069346 30477 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:23.071938 30596 leader_election.cc:343] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.079879 30478 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:23.082778 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:23.085126 30754 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:23.085541 30753 tablet_bootstrap.cc:654] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.092087 31176 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.094919 30752 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.095841 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:23.097617 30670 leader_election.cc:343] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.103368 30754 tablet_bootstrap.cc:654] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.135475 30484 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:23.137374 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:23.166637 30752 ts_tablet_manager.cc:1434] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.201s	user 0.006s	sys 0.004s
I20260709 13:13:23.168215 30789 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.168821 30789 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.144330 31017 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:23.172526 30789 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:23.172761 31017 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:23.173758 31017 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.158288 31176 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:23.196270 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:23.197397 30557 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:23.169457 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46740.9us
I20260709 13:13:23.188385 31178 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.227831 30557 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
W20260709 13:13:23.228701 30596 leader_election.cc:343] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.207742 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:23.239763 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:23.240329 31174 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.240796 30844 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:23.241353 30844 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:23.242216 30844 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:23.242702 30596 leader_election.cc:343] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.242744 31166 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:23.245018 30597 leader_election.cc:343] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.246091 31166 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:23.247026 30559 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 1: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:23.247476 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:23.254473 30522 leader_election.cc:343] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.252810 30702 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:23.256680 30754 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:23.257319 30754 ts_tablet_manager.cc:1403] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.172s	user 0.021s	sys 0.027s
I20260709 13:13:23.219017 31178 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.278939 30751 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: No bootstrap required, opened a new log
I20260709 13:13:23.279788 30751 ts_tablet_manager.cc:1403] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent bootstrapping tablet: real 0.486s	user 0.030s	sys 0.020s
I20260709 13:13:23.280974 31178 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:23.284643 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:23.294670 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:23.306602 30753 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:23.307562 30753 ts_tablet_manager.cc:1403] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.329s	user 0.022s	sys 0.003s
I20260709 13:13:23.303689 31172 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:23.327728 30482 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:23.330255 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:23.325531 31172 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:23.333698 30450 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:23.336131 30751 raft_consensus.cc:359] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.337100 30751 raft_consensus.cc:385] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.337417 30751 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Initialized, Role: FOLLOWER
I20260709 13:13:23.337541 31172 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.338137 30751 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.339161 30754 raft_consensus.cc:359] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.341566 30754 raft_consensus.cc:385] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.341827 30754 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:23.344179 31030 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:23.343132 30753 raft_consensus.cc:359] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.345824 30753 raft_consensus.cc:385] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.346143 30753 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:23.346911 30753 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:23.356652 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 383180us
I20260709 13:13:23.362182 30482 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:23.364653 30596 leader_election.cc:343] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.365365 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:23.366484 30789 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:23.370393 30754 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.374084 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:23.374876 30707 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:23.370447 30381 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:23.387679 30754 ts_tablet_manager.cc:1434] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.000s
I20260709 13:13:23.398397 30751 ts_tablet_manager.cc:1434] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent starting tablet: real 0.118s	user 0.009s	sys 0.000s
I20260709 13:13:23.400393 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:23.432395 30707 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:23.439100 30633 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:23.440189 30752 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:23.442529 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:23.450704 30754 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:23.452611 30911 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.455734 30753 ts_tablet_manager.cc:1434] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.148s	user 0.004s	sys 0.006s
I20260709 13:13:23.454651 31183 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:13:23.471715 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:23.472868 30789 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:13:23.464110 30596 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:23.481141 30753 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:23.495312 30753 tablet_bootstrap.cc:654] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.513144 30752 tablet_bootstrap.cc:654] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.518308 30754 tablet_bootstrap.cc:654] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.544888 30382 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:23.546996 30844 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.547806 30844 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.555190 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:23.556020 30628 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:23.556777 30844 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:23.557591 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:23.562899 30844 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:23.563436 30844 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.563796 30844 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:23.573052 30844 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.572651 31031 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.596470 30844 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:23.587972 31183 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.604063 31183 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.598942 31052 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.675729 30753 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
W20260709 13:13:23.674450 31202 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:13:23.678260 30753 ts_tablet_manager.cc:1403] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.208s	user 0.037s	sys 0.003s
I20260709 13:13:23.676036 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:23.674185 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:23.680233 31052 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.692754 30479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:23.676431 30483 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:23.678821 31183 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:23.711615 31200 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:13:23.715703 30522 leader_election.cc:343] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.717753 31052 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:23.690836 30629 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:23.719041 30752 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:23.720021 30752 ts_tablet_manager.cc:1403] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.280s	user 0.014s	sys 0.008s
I20260709 13:13:23.719411 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:23.742573 31143 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:23.697151 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 98210.4us
I20260709 13:13:23.744551 30752 raft_consensus.cc:359] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:23.745961 30752 raft_consensus.cc:385] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.746390 30752 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:23.745765 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:23.751672 30706 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 1: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:23.747017 30752 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:23.755683 30753 raft_consensus.cc:359] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.771907 30752 ts_tablet_manager.cc:1434] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.001s
I20260709 13:13:23.772193 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:23.774825 30752 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Bootstrap starting.
I20260709 13:13:23.771919 30753 raft_consensus.cc:385] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.778038 30753 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:23.755954 30448 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:23.782708 31195 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:23.783285 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:23.786845 30479 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:23.793196 30754 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:23.793865 30754 ts_tablet_manager.cc:1403] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.344s	user 0.015s	sys 0.007s
I20260709 13:13:23.795017 30477 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:23.795902 30670 leader_election.cc:343] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.796336 31166 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:23.799080 30707 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:23.803797 31183 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.805153 31195 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:23.807185 30380 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 2, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:23.785511 30753 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:23.816428 30705 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 0.
W20260709 13:13:23.828999 30670 leader_election.cc:343] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:23.834324 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:23.836911 30754 raft_consensus.cc:359] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:23.837494 31135 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:23.838640 30754 raft_consensus.cc:385] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:23.838807 31135 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:23.839021 30754 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
W20260709 13:13:23.840390 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57220.1us
I20260709 13:13:23.845009 31181 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:23.847150 31135 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:23.826421 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:23.853432 31181 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:23.862135 31181 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:23.847759 30754 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:23.865365 30754 ts_tablet_manager.cc:1434] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.071s	user 0.009s	sys 0.000s
I20260709 13:13:23.866556 30754 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:23.878512 31073 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:13:23.899498 30753 ts_tablet_manager.cc:1434] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.220s	user 0.010s	sys 0.001s
I20260709 13:13:23.911679 30382 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:23.916488 30382 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:23.914192 31073 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:23.936506 30753 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
I20260709 13:13:23.950054 30754 tablet_bootstrap.cc:654] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:23.953817 30844 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:23.960081 31073 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:24.008742 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:24.010847 30597 leader_election.cc:343] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:24.011932 31073 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:24.012148 30752 tablet_bootstrap.cc:654] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:24.017634 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51107.6us
I20260709 13:13:24.053234 31135 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.066385 30753 tablet_bootstrap.cc:654] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
W20260709 13:13:24.071587 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115060us
I20260709 13:13:24.078578 30754 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:24.079321 30754 ts_tablet_manager.cc:1403] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.004s
I20260709 13:13:24.112955 30789 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:13:24.115324 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:24.140395 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:24.157956 30704 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.176298 31052 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.189823 30753 tablet_bootstrap.cc:492] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:24.190685 30753 ts_tablet_manager.cc:1403] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.270s	user 0.033s	sys 0.029s
I20260709 13:13:24.195188 31135 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:24.195590 31222 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:13:24.213676 30754 raft_consensus.cc:359] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.206168 31052 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:24.216675 31052 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:24.218155 30480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:24.219731 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:24.228427 30522 leader_election.cc:343] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:24.230197 30754 raft_consensus.cc:385] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.230854 30754 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.237216 30754 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.228468 30753 raft_consensus.cc:359] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:24.240613 30753 raft_consensus.cc:385] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.241081 30753 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.249383 30753 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:24.252044 30753 ts_tablet_manager.cc:1434] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.002s
I20260709 13:13:24.276255 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:24.287164 30754 ts_tablet_manager.cc:1434] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.207s	user 0.009s	sys 0.001s
W20260709 13:13:24.317060 30429 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:13:24.289590 30704 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:24.328507 30478 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.330948 30752 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: No bootstrap required, opened a new log
I20260709 13:13:24.331640 30752 ts_tablet_manager.cc:1403] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent bootstrapping tablet: real 0.557s	user 0.027s	sys 0.053s
I20260709 13:13:24.330948 30754 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:24.331530 30753 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Bootstrap starting.
W20260709 13:13:24.341116 31211 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.306s	user 0.006s	sys 0.000s
I20260709 13:13:24.356361 30752 raft_consensus.cc:359] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.357362 30752 raft_consensus.cc:385] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.357705 30752 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.363234 30752 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.369266 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:24.370208 31181 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:24.370891 30754 tablet_bootstrap.cc:654] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:24.372772 30753 tablet_bootstrap.cc:654] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:24.375973 30752 ts_tablet_manager.cc:1434] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.006s
I20260709 13:13:24.385110 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:24.386196 31017 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:24.386981 31017 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:24.388198 31017 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.370234 30555 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:24.414451 30378 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:24.435878 30448 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:13:24.445823 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59991.3us
W20260709 13:13:24.471331 30670 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:24.489578 31218 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:13:24.495100 30448 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57016.7us
I20260709 13:13:24.520503 30753 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: No bootstrap required, opened a new log
I20260709 13:13:24.521209 30753 ts_tablet_manager.cc:1403] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent bootstrapping tablet: real 0.206s	user 0.015s	sys 0.008s
I20260709 13:13:24.527415 31231 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.528056 31231 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.530929 31231 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:24.532323 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:24.534802 30597 leader_election.cc:343] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:13:24.546662 30448 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50218.5us
I20260709 13:13:24.536278 30753 raft_consensus.cc:359] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.551222 30753 raft_consensus.cc:385] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.551693 30753 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Initialized, Role: FOLLOWER
W20260709 13:13:24.556396 30450 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46137us
I20260709 13:13:24.563931 30481 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:24.576555 30596 leader_election.cc:343] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:24.577127 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:24.578657 30789 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:24.574050 30753 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.588698 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:24.594786 30753 ts_tablet_manager.cc:1434] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent starting tablet: real 0.073s	user 0.012s	sys 0.000s
I20260709 13:13:24.591686 31143 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:24.595427 31143 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:24.595794 31143 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.610560 31229 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:24.611331 31229 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.613803 31229 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:24.615989 30754 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:24.616633 30754 ts_tablet_manager.cc:1403] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.302s	user 0.048s	sys 0.048s
I20260709 13:13:24.617102 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:24.618901 31143 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:24.620647 30754 raft_consensus.cc:359] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.632092 30754 raft_consensus.cc:385] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.632478 30754 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.633276 30754 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.634575 31143 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:24.636868 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:24.637647 30557 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.640795 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:24.641517 30705 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.646215 30557 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:24.650279 30754 ts_tablet_manager.cc:1434] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:13:24.651880 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:24.651567 30381 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:24.652850 31174 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:24.653450 31174 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:24.654345 31174 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:24.655158 30754 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:24.668421 30705 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:24.679445 30754 tablet_bootstrap.cc:654] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:24.698727 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:24.731031 30382 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:24.732676 30477 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:24.847517 30448 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:24.853149 30754 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:24.853804 31192 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:24.855645 30754 ts_tablet_manager.cc:1403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.201s	user 0.029s	sys 0.066s
I20260709 13:13:24.883821 30754 raft_consensus.cc:359] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:24.898931 30754 raft_consensus.cc:385] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.900357 30754 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.901701 30754 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:24.902710 31229 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:24.906240 31229 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:24.908768 30754 ts_tablet_manager.cc:1434] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 13:13:24.911479 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:24.910329 31229 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:24.917323 30754 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:24.925217 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:24.927168 30450 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:24.928084 31229 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:24.930176 30754 tablet_bootstrap.cc:654] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:24.957106 30754 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:24.958070 30754 ts_tablet_manager.cc:1403] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.042s	user 0.015s	sys 0.010s
I20260709 13:13:24.961524 30754 raft_consensus.cc:359] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.962365 30754 raft_consensus.cc:385] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:24.962692 30754 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:24.963500 30754 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:24.966190 30754 ts_tablet_manager.cc:1434] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:13:24.967347 31183 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:24.967511 30754 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
W20260709 13:13:24.973548 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:24.978574 30754 tablet_bootstrap.cc:654] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:25.007688 31229 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.008456 31229 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.012053 30482 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.017095 30754 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:25.017756 30754 ts_tablet_manager.cc:1403] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.051s	user 0.030s	sys 0.006s
I20260709 13:13:25.020076 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:25.020982 30754 raft_consensus.cc:359] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.021747 30754 raft_consensus.cc:385] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:25.022042 30754 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:25.022783 30754 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.023470 31229 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:25.026175 30754 ts_tablet_manager.cc:1434] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:13:25.027498 30754 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:25.029973 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:25.033062 31181 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.034126 31181 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.037119 31181 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:25.038565 30754 tablet_bootstrap.cc:654] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:25.039217 30448 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:25.040689 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:25.041466 30629 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 0.
I20260709 13:13:25.044983 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:25.046576 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:25.047621 31173 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:13:25.048018 31173 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.048374 31173 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:13:25.049503 30670 leader_election.cc:343] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:13:25.054570 31228 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:25.055385 30754 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:25.055977 30754 ts_tablet_manager.cc:1403] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.004s
I20260709 13:13:25.059326 30754 raft_consensus.cc:359] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.060145 30754 raft_consensus.cc:385] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:25.060461 30754 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:25.061478 30754 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.066020 30754 ts_tablet_manager.cc:1434] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:13:25.067286 30754 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:25.072994 30754 tablet_bootstrap.cc:654] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:13:25.083963 31173 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.089979 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:25.094285 30482 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.102278 31173 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:25.107082 30629 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.102029 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:25.108268 30484 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.107071 30754 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: No bootstrap required, opened a new log
I20260709 13:13:25.111409 30754 ts_tablet_manager.cc:1403] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.015s
I20260709 13:13:25.114720 30754 raft_consensus.cc:359] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:25.115468 30754 raft_consensus.cc:385] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:13:25.115779 30754 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:25.116513 30754 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:25.124969 30754 ts_tablet_manager.cc:1434] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:13:25.152098 30484 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:25.154497 30629 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:25.154868 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:25.158291 31227 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:13:25.158927 31227 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:25.159965 31227 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:25.175127 31174 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:25.187124 30381 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 1 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:25.204890 31174 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:25.210582 31229 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.211216 31229 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.214913 31229 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:25.215278 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:25.216053 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:25.217844 30450 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:25.219398 31229 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:13:25.273198 31174 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:25.273774 31174 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.275949 31174 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:25.276818 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:25.277017 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:25.277585 30484 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 0.
I20260709 13:13:25.277786 30555 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:25.278810 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:25.279667 31174 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:25.280099 31174 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:25.280431 31174 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.287068 31174 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.289284 31174 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:25.290046 30484 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:25.290347 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:25.291231 30484 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.291131 30555 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:25.303388 30555 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:25.304481 30484 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:25.304554 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:25.305778 31174 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:25.306269 31174 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:25.307185 31174 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:25.318990 30381 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 0 to 2, leader changed from <none> to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:25.372825 30482 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.467636 30482 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:13:25.552381 31249 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:25.553236 31227 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:25.877558 30477 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:25.879060 31174 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:13:25.884404 30555 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:13:25.886317 31174 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:13:26.873524 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
W20260709 13:13:26.879750 30522 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 167 similar messages]
W20260709 13:13:26.884706 30522 consensus_peers.cc:597] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:26.892329 30596 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:26.911352 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:26.912274 30596 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:26.928874 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:26.930925 30670 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:26.937585 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:13:26.938855 21526 tablet_replica.cc:333] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.940093 21526 raft_consensus.cc:2243] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.940762 21526 raft_consensus.cc:2272] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.943884 21526 tablet_replica.cc:333] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
W20260709 13:13:26.944480 30670 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:26.945252 21526 raft_consensus.cc:2243] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.945938 21526 raft_consensus.cc:2272] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.948235 21526 tablet_replica.cc:333] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.948864 21526 raft_consensus.cc:2243] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.949334 21526 raft_consensus.cc:2272] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.951557 21526 tablet_replica.cc:333] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.952293 21526 raft_consensus.cc:2243] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.952912 21526 raft_consensus.cc:2272] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.955364 21526 tablet_replica.cc:333] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.955982 21526 raft_consensus.cc:2243] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.956452 21526 raft_consensus.cc:2272] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:26.957719 30596 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:26.958746 21526 tablet_replica.cc:333] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.959496 21526 raft_consensus.cc:2243] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.959975 21526 raft_consensus.cc:2272] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.962533 21526 tablet_replica.cc:333] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.963310 21526 raft_consensus.cc:2243] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.963862 21526 raft_consensus.cc:2272] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.965986 21526 tablet_replica.cc:333] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.966785 21526 raft_consensus.cc:2243] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.967264 21526 raft_consensus.cc:2272] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.970019 21526 tablet_replica.cc:333] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.971100 21526 raft_consensus.cc:2243] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.971742 21526 raft_consensus.cc:2272] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:26.973376 30522 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:26.974375 21526 tablet_replica.cc:333] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.975559 21526 raft_consensus.cc:2243] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.976208 21526 raft_consensus.cc:2272] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.979458 21526 tablet_replica.cc:333] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.980499 21526 raft_consensus.cc:2243] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.981117 21526 raft_consensus.cc:2272] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.984045 21526 tablet_replica.cc:333] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.985162 21526 raft_consensus.cc:2243] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.985793 21526 raft_consensus.cc:2272] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.988226 21526 tablet_replica.cc:333] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.989117 21526 raft_consensus.cc:2243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:26.989724 21526 raft_consensus.cc:2272] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.992022 21526 tablet_replica.cc:333] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.992995 21526 raft_consensus.cc:2243] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:26.994119 21526 raft_consensus.cc:2272] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:26.997489 21526 tablet_replica.cc:333] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:26.998473 21526 raft_consensus.cc:2243] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:26.999734 21526 raft_consensus.cc:2272] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.002913 21526 tablet_replica.cc:333] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.003922 21526 raft_consensus.cc:2243] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.004562 21526 raft_consensus.cc:2272] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.007392 21526 tablet_replica.cc:333] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
W20260709 13:13:27.007829 30670 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.008466 21526 raft_consensus.cc:2243] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.009203 21526 raft_consensus.cc:2272] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.009899 30522 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.010839 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.011510 30522 consensus_peers.cc:597] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.012629 21526 tablet_replica.cc:333] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.013638 21526 raft_consensus.cc:2243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.014334 21526 raft_consensus.cc:2272] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.017397 21526 tablet_replica.cc:333] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.018267 21526 raft_consensus.cc:2243] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.018914 21526 raft_consensus.cc:2272] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.022090 21526 tablet_replica.cc:333] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.023012 21526 raft_consensus.cc:2243] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.023651 21526 raft_consensus.cc:2272] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.025009 30596 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.026201 21526 tablet_replica.cc:333] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
W20260709 13:13:27.026229 30670 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.027160 21526 raft_consensus.cc:2243] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.027702 21526 raft_consensus.cc:2272] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.029965 21526 tablet_replica.cc:333] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.030956 21526 raft_consensus.cc:2243] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.031569 21526 raft_consensus.cc:2272] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.034963 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.037919 21526 tablet_replica.cc:333] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.039217 21526 raft_consensus.cc:2243] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.039811 21526 raft_consensus.cc:2272] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.041353 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.042943 21526 tablet_replica.cc:333] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.043931 21526 raft_consensus.cc:2243] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.044608 21526 raft_consensus.cc:2272] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.047854 21526 tablet_replica.cc:333] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.048720 21526 raft_consensus.cc:2243] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.049341 21526 raft_consensus.cc:2272] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.052738 21526 tablet_replica.cc:333] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.053367 21526 raft_consensus.cc:2243] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.053979 21526 raft_consensus.cc:2272] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.057333 21526 tablet_replica.cc:333] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
W20260709 13:13:27.058189 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.058203 21526 raft_consensus.cc:2243] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.059253 21526 raft_consensus.cc:2272] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.061781 21526 tablet_replica.cc:333] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.062568 21526 raft_consensus.cc:2243] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.063669 21526 raft_consensus.cc:2272] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.067150 21526 tablet_replica.cc:333] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.068020 21526 raft_consensus.cc:2243] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.068588 21526 raft_consensus.cc:2272] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.071412 21526 tablet_replica.cc:333] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.072290 21526 raft_consensus.cc:2243] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.073534 21526 raft_consensus.cc:2272] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.076956 21526 tablet_replica.cc:333] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.077885 21526 raft_consensus.cc:2243] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.078527 21526 raft_consensus.cc:2272] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.079048 30596 consensus_peers.cc:597] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.081817 21526 tablet_replica.cc:333] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.082906 21526 raft_consensus.cc:2243] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.083637 21526 raft_consensus.cc:2272] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.087064 21526 tablet_replica.cc:333] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.087836 21526 raft_consensus.cc:2243] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.088315 21526 raft_consensus.cc:2272] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.090796 21526 tablet_replica.cc:333] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.091468 21526 raft_consensus.cc:2243] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.091970 21526 raft_consensus.cc:2272] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.094132 21526 tablet_replica.cc:333] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.094898 21526 raft_consensus.cc:2243] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.095397 21526 raft_consensus.cc:2272] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.097975 21526 tablet_replica.cc:333] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.098691 21526 raft_consensus.cc:2243] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.099213 21526 raft_consensus.cc:2272] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.101727 21526 tablet_replica.cc:333] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.102311 21526 raft_consensus.cc:2243] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.103232 21526 raft_consensus.cc:2272] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.105969 21526 tablet_replica.cc:333] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.106762 21526 raft_consensus.cc:2243] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.107340 21526 raft_consensus.cc:2272] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.108039 30670 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.109843 21526 tablet_replica.cc:333] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.110598 21526 raft_consensus.cc:2243] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.111616 21526 raft_consensus.cc:2272] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.114076 21526 tablet_replica.cc:333] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.114719 21526 raft_consensus.cc:2243] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.115731 21526 raft_consensus.cc:2272] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.118536 21526 tablet_replica.cc:333] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.119395 21526 raft_consensus.cc:2243] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.120000 21526 raft_consensus.cc:2272] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.123039 21526 tablet_replica.cc:333] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.123796 21526 raft_consensus.cc:2243] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.124356 21526 raft_consensus.cc:2272] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.127468 21526 tablet_replica.cc:333] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.128150 21526 raft_consensus.cc:2243] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.129338 21526 raft_consensus.cc:2272] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:13:27.132503 21526 tablet_replica.cc:333] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.133324 21526 raft_consensus.cc:2243] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:13:27.133894 21526 raft_consensus.cc:2272] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.136246 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.136672 21526 tablet_replica.cc:333] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: stopping tablet replica
I20260709 13:13:27.137282 21526 raft_consensus.cc:2243] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:13:27.138185 21526 raft_consensus.cc:2272] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:13:27.149619 30596 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.157521 30596 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.160130 30596 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.169284 30670 consensus_peers.cc:597] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.175165 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.198215 30596 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.199110 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.226337 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
W20260709 13:13:27.228613 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.229302 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.247205 30596 consensus_peers.cc:597] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.269824 30670 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.274343 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.319852 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:13:27.333600 31271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:27.356261 21526 server_base.cc:1061] running on GCE node
W20260709 13:13:27.354851 31273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:13:27.363655 30522 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:27.384970 31278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:13:27.388959 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:13:27.389331 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:13:27.389705 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783602807389673 us; error 0 us; skew 500 ppm
I20260709 13:13:27.395140 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260709 13:13:27.407943 30596 consensus_peers.cc:597] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.440779 21526 webserver.cc:533] Webserver started at http://127.21.5.129:42141/ using document root <none> and password file <none>
I20260709 13:13:27.441984 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:13:27.442265 21526 fs_manager.cc:365] Using existing metadata directory in first data directory
W20260709 13:13:27.443048 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:27.453410 21526 fs_manager.cc:714] Time spent opening directory manager: real 0.010s	user 0.000s	sys 0.007s
I20260709 13:13:27.467525 31290 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:13:27.470741 21526 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.000s	sys 0.003s
I20260709 13:13:27.473209 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "015ca80eba674552b34856a4f20271f1"
format_stamp: "Formatted at 2026-07-09 13:13:11 on dist-test-slave-h0rz"
I20260709 13:13:27.473739 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:13:27.504233 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:13:27.506210 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:13:27.508812 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:13:27.516381 31297 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:13:27.917384 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:13:27.917850 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.403s	user 0.009s	sys 0.000s
I20260709 13:13:27.918215 21526 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:13:27.926213 31297 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:27.947685 31297 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:27.948673 31297 tablet_bootstrap.cc:492] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:27.949359 31297 ts_tablet_manager.cc:1403] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 13:13:27.952450 31297 raft_consensus.cc:359] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:27.953327 31297 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:27.954172 31297 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:27.957015 31297 ts_tablet_manager.cc:1434] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:13:27.958009 31297 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:27.977016 31297 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:27.978068 31297 tablet_bootstrap.cc:492] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:27.978785 31297 ts_tablet_manager.cc:1403] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260709 13:13:27.981067 31297 raft_consensus.cc:359] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:27.981652 31297 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:27.982220 31297 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:27.983884 31297 ts_tablet_manager.cc:1434] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:13:27.984760 31297 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.007751 31297 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.008816 31297 tablet_bootstrap.cc:492] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.009541 31297 ts_tablet_manager.cc:1403] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.004s
I20260709 13:13:28.012302 31297 raft_consensus.cc:359] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.013101 31297 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.013797 31297 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.015919 31297 ts_tablet_manager.cc:1434] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:13:28.016837 31297 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.033453 31297 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.034543 31297 tablet_bootstrap.cc:492] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.035477 31297 ts_tablet_manager.cc:1403] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:13:28.038283 31297 raft_consensus.cc:359] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.039074 31297 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.039755 31297 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.041066 31298 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.041913 31297 ts_tablet_manager.cc:1434] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:13:28.041674 31298 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.042883 31297 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.049254 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:28.053896 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.054455 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:28.055383 31268 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.057126 31298 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.063764 31297 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.064926 31297 tablet_bootstrap.cc:492] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.065671 31297 ts_tablet_manager.cc:1403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:13:28.068553 31297 raft_consensus.cc:359] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.069298 31297 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.070063 31297 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.072193 31297 ts_tablet_manager.cc:1434] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:13:28.073068 31297 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.090971 31297 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.092172 31297 tablet_bootstrap.cc:492] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.092916 31297 ts_tablet_manager.cc:1403] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 13:13:28.095579 31297 raft_consensus.cc:359] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.096330 31297 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.097162 31297 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.098861 31297 ts_tablet_manager.cc:1434] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:13:28.099663 31297 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.108777 31285 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.109509 31285 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.117800 31285 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:28.119758 30670 leader_election.cc:336] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.121163 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.123085 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:28.124368 31285 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.131244 31297 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.132257 31297 tablet_bootstrap.cc:492] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.132916 31297 ts_tablet_manager.cc:1403] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.033s	user 0.022s	sys 0.004s
I20260709 13:13:28.135422 31297 raft_consensus.cc:359] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.136019 31297 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.136778 31297 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.138969 31297 ts_tablet_manager.cc:1434] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:13:28.139899 31297 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.149438 31285 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.150177 31285 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.159761 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:28.165005 30670 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.165668 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:28.169700 31285 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.170576 31279 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.188251 31268 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.188941 31268 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.197517 31268 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.200562 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.201315 30705 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.205647 31297 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.206764 31297 tablet_bootstrap.cc:492] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.207449 31297 ts_tablet_manager.cc:1403] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.068s	user 0.037s	sys 0.012s
I20260709 13:13:28.211640 31297 raft_consensus.cc:359] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.212357 31297 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.213120 31297 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:28.214962 30596 leader_election.cc:336] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.215356 31297 ts_tablet_manager.cc:1434] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:13:28.216261 31297 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.216770 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.218030 31268 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.218420 31268 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.218819 31268 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.231719 31268 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.239490 31073 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.240204 31073 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.240835 31285 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.241707 31285 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.245275 31268 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.245762 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.246654 30702 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.248029 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:28.248695 21526 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:13:28.249122 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.331s	user 0.316s	sys 0.007s
I20260709 13:13:28.249887 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:28.248267 31297 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.251138 31174 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.251606 31174 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.252056 31174 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.250560 30705 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.242376 31073 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:28.252013 30596 leader_election.cc:336] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.245877 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.257090 31249 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.257776 31249 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.260212 31249 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.263142 31297 tablet_bootstrap.cc:492] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.264170 31297 ts_tablet_manager.cc:1403] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.011s
I20260709 13:13:28.266841 31284 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.267539 31284 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.267647 30705 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:28.269090 31297 raft_consensus.cc:359] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.270148 31297 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.271266 31285 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.271023 31297 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:28.272413 30596 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:13:28.272665 30670 leader_election.cc:336] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.274531 31297 ts_tablet_manager.cc:1434] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:13:28.275970 31297 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.279529 31284 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:28.289319 30670 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.289577 31310 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.290344 31310 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.291554 31174 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.292904 30629 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:28.298988 31174 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.299141 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:13:28.301088 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:13:28.303117 30596 leader_election.cc:336] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.307272 31297 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.308884 31297 tablet_bootstrap.cc:492] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.309906 31297 ts_tablet_manager.cc:1403] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.008s
W20260709 13:13:28.311498 30596 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.313231 31297 raft_consensus.cc:359] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.314301 31297 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.315357 31297 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.316107 31298 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.316931 31298 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:28.318305 31297 ts_tablet_manager.cc:1434] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:13:28.317886 31298 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.319411 31297 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.317796 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.321036 30629 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:28.321751 31310 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.322713 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.323460 30629 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:28.327980 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.328708 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.332741 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:28.334110 31284 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.346815 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:28.348033 31285 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.348973 31297 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.348996 31285 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.350737 31297 tablet_bootstrap.cc:492] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.351032 31276 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.351665 31276 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.352007 31297 ts_tablet_manager.cc:1403] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.007s
I20260709 13:13:28.353878 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:28.356217 31285 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:28.356460 31297 raft_consensus.cc:359] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.357651 31297 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.359163 31299 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.359009 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.359754 31299 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.360672 31285 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.361233 31285 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.361755 31285 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.362171 31299 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:28.368696 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.358765 31297 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.369985 31279 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.370388 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.370589 31279 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.371260 31279 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.371865 31297 ts_tablet_manager.cc:1434] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:13:28.372843 31297 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.375150 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:28.368703 30382 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:28.382846 31276 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:28.386914 30522 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.386785 31285 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.389945 31285 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.390976 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:13:28.395830 30522 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.397791 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:28.393306 31299 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.396385 31276 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.401854 31299 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.403422 31276 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.404659 31299 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.406699 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.407492 30705 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:28.408774 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:28.409482 31279 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.411134 31299 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.400904 31284 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.426561 31299 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:13:28.418032 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.427114 31299 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.411212 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:28.422322 31297 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.431098 30633 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.431864 31297 tablet_bootstrap.cc:492] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.432854 31297 ts_tablet_manager.cc:1403] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.004s
I20260709 13:13:28.436473 31297 raft_consensus.cc:359] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.437271 31297 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.422421 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:13:28.425734 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.440824 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:28.447690 31299 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.451838 31297 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.465842 31297 ts_tablet_manager.cc:1434] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.001s
I20260709 13:13:28.467590 31297 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.452476 30704 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:28.454026 31285 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.477523 31299 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.480261 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:28.480947 30705 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.493434 30705 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:28.453948 30633 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:28.504196 31073 raft_consensus.cc:2764] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:28.507433 31317 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.508337 31317 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.511478 31317 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.494261 31284 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.456354 31279 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:28.502463 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.512883 31284 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.515412 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.518392 31317 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.520932 31317 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:28.522589 31317 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.527014 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.527853 30705 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
W20260709 13:13:28.523746 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59540.4us
I20260709 13:13:28.531214 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:28.532830 31299 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.533413 31299 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.534107 31299 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.539278 31298 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.540194 31298 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.576432 31073 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.577312 31298 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:28.581137 31073 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:28.589346 30670 leader_election.cc:336] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:13:28.590603 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.597280 31174 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.598270 31299 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.598004 31174 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.600100 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:28.600649 31299 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.601151 30633 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.601325 31174 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:28.603652 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:28.604568 30705 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.605906 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:28.606770 30557 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:28.608500 31073 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.610276 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:28.611213 30557 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.620956 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.603878 31284 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:28.626689 30522 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.652271 31174 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.654521 31174 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:28.626323 30382 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:28.655512 31174 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.658569 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.634244 30705 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:28.662228 31324 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.638581 30633 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:28.643916 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:28.663326 30555 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.662755 31324 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.664767 31324 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:28.665875 30522 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.669137 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.670542 31299 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.671329 31299 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:13:28.646461 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.648104 31297 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:13:28.673859 30596 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.673521 31299 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:28.674573 31297 tablet_bootstrap.cc:492] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.675504 31297 ts_tablet_manager.cc:1403] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.208s	user 0.087s	sys 0.086s
W20260709 13:13:28.649813 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:13:28.678679 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.678376 31297 raft_consensus.cc:359] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.679471 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:28.679843 31297 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.680644 31284 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:28.681236 31297 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.684257 31297 ts_tablet_manager.cc:1434] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:13:28.685500 31297 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.685177 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.691319 31268 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.691882 31268 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.692245 31268 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.721487 31268 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.743687 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:28.767000 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:13:28.781548 31336 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:13:28.785492 31324 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.787145 31268 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:28.789076 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 134648us
I20260709 13:13:28.793351 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.793187 30382 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:28.794833 31284 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.795709 31284 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:28.796638 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:28.797434 30559 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.797039 31284 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.802640 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:28.803700 30705 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:28.806454 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.806847 30559 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:28.808215 31285 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.808961 31285 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:28.817348 31324 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:28.832214 30704 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:28.837589 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:28.810312 31285 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.859031 31297 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:28.860687 31297 tablet_bootstrap.cc:492] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:28.861595 31297 ts_tablet_manager.cc:1403] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.176s	user 0.074s	sys 0.052s
I20260709 13:13:28.864161 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.869148 31268 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:28.869666 31268 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:28.870056 31268 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.870412 30382 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:28.877915 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.882366 31297 raft_consensus.cc:359] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.883581 31297 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:28.884362 31297 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.884697 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:28.885380 30559 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:28.887126 31297 ts_tablet_manager.cc:1434] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 13:13:28.888160 31297 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:28.882094 31310 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.891319 31310 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:28.892611 31310 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:28.909132 30382 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:28.911290 30559 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:28.913089 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:28.913929 31343 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:28.914677 31343 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:28.915629 31343 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:28.930716 30596 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:28.936597 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 68992.6us
I20260709 13:13:28.938505 31333 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:13:28.966789 31268 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:28.973184 31268 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:28.981055 30382 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:28.992195 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:28.993103 30704 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:29.023294 30522 consensus_peers.cc:597] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:29.027478 30596 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:13:29.058779 30704 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:29.077775 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:29.082947 31297 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:29.084333 31297 tablet_bootstrap.cc:492] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:29.085736 31297 ts_tablet_manager.cc:1403] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.198s	user 0.037s	sys 0.011s
I20260709 13:13:29.087379 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:29.088680 31073 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:29.088492 31297 raft_consensus.cc:359] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.089382 31073 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:29.089596 31297 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:29.090389 31073 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:29.090564 31297 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.094489 31297 ts_tablet_manager.cc:1434] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:13:29.095782 31297 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:29.103147 30705 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.121582 31317 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:29.125186 31297 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:29.126407 31297 tablet_bootstrap.cc:492] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
W20260709 13:13:29.126240 30522 consensus_peers.cc:597] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:29.127532 31297 ts_tablet_manager.cc:1403] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.032s	user 0.024s	sys 0.002s
I20260709 13:13:29.132527 31297 raft_consensus.cc:359] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.134534 31297 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:29.135557 31297 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.138602 31297 ts_tablet_manager.cc:1434] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:13:29.139576 31297 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
W20260709 13:13:29.140322 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51028.7us
I20260709 13:13:29.143604 30704 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.143242 30382 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:29.157616 31301 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.158587 31301 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:29.162750 31344 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:29.169469 31301 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:29.201042 30596 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.212522 30629 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:29.219128 30596 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:29.253855 31297 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:29.255239 31297 tablet_bootstrap.cc:492] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:29.256098 31297 ts_tablet_manager.cc:1403] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.117s	user 0.016s	sys 0.006s
I20260709 13:13:29.258682 30704 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.262151 31320 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:29.259841 31297 raft_consensus.cc:359] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.263605 31297 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:29.264652 31297 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.265048 31341 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:29.267365 31297 ts_tablet_manager.cc:1434] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:13:29.268294 31297 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
W20260709 13:13:29.276497 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.328635 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.351339 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:29.366580 30522 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.368101 30670 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:29.370522 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:29.378880 30522 consensus_peers.cc:597] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.385888 31297 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:13:29.393168 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:13:29.401997 31297 tablet_bootstrap.cc:492] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:29.403540 31297 ts_tablet_manager.cc:1403] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.135s	user 0.047s	sys 0.029s
W20260709 13:13:29.405804 30670 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.407123 31297 raft_consensus.cc:359] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:29.408169 31297 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:29.409072 31297 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:29.475044 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:29.483959 31301 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.496870 30555 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.505960 31301 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.517932 30629 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.524530 31301 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:29.531286 31301 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:29.546939 30670 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.548209 31284 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:13:29.552357 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.532356 31297 ts_tablet_manager.cc:1434] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.128s	user 0.011s	sys 0.001s
I20260709 13:13:29.557173 31301 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:29.568423 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94131.9us
I20260709 13:13:29.573431 31297 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:29.585572 31333 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:13:29.561203 31301 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:29.606714 31301 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:29.607445 31301 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.608489 31301 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.613492 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.617257 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:29.619519 30555 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:29.623292 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:29.629812 31301 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:29.620882 31386 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.630411 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.632323 31343 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:29.636137 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:29.636386 30670 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.638697 31301 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.631268 31386 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:29.647536 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.647660 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.655936 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.659364 31301 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.659917 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:29.660571 31301 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.654250 31386 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:29.675218 31301 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:29.669720 31386 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:29.682382 30702 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:29.680269 30522 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.683876 30522 consensus_peers.cc:597] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.699266 31344 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:29.701920 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.699570 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:29.742816 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 65428.5us
W20260709 13:13:29.757633 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.775097 31386 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.775931 31386 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:29.783226 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 122399us
I20260709 13:13:29.809124 31301 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.810163 31301 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:29.804864 30670 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.824476 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.825245 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.825737 30522 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.828588 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:29.832190 31386 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:29.837745 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.849336 31301 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:29.867688 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:29.819348 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:29.873138 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:29.874015 31386 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.874984 31386 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:29.876329 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61786.3us
I20260709 13:13:29.899690 31301 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.900673 31301 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.912155 31301 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:29.918438 31297 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:13:29.914332 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:29.919971 30670 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.919550 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:29.930094 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.932624 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.933141 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.935324 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:29.936244 31386 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.937070 31386 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.938421 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:29.939477 31386 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.941567 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.943648 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:29.945014 31301 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:29.950894 31301 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:29.951716 31301 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.958035 31297 tablet_bootstrap.cc:492] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:29.960985 31297 ts_tablet_manager.cc:1403] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.389s	user 0.032s	sys 0.010s
I20260709 13:13:29.968930 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:29.972473 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:29.976033 31301 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:29.979182 31297 raft_consensus.cc:359] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.980204 31297 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:29.981194 31297 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:29.984272 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:29.985677 31301 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:29.983520 30670 consensus_peers.cc:597] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:29.998811 31297 ts_tablet_manager.cc:1434] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.037s	user 0.004s	sys 0.006s
I20260709 13:13:30.000794 31297 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
W20260709 13:13:30.007400 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.009128 30596 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:30.009776 30596 consensus_peers.cc:597] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:30.010737 30596 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.011224 30596 consensus_peers.cc:597] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:13:30.012264 30596 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.012967 30596 consensus_peers.cc:597] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.020172 30670 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.030177 30596 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.030947 30596 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.031407 30596 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.031598 31297 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:13:30.031875 30596 consensus_peers.cc:597] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.034366 31297 tablet_bootstrap.cc:492] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.035266 31297 ts_tablet_manager.cc:1403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.035s	user 0.016s	sys 0.012s
I20260709 13:13:30.038704 31297 raft_consensus.cc:359] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.039501 31297 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.040547 31297 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:30.044382 30596 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.044436 31297 ts_tablet_manager.cc:1434] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 13:13:30.045158 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:13:30.045681 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.046267 31297 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
W20260709 13:13:30.050408 30596 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.056612 31301 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.057415 31301 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.061561 31301 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:30.063354 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:30.084048 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:30.085616 31297 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.086899 31297 tablet_bootstrap.cc:492] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.086601 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:30.087913 31297 ts_tablet_manager.cc:1403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.042s	user 0.026s	sys 0.009s
I20260709 13:13:30.088491 31301 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:30.091539 31297 raft_consensus.cc:359] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.092648 31297 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.099737 31297 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.103086 31297 ts_tablet_manager.cc:1434] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 13:13:30.104455 31297 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.136508 31297 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.137349 31297 tablet_bootstrap.cc:492] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.138027 31297 ts_tablet_manager.cc:1403] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.000s
I20260709 13:13:30.141340 31297 raft_consensus.cc:359] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.142076 31426 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:41021 every 8 connection(s)
I20260709 13:13:30.142176 31297 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.142752 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:41021
I20260709 13:13:30.143379 31297 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.149225 31297 ts_tablet_manager.cc:1434] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.004s
I20260709 13:13:30.150506 31297 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.165201 31301 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.166250 31301 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.171304 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:30.177875 31301 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:30.183678 31297 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.185184 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:30.185828 31297 tablet_bootstrap.cc:492] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.186897 31297 ts_tablet_manager.cc:1403] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.010s
I20260709 13:13:30.190222 31297 raft_consensus.cc:359] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.192448 31297 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.193390 31297 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.196033 31297 ts_tablet_manager.cc:1434] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:13:30.197189 31297 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.225281 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:30.226449 31301 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:30.263736 31301 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.266708 31301 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.269155 31301 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:30.271070 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:30.276474 31297 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.277576 31394 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.277997 31297 tablet_bootstrap.cc:492] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.279170 31297 ts_tablet_manager.cc:1403] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.082s	user 0.025s	sys 0.005s
I20260709 13:13:30.282100 31297 raft_consensus.cc:359] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.282727 31297 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.284004 31297 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.285907 31297 ts_tablet_manager.cc:1434] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:30.287261 31297 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.311921 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:30.319526 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:30.321704 31427 heartbeater.cc:344] Connected to a master server at 127.21.5.190:45121
I20260709 13:13:30.322273 31427 heartbeater.cc:461] Registering TS with master...
I20260709 13:13:30.323570 31427 heartbeater.cc:507] Master 127.21.5.190:45121 requested a full tablet report, sending...
I20260709 13:13:30.353931 31301 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:30.367969 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:13:30.379722 31297 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.381107 31297 tablet_bootstrap.cc:492] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.381929 31297 ts_tablet_manager.cc:1403] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.095s	user 0.027s	sys 0.004s
I20260709 13:13:30.385020 31297 raft_consensus.cc:359] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.385625 30378 ts_manager.cc:194] Re-registered known tserver with Master: 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:30.385859 31297 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.386811 31297 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.389114 31297 ts_tablet_manager.cc:1434] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:13:30.390035 31297 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.409914 31297 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.411089 31297 tablet_bootstrap.cc:492] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.411851 31297 ts_tablet_manager.cc:1403] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:13:30.414887 31297 raft_consensus.cc:359] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.415678 31297 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.416462 31297 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.419185 31297 ts_tablet_manager.cc:1434] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:13:30.420198 31297 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.437078 30378 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35874
I20260709 13:13:30.440140 31297 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.441255 31297 tablet_bootstrap.cc:492] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.441879 31297 ts_tablet_manager.cc:1403] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 13:13:30.443715 31427 heartbeater.cc:499] Master 127.21.5.190:45121 was elected leader, sending a full tablet report...
I20260709 13:13:30.444634 31297 raft_consensus.cc:359] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.445395 31297 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.446061 31297 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.448508 31297 ts_tablet_manager.cc:1434] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:13:30.449482 31297 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.460947 31391 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.467077 31297 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.468317 31297 tablet_bootstrap.cc:492] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.469118 31297 ts_tablet_manager.cc:1403] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:13:30.472280 31297 raft_consensus.cc:359] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.473073 31297 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.473929 31297 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.476614 31297 ts_tablet_manager.cc:1434] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:13:30.477584 31297 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.499383 31297 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.500077 31301 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.500762 31301 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.503755 31301 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:30.505095 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:30.505620 31297 tablet_bootstrap.cc:492] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.506317 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:30.506660 31297 ts_tablet_manager.cc:1403] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.012s
I20260709 13:13:30.507747 31301 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:30.508039 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:30.508356 31301 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.509744 31297 raft_consensus.cc:359] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.510483 31297 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.511428 31297 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.513751 31297 ts_tablet_manager.cc:1434] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:13:30.515517 31297 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.527789 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:30.528234 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:30.530764 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:30.531889 31444 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:30.533576 31386 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:30.535663 31301 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:30.536597 31297 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.537777 31297 tablet_bootstrap.cc:492] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.538550 31297 ts_tablet_manager.cc:1403] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.001s
I20260709 13:13:30.541734 31297 raft_consensus.cc:359] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.542483 31297 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.543187 31297 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.545457 31297 ts_tablet_manager.cc:1434] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:13:30.546451 31391 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.547442 31297 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.555666 31396 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.642365 31297 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.643538 31297 tablet_bootstrap.cc:492] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.644459 31297 ts_tablet_manager.cc:1403] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.097s	user 0.029s	sys 0.001s
I20260709 13:13:30.647943 31297 raft_consensus.cc:359] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.648705 31297 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.649493 31297 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.652072 31297 ts_tablet_manager.cc:1434] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:13:30.653599 31297 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.671363 31297 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.672616 31297 tablet_bootstrap.cc:492] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.673522 31297 ts_tablet_manager.cc:1403] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:13:30.677290 31297 raft_consensus.cc:359] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.678169 31297 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.678858 31297 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.681665 31297 ts_tablet_manager.cc:1434] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:13:30.682854 31297 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.701129 31297 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.702008 31297 tablet_bootstrap.cc:492] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.702746 31297 ts_tablet_manager.cc:1403] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:13:30.705410 31297 raft_consensus.cc:359] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.706161 31297 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.706883 31297 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.708892 31297 ts_tablet_manager.cc:1434] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:30.710192 31297 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.727527 31297 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.728600 31297 tablet_bootstrap.cc:492] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.729385 31297 ts_tablet_manager.cc:1403] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:13:30.732574 31297 raft_consensus.cc:359] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.733413 31297 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.734279 31297 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:30.736721 31297 ts_tablet_manager.cc:1434] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:13:30.737644 31297 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.760401 31297 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.761616 31297 tablet_bootstrap.cc:492] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.762427 31297 ts_tablet_manager.cc:1403] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.000s
I20260709 13:13:30.765309 31297 raft_consensus.cc:359] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.766057 31297 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.766812 31297 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:30.769119 31396 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.769585 31297 ts_tablet_manager.cc:1434] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:13:30.770670 31297 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.787575 31391 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.825314 31297 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.826421 31297 tablet_bootstrap.cc:492] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.827260 31297 ts_tablet_manager.cc:1403] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.027s
I20260709 13:13:30.830344 31297 raft_consensus.cc:359] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.831192 31297 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.831732 31297 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.833407 31297 ts_tablet_manager.cc:1434] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:13:30.834091 31297 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.844414 31391 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:30.866887 31297 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:30.868127 31297 tablet_bootstrap.cc:492] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:30.869278 31297 ts_tablet_manager.cc:1403] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.016s
I20260709 13:13:30.872720 31297 raft_consensus.cc:359] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.873533 31297 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:30.874419 31297 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:30.877104 31297 ts_tablet_manager.cc:1434] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:13:30.878729 31297 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:30.999807 31297 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.000952 31297 tablet_bootstrap.cc:492] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.001677 31297 ts_tablet_manager.cc:1403] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.123s	user 0.050s	sys 0.040s
I20260709 13:13:31.004755 31297 raft_consensus.cc:359] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:31.005472 31297 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.012176 31297 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:31.020206 31297 ts_tablet_manager.cc:1434] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.001s
I20260709 13:13:31.021332 31297 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.067529 31297 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.068805 31297 tablet_bootstrap.cc:492] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.069628 31297 ts_tablet_manager.cc:1403] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.049s	user 0.020s	sys 0.020s
I20260709 13:13:31.072522 31297 raft_consensus.cc:359] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.073279 31297 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.073997 31297 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.076442 31297 ts_tablet_manager.cc:1434] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:13:31.077687 31297 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.098374 31297 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.099876 31297 tablet_bootstrap.cc:492] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.100873 31297 ts_tablet_manager.cc:1403] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 13:13:31.105044 31297 raft_consensus.cc:359] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.105695 31297 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.106449 31297 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.108395 31297 ts_tablet_manager.cc:1434] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:13:31.109819 31297 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.180300 31297 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.181452 31297 tablet_bootstrap.cc:492] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.182255 31297 ts_tablet_manager.cc:1403] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.073s	user 0.037s	sys 0.016s
I20260709 13:13:31.185245 31297 raft_consensus.cc:359] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:31.186040 31297 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.188730 31297 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:31.194058 31297 ts_tablet_manager.cc:1434] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 13:13:31.195120 31297 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.219676 31391 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:31.244431 31297 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.248389 31297 tablet_bootstrap.cc:492] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.249178 31297 ts_tablet_manager.cc:1403] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.054s	user 0.023s	sys 0.015s
I20260709 13:13:31.252444 31297 raft_consensus.cc:359] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.253238 31297 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.253984 31297 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.256016 31297 ts_tablet_manager.cc:1434] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:13:31.258926 31297 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.288695 31396 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:31.290721 31297 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.291939 31297 tablet_bootstrap.cc:492] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.292696 31297 ts_tablet_manager.cc:1403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.004s
I20260709 13:13:31.295518 31297 raft_consensus.cc:359] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.296213 31297 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.296968 31297 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.298614 31297 ts_tablet_manager.cc:1434] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:13:31.299571 31297 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.338773 31297 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.348045 31297 tablet_bootstrap.cc:492] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.349042 31297 ts_tablet_manager.cc:1403] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.050s	user 0.016s	sys 0.022s
I20260709 13:13:31.352701 31297 raft_consensus.cc:359] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.353506 31297 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.354271 31297 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:31.356554 31297 ts_tablet_manager.cc:1434] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:13:31.357517 31297 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Bootstrap starting.
I20260709 13:13:31.390518 31297 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:13:31.391531 31297 tablet_bootstrap.cc:492] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Bootstrap complete.
I20260709 13:13:31.392206 31297 ts_tablet_manager.cc:1403] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.012s
I20260709 13:13:31.394444 31297 raft_consensus.cc:359] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.395092 31297 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Initialized, Role: FOLLOWER
I20260709 13:13:31.395733 31297 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:31.398298 31297 ts_tablet_manager.cc:1434] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
W20260709 13:13:34.130424 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08003e28a0 after lost signal to thread 21529
W20260709 13:13:34.131923 30654 debug-util.cc:398] Leaking SignalData structure 0x7b080035f4a0 after lost signal to thread 30416
W20260709 13:13:34.133590 30654 debug-util.cc:398] Leaking SignalData structure 0x7b0800369f40 after lost signal to thread 30583
W20260709 13:13:34.135289 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08000bbce0 after lost signal to thread 30657
W20260709 13:13:34.136986 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08001d2660 after lost signal to thread 30731
W20260709 13:13:34.139278 30654 debug-util.cc:398] Leaking SignalData structure 0x7b080021d340 after lost signal to thread 31426
I20260709 13:13:40.350528 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:40.413007 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "536e99f894744c35a0490416a32a875e"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:40.414151 30632 raft_consensus.cc:606] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:40.417073 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:40.417689 30632 raft_consensus.cc:606] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:40.443703 30703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:13:40.444455 30703 raft_consensus.cc:606] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:40.447286 31496 raft_consensus.cc:993] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.447817 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:40.448089 31486 raft_consensus.cc:1081] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.448555 30632 raft_consensus.cc:606] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:40.450539 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "536e99f894744c35a0490416a32a875e"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:40.451270 31361 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:40.451684 31361 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.455945 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:40.456926 30632 raft_consensus.cc:606] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:40.458679 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:40.460999 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:40.464013 31361 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.466466 31361 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:40.467958 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:40.468727 30702 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.467434 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:40.469648 30632 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:40.470029 30632 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:40.471117 30632 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.472746 30632 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.474948 31490 raft_consensus.cc:993] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:40.475711 31478 raft_consensus.cc:1081] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Signalling peer 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:40.477658 30558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:49258
I20260709 13:13:40.478276 30558 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:40.478672 30558 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.483312 30632 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.484656 30702 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.485484 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:40.485994 30558 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:40.487876 31502 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:40.489059 30558 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:40.489740 31502 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:40.490135 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:40.490135 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:40.491117 30632 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.491253 30702 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:40.491091 31502 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.492026 30702 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:40.493028 30702 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:40.494665 30702 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.499366 30632 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:40.501128 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:40.502199 31497 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:40.502247 30702 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:40.502789 31497 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:40.503774 31497 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:40.514806 30378 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.541603 30382 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 2, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:40.587198 31496 raft_consensus.cc:993] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.587694 31486 raft_consensus.cc:1081] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.589413 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:40.589978 31361 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:40.590307 31361 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.598274 31361 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.600546 31361 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:40.602561 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:40.603049 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:40.603345 30552 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.603781 30632 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:40.604143 30632 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:40.604944 30632 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.606063 30632 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.610282 30552 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.612195 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:40.612582 30632 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.613269 31502 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:40.613829 31502 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:40.614880 31502 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.640260 30382 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:40.796511 31496 raft_consensus.cc:993] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.797078 31496 raft_consensus.cc:1081] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.798779 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:40.799365 31361 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:40.799715 31361 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.815753 31361 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.817957 31361 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:40.820722 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:40.821385 30632 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:40.821759 30632 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:40.823096 31496 raft_consensus.cc:993] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.822713 30632 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.819567 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:40.826205 30632 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.823850 31501 raft_consensus.cc:1081] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:40.834020 30552 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:40.837730 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:40.838343 31361 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:40.838852 31361 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:40.846393 30632 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.847714 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:40.848711 31502 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:40.849242 31502 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:40.850251 31502 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:40.852043 31361 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.853737 30552 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:40.854524 31361 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:40.855650 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:40.855650 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:40.856395 30702 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:40.856760 30632 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:40.857075 30632 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:40.857944 30632 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.859431 30632 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:40.867516 30632 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:40.867539 30382 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:40.871173 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:40.871580 30702 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:40.872922 31502 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:40.873448 31502 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:40.874320 31502 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:40.889846 30382 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:40.945679 30628 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:40.947582 31502 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:40.978292 30702 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:40.981083 31503 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:41.019019 30632 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.020633 31497 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:41.038769 30629 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.039242 30702 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.040535 31503 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:41.044088 31489 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:41.082167 30552 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.087893 31503 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:41.285250 30628 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:41.286861 31503 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:41.296741 30703 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:41.298740 31527 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:41.312944 30552 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.314424 31527 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:41.341346 30628 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:41.343163 31502 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:41.462574 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:41.512646 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:41.559252 30628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:41.559942 30628 raft_consensus.cc:606] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:41.563141 30628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3511f2706d78410aa91eb654ab345803"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:41.563686 30628 raft_consensus.cc:606] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:41.566818 30628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:41.568641 30628 raft_consensus.cc:606] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:41.571549 30628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:41.572184 30628 raft_consensus.cc:606] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:41.575398 30628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:41.576195 30628 raft_consensus.cc:606] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:41.577934 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:41.580365 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:41.584347 31486 raft_consensus.cc:993] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: : Instructing follower 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:41.585004 31519 raft_consensus.cc:1081] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Signalling peer 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:41.588172 30552 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:49250
I20260709 13:13:41.588891 30552 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:41.589313 30552 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:41.608840 30552 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:41.610560 30552 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:41.611617 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:41.611944 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:41.612501 30628 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:41.612541 30703 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:41.613073 30628 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:41.614037 30628 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:41.615581 30628 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:41.618063 31486 raft_consensus.cc:993] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:41.618721 31486 raft_consensus.cc:1081] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:41.621001 31391 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:41.621799 31391 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:41.622172 31391 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:41.624459 30703 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:41.626194 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:41.627324 31489 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:41.627843 31489 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:41.628937 31489 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:41.632583 31391 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:41.635200 30628 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:41.636406 31391 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:41.639096 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:41.640024 30628 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:41.640154 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:41.640508 30628 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:41.640930 30552 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:41.641598 30628 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:41.642583 30382 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:41.643185 30628 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:41.701218 30628 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:41.717193 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:41.718225 31527 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:41.718870 31527 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:41.720086 31527 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:42.585877 30382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.007s	sys 0.020s
W20260709 13:13:42.586452 30382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.007s	sys 0.020s
I20260709 13:13:42.776330 31537 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:42.777642 31537 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:42.787416 31522 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:42.788617 31522 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:42.792199 31522 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:42.837987 31540 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:42.839103 31540 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:42.843272 31540 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:42.870090 31497 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:42.871261 31497 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:42.874377 31497 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:42.902364 31497 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:42.903323 31497 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:42.907024 31497 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:42.962875 31489 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.048157 31489 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.051692 31489 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.051808 31539 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.052521 31489 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.053095 31539 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.053547 31489 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.056648 31489 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.057489 31489 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.058126 31489 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:42.964490 30552 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.323s	user 0.021s	sys 0.008s
I20260709 13:13:43.065364 30552 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:42.968498 31522 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.067649 31522 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.038031 31540 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.071171 31540 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.074640 31540 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:42.966027 31537 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.075693 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 19) took 1436 ms. Trace:
I20260709 13:13:43.079558 30552 rpcz_store.cc:276] 0709 13:13:41.639690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.639889 (+   199us) service_pool.cc:224] Handling call
0709 13:13:43.075557 (+1435668us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.086445 31537 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.087393 31537 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.092520 31497 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.093242 31497 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.097015 31537 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.108543 31537 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.126080 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 458) took 1358 ms. Trace:
I20260709 13:13:43.126811 30558 rpcz_store.cc:276] 0709 13:13:41.768127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.124879 (+1356752us) service_pool.cc:224] Handling call
0709 13:13:43.125343 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.125607 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.125648 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:43.125725 (+    77us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:43.125771 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.125845 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.125993 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.129806 31542 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.131770 31542 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.111162 31537 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.136919 31522 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.137171 31537 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.138145 31544 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.138970 31544 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.142554 31544 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.146178 31546 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.147192 31546 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.150170 31546 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.155125 31541 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.155922 31541 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.162396 31542 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:43.163623 30671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.018s	sys 0.093s
W20260709 13:13:43.164155 30671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.018s	sys 0.093s
I20260709 13:13:43.168134 31540 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.168794 31540 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.170138 31541 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.171275 31540 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.174008 31544 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.174808 31490 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.175225 31544 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.175562 31490 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.174069 31552 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.176955 31552 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.178012 31544 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.178012 31490 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.179006 31490 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.179788 31490 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.174562 31541 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.182219 31541 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.186899 31542 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.144474 31539 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.185204 31540 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.185950 31543 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.187558 31542 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.187985 31540 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.188037 31543 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:42.646659 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:43.220369 31502 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.221884 31527 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.222774 31527 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.225996 31503 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.226737 31503 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.228111 31545 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.228775 31545 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.239491 31538 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.240868 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 457) took 1509 ms. Trace:
I20260709 13:13:43.241511 30557 rpcz_store.cc:276] 0709 13:13:41.731209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:43.239354 (+1508145us) service_pool.cc:224] Handling call
0709 13:13:43.239891 (+   537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:43.240173 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:43.240214 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:43.240282 (+    68us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:43.240331 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:43.240404 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:43.240611 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:43.240525 31538 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.259022 31550 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.259913 31550 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.289983 31549 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.308218 31549 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.357750 31539 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.358654 31539 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.373492 31551 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.374459 31551 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.377166 31544 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.379170 31537 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.380332 31537 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.382982 31522 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.383862 31522 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.412936 31546 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.413993 31546 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.460357 30382 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 121) took 1821 ms. Trace:
I20260709 13:13:43.461153 31543 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.377923 31544 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.462023 31489 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.462752 31489 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.463354 31489 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.464176 31547 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.464593 31537 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.464879 31547 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.467296 31547 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.467296 31546 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.468482 31559 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.469022 31559 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.466145 31497 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.470149 31550 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.484900 31555 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.475790 31554 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.488956 31541 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.489852 31542 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.490365 31551 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.490973 31522 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.491405 31545 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.491195 31489 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.492389 31538 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.493323 31552 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.460988 31549 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.495476 31490 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.496771 31550 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.464253 31544 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.492092 31545 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.493404 31539 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.486055 31554 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.492426 31540 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.494267 31538 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.537581 31502 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.541872 31486 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.542546 31486 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.543196 31549 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.544028 31549 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.544929 31486 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.545488 31486 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.545899 31486 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.546795 31547 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.485950 31555 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.550170 31555 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.547391 31547 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.551232 31503 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.497527 31559 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.460973 30382 rpcz_store.cc:276] 0709 13:13:41.639273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:41.639440 (+   167us) service_pool.cc:224] Handling call
0709 13:13:42.586828 (+947388us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:42.586858 (+    30us) write_op.cc:454] Released schema lock
0709 13:13:43.460232 (+873374us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:41.715637 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:41.717293 (+  1656us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:41.717332 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:13:41.717350 (+    18us) tablet.cc:662] Decoding operations
0709 13:13:41.717913 (+   563us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:41.718004 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:41.718030 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:41.718265 (+   235us) tablet.cc:701] Row locks acquired
0709 13:13:41.718285 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:13:41.718454 (+   169us) write_op.cc:270] Start()
0709 13:13:41.720314 (+  1860us) write_op.cc:276] Timestamp: P: 1783602821718400 usec, L: 0
0709 13:13:41.720341 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:13:41.721152 (+   811us) log.cc:844] Serialized 537 byte log entry
0709 13:13:41.723881 (+  2729us) op_driver.cc:464] REPLICATION: finished
0709 13:13:41.775986 (+ 52105us) write_op.cc:301] APPLY: starting
0709 13:13:41.776125 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:41.776292 (+   167us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:41.776479 (+   187us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:41.776851 (+   372us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:41.777093 (+   242us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:41.777163 (+    70us) write_op.cc:312] APPLY: finished
0709 13:13:41.777505 (+   342us) log.cc:844] Serialized 22 byte log entry
0709 13:13:41.778738 (+  1233us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:41.778898 (+   160us) write_op.cc:454] Released schema lock
0709 13:13:41.778959 (+    61us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":51729,"apply.run_cpu_time_us":2566,"apply.run_wall_time_us":3526,"num_ops":1,"prepare.queue_time_us":60631,"prepare.run_cpu_time_us":3062,"prepare.run_wall_time_us":8150,"replication_time_us":3414,"thread_start_us":1362223,"threads_started":2}]]}
I20260709 13:13:43.466084 31527 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.497239 31562 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.497371 31550 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.563984 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.546836 31549 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.519382 31545 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.595789 31486 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.597697 31566 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.570433 31562 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.598533 31566 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.600895 31562 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.601379 31566 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.602200 31527 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.602857 31527 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.604074 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.604647 31554 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.605154 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.605296 31564 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.605876 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:43.606133 31559 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.605973 31564 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.606762 31559 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.607261 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.608875 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.609163 31559 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.609299 31503 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.609751 31559 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.609925 31545 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.610663 31545 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.561522 31547 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.610996 31548 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.620383 31565 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.620646 31548 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.622836 31543 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.623044 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.605244 31527 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.623622 31543 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.534003 31538 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.610010 31503 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.624320 31560 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.624946 31560 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.626031 31543 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.612608 31570 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.626330 31503 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.611526 31564 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.612072 31568 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.627827 31570 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.621136 31565 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.617731 30702 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:43.628290 31568 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.619654 31558 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.644690 31490 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.644501 31558 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.645351 31490 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.646936 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.639178 31548 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.626125 31538 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.618136 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.640674 31570 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.614787 31550 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.637607 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.647118 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.664304 31503 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.682533 31490 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.684016 31565 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.610339 31559 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.685523 31560 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.690903 31527 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.691601 31527 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.689090 31538 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.693929 31527 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.693974 31538 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.694553 31559 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.694950 31556 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.695215 31559 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.695564 31556 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.695869 31559 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.696292 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.696556 31538 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.697077 31364 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.697180 31538 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.698038 31556 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.697803 31538 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.699226 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.699950 31364 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.700448 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.695595 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.701171 31394 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.701313 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.702121 31548 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.702720 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.703358 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.704159 31503 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.704550 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.704897 31567 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.705308 30626 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:43.705554 31567 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.707314 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.708055 30706 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.611178 31545 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.715118 31565 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.715444 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.715644 31557 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.715849 31565 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.716316 31557 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.717075 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.717845 31389 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.718461 31568 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.718971 31490 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.719182 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.709650 31556 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.719568 31490 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.720024 31378 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.720305 31556 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.684247 31502 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.721040 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.721606 31553 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.721882 31389 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.722352 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.722227 31553 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.722946 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.723176 31394 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.723599 31361 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:43.724804 31553 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.731277 31574 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.732029 31574 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.732769 31548 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.733707 31490 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.718312 31565 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.734675 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.734864 31502 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.735055 31548 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.734900 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.735569 31502 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.713208 31559 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.736907 31558 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.737128 31559 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.737530 31558 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.738021 31502 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.738128 31558 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.735523 31389 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:43.739434 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.751921 31361 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.741295 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.754061 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.756825 31570 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.757491 31570 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.759510 31486 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.759876 31570 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.760164 31486 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.763150 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.764248 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.737867 31559 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.768146 31559 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.768719 31559 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.771814 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.835501 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.835314 31559 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.770457 30378 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:43.772182 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.793859 31558 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.837994 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.805084 31527 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.838899 30702 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.839184 31527 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.792428 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.839872 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.841120 31557 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.779206 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.841595 31527 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.775414 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.805768 31574 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.842289 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.708056 31567 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.843220 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.749502 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.843930 30703 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.844072 31364 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.809576 31572 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:43.753490 31502 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.845374 31502 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:43.773675 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.846606 31361 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.847410 31486 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.847739 31502 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.750089 31503 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.755837 31388 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:43.848775 31556 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.837952 31559 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.849860 31545 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.851262 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.853250 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.854113 31552 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.854564 31552 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.855028 31552 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.856098 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.857698 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.859977 31564 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.860533 31564 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.860368 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.861017 31564 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.845288 31572 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.863937 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.864717 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.865569 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.867993 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.876849 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.875638 31486 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:43.871624 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.877388 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.878424 31486 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.881371 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.886314 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.824520 31538 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.904307 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.891877 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.897444 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.898428 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.898602 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.912149 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.913214 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.914206 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.903139 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.898741 31563 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.915221 31563 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:43.899593 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.920101 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.921639 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.900557 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.876296 31550 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:43.901341 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.901837 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.902096 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.922494 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.882325 31544 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.924013 31544 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.924503 31544 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.924721 30559 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:43.922924 31486 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.886113 31539 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.903653 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.903923 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.904134 31572 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.905676 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.930754 31539 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.937593 31539 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.937722 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.906050 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.907284 31489 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.939369 31489 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.939815 31489 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.940860 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.908200 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.921738 31564 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:43.931768 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.943743 30632 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:43.934749 30625 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:43.934971 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.944131 31544 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.936707 31550 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.937875 31522 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.945051 31564 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:43.945322 31522 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:43.909188 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.947216 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:43.947755 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.943948 30624 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:43.950565 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:43.951352 31550 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:43.922508 30550 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:43.952128 31551 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:43.952646 31551 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:43.952471 31552 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:43.952795 31544 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.953111 31551 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.954720 31522 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:43.918329 31563 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:43.959215 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:43.962450 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.967487 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.968492 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.971530 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.973299 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.974318 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.975412 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.975724 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.976434 30632 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:43.977010 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.977200 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.977382 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.977386 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.979370 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.980867 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.980865 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.981870 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.980413 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.982479 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.979395 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.983274 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.979720 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.984010 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.980137 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.979429 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:43.985798 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.986255 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.986668 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.987811 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:43.991806 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:43.996165 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.000945 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.001488 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.005280 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.005951 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.006798 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:44.006994 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:43.957357 31552 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:43.959990 31544 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.015074 31544 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:44.015580 31544 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.017524 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.018664 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.019532 31394 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:44.021093 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.024955 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.028052 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.028754 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.029712 31560 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.030103 31560 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:44.030519 31560 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.031177 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:44.039520 31553 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.040071 31553 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.040445 31553 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.051599 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.052495 31389 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:44.053318 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.053818 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.054224 31364 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:44.055063 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.055511 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.055874 31394 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:44.058853 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.075606 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.076316 31489 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.081202 31551 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.083360 31539 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.087750 31539 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.050383 31553 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.099669 31553 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.101291 31522 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.109455 31522 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.103824 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:13:44.104955 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52414.5us
I20260709 13:13:44.113785 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:44.114506 30627 raft_consensus.cc:606] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.114914 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.103552 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:44.111394 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82729.6us
I20260709 13:13:44.118010 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.127254 31573 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.127487 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.128270 30698 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:13:44.126277 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71995.6us
I20260709 13:13:44.117861 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:44.138906 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 179819us
I20260709 13:13:44.143190 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.143539 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.145149 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.148335 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:44.130445 31553 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.159505 30556 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.160238 31573 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.160714 31573 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:44.161130 31573 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.161007 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:44.161831 30627 raft_consensus.cc:606] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:44.143265 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.163401 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.163610 31572 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:44.158484 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:44.165891 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.133625 31389 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
W20260709 13:13:44.170353 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51251.3us
I20260709 13:13:44.170941 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.064800 31560 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.157367 31564 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.155509 31489 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.176234 31577 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.202752 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.203902 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.204484 30554 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.204888 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.205678 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.205844 31541 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:44.206756 31577 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.203958 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.207041 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.207073 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.206081 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:44.199431 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59800.4us
I20260709 13:13:44.211297 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.196355 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.211329 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.216943 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.219008 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.219296 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.198586 31548 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.220074 31551 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.221047 31569 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.220950 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.221591 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.199530 31564 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.222312 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.222707 31564 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.207041 31541 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.223569 31579 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.223755 31541 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.224155 31579 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.224632 31579 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.224903 31541 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.226325 31541 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.223587 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.217382 31543 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:13:44.227398 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:44.237031 31423 debug-util.cc:398] Leaking SignalData structure 0x7b08002df380 after lost signal to thread 21529
W20260709 13:13:44.239152 31423 debug-util.cc:398] Leaking SignalData structure 0x7b08000fa7e0 after lost signal to thread 30416
I20260709 13:13:44.239383 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:44.237592 31543 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.240245 30627 raft_consensus.cc:606] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:44.241017 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.242094 31563 raft_consensus.cc:993] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: : Instructing follower 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:44.235909 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.242724 31563 raft_consensus.cc:1081] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:13:44.234447 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.243192 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:44.237526 31540 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.243988 31540 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.244314 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.246062 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:44.246474 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.246904 30627 raft_consensus.cc:606] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.247134 31489 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.220860 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.245365 31540 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.247644 31489 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.248754 31489 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.248816 31568 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.249145 31540 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.252069 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.252887 31543 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.253438 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.245359 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.255738 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.259646 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.260605 31539 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.263096 31539 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.263485 31539 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.263695 31522 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.267251 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:44.257365 31423 debug-util.cc:398] Leaking SignalData structure 0x7b080015a440 after lost signal to thread 30583
I20260709 13:13:44.269491 30624 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:44.270059 31582 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.270536 31582 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:13:44.270665 31423 debug-util.cc:398] Leaking SignalData structure 0x7b0800273440 after lost signal to thread 30657
I20260709 13:13:44.271004 31582 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:44.272243 31423 debug-util.cc:398] Leaking SignalData structure 0x7b080033d320 after lost signal to thread 30731
W20260709 13:13:44.273903 31423 debug-util.cc:398] Leaking SignalData structure 0x7b08001d99e0 after lost signal to thread 31426
I20260709 13:13:44.262300 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:44.275095 31552 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.275499 31552 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.275907 31552 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.277586 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.280301 30624 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:44.281041 30624 raft_consensus.cc:606] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:44.282569 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.282939 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:44.283895 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.285223 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:44.285979 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.286731 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.288069 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.289626 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.291093 31551 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.291596 31551 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:44.291970 31551 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.292618 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.291167 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.295804 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.298561 31560 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.282079 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.300683 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.303078 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:44.285101 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.307575 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.307029 31522 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.308889 31522 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:44.309319 31522 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.309785 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.294138 31573 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:44.274482 31423 debug-util.cc:398] Leaking SignalData structure 0x7b08003f51a0 after lost signal to thread 31514
I20260709 13:13:44.315773 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:44.328693 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.328922 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.329079 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:44.330862 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.335223 31542 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.335955 31542 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.336465 31542 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:44.338809 31423 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1960 after lost signal to thread 31520
I20260709 13:13:44.321436 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:44.343381 31522 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.344974 31551 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.324913 31544 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:44.326794 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71550.9us
I20260709 13:13:44.326555 31489 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.338470 31539 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.324060 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:44.346148 31522 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.347271 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.347365 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.347750 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.321332 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.348301 31579 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.373062 31572 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.370541 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:44.374985 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 143985us
I20260709 13:13:44.375073 31568 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:44.373355 31522 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.376678 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.375691 31568 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:44.377758 31568 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:44.377319 31522 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.378409 31522 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.378638 31489 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.376643 31389 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:44.384234 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.376842 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.375345 31539 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.351089 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:44.376694 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55466.9us
I20260709 13:13:44.399654 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.380752 30699 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.366557 31581 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.402906 30555 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.404688 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.401476 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.379987 31579 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:44.410646 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 103144us
I20260709 13:13:44.412744 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.414170 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.414546 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.412468 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.416326 31579 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.348338 31564 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.418053 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.401983 31572 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.363126 31552 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.420641 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.406544 31541 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.425485 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.426074 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.408977 31581 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.429626 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.413599 31556 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.438676 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.424386 31577 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.440716 31577 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.419041 31538 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.423589 31548 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.442478 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.426924 31573 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.436267 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.431540 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.434589 31581 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.467949 31551 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.468643 31544 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.469774 31564 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:44.438431 31560 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.470851 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.474845 31582 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.477320 31582 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.478873 30554 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.446058 31577 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.437029 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:44.490254 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69717.8us
I20260709 13:13:44.506367 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.480731 31542 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.481357 31569 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:13:44.516458 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.517868 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:44.518976 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.519335 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.509430 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.520459 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.518095 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.555668 30558 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.517884 31540 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:44.510569 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 96104.4us
I20260709 13:13:44.558769 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.561506 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.568300 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.575089 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.575845 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.576915 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.588699 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.591818 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.593418 31486 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.593938 31486 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.561444 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.594594 31577 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.564378 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.597416 31577 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.499706 31543 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:13:44.598992 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.600169 30553 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.601858 31543 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.602337 31543 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.603322 31543 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.604931 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.605872 31573 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.606324 31573 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.606807 31573 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.606599 31486 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.607477 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.607993 31486 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.608567 31553 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.608996 31553 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.609359 31553 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.603801 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.613157 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:44.615451 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.616885 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.617733 31541 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.585283 31576 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.586928 31542 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:44.587432 31552 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:44.588001 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.618642 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.621730 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.621994 30554 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:44.638859 31552 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.638188 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.583586 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.625137 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:13:44.644937 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61466.4us
I20260709 13:13:44.645998 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.645318 31556 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.646972 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.627128 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.627306 31522 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.580413 31568 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.649955 31552 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.629182 31581 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.651500 31522 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.652424 31581 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:44.654223 31568 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:44.655645 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.656000 30698 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:44.632287 31543 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.632777 31553 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.618292 31577 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.615547 30558 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:44.663892 31577 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.664274 30551 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:44.637128 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49215.6us
I20260709 13:13:44.667325 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.668187 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.669559 31553 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.670223 31522 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.641839 31576 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.643002 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:44.644512 31548 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.671876 31548 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.672813 31548 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.674113 31548 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.581102 31540 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.670333 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.664402 31577 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.678824 31544 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.674855 31540 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:44.680560 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.658914 31568 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:44.683027 31568 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:44.682883 31588 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.683642 31568 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.684624 31568 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.685377 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.685936 31568 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.689474 31573 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.691629 30558 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:49250
I20260709 13:13:44.692443 30558 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:44.692899 30558 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.693091 31577 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.636078 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.695454 31576 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:44.695854 31576 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:44.696220 31576 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.696461 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.699399 31577 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.705730 31576 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.706414 31540 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.706991 31581 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.708186 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.727807 31540 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:44.711341 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75354.1us
I20260709 13:13:44.713068 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.717937 30551 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:44.729672 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.721541 30558 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:44.730819 31588 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:44.718878 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.722195 30556 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.721583 30552 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.734102 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.723704 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.734967 30558 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:44.725168 31573 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.725553 31576 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:44.736265 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:44.742950 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.746232 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.746927 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.748661 30552 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.748481 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.749442 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.750592 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.752239 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.753012 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.728400 31581 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.728542 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.750901 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.751595 30552 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.754405 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:44.751078 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.746341 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:44.754271 30558 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.759572 30552 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.762020 31573 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.764858 30552 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.765875 30698 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:44.766772 31540 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.772930 31592 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:44.774492 31592 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:44.773713 31540 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.709184 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:44.776069 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90819.5us
I20260709 13:13:44.776912 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.771535 31563 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.778290 31563 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.778699 31563 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.767669 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:44.772861 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.779736 31563 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.774966 31592 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.781293 31563 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.782310 31592 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.782792 31556 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.783501 31588 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.784458 31592 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.787097 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.788316 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.795482 30701 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.796754 31588 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.797415 30698 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.803377 31540 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.804821 31540 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.805775 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.807240 31577 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.772853 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.816207 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:44.818101 30700 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.821283 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.822685 31577 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.826923 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.828697 31573 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.829927 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.830600 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.830922 31588 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.832023 31577 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.834158 30552 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:44.835775 31575 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:44.838821 31569 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:44.839095 31550 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.843029 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.844296 31543 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:44.844970 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.842535 31486 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.846140 31543 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.847136 31486 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.848688 31486 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.851435 31486 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.860289 31486 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.863485 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.848875 31568 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.864472 31588 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.864555 31568 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.858028 31548 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.867151 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.867933 31543 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.868590 31543 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:44.868726 31588 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.848009 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.869927 31543 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:44.872174 31577 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.872373 31543 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.879689 31563 raft_consensus.cc:993] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: : Instructing follower 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:13:44.880476 31563 raft_consensus.cc:1081] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:13:44.842392 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:44.882997 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 110230us
I20260709 13:13:44.885215 31570 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:44.883173 30700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:53928
I20260709 13:13:44.885751 31570 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.885891 31588 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.886061 30700 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:44.886835 30700 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:44.887327 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45069us
I20260709 13:13:44.888357 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.888643 31570 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.889428 31550 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.891031 31600 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:44.891461 31600 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.890216 31570 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.892462 31600 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.908486 31600 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.915253 31556 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.905887 31598 raft_consensus.cc:993] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:44.916465 31563 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:13:44.917194 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:44.918063 31598 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.918550 31598 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.919112 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.919636 31598 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.920696 31538 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.920783 31592 raft_consensus.cc:1081] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:44.866525 31568 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:44.923357 31598 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.925005 31568 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.904608 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:44.909344 31550 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:44.911573 31601 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.927493 31569 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.928057 31601 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.928678 31601 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.928983 31550 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:44.929534 31550 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.929934 31601 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.930959 31550 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.931958 31601 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.912325 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.912431 31575 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.913138 31588 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.934300 31575 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.935262 31575 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.935364 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.935873 31544 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.936585 31569 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.937722 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:44.937784 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.923609 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.940750 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:44.943023 31588 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:44.944319 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.945583 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.945943 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:44.946985 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.944033 31538 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:44.944706 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.950484 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:44.951851 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.952670 31564 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.952735 31552 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:44.952986 31522 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.953377 31552 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:44.953815 31552 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.952818 31592 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.954840 31592 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:44.957929 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.958599 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.960569 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.961532 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.932761 31550 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:44.957165 31569 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 13:13:44.985260 31539 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.987196 31551 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.947608 31581 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.985935 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:44.957755 31538 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.950136 31563 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.990029 31522 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:44.990446 31538 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:44.990939 31563 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:44.991315 31563 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:44.991358 31538 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:44.992281 31563 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:44.936205 31575 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.000530 31575 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.950646 31544 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.005575 31544 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:45.006062 31544 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.014343 31569 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.014979 31569 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.016018 31569 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.017766 31569 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.020712 31581 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.020738 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.024784 30700 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.031402 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:45.032162 30701 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.035876 30700 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.036207 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.036866 30552 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:44.962875 31599 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:45.038542 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.039283 30631 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.039587 30631 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.040369 31552 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:45.038764 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77430.7us
W20260709 13:13:45.042948 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91550.9us
I20260709 13:13:45.043165 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.028424 31581 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.053602 31563 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.059206 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.061439 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.073555 30552 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:45.080519 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.040436 30631 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.082185 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:45.084142 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 138232us
I20260709 13:13:45.085235 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.087220 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.088444 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.089493 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.090915 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.084139 31606 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.092340 31606 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.093786 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.095219 30556 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 3. Ops: []
I20260709 13:13:45.098403 30553 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:45.098755 30551 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:45.097440 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.085860 31556 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.085567 31575 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:13:45.105304 31575 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.105685 31575 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.081120 31538 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.099354 31598 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.115713 30631 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.100387 31550 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.114246 30628 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:45.119977 30558 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:45.090993 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.099610 31573 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:45.128376 31573 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:45.128199 31603 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:13:45.129468 31573 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.129468 31601 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.136737 31601 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.126273 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.141637 30632 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:45.145187 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.146723 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.147197 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:45.107688 31592 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.106523 31544 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.156338 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.137763 31601 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.148548 31550 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.149535 31569 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.166983 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.169727 31570 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.148592 31603 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
W20260709 13:13:45.172884 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90801.9us
I20260709 13:13:45.106557 31575 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.174593 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.149950 31588 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.159771 31592 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:45.150208 31568 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:13:45.177033 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.181592 31601 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.155364 31539 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:13:45.182898 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.183447 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.186383 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.186985 31538 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.182507 31544 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.191613 31568 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.192389 31568 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.193778 31539 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.194305 31539 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:45.191797 31556 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.183076 31552 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.185525 31569 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.195365 31539 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.197299 31538 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.197986 31539 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.198468 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.193562 31568 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.199875 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.200973 31568 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.201922 31608 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.202481 31556 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.202905 31604 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.205538 31588 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.206789 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.210023 31575 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.179699 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:45.217604 30700 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:45.220230 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:45.225430 30698 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:45.227553 31556 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:45.227698 31600 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.228783 31556 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:45.228962 31600 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.229235 31556 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.229494 31600 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.230278 31556 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.231835 31556 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.232908 31570 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:45.233423 31570 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.239037 31599 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.230432 31600 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.241164 31570 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.246999 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:45.247539 31570 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.248003 31378 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:13:45.247406 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:45.248559 31378 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.250198 30698 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:45.250797 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79698.3us
I20260709 13:13:45.257578 31486 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:45.242470 31600 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.243697 30701 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
W20260709 13:13:45.245450 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 91400.7us
I20260709 13:13:45.266218 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.266552 31486 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:45.268424 31486 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:45.269436 31486 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.257733 30378 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:45.271124 31486 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:45.275369 30625 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:45.280611 31564 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:45.281255 31564 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:45.286463 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:45.282567 31564 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.294423 30627 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:45.297961 31588 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:45.299180 31556 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:45.299566 31556 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.304492 30382 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.313082 31378 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.317044 31556 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.342367 31556 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.342761 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.345666 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.350575 30381 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.333976 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.390187 31551 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:13:45.392328 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58474.4us
I20260709 13:13:45.393290 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.394665 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.395529 31539 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.397116 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.398030 31551 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.398680 31378 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.396811 31564 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.401538 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.401933 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.403115 31564 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.411499 30625 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.413686 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:45.414849 31556 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:45.415436 31556 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.404726 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.416402 31556 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.418540 31564 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.419725 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.421306 31564 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.421514 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.423188 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.424023 31564 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.424562 31539 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:45.431808 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:45.434706 31539 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.435920 31539 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:45.436321 31539 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:45.437196 31539 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.442250 31564 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:45.449622 31564 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:45.450675 31564 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.449046 31539 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:45.447530 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.457952 31551 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.461467 30381 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.463012 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:45.464210 31610 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:45.481827 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:45.494258 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.495445 30553 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.497054 30554 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:45.498736 30558 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:45.503175 31556 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:45.504379 31556 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:45.504717 31556 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.505605 31556 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.502431 30381 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.509464 31556 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.528458 30558 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:45.530378 31576 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.554407 31556 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:45.555212 31556 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.558450 31556 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.558733 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:45.559610 30698 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.559955 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.561085 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.562708 31556 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:45.563277 31556 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:45.563678 31556 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.593040 31556 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.596659 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:45.599093 31556 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.599298 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.603492 30631 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:45.609583 31540 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.616564 30701 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.621330 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.621465 30625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:45.622166 30625 raft_consensus.cc:606] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.622574 30625 raft_consensus.cc:614] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:45.622740 31556 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:45.624356 31556 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.625761 31556 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.645037 30381 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.670316 30631 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:45.672290 31556 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:45.682168 30554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:45.682969 30554 raft_consensus.cc:606] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:45.689635 30625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:45.690470 30625 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:45.690994 30625 raft_consensus.cc:614] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:45.694346 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:13:45.695186 30698 raft_consensus.cc:606] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:45.702184 30554 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:45.695616 30698 raft_consensus.cc:614] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:45.708835 31608 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.710829 30625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:45.711782 30625 raft_consensus.cc:606] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:45.712251 30625 raft_consensus.cc:614] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:45.713600 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:13:45.720538 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:45.735355 31608 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.736099 31608 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.738371 31608 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.740839 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.741400 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:45.741946 30554 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.742283 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.743502 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:45.744714 31608 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:45.745455 31608 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.745853 31608 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.749249 31540 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.749984 31540 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.752875 31540 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.754513 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:45.755462 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.755925 31394 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:45.756210 30554 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:45.757637 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.758478 31573 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.758883 31573 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.759200 31573 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.760905 31608 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.763671 31608 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.766425 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.766425 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.770973 30554 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.773914 31573 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.775873 30631 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.776877 31573 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.777941 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.778342 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.778707 30554 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.779446 31608 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:45.780028 31608 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:45.781085 31608 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.779479 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:45.784010 31394 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.789392 30554 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:45.790411 31600 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.791030 31600 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.793308 31600 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:45.795135 31556 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.795816 31556 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.796955 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:45.797719 30698 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:45.799427 31394 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:45.798280 31556 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.800146 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.801043 30558 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.801627 31556 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.801071 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.802373 31556 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.804083 31573 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:45.799329 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:45.804606 31573 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:45.805153 31588 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.805586 31573 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.806378 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:45.807133 30631 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.807557 31576 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.808056 31576 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.808406 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.810832 31538 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:45.811100 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:45.811324 31538 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.811714 31538 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.811821 30631 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:45.814405 31601 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.814884 31601 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.815253 31601 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.816713 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:45.817001 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.805756 31588 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.817914 30553 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.819577 31588 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.819833 31605 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:45.820295 31605 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.820658 31605 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:45.821244 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.822114 30553 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.810566 31576 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.823848 30381 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.823894 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.851675 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:45.852716 30631 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:45.854252 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:45.855786 31632 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.856299 31632 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:45.856727 31632 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.863071 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:45.863981 31394 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:45.865233 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.866086 30554 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:45.865522 31538 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.810874 31556 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.867337 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.868948 31540 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.869513 31573 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:45.869892 31540 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.870155 31573 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.870442 31540 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.872478 31573 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.874224 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:45.875074 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:45.876292 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.877164 31573 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:45.877585 31573 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:45.877990 31573 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.882421 31601 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:45.883330 31605 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:45.881517 30381 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:45.885720 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:45.886085 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:45.884981 31601 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:45.886826 30698 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.890885 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.891294 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.894232 31573 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.867803 31632 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:45.897140 30625 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:45.898876 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.899075 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:45.899616 30554 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.900173 31538 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.900811 31573 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.901932 31632 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:45.890933 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.903810 30553 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.903635 31540 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.910782 31540 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:45.910951 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:45.911408 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.911737 31394 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.912145 30558 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:45.912534 30701 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:45.914700 31551 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.916704 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:45.922112 31394 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:45.924360 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:45.925405 30553 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:45.926726 31551 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.927759 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:45.928791 31540 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:45.929327 31540 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:45.930297 31540 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.955353 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:45.960912 30632 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:45.965070 30558 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:45.978266 31551 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:45.978783 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:45.980171 31573 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:45.980779 31573 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:45.981859 31573 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:45.999805 30631 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:46.008766 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:46.008270 30381 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:46.009835 30700 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:46.010118 31632 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:46.010766 31632 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:46.011689 31632 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.018195 31605 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:46.021879 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:46.021922 30628 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.029871 30698 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:46.031869 30700 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.032631 31610 raft_consensus.cc:993] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:46.034679 31489 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.038709 31610 raft_consensus.cc:1081] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:46.042181 31394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:13:46.038543 31605 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:13:46.042922 31394 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:46.043397 31394 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:46.047055 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:46.048436 31570 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:46.048477 31600 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:46.049232 31600 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.054903 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:46.056710 30558 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:46.054949 30632 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:46.058279 30554 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.060479 30625 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:46.062084 31600 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:46.065430 31632 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.067373 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.067832 31600 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.068656 31600 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.069739 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.070525 30553 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.072172 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.073426 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:46.078850 31538 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:46.079445 31538 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:46.079872 31556 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:46.080559 31556 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:46.080451 31538 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.081663 31556 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.087679 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.088675 31367 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:46.084142 30378 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:46.089932 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.066332 30554 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.097889 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:46.108409 30705 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:46.090353 31601 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:46.132308 30382 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:46.142381 31608 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.145041 31641 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:46.145632 31641 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.145998 31641 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.134007 31601 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:46.154244 31601 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:46.155368 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.149020 31600 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.130017 31570 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:13:46.197948 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46155.3us
W20260709 13:13:46.204964 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 104114us
I20260709 13:13:46.202353 30553 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.203424 31394 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:46.268064 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61366.6us
I20260709 13:13:46.240464 31632 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:46.279495 31605 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:46.283939 31605 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:46.305482 31394 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.307956 31641 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.312574 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:46.335369 31657 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.336383 31657 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.340029 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:46.367591 30554 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.324074 31389 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:46.341779 31643 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.371189 31643 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.373540 31657 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.284615 31632 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:46.384599 31632 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.349011 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:46.312731 31573 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.355492 30632 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.386878 31573 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.387948 31540 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:46.389143 31573 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:46.389250 30552 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.357592 31605 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.362313 30553 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:46.366111 31576 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:46.365856 31601 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.391891 31576 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.324098 31641 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.394023 30553 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:46.393632 31601 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:46.395061 30553 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:46.396363 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:46.273399 31570 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.397514 30553 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:46.400051 31649 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.376369 31600 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.377100 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.403152 31389 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:46.402935 31600 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.378472 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.381181 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.385221 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.404026 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.404660 30556 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:46.404851 30555 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:46.405556 30551 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.386349 31394 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.396562 31576 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:46.373612 31643 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.409420 31600 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.414927 31367 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:46.417449 31542 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.439057 31542 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:46.439445 31643 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.437930 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:46.440291 31605 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.439368 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.440471 30700 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.436062 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:46.442117 30705 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:46.428189 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.440641 31650 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.446182 31542 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.449019 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.458817 31540 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.381217 30554 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:46.464126 31540 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.466364 31600 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:46.466940 31600 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.467463 31600 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.468140 31540 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.468823 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.469610 31367 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:46.466810 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.470944 30556 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:46.453660 31576 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:46.472064 31576 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.465741 31389 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:46.474385 31576 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:46.478976 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:46.466277 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.450450 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:46.482842 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47815.5us
I20260709 13:13:46.485942 31540 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.486231 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.497799 30552 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
W20260709 13:13:46.504411 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44511.2us
I20260709 13:13:46.512701 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.513602 30631 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:46.512192 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:46.518301 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.520020 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:46.440306 31643 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.523622 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.530921 30631 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:46.499207 31551 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:46.500047 31570 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.528434 31664 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:46.495823 31605 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.557240 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.558250 30628 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:13:46.558058 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59492.9us
I20260709 13:13:46.500139 31394 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:46.546030 31551 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.546944 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.567381 30631 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:46.510994 31651 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:46.511132 30551 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.569721 31551 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.512909 31540 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:46.459208 31650 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.568053 31601 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:46.602090 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 135950us
I20260709 13:13:46.567760 31664 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:46.583433 31641 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:46.617007 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:46.560444 31570 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:46.627183 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115124us
I20260709 13:13:46.591997 31364 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:13:46.614243 31658 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:13:46.565186 31605 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.592166 31646 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.600131 31556 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.637312 30558 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.599829 31653 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.607226 31601 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.607120 31657 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.623862 31641 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:46.591672 31651 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.673628 30378 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:46.710044 31651 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:46.711052 31651 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:46.659042 31588 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:46.670717 31601 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.706053 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:46.747607 30382 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 3 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:46.671062 31641 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:46.677959 31657 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.757158 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:46.695480 31634 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.798099 31634 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.801034 31634 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.656306 31556 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:46.805501 31556 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:46.782842 30632 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:46.807495 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:46.797320 31657 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.669131 31653 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.654299 31646 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:46.774507 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 118985us
I20260709 13:13:46.815990 30632 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.809893 31601 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.751256 31669 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:46.780409 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:46.751369 31588 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:46.833236 31657 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:46.835629 31643 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.837347 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.824098 31556 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:13:46.900580 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.902001 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.900918 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:46.902395 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 398439us
I20260709 13:13:46.866110 31669 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:46.862560 30698 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:46.922529 31361 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.897480 31672 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:46.896571 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.898638 31552 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.953905 30558 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:46.922910 31391 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.922106 31396 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:46.956275 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:46.950856 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:46.957274 31361 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.919035 31608 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:46.961431 30632 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:46.961984 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.933171 31540 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.744745 31664 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:46.953979 31661 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 13:13:46.918319 30700 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:46.948460 31570 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:46.970978 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:46.980736 31669 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:46.957295 31650 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:46.934700 31600 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.012287 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:46.935397 30625 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:46.954843 31672 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.018121 31653 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.019523 31646 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:47.020296 31671 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:13:47.021929 31576 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:46.983204 31684 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:46.964057 30556 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:46.920238 31657 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.112314 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:47.112789 31657 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.041250 31588 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.010016 31551 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:47.052423 30671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.003s	sys 0.012s
W20260709 13:13:47.114024 30671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.004s	sys 0.013s
I20260709 13:13:47.115516 31551 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.118997 30558 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:47.121003 30552 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:13:47.121452 30553 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.723s	user 0.012s	sys 0.002s
I20260709 13:13:47.121958 30553 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:46.964020 31552 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:47.071596 31539 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:47.123816 31539 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.124008 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.124656 31552 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.125051 31388 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.125499 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:47.126070 31539 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:46.971801 31608 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:47.102310 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.103247 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.027551 31665 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:47.108552 31684 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.126238 30555 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.132730 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:47.133383 30552 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:47.134533 31396 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:47.076762 30551 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:47.083268 31610 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.142709 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:47.048255 31666 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:13:47.070912 31556 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:13:47.105937 31576 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.143678 31396 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.142810 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.133987 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.156751 31391 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:47.158241 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:47.158949 31391 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.161058 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:13:47.157889 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 207117us
I20260709 13:13:47.172313 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:47.126314 30554 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.173736 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:47.148054 31601 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.149497 31664 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.127249 31652 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:13:47.160584 31576 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.153596 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:47.171430 31651 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.203804 31391 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:13:47.204602 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 88707.6us
I20260709 13:13:47.215818 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.209249 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.218102 30703 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.219199 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:47.219988 30702 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.221871 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.175127 31540 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 13:13:47.226279 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:47.227497 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.140272 31665 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.238790 30703 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:47.206482 31651 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:47.149905 31610 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.284633 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:47.285339 31391 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.285892 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.286769 30704 raft_consensus.cc:2368] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:47.269037 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.272150 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.258718 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:47.251109 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:47.226785 31601 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.290066 30707 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:47.226207 31650 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.198652 31664 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:47.292168 31584 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:13:47.307678 31686 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.293172 31680 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.278396 30553 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:47.309078 31680 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.310864 31687 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.311471 31687 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.311861 31687 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.312232 31680 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:47.248615 31657 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.313267 31657 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.313876 31657 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.314471 31665 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.314903 31670 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.315402 31670 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.315760 31670 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.242164 31489 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.316715 31489 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.317325 31489 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:47.291728 31685 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:13:47.304606 31391 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:13:47.305155 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 131517us
I20260709 13:13:47.249624 31651 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.292475 31682 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.155s
I20260709 13:13:47.225943 31653 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.073s
I20260709 13:13:47.308305 31686 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:47.242735 31588 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.330245 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.235008 30550 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:47.339720 30702 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.296762 31601 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:47.350915 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 145587us
I20260709 13:13:47.302110 31650 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.354981 30555 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:47.356421 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:47.361912 31693 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:13:47.296579 31643 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:13:47.363873 31684 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.365790 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.366564 31364 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:47.367483 31668 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.367982 31668 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.368361 31668 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.368665 31687 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:47.369123 31600 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.370800 31692 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.337652 31540 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.361034 31588 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.371492 31692 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.371686 31540 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.373947 31540 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.374806 30704 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:47.352430 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.375447 31588 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:47.375593 31367 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.375799 31688 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:47.376070 31687 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.376426 31688 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:47.376793 30703 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:47.353993 31646 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:47.381772 31646 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.383074 31605 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.359280 31601 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.384838 31646 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.354867 31686 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.387553 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.388386 31378 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.402822 31396 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:47.404945 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:47.406877 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.407425 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.407621 30631 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:47.408071 30628 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:47.404160 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.417842 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.418061 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:47.419152 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 129605us
I20260709 13:13:47.419948 30628 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
W20260709 13:13:47.420083 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63246.9us
I20260709 13:13:47.370997 31608 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.424382 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.426442 31696 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:47.427083 31696 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:47.427755 31600 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.428220 31600 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:47.428797 31600 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.428164 31696 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:47.422132 31643 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.429768 31643 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.430141 31643 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.387413 31651 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.433248 31651 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.434854 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:47.435729 30558 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:47.421639 31672 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.437767 31610 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.377641 31688 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.405290 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:47.425920 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.421049 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:47.421770 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.360827 31650 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.452382 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.459872 31649 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:47.460191 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.460521 31649 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.460912 31649 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.439126 31692 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:47.461774 31681 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.462306 31681 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.462774 31672 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:47.463421 31672 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:47.464407 31672 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.465008 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:47.465762 30552 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.453780 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.467248 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.468643 31608 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.456844 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.469237 31608 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.467274 31651 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:47.469990 30551 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:13:47.470191 31651 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:47.471001 31681 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.471577 31608 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:47.472616 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51614.5us
I20260709 13:13:47.425369 31664 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.455376 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:47.478714 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.479642 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:47.482785 30552 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.476902 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:47.442582 31641 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.005s
I20260709 13:13:47.497102 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:47.500130 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79154.6us
I20260709 13:13:47.501700 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:47.510890 30703 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:47.512105 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.512244 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.512423 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:47.516634 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.518703 30707 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:47.512831 30703 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:47.521773 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.519778 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.528358 30707 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:47.495460 31651 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:47.537046 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:47.534396 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.539223 30523 leader_election.cc:400] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:47.539650 30523 leader_election.cc:403] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:47.539798 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.498018 31664 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.538137 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:47.544716 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.551227 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:47.534291 31653 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:47.529771 31687 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:47.533116 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.528930 31692 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.534890 31632 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.562232 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.564227 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:47.506758 31641 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.583069 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.585063 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.547812 30551 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:47.551180 31669 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.546217 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:47.563397 31653 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:47.596202 30700 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:47.601197 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.590593 30632 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:47.547677 31664 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.606763 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.592216 31605 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:47.610760 31695 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.611290 31689 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.611760 31705 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.612202 31676 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:47.613907 31700 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:13:47.602665 31653 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:47.611306 31610 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:47.623207 31683 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.623768 31698 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:47.627130 31540 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:47.568081 31687 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.691973 31676 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:13:47.692467 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 402120us
I20260709 13:13:47.735215 31676 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:47.735602 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:13:47.693424 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45901.8us
I20260709 13:13:47.577606 31692 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.738721 31692 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:47.741128 31692 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:47.742427 31697 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.743136 31715 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:47.703155 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 166174us
I20260709 13:13:47.591576 31542 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:47.743825 31712 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:47.743767 31715 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:47.744439 31712 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.745216 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:47.578053 31632 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:47.649475 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 143970us
I20260709 13:13:47.632630 31605 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.599377 31669 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.731259 31698 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:47.731467 31675 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.190s
I20260709 13:13:47.696605 31705 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:13:47.733249 31706 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:13:47.772364 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:47.746347 31715 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:47.746778 31692 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:47.825261 31396 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:13:47.825325 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51209.5us
I20260709 13:13:47.721287 31672 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:47.704638 31576 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.754711 31605 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:47.748345 31683 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:47.776185 31698 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.726133 31610 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.750824 31540 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.825942 31705 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.707407 31695 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:47.706702 31689 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:47.857642 31683 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:47.979600 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:47.979739 31396 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:47.743137 31697 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:47.907083 31641 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:47.864705 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:47.958446 31394 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:47.862565 31672 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.016213 30707 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:13:47.991535 31584 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.162s	user 0.001s	sys 0.006s
W20260709 13:13:48.017021 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186297us
I20260709 13:13:47.999498 31688 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.051795 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 359697us
I20260709 13:13:47.866115 31643 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:48.008857 31489 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.691s	user 0.018s	sys 0.006s
I20260709 13:13:47.874552 31719 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.182s
I20260709 13:13:48.068912 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:48.068790 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 81339us
W20260709 13:13:48.072145 31686 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.685s	user 0.007s	sys 0.007s
I20260709 13:13:48.072585 31686 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.073951 30705 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:13:48.074059 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.043771 31632 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.090449 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:13:48.071494 31600 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.004s
I20260709 13:13:48.059166 31682 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:48.062110 31564 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:48.092119 31601 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.001s
I20260709 13:13:48.096946 30631 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:48.113812 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.057444 31715 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:13:48.096657 31668 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.728s	user 0.014s	sys 0.000s
I20260709 13:13:48.079288 31641 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:48.096901 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.121106 31600 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.070045 31688 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.144273 31720 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:48.145458 31722 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:48.146989 31689 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.147328 31669 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.159621 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.147650 31648 raft_consensus.cc:476] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Not starting pre-election -- already a leader
I20260709 13:13:48.188033 30698 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:48.188688 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 587) took 1117 ms. Trace:
I20260709 13:13:48.189198 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:48.148236 31718 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:13:48.095610 31692 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:48.137450 31668 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.195004 31692 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.195909 30706 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:48.197104 31668 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.070883 31489 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:48.197613 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 588) took 1125 ms. Trace:
I20260709 13:13:48.198205 30701 rpcz_store.cc:276] 0709 13:13:47.072306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.078321 (+  6015us) service_pool.cc:224] Handling call
0709 13:13:48.196728 (+1118407us) spinlock_profiling.cc:243] Waited 74.3 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:48.196966 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.197187 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.197218 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:48.197262 (+    44us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:48.197300 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.197363 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.197511 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170950016}
I20260709 13:13:48.199437 31489 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.200644 31720 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:48.143851 31716 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:13:48.147117 31672 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.148893 31641 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.120944 31714 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:48.155860 31600 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.156956 31540 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.158771 31713 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:48.159303 31729 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:48.165722 31695 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:48.167070 31650 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
W20260709 13:13:48.149664 31657 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.835s	user 0.006s	sys 0.009s
I20260709 13:13:48.167987 31576 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.168637 31610 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.129480 31601 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.238286 30701 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:48.239874 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.240438 31601 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.129562 31564 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.241506 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.235142 31551 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:48.242718 31657 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.135239 31715 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.124158 31682 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.152006 31664 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:48.172884 31670 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.857s	user 0.012s	sys 0.003s
W20260709 13:13:48.172998 31651 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.003s
W20260709 13:13:48.172883 31681 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.701s	user 0.004s	sys 0.007s
W20260709 13:13:48.174764 30700 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.003s
I20260709 13:13:48.177627 31697 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.179236 31683 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.164207 31687 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.180280 31705 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.279726 31641 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.267108 31670 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.280818 31705 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.281582 30630 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:48.180595 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:48.150108 31649 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.000s
I20260709 13:13:48.191102 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.192993 31722 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.208065 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:48.218104 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 470543us
I20260709 13:13:48.218149 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:48.247232 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61686.9us
I20260709 13:13:48.189036 30698 rpcz_store.cc:276] 0709 13:13:47.071182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.076649 (+  5467us) service_pool.cc:224] Handling call
0709 13:13:48.076859 (+1000210us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:13:48.188579 (+111720us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169935744}
I20260709 13:13:48.220057 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.222039 31714 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:48.263025 30633 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:48.256744 31601 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:13:48.299399 30632 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:48.299888 31683 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.299803 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.300702 30698 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.301159 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:48.301705 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.302724 31702 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:48.303192 31702 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.303604 31702 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.303270 31608 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:48.293058 31649 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.305346 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:48.306560 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.308410 31722 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.269199 31651 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:48.310053 31676 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.003s
I20260709 13:13:48.310652 31651 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.243628 31551 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.311988 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.330116 30706 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.271056 30700 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:48.272680 31682 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.235813 31713 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.272589 31657 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.272614 31715 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.253163 31729 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.273824 31681 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.276322 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.231235 31650 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.257807 31564 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:48.288538 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70523.6us
I20260709 13:13:48.292640 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.294257 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:48.365733 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73220us
W20260709 13:13:48.308512 31653 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.581s	user 0.010s	sys 0.001s
I20260709 13:13:48.312963 31670 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.313658 31605 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:48.375958 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.376798 30550 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.377094 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 515) took 1256 ms. Trace:
I20260709 13:13:48.377534 30556 rpcz_store.cc:276] 0709 13:13:47.120883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.135125 (+ 14242us) service_pool.cc:224] Handling call
0709 13:13:47.854680 (+719555us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:47.989365 (+134685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.067101 (+ 77736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.070363 (+  3262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:48.075831 (+  5468us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:48.090220 (+ 14389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.376815 (+286595us) spinlock_profiling.cc:243] Waited 59.6 ms on lock 0x7b54002d0ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:48.376872 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.377005 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152820224}
I20260709 13:13:48.368626 31682 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:48.379393 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:48.307758 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.379786 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.319789 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.328136 31608 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:13:48.247984 31696 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.005s	sys 0.012s
I20260709 13:13:48.331005 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.388722 30629 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:48.339334 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:48.302515 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:48.310528 31676 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.365797 30631 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:48.367770 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.369807 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.372731 31643 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.396859 31676 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.397312 31643 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:48.373023 31698 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.399260 31650 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.399462 31686 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.399582 31698 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.400043 30702 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:48.400624 31396 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:48.401028 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.401904 30699 raft_consensus.cc:2368] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:48.402038 31729 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.376161 31713 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.306357 31649 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.297892 31714 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:48.380391 31383 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:48.380414 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.343096 31675 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:48.378960 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.421736 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.386004 31605 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.397003 30700 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:48.442683 31388 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.387988 31681 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:48.387377 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 43842.1us
I20260709 13:13:48.451177 31745 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:48.382212 31653 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.452910 30703 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:48.453127 31745 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.453670 31745 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.391117 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.392686 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.359509 31697 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:48.421072 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.392324 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.455191 30557 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:48.455092 31697 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.455776 31367 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:48.456370 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.456828 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.457319 31697 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:48.394359 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55156.2us
I20260709 13:13:48.457680 31367 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:48.395546 31551 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.458535 31634 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:48.405372 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.402460 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.459739 31394 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:48.460034 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.460209 31396 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:48.460697 31383 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:48.460747 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.461073 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:48.461714 31396 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.395922 31608 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:48.392700 31696 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.005s	sys 0.012s
I20260709 13:13:48.389894 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:13:48.463955 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74139us
I20260709 13:13:48.464324 31608 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.457032 30557 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.465628 30706 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:48.466563 31696 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:48.402894 31692 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.403580 31720 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.468801 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.469295 30554 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.396739 30628 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:48.443740 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:48.383225 31688 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.473604 30698 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:48.446391 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:48.475452 31653 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.416332 31712 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:48.476068 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:48.455304 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64280us
I20260709 13:13:48.476961 31720 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:48.478072 31720 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:48.455343 30558 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:48.478981 31720 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:48.409991 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.480032 31720 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.481195 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.482059 31681 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:48.482537 31681 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:48.483080 30378 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 152) took 1852 ms. Trace:
I20260709 13:13:48.483683 31653 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:48.483467 31681 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.483980 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.484956 31681 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.485951 31698 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:48.437093 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.483512 30378 rpcz_store.cc:276] 0709 13:13:46.649755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.650327 (+   572us) service_pool.cc:224] Handling call
0709 13:13:48.483004 (+1832677us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:46.775195 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:46.807448 (+ 32253us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:46.807489 (+    41us) write_op.cc:435] Acquired schema lock
0709 13:13:46.807504 (+    15us) tablet.cc:662] Decoding operations
0709 13:13:46.807900 (+   396us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:46.807966 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:46.807991 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:46.808193 (+   202us) tablet.cc:701] Row locks acquired
0709 13:13:46.808208 (+    15us) write_op.cc:260] PREPARE: finished
0709 13:13:46.808381 (+   173us) write_op.cc:270] Start()
0709 13:13:46.808547 (+   166us) write_op.cc:276] Timestamp: P: 1783602826808324 usec, L: 0
0709 13:13:46.808573 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:13:46.809330 (+   757us) log.cc:844] Serialized 537 byte log entry
0709 13:13:47.188772 (+379442us) op_driver.cc:464] REPLICATION: finished
0709 13:13:47.561913 (+373141us) write_op.cc:301] APPLY: starting
0709 13:13:47.570526 (+  8613us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:47.592578 (+ 22052us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:47.624355 (+ 31777us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:47.624689 (+   334us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:47.689951 (+ 65262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:47.714270 (+ 24319us) write_op.cc:312] APPLY: finished
0709 13:13:47.753951 (+ 39681us) log.cc:844] Serialized 22 byte log entry
0709 13:13:47.850490 (+ 96539us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:47.863239 (+ 12749us) write_op.cc:454] Released schema lock
0709 13:13:47.865910 (+  2671us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:48.014948 (+149038us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:48.043183 (+ 28235us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":345422,"apply.run_cpu_time_us":15146,"apply.run_wall_time_us":440278,"num_ops":1,"prepare.queue_time_us":6637,"prepare.run_cpu_time_us":2974,"prepare.run_wall_time_us":54372,"replication_time_us":350303,"thread_start_us":283513,"threads_started":1}]]}
I20260709 13:13:48.490326 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.487829 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 92) took 1079 ms. Trace:
I20260709 13:13:48.492122 30382 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 66) took 1830 ms. Trace:
I20260709 13:13:48.492973 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:48.492568 30382 rpcz_store.cc:276] 0709 13:13:46.674875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.683353 (+  8478us) service_pool.cc:224] Handling call
0709 13:13:48.491811 (+1808458us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b2400567cc8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f1803f88970 00007f1803f95a8f 00007f180cd9ae14 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc
0709 13:13:48.492056 (+   245us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:46.809839 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:46.810421 (+   582us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:46.810490 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:13:46.810510 (+    20us) tablet.cc:662] Decoding operations
0709 13:13:46.897202 (+ 86692us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:46.901799 (+  4597us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:46.920189 (+ 18390us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:13:46.941088 (+ 20899us) tablet.cc:701] Row locks acquired
0709 13:13:46.948346 (+  7258us) write_op.cc:260] PREPARE: finished
0709 13:13:46.964448 (+ 16102us) write_op.cc:270] Start()
0709 13:13:46.973229 (+  8781us) write_op.cc:276] Timestamp: P: 1783602826959296 usec, L: 0
0709 13:13:46.983224 (+  9995us) op_driver.cc:348] REPLICATION: starting
0709 13:13:47.132164 (+148940us) log.cc:844] Serialized 918 byte log entry
0709 13:13:48.202361 (+1070197us) op_driver.cc:464] REPLICATION: finished
0709 13:13:48.202798 (+   437us) write_op.cc:301] APPLY: starting
0709 13:13:48.202901 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:48.203037 (+   136us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:48.203159 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:48.203562 (+   403us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:48.203751 (+   189us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:13:48.203808 (+    57us) write_op.cc:312] APPLY: finished
0709 13:13:48.219967 (+ 16159us) log.cc:844] Serialized 28 byte log entry
0709 13:13:48.256673 (+ 36706us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:48.263833 (+  7160us) write_op.cc:454] Released schema lock
0709 13:13:48.267032 (+  3199us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:48.292695 (+ 25663us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:48.302911 (+ 10216us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":155167,"reported_tablets":2,"spinlock_wait_cycles":136218496,"child_traces":[["op",{"apply.queue_time_us":164,"apply.run_cpu_time_us":2671,"apply.run_wall_time_us":79960,"num_ops":2,"prepare.queue_time_us":31812,"prepare.run_cpu_time_us":5128,"prepare.run_wall_time_us":334801,"replication_time_us":1218922}]]}
I20260709 13:13:48.496940 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.499120 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.498216 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.502763 31361 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.521306 31692 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:48.521958 31692 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:48.523023 31692 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.525605 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.526775 30706 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:48.503897 31702 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.417515 30550 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:48.505479 31688 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:48.530016 31688 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:48.531266 31702 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:48.531045 31688 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.532107 30558 raft_consensus.cc:2368] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:48.533926 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:13:48.505915 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 125593us
I20260709 13:13:48.384809 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.475471 31378 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.536197 31657 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:48.536199 30556 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:48.509466 30380 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 130) took 1565 ms. Trace:
I20260709 13:13:48.538507 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.551350 30700 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:48.509590 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:48.515332 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.552054 30698 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.552556 31653 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:48.553032 31653 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:48.553408 31653 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:48.554065 30554 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:13:48.558844 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43612.6us
W20260709 13:13:48.516136 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 124397us
I20260709 13:13:48.515978 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:48.563750 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.538936 30380 rpcz_store.cc:276] 0709 13:13:46.943617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:46.943850 (+   233us) service_pool.cc:224] Handling call
0709 13:13:47.423596 (+479746us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000f32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:13:48.508987 (+1085391us) spinlock_profiling.cc:243] Waited 30.6 ms on lock 0x7b2400567cc8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f1803f88970 00007f1803f95a8f 00007f180cd9ae14 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc
0709 13:13:48.509246 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50434,"reported_tablets":2,"spinlock_wait_cycles":358706304}
I20260709 13:13:48.540077 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.564754 30555 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:48.565152 30630 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.545482 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.568066 31745 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:48.547322 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.570082 30626 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.572212 31713 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:48.575992 31388 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:48.547822 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:48.510299 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.516683 31696 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:48.579411 31665 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:48.580279 31696 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:48.580750 31696 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.581241 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:48.581888 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.582314 31665 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:48.584951 31634 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:48.585687 31634 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:48.590319 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.590495 31634 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.591756 31720 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:48.592213 31720 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:48.592617 31720 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.594559 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:48.602047 31650 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:48.602721 31650 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:48.603838 31650 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:48.509920 30557 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:13:48.605494 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62360.2us
I20260709 13:13:48.606235 31643 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:48.606820 31643 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.607506 30698 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:48.610002 31643 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:48.610483 31755 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:13:48.615723 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.618016 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:48.621234 31712 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:48.621783 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.622679 31712 raft_consensus.cc:2764] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:48.620296 31634 raft_consensus.cc:2764] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:13:48.623024 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.626161 30673 leader_election.cc:400] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:48.626726 30673 leader_election.cc:403] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:48.628094 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:48.629153 31712 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:48.630074 31634 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:48.630728 31634 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:48.631927 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.632494 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.633153 30550 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.637935 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.638963 30629 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:48.638844 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.639761 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.450235 31675 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:48.641454 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:48.639221 31712 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3.
I20260709 13:13:48.643988 31675 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:48.644340 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:48.645773 31667 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:48.651845 31667 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:48.653038 31667 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:48.654523 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72716.9us
I20260709 13:13:48.645478 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.645642 30553 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:48.658982 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.665969 30555 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:48.681318 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:48.682051 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:48.683611 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:48.672046 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:13:48.684644 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90185.8us
I20260709 13:13:48.685194 30552 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:48.689132 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.690476 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.691681 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.698429 31361 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:48.700443 31717 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:48.700320 30559 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:48.691171 31745 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:48.762511 30628 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:48.708349 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:48.743580 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.702728 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:48.761327 31396 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
W20260709 13:13:48.762511 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59618.2us
I20260709 13:13:48.995375 30552 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:48.774549 30554 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:13:48.922201 31759 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:13:48.746454 30630 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:48.772208 30626 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
W20260709 13:13:48.922891 31752 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:13:48.771757 31552 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:48.491390 30624 rpcz_store.cc:276] 0709 13:13:47.408128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:47.411820 (+  3692us) service_pool.cc:224] Handling call
0709 13:13:48.173691 (+761871us) spinlock_profiling.cc:243] Waited 975 us on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:48.173961 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.182968 (+  9007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.183005 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.183591 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:13:48.476481 (+292890us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:48.476616 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:48.483269 (+  6653us) spinlock_profiling.cc:243] Waited 8.47 ms on lock 0x7b54005f37d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:48.483332 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:48.483385 (+    53us) raft_consensus.cc:1685] finished
0709 13:13:48.483403 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:48.487743 (+  4340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1413,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":12016,"spinlock_wait_cycles":21723776,"thread_start_us":292446,"threads_started":1,"wal-append.queue_time_us":285624}
I20260709 13:13:48.702978 31712 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:48.632007 31634 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:48.896266 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.102046 31364 raft_consensus.cc:2368] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:49.103502 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:48.696878 31692 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.202s
I20260709 13:13:48.691159 31699 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:49.006875 30631 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:48.905872 31367 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:49.115841 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:49.031940 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:49.030238 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:48.775909 31720 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:49.066339 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:49.036804 31292 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 2.125s (SENT) [suppressed 271 similar messages]
I20260709 13:13:49.071372 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:49.099452 31653 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.002s
I20260709 13:13:49.032286 31689 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.069s
W20260709 13:13:49.107983 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 103083us
I20260709 13:13:49.170539 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:49.171777 31697 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:49.172487 31697 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:13:49.102363 31760 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:13:49.121609 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.175302 31391 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:49.175949 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:49.178182 31294 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.178325 31667 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:49.135295 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.180228 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.180791 31396 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:49.183169 31391 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:49.104375 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:49.173635 31697 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:49.115414 31364 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:49.146919 31665 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.184762 31665 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:49.187060 31665 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:49.147774 31712 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:49.188244 31712 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:49.190922 31653 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:49.192778 31364 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:49.193701 31653 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:49.193981 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:49.194990 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:49.195250 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.195972 31391 raft_consensus.cc:2368] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:49.196779 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.197407 31391 raft_consensus.cc:2368] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:49.197576 31720 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:49.198105 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.178442 31294 leader_election.cc:403] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:49.199414 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:13:49.200026 31696 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.001s
I20260709 13:13:49.200608 31649 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:49.125090 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:49.205596 31396 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:13:49.161957 31378 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.001s
I20260709 13:13:49.141899 31361 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:13:49.178597 30550 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.545s	user 0.015s	sys 0.006s
I20260709 13:13:49.209549 31391 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:49.269546 30556 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:49.270542 30556 raft_consensus.cc:606] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Received request to transfer leadership to TS 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:49.271611 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:49.273272 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:49.274118 30706 raft_consensus.cc:2368] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.299717 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:49.327983 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:49.200562 31696 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.109656 31744 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.269583 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:13:49.383746 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 114289us
I20260709 13:13:49.385044 31489 raft_consensus.cc:2764] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:13:49.093003 31552 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.200079 31713 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.137209 31681 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:49.206940 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:49.329696 31389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.009s	sys 0.064s
I20260709 13:13:49.217824 31649 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:49.404515 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:49.214157 30550 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:49.405768 30555 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:49.404215 31681 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.213724 31378 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:49.433713 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:49.387569 31682 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:49.400405 31702 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.216441 31667 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.194s
I20260709 13:13:49.473460 31489 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:49.406490 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:49.471562 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 386) took 1084 ms. Trace:
I20260709 13:13:49.428256 30626 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:49.478735 31489 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.475306 31682 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:49.397903 31744 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.480701 30698 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.480854 31378 rpcz_store.cc:276] 0709 13:13:48.386908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.446064 (+ 59156us) service_pool.cc:224] Handling call
0709 13:13:49.445275 (+999211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:49.515443 31489 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:49.517120 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:49.518013 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 481224us
I20260709 13:13:49.479014 31649 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.482200 31702 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:49.546844 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:49.548221 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 824) took 1271 ms. Trace:
I20260709 13:13:49.553565 31292 leader_election.cc:400] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.554260 31292 leader_election.cc:403] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:49.562546 31646 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.564023 31681 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:49.565611 31767 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:13:49.567920 31692 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:49.569604 31552 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:49.570379 31692 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:13:49.574694 31763 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:13:49.576566 31696 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:49.579432 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:49.580591 30700 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:49.562546 31720 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:49.563760 31646 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:49.583799 31744 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:49.585078 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 382) took 1199 ms. Trace:
I20260709 13:13:49.585433 31646 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:49.585628 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:49.585716 31383 rpcz_store.cc:276] 0709 13:13:48.385631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.461636 (+ 76005us) service_pool.cc:224] Handling call
0709 13:13:48.461991 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:48.645836 (+183845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:48.645874 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:48.646392 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.000793 (+354401us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:49.000841 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.106810 (+105969us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540057d854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:49.106873 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:49.584648 (+477775us) raft_consensus.cc:1685] finished
0709 13:13:49.584681 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.584967 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2913,"raft.queue_time_us":3224,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":414937984,"thread_start_us":353855,"threads_started":1,"wal-append.queue_time_us":748868}
I20260709 13:13:49.588992 31692 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.589635 31692 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:49.601212 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58521.7us
I20260709 13:13:49.606271 31292 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:49.608534 31720 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:49.607121 31692 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:49.610213 31675 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
W20260709 13:13:49.612217 31608 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.042s	sys 0.082s
I20260709 13:13:49.609882 30625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
W20260709 13:13:49.615589 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 608155us
W20260709 13:13:49.613601 31683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.008s	sys 0.018s
W20260709 13:13:49.630015 31608 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.043s	sys 0.082s
I20260709 13:13:49.652796 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:49.656502 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:49.656922 31673 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:49.657487 31673 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:49.658538 30699 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:49.659448 30700 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:49.655632 31608 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:49.663817 31383 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:49.626823 31682 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:49.626458 31720 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:49.631068 31683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.008s	sys 0.018s
I20260709 13:13:49.548908 30705 rpcz_store.cc:276] 0709 13:13:48.276935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.302752 (+ 25817us) service_pool.cc:224] Handling call
0709 13:13:48.303060 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.303283 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:48.677747 (+374464us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.543016 (+865269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.543096 (+    80us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:49.543160 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.544585 (+  1425us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.548129 (+  3544us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137483,"prepare.queue_time_us":68743,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":126276,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":15240,"thread_start_us":1116958,"threads_started":3,"wal-append.queue_time_us":801293}
I20260709 13:13:49.663736 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:49.662590 31489 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:49.667500 31391 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:49.667971 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:49.683928 31489 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:49.687291 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:49.668744 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 458971us
I20260709 13:13:49.689512 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:13:49.628368 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.007s	sys 0.021s
W20260709 13:13:49.692981 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.007s	sys 0.021s
W20260709 13:13:49.693432 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 734456us
I20260709 13:13:49.694370 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:49.629102 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.016s
W20260709 13:13:49.695562 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.014s	sys 0.016s
I20260709 13:13:49.634218 30625 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:49.705888 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:49.719482 31489 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:49.719852 31673 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:49.714859 31681 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:49.721971 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:49.722805 31675 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:49.663882 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:49.731886 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44733.2us
I20260709 13:13:49.754076 30596 leader_election.cc:400] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:49.754571 30596 leader_election.cc:403] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:49.758378 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:49.759238 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:49.759955 31396 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:49.761652 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:49.771260 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:49.700635 31605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.012s	sys 0.016s
I20260709 13:13:49.724557 31712 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:49.715153 31682 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:49.785449 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:49.807627 30596 leader_election.cc:400] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:49.808163 30596 leader_election.cc:403] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:49.684451 31608 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.077s
I20260709 13:13:49.734517 31689 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:49.754567 31646 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:49.842161 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:49.843046 30631 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:49.768440 31683 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:49.798804 31605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.014s	sys 0.018s
I20260709 13:13:49.907351 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.090761 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:49.882761 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:50.082244 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 111162us
I20260709 13:13:49.867093 30631 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
W20260709 13:13:49.952050 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.815s	user 0.006s	sys 0.055s
W20260709 13:13:50.097599 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.013s	sys 0.056s
I20260709 13:13:50.100893 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:50.079669 30554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.002s	sys 0.014s
I20260709 13:13:50.102208 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.103932 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:50.105402 30625 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:50.102442 30554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.002s	sys 0.015s
I20260709 13:13:50.077874 30550 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:13:50.089462 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 281852us
W20260709 13:13:50.114030 31291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.013s	sys 0.098s
W20260709 13:13:50.114823 31291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.014s	sys 0.098s
I20260709 13:13:50.116024 30596 leader_election.cc:400] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:50.116670 30596 leader_election.cc:403] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:13:49.759109 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:49.963790 31293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.042s	sys 0.055s
W20260709 13:13:50.127506 31293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.042s	sys 0.056s
I20260709 13:13:49.866906 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:49.950826 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 389) took 1563 ms. Trace:
I20260709 13:13:49.814262 31601 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:49.861281 31646 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:13:50.180625 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 421478us
I20260709 13:13:50.178292 31389 rpcz_store.cc:276] 0709 13:13:48.387805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.465910 (+ 78105us) service_pool.cc:224] Handling call
0709 13:13:48.466209 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:48.466450 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:48.712753 (+246303us) log.cc:844] Serialized 14 byte log entry
0709 13:13:49.902199 (+1189446us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:49.907867 (+  5668us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:49.908500 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.934246 (+ 25746us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:49.950705 (+ 16459us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25824,"prepare.queue_time_us":612169,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":75708,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":1245368,"threads_started":3,"wal-append.queue_time_us":679958}
I20260709 13:13:49.843902 31675 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.816917 31712 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:49.932801 31702 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:49.969590 31720 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:49.782917 31726 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:49.810272 31744 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
W20260709 13:13:49.932358 31767 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:13:50.011270 31774 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:13:49.850040 31775 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:13:50.166292 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47359.3us
W20260709 13:13:50.012295 31777 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:13:50.025183 31782 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.066s	user 0.001s	sys 0.001s
W20260709 13:13:50.020329 31784 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 13:13:50.018090 31785 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 13:13:50.014671 31634 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.017s	sys 0.011s
W20260709 13:13:50.166605 31667 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.009s	sys 0.042s
W20260709 13:13:50.167392 31364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.031s	sys 0.049s
I20260709 13:13:50.168104 31781 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:50.170710 31695 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.184656 31489 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.192314 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.211328 30627 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:50.211010 31489 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.191826 31726 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.213992 31489 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.183668 31675 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.194911 31775 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.217684 31675 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.217815 31775 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:50.192997 31794 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
W20260709 13:13:50.219859 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60502.6us
I20260709 13:13:50.192988 31652 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.222182 31652 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.185086 31712 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.251582 31695 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.254266 31790 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.192646 31649 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.255316 31790 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.256484 31649 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.255546 30380 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 3, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:50.182021 31646 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.269716 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.271278 31813 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:50.198344 31679 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.276305 31679 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.277107 31789 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:50.277660 31789 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:13:50.254552 31667 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.011s	sys 0.042s
I20260709 13:13:50.182353 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:50.285836 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.288100 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 382) took 1679 ms. Trace:
I20260709 13:13:50.288744 31388 rpcz_store.cc:276] 0709 13:13:48.608572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:48.691224 (+ 82652us) service_pool.cc:224] Handling call
0709 13:13:48.775192 (+ 83968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.059474 (+284282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.066142 (+  6668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.144406 (+ 78264us) log.cc:844] Serialized 26 byte log entry
0709 13:13:49.616345 (+471939us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:49.616911 (+   566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:49.617940 (+  1029us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.287576 (+669636us) raft_consensus.cc:1685] finished
0709 13:13:50.287625 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.287853 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":469935,"threads_started":1,"wal-append.queue_time_us":756225}
I20260709 13:13:50.291500 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:50.182935 31793 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:13:50.249450 31364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.032s	sys 0.050s
I20260709 13:13:50.258366 31790 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.187813 31814 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:13:50.305778 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 123645us
I20260709 13:13:50.291749 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:50.320199 31652 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.322928 31649 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:50.323796 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 208256us
I20260709 13:13:50.324321 31814 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.325239 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.326498 31788 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.327940 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 550) took 1469 ms. Trace:
W20260709 13:13:50.329267 31634 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.018s	sys 0.012s
I20260709 13:13:50.331274 31679 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.255205 31781 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.335407 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.336661 31383 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:50.322963 31793 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.337695 31781 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.337977 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.339478 31793 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.340597 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.339313 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.344035 31378 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:50.348528 31788 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.328684 31364 rpcz_store.cc:276] 0709 13:13:48.859041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.199204 (+340163us) service_pool.cc:224] Handling call
0709 13:13:49.199584 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:49.200024 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:49.200058 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:49.200656 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:13:50.327126 (+1126470us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:50.327257 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.327418 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:50.327506 (+    88us) raft_consensus.cc:1685] finished
0709 13:13:50.327527 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.327840 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":403,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":122,"thread_start_us":1099434,"threads_started":1,"wal-append.queue_time_us":825754}
I20260709 13:13:50.331142 31726 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:50.362407 31564 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:50.363510 31564 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.380771 31789 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.381800 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.386731 31744 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.385870 31794 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.371248 31790 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:50.393429 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 96) took 1003 ms. Trace:
I20260709 13:13:50.393522 31790 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:50.396642 31790 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.397955 31649 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.386989 31781 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.408690 31794 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.409592 31794 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.414157 31794 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.394129 30554 rpcz_store.cc:276] 0709 13:13:49.396151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.399411 (+  3260us) service_pool.cc:224] Handling call
0709 13:13:49.476112 (+ 76701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:49.499069 (+ 22957us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:50.111287 (+612218us) log.cc:844] Serialized 14 byte log entry
0709 13:13:50.219999 (+108712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:50.220132 (+   133us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:50.220275 (+   143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:50.392338 (+172063us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b18003ddaa0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419
0709 13:13:50.393049 (+   711us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b540013a954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:50.393103 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:50.393311 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41334,"prepare.queue_time_us":529111,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":48449,"spinlock_wait_cycles":122469120,"thread_start_us":636451,"threads_started":2,"wal-append.queue_time_us":90783}
I20260709 13:13:50.416877 31781 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:50.417671 31781 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.420492 31781 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:50.424436 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46230us
I20260709 13:13:50.425267 31794 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:50.426187 31738 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:50.427325 31738 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.430817 31788 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:50.431491 31738 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:50.432430 30631 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.432560 31794 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.434446 31814 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.430049 30625 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:50.453914 31683 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:50.439867 31789 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.440819 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.439939 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.439035 31794 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:50.443010 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:13:50.443351 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.444883 31564 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:50.445438 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.447220 31702 raft_consensus.cc:993] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:13:50.447368 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:50.450922 31738 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.379660 31388 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.447602 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:50.472525 30627 raft_consensus.cc:606] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:50.477326 30552 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:50.479360 31697 raft_consensus.cc:1081] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Signalling peer 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:13:50.484333 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.500675 31772 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.513959 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.515492 31388 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.510921 31726 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.519438 31726 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.519870 31695 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:50.528414 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81008.7us
I20260709 13:13:50.526082 31772 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.537518 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.538846 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.537541 31786 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:50.539861 31786 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.540304 31786 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:50.540349 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.540637 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.542447 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.547554 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:50.547924 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.549587 30703 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.550933 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:50.551684 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 67550.1us
I20260709 13:13:50.551893 30627 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:50.546164 31726 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:50.551157 30698 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.498962 31788 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.552839 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.560777 31646 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:50.561491 31646 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.561611 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.562047 31646 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:50.562135 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.564245 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.564862 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.566150 31815 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:50.566773 31815 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:50.566241 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.568044 31815 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.567034 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:50.571252 30705 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.735s	user 0.018s	sys 0.013s
I20260709 13:13:50.571664 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.571880 30705 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:50.573072 31772 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.532415 31738 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.580287 30556 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:50.591421 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:50.598084 31702 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.598603 31779 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:50.599102 31779 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:50.599521 31779 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:50.603076 31788 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.604888 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.606973 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.608451 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.608791 30555 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.611110 30553 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.611228 31780 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.603080 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.621564 31794 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:50.622488 31794 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:50.616967 31826 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
W20260709 13:13:50.630916 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78315.8us
I20260709 13:13:50.633044 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.635680 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.636842 31726 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:50.637403 31726 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.637811 31726 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:50.628226 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.643129 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:50.644237 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82241.2us
I20260709 13:13:50.645370 31702 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.649771 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.651167 31551 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.660480 30552 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:50.687477 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.689020 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.690438 31683 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.691067 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.692756 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.695343 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.696290 30633 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:50.689653 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.662668 31697 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.701419 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.694247 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.679970 31695 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:50.714305 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 48845.7us
I20260709 13:13:50.721933 31489 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:50.715907 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.720191 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:50.677062 31788 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:13:50.729687 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.737324 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.737895 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.738790 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.738921 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.739166 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.714600 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:50.754079 31673 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.755118 31673 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.756794 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.757916 31673 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.759001 31791 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.759068 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.762594 31367 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:50.747967 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:50.763710 30630 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:50.764252 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.767269 31772 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:50.767863 31772 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:50.768290 31772 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:50.769316 31738 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.740276 31683 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:13:50.779055 31780 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:50.770385 31820 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:13:50.781695 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 104904us
I20260709 13:13:50.723580 31794 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:50.785570 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:50.788141 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:50.796787 31367 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.798702 30553 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:50.798491 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:50.799408 30624 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:50.799889 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:50.800339 31771 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.801380 30705 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.801657 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.804961 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.805315 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.789052 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.805980 31396 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:50.736155 31695 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.789695 31781 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:13:50.813468 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 92385.7us
I20260709 13:13:50.812151 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:50.820642 31834 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:50.819000 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:50.815759 31781 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:50.837647 30552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:50.815320 30700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:53938
I20260709 13:13:50.814145 31788 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:13:50.831763 31697 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.847304 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:50.850208 31830 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.002s
I20260709 13:13:50.850421 31810 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:50.851136 31702 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.852882 31779 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:50.838651 31834 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.838382 30552 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:50.853878 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:50.849386 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:50.844895 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:50.849493 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.862932 31695 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:50.864162 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107590us
I20260709 13:13:50.865289 31652 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.869774 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.876976 31775 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:50.868552 31794 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.877285 31652 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:50.876717 31696 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:50.860800 30630 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:50.885391 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.881389 31652 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:50.886504 31652 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:50.887326 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.877671 31775 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:50.890938 30700 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:50.891436 31775 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.893150 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.868625 31810 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.884079 30556 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.860195 31697 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.873368 31738 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:50.850514 31781 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:50.906810 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.916524 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:50.891510 30700 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:50.919842 31781 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:50.918090 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:50.921445 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.933313 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:50.935149 31653 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.935854 31744 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:50.937942 31653 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.938983 31779 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:50.949673 31790 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.950954 31789 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:50.951428 31789 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:50.952517 31552 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:50.953337 31552 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:50.952823 31789 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:50.930246 31712 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:50.955866 31696 raft_consensus.cc:993] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:50.956610 31696 raft_consensus.cc:1081] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:50.956962 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:50.958848 31744 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:50.961334 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:50.920680 31810 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:50.963253 31788 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.963268 31794 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.954511 31552 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.969856 31810 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:50.916225 31726 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:50.977991 31726 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:50.983647 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 101740us
W20260709 13:13:50.987661 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 107184us
I20260709 13:13:50.993541 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:50.984580 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:51.001582 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:51.005851 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:13:50.996114 31389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:13:51.011225 31389 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:51.011751 31389 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:51.016906 31697 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:51.018682 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:13:51.023581 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 88034.7us
I20260709 13:13:50.998843 31394 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:50.999850 31789 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:51.029955 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.035405 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:51.039661 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55176.7us
I20260709 13:13:51.041744 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.050765 30550 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:51.052793 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.078519 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.078540 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.079767 30632 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:13:51.079770 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.079036 30628 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:51.081974 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.082906 30627 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:51.042546 31794 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:51.087318 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.095259 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.099382 31551 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.100507 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.102516 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.103044 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.104148 31489 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.099188 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.059779 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.109534 31652 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.099220 31794 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:51.104820 31489 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:51.111379 31794 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.112098 31652 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:51.113065 30698 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:51.114032 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.114023 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:51.114966 31391 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:51.122376 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.140048 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:51.146708 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.115461 31771 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.058458 31646 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:51.114897 31810 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:51.115114 31744 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.142071 31652 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:51.172976 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 85781.8us
I20260709 13:13:51.117789 31675 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 13:13:51.184064 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 88892.7us
I20260709 13:13:51.121322 31712 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:13:51.161922 31810 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:51.191322 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:51.202085 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 182179us
I20260709 13:13:51.207067 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:51.178653 31786 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.638s	user 0.010s	sys 0.010s
I20260709 13:13:51.191315 31652 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:51.196830 31389 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:51.166875 31771 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:51.202876 30626 pending_rounds.cc:85] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 1
I20260709 13:13:51.203394 30704 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:51.228439 31771 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:51.228544 31489 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:51.208801 31772 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.231793 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:51.233345 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.194092 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.208535 31810 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:51.203467 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.237272 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.209419 31779 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.248553 31702 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.250903 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.233144 31552 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.233596 30626 pending_rounds.cc:107] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:13:51.216153 31786 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:51.250239 31675 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:51.231798 31695 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 13:13:51.295109 30629 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:51.268957 31744 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.280644 31781 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.288362 30626 raft_consensus.cc:2889] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.279502 30700 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.306064 31772 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:51.334791 31850 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:13:51.336361 30557 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:51.337083 31646 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:51.316905 30625 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:51.300405 31675 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.347015 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.345423 31389 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:51.353325 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 130719us
I20260709 13:13:51.357040 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:51.359449 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 132680us
W20260709 13:13:51.363456 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90763.6us
I20260709 13:13:51.364701 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:51.368495 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63724.2us
I20260709 13:13:51.368760 30703 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:51.373981 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.376730 31836 raft_consensus.cc:2764] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:13:51.377903 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:13:51.380055 31844 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:13:51.381387 31675 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:51.388417 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:51.345964 31847 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.392704 30703 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:51.395274 31857 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:51.403692 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.406332 30701 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:51.408273 31843 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 13:13:51.408800 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:51.418315 30628 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:51.421783 31652 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:51.400718 31848 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:13:51.437517 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:13:51.445369 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 141198us
I20260709 13:13:51.443795 31391 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:13:51.407702 31794 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.442732 31652 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.400292 31781 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 13:13:51.441783 31552 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:51.460958 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.456291 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:51.463470 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59279.1us
I20260709 13:13:51.451391 31673 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.468235 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.478989 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:51.479826 30555 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:51.459815 31552 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.480211 30555 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:51.443607 31847 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
W20260709 13:13:51.483129 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56348.4us
I20260709 13:13:51.470499 31675 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.492388 30629 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:51.481719 31862 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:51.485988 31552 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:51.494774 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.507054 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.396293 31857 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:51.521106 31838 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.368s	user 0.005s	sys 0.000s
W20260709 13:13:51.522017 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 72095.3us
I20260709 13:13:51.527767 31857 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:51.529245 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.531418 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.510056 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.523393 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.524734 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:51.535976 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.538368 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.513599 30555 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.540213 30555 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:51.538822 30632 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:51.543579 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.490828 31794 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:13:51.555888 31794 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:51.537487 31652 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.556350 31794 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:51.530418 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:51.541754 31702 consensus_queue.cc:1175] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acf852f85f47483d99cbe4020baee627" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:13:51.558863 31842 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.544523 31789 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.554009 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.538471 31675 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:51.544519 31697 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:51.567602 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.565141 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.557782 31794 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.580379 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.565816 31815 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.531904 31862 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.567085 31697 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.578672 31814 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.589808 31714 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.589994 31864 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:51.590464 31794 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:51.590580 31714 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.573312 31779 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:13:51.594663 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.593763 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.587419 31697 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:51.586442 31726 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.586442 31772 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.599018 31389 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:51.599454 31772 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:51.598271 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44374.1us
I20260709 13:13:51.604100 31772 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:51.611625 31860 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.153s	user 0.003s	sys 0.000s
I20260709 13:13:51.611856 31847 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:51.610669 31552 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:51.612986 31810 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.613965 30555 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:51.615118 31552 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:51.615393 31810 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:51.616962 31788 raft_consensus.cc:993] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:51.617579 31788 raft_consensus.cc:1081] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:51.618425 31857 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:51.618935 31857 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:51.619992 31857 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.621420 31857 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:51.621856 31837 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.254s	user 0.000s	sys 0.002s
I20260709 13:13:51.612579 31847 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:51.624843 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:51.626194 31702 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:51.601094 31868 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:13:51.626571 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.626667 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.639006 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.640957 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:51.626655 31702 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:51.646598 31702 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:51.651206 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:51.667981 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.629380 31847 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:51.765558 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:51.655519 31551 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:51.711359 30698 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:13:51.712186 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:51.714008 30704 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:51.714051 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.718083 31367 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:51.664253 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:51.721215 30632 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:51.777277 30555 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:51.663028 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.727617 31702 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:51.670143 31786 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:51.733232 31697 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:51.628942 31862 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:51.743708 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 87476.3us
W20260709 13:13:51.745401 31845 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.479s	user 0.000s	sys 0.002s
I20260709 13:13:51.761056 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:51.657689 31789 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.778578 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:51.796896 30557 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:51.782212 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:51.786438 31772 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:51.815676 31396 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:51.816207 31396 pending_rounds.cc:85] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:13:51.816437 31396 pending_rounds.cc:107] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:51.790721 31779 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.815055 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:51.816680 31396 raft_consensus.cc:2889] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:51.799126 31862 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.799433 31815 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:13:51.823267 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.799890 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:51.810042 31775 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:51.845814 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:51.854521 30632 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:51.826115 31789 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:51.857342 30628 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:13:51.847096 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.831893 30553 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:51.815629 31675 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:13:51.815795 31868 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:13:51.840054 31772 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:51.862542 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:51.862310 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:51.860867 31775 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:51.863620 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.863838 31789 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:51.863097 31862 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.846913 31779 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:51.867400 31781 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:13:51.874571 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:51.887830 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:51.926334 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49919.3us
I20260709 13:13:51.899967 30698 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:51.931166 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:51.932364 31842 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:51.908262 31789 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:51.906078 31857 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:51.911993 30626 raft_consensus.cc:2376] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:13:51.925853 31868 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:51.952921 30704 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:51.936224 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:51.922910 31862 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:51.953685 31702 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:51.941912 31794 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:13:51.890949 31839 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:13:51.948153 31857 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.010071 31697 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:52.055536 31872 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
W20260709 13:13:52.057710 31875 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:13:51.991897 31868 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.061296 31868 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.062103 31850 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.062979 31850 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.067106 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:51.958629 31772 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.075245 31868 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:52.095990 31881 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:13:52.096277 31882 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:52.096942 31878 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:13:52.096979 31876 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.229s	user 0.002s	sys 0.000s
W20260709 13:13:52.120046 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 333693us
I20260709 13:13:52.103892 31714 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:52.044299 31775 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:52.105419 31850 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:52.107336 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:52.108767 31696 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:13:52.085665 31367 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:52.097358 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:52.129940 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:52.137377 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:52.137924 31862 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.132145 31714 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:13:52.141211 31779 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:52.119318 31646 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:13:52.136058 31834 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:52.142581 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.119676 31882 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.145913 31882 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:52.147718 31887 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:13:52.149382 31839 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:52.114454 31868 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.153740 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:52.152997 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:52.185353 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.190462 31882 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:52.150337 31839 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:52.202422 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49594.2us
I20260709 13:13:52.183696 31696 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:52.215438 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:52.216276 30627 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:52.217412 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:52.193034 31834 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.183259 31702 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 13:13:52.203073 30700 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:52.206837 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.225201 31890 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:13:52.219758 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:52.087780 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:52.220913 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.224961 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.246251 30555 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:52.194801 31789 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.201889 31788 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:13:52.238854 31862 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:52.339392 30554 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:52.339718 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:53938 (request call id 636) took 1450 ms. Trace:
I20260709 13:13:52.341576 30556 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:52.227803 31857 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.344388 31394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:13:52.368255 31394 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:52.368714 31394 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:52.255456 31779 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:52.335244 31868 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.375226 31886 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:52.376055 31886 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:52.377379 31886 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.379976 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:52.381485 30550 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:52.381655 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:52.383534 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:52.386262 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:52.386775 31867 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:52.387355 31867 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:52.384706 31886 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:52.388357 31867 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.389456 31867 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.392688 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:52.393611 31869 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:52.393895 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:52.398854 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:52.410575 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.421433 31391 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:52.371758 31847 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:52.428892 31779 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:52.433808 31675 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:52.226675 31834 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.440721 31394 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.413884 30625 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:52.459414 31394 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:52.423491 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:52.470299 31883 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:52.472254 31893 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:13:52.414919 31882 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:13:52.369174 30700 rpcz_store.cc:276] 0709 13:13:50.793677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.806337 (+ 12660us) service_pool.cc:224] Handling call
0709 13:13:50.890711 (+ 84374us) spinlock_profiling.cc:243] Waited 6.68 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb9a40 00007f180bea51e9 00007f180bea4c85 00007f1805429254
0709 13:13:52.193245 (+1302534us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4c0063f9c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18051a9a20 00007f18051baa29 00007f18051bb13f 00007f180517f2ab 00007f18051acf3c 00007f18051ad3fe 00007f180543a88b 00007f1806e39495 00007f1806e8a30f 00007f1806ed4a55 00007f180bea4cd0
0709 13:13:52.243979 (+ 50734us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19675776}
I20260709 13:13:52.478435 31789 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:52.470301 31378 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:52.415072 31712 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:13:52.424129 31790 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:52.414716 31862 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:52.503903 31779 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.527920 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.511812 30626 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:52.522552 31744 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:52.502187 31790 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:52.570122 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.543860 31839 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:52.593319 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 160352us
I20260709 13:13:52.560683 30627 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:52.546162 31744 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:52.553751 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.532534 31779 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:52.597641 30698 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.652s	user 0.003s	sys 0.015s
W20260709 13:13:52.675844 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 182083us
I20260709 13:13:52.677119 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:13:52.679003 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 130441us
W20260709 13:13:52.679574 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57926.4us
I20260709 13:13:52.617978 30698 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:52.687340 30706 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:52.702184 31898 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:13:52.699682 31744 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.689992 31714 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:13:52.705503 31744 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:52.688058 31854 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:52.720986 31854 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:52.723485 31854 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:52.715704 31862 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:52.690726 31902 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:13:52.725399 31902 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:52.725822 31902 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:52.729194 31790 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:52.726807 31902 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.731774 31902 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:52.719194 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.696182 31834 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:52.719914 31781 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:52.736819 31781 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:52.739157 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52971.9us
I20260709 13:13:52.696182 31712 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.739593 30551 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:52.742141 31911 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.742877 31911 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.744931 31634 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:52.745553 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.746433 30559 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:13:52.742154 31888 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.487s	user 0.004s	sys 0.000s
I20260709 13:13:52.748723 31789 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:52.749380 31789 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.749951 31911 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:52.751675 31789 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.752628 31781 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.755959 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.758390 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.767724 31908 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.768955 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.769345 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.770002 31908 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:52.770823 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.772167 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.759454 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.760468 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.774189 30554 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:52.761281 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.775460 30550 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:52.768955 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.779009 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:52.779956 31781 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:52.780097 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:52.780483 31781 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:52.781100 31781 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.770342 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.782276 31908 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:52.782815 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.761888 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.786340 31744 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:52.755975 31885 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:52.786909 31744 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:52.787202 31885 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:52.787341 31744 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.788300 31885 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.789234 31908 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.789582 31885 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:52.765962 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.792789 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:52.797340 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:52.798169 30701 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:52.803507 31738 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:52.809441 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:52.794411 31696 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:52.811901 31696 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:52.811992 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:52.813014 31696 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.813596 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:52.814283 31696 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:52.814822 31794 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:52.818868 31901 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:13:52.820412 31815 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:52.820874 31815 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:13:52.823925 31897 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:13:52.828576 31900 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:52.829344 31900 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.829866 31836 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:52.830721 31836 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.834719 31726 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:52.835326 31726 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:52.836294 31726 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:52.837515 30555 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:52.837718 31726 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:52.846414 31900 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:52.816854 31903 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:13:52.817303 31867 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:52.821841 31815 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:52.849531 31815 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:52.851418 31836 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:52.822321 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:52.863072 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:52.864682 31884 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:52.865262 31884 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:52.865711 31884 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:52.867892 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:52.870781 30632 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:52.873410 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:52.883625 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.884908 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.885285 31842 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:52.886565 31839 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:52.887025 31839 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:52.887941 31839 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.888607 31775 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:52.889194 31839 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:52.889935 31842 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:52.890347 31842 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:13:52.890753 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 68535.5us
I20260709 13:13:52.891237 31842 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:52.902987 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:52.945827 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:52.955216 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:52.956313 31364 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:52.959833 31367 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:52.970938 31790 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:52.971719 31790 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:52.995316 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:53.165541 31913 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:13:53.216835 31884 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:53.263550 31902 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.004s
W20260709 13:13:53.837689 31396 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.010s	sys 0.029s
W20260709 13:13:53.838903 31396 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.013s	sys 0.029s
I20260709 13:13:53.936721 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:53.937700 30633 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:53.937574 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:53.941223 31926 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:53.942030 31926 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:53.944558 31926 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:53.945217 31926 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:53.945950 31926 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:53.948463 31926 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:53.948385 30633 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:53.951347 31926 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:53.952037 31926 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:53.955399 31926 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:53.956126 31926 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:53.956673 31926 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:53.959331 31926 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:53.959863 31926 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:53.960296 31926 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:53.961720 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:53.962045 31926 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:53.962728 31926 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:53.963182 31926 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:53.965194 31926 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:53.968902 31928 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:53.970070 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:53.969875 31920 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:53.971896 31920 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:53.971920 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:53.975358 31920 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:53.976154 31920 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:13:53.977224 31675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.000s	sys 0.031s
W20260709 13:13:53.977882 31675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.000s	sys 0.033s
I20260709 13:13:53.977603 31920 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:53.982363 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:53.983945 31790 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:53.995178 31928 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:53.997718 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:53.998792 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 589) took 1008 ms. Trace:
I20260709 13:13:53.999222 31367 rpcz_store.cc:276] 0709 13:13:52.990729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:53.997345 (+1006616us) service_pool.cc:224] Handling call
0709 13:13:53.998716 (+  1371us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.003630 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 438) took 1252 ms. Trace:
I20260709 13:13:53.996449 31789 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.004920 31789 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.004243 31383 rpcz_store.cc:276] 0709 13:13:52.751096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.751225 (+   129us) service_pool.cc:224] Handling call
0709 13:13:52.751548 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.751814 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:53.006584 (+254770us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.003076 (+996492us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.003172 (+    96us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:54.003243 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.003349 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.003539 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":494283,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"raft.queue_time_us":1486,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":57,"thread_start_us":292922,"threads_started":3,"wal-append.queue_time_us":1061}
I20260709 13:13:53.995368 31790 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.007069 31790 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.009586 31790 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.011118 31789 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.007314 31884 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:53.997448 31928 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.017319 31293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.011s	sys 0.014s
W20260709 13:13:54.017823 31293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.011s	sys 0.014s
I20260709 13:13:54.020457 31675 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.021855 31675 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.024485 31675 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.025241 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.026099 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.018404 31928 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.027112 31926 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.028064 31926 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.030856 31790 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:54.031594 31790 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:54.032661 31790 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.035507 30699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.203s	user 0.009s	sys 0.020s
W20260709 13:13:54.036128 30699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.204s	user 0.010s	sys 0.020s
I20260709 13:13:54.038681 31926 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.040731 31714 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.041523 31714 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.043099 31702 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.043365 31926 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.044119 31714 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.044318 31926 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.046422 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.043785 31702 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.049364 31702 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.049963 31702 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.050545 31714 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.051180 31714 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.049413 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.053462 31702 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.054315 31714 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.056231 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:54.057468 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63044.9us
I20260709 13:13:54.057237 30550 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:54.058869 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.059495 31702 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:54.060580 31910 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.351s	user 0.005s	sys 0.006s
I20260709 13:13:54.062222 31936 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:54.065887 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 664) took 1638 ms. Trace:
I20260709 13:13:54.066056 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.066855 30630 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:13:54.067561 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.068802 30631 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:13:54.069041 31744 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.281s	user 0.012s	sys 0.020s
I20260709 13:13:54.069587 31744 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.066500 30707 rpcz_store.cc:276] 0709 13:13:52.429409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.429575 (+   166us) service_pool.cc:224] Handling call
0709 13:13:52.430009 (+   434us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.525545 (+ 95536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.525580 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.541624 (+ 16044us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.666763 (+125139us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:52.666821 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.666921 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.059737 (+1392816us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:13:54.065542 (+  5805us) raft_consensus.cc:1685] finished
0709 13:13:54.065568 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.065797 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5441,"raft.queue_time_us":5311,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":13324,"spinlock_wait_cycles":5127424,"thread_start_us":154514,"threads_started":2,"wal-append.queue_time_us":136348}
I20260709 13:13:54.067070 31937 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.071259 30625 raft_consensus.cc:2368] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:54.072806 31744 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.074975 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:54.071511 31937 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.069597 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.077569 30632 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:54.080361 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.081149 31383 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.080478 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.083061 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.083828 31937 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.084916 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.085902 31383 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:54.088968 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.093225 31383 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.095310 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.097581 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.098978 31675 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.099143 31292 leader_election.cc:400] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:54.099505 31675 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.099812 31292 leader_election.cc:403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:54.100032 31675 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.100895 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.102783 31934 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:54.104669 31934 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.105093 31934 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:54.109061 31834 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:54.112993 31744 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.113425 31744 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.113785 31744 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.116077 31383 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:54.116762 31790 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:54.120767 31675 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.126485 31926 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.130431 31675 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.131500 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:54.133759 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.131067 31779 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.143329 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:54.151765 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.152706 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.149444 31675 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:54.150516 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.149099 31936 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:54.149976 31834 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.154839 31934 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.174574 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:54.175370 30629 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:54.176095 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.171443 31936 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:54.174432 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.185503 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.184013 31937 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.187764 31937 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.188338 31933 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.188958 31933 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.183319 31779 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.191215 31779 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.193404 31779 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:54.194968 31815 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.345s	user 0.011s	sys 0.007s
I20260709 13:13:54.196857 31786 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.197319 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:54.197535 31786 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.198339 31933 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.198851 30625 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:54.197999 30628 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:54.199772 30628 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:54.199977 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.200686 30625 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:54.200850 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1085) took 1438 ms. Trace:
W20260709 13:13:54.201320 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.940s (SENT) [suppressed 6 similar messages]
I20260709 13:13:54.200752 30628 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.190299 31937 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.204157 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.205420 31938 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.201354 30552 rpcz_store.cc:276] 0709 13:13:52.762595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.763044 (+   449us) service_pool.cc:224] Handling call
0709 13:13:52.763365 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.763901 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 13:13:52.763943 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.764611 (+   668us) log.cc:844] Serialized 26 byte log entry
0709 13:13:52.903370 (+138759us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:13:52.903413 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:52.903486 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.200561 (+1297075us) raft_consensus.cc:1685] finished
0709 13:13:54.200585 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.200753 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1840,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":138294,"threads_started":1,"wal-append.queue_time_us":78948,"wal-append.run_cpu_time_us":3163,"wal-append.run_wall_time_us":1324042}
I20260709 13:13:54.218502 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:54.204252 30701 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.405s	user 0.021s	sys 0.001s
I20260709 13:13:54.220789 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.171221 31834 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.217815 31781 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.435s	user 0.020s	sys 0.001s
I20260709 13:13:54.222316 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.221376 30629 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:13:54.220702 30701 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:54.236320 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:54.228356 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:54.233615 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08003dc7a0 after lost signal to thread 21526
W20260709 13:13:54.240142 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08003bcd80 after lost signal to thread 21529
I20260709 13:13:54.227492 30552 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:54.237304 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.205999 31938 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.254094 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52681us
I20260709 13:13:54.253145 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:54.240864 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08002afee0 after lost signal to thread 30382
I20260709 13:13:54.259637 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.223832 31942 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.261448 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:54.264190 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:54.241775 31367 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:54.240784 31815 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:54.226625 31934 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.252588 31937 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.254417 31738 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:54.240255 30701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 89) took 1444 ms (client timeout 1786 ms). Trace:
I20260709 13:13:54.282344 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.279603 30559 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.237026 31781 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.290184 31946 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:54.296324 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.278280 31933 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.310325 30629 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:54.285027 31815 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:54.284523 31738 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.317788 30633 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:54.349728 31738 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:54.292249 30701 rpcz_store.cc:269] 0709 13:13:52.795856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.796997 (+  1141us) service_pool.cc:224] Handling call
0709 13:13:54.240141 (+1443144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:13:54.352981 31847 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.353595 31847 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.356364 31675 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.357128 31675 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.330989 31884 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.359105 31884 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.361366 31884 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.361943 31884 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.364549 31675 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.362424 31884 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.333276 31779 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.371513 31779 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.380273 30559 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:13:54.380699 31291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.413s	user 0.000s	sys 0.023s
I20260709 13:13:54.382083 31675 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.383045 31675 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.334332 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 661) took 1985 ms. Trace:
I20260709 13:13:54.255052 31936 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.340857 31786 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.344803 31673 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.385097 30706 rpcz_store.cc:276] 0709 13:13:52.348663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.348787 (+   124us) service_pool.cc:224] Handling call
0709 13:13:52.684982 (+336195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.687913 (+  2931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.334062 (+1646149us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b540037d2d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:13:54.334253 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735028608}
I20260709 13:13:54.385686 31673 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.388265 30380 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 139) took 4736 ms. Trace:
I20260709 13:13:54.388327 31673 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.392016 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.388803 30380 rpcz_store.cc:276] 0709 13:13:49.652306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:49.658859 (+  6553us) service_pool.cc:224] Handling call
0709 13:13:54.388153 (+4729294us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:50.583474 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:50.649861 (+ 66387us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:50.649922 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:13:50.649963 (+    41us) tablet.cc:662] Decoding operations
0709 13:13:50.650454 (+   491us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:50.650565 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:50.650603 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:13:50.651012 (+   409us) tablet.cc:701] Row locks acquired
0709 13:13:50.651092 (+    80us) write_op.cc:260] PREPARE: finished
0709 13:13:50.651340 (+   248us) write_op.cc:270] Start()
0709 13:13:50.651540 (+   200us) write_op.cc:276] Timestamp: P: 1783602830651267 usec, L: 0
0709 13:13:50.651586 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:13:50.652613 (+  1027us) log.cc:844] Serialized 537 byte log entry
0709 13:13:51.208383 (+555770us) op_driver.cc:464] REPLICATION: finished
0709 13:13:51.417151 (+208768us) write_op.cc:301] APPLY: starting
0709 13:13:51.417303 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:51.417480 (+   177us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:51.417679 (+   199us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:51.418101 (+   422us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:51.418521 (+   420us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:13:51.418636 (+   115us) write_op.cc:312] APPLY: finished
0709 13:13:51.419075 (+   439us) log.cc:844] Serialized 22 byte log entry
0709 13:13:51.419313 (+   238us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:51.419417 (+   104us) write_op.cc:454] Released schema lock
0709 13:13:51.419478 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:51.420057 (+   579us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:51.420100 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16482,"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":176882,"apply.run_cpu_time_us":2931,"apply.run_wall_time_us":2928,"num_ops":1,"prepare.queue_time_us":261726,"prepare.run_cpu_time_us":54156,"prepare.run_wall_time_us":153421,"raft.queue_time_us":219070,"raft.run_cpu_time_us":9942,"raft.run_wall_time_us":140484,"replication_time_us":524874,"thread_start_us":355697,"threads_started":4,"wal-append.queue_time_us":173097,"wal-append.run_cpu_time_us":11962,"wal-append.run_wall_time_us":1596089}]]}
I20260709 13:13:54.283161 31942 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:54.348385 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800324840 after lost signal to thread 30416
I20260709 13:13:54.313449 31933 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.394790 31779 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.396922 31933 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.310848 31702 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.397316 31884 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.397933 31702 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.391146 30382 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
W20260709 13:13:54.399909 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.506s	user 0.009s	sys 0.026s
W20260709 13:13:54.400302 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.506s	user 0.009s	sys 0.027s
I20260709 13:13:54.400343 31702 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:54.400604 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 1.50635e+06us
I20260709 13:13:54.400483 30382 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:54.403438 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.402935 30382 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:54.405150 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.405964 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.332674 31934 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.367928 31928 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.406710 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.370558 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.408219 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.407069 31928 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.408905 31383 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:54.410359 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.410475 31928 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.411149 31928 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.368896 31890 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.409142 30382 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.411829 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.411912 31890 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.413010 31790 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.412667 30382 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:54.413551 31790 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.414654 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:54.415719 31790 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.414346 31890 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.416884 31928 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.418308 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.419085 31790 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.340338 31815 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.420727 31291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.453s	user 0.000s	sys 0.024s
I20260709 13:13:54.442656 31790 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.423310 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.424324 31714 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.444398 31714 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.444898 31790 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.317709 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.446866 31714 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.418372 31847 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.449030 31847 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.449100 31928 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.449370 30628 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.449723 31847 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.451640 31951 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.452347 31834 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.452276 31951 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.453147 31834 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.453550 31834 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.463984 31834 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.465351 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 662) took 2116 ms. Trace:
I20260709 13:13:54.431641 31929 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.433702 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:54.434871 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.431351 31942 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.390858 31675 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:54.437366 30551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 120) took 2098 ms (client timeout 1940 ms). Trace:
I20260709 13:13:54.441231 31744 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.408013 31934 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.430738 31938 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:54.457688 30557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.655s	user 0.034s	sys 0.027s
I20260709 13:13:54.429517 31884 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.456535 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:54.468935 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.450301 31847 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.472327 31920 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.475291 31884 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.475713 31920 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.477036 31847 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.477663 31884 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.477907 31920 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.478343 31890 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:54.478598 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08002d5860 after lost signal to thread 30583
W20260709 13:13:54.472568 30551 rpcz_store.cc:269] 0709 13:13:52.342598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.355963 (+ 13365us) service_pool.cc:224] Handling call
0709 13:13:54.437017 (+2081054us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b540067b6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:13:54.437257 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633167232}
I20260709 13:13:54.479094 31890 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.479692 31890 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:54.480329 30632 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:54.469969 31929 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.483420 31929 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.483369 31744 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.487349 31926 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:54.487983 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 170387us
I20260709 13:13:54.488425 31948 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:13:54.488610 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86702.3us
I20260709 13:13:54.488844 31950 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.487198 31794 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.489320 31847 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:54.489547 31950 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.489789 31794 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:54.490880 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.488426 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90147us
I20260709 13:13:54.491926 31726 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.488054 31926 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.492588 31726 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.494031 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:54.487160 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68918.4us
I20260709 13:13:54.496286 31726 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.501966 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.489065 31948 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.496948 31712 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.516629 31726 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.498559 31367 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:54.518532 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.497812 31383 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:54.519434 31907 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.504674 31794 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.518249 31948 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:54.520084 31907 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:54.480437 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08001e3480 after lost signal to thread 30657
I20260709 13:13:54.522791 31653 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.492587 31951 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.503223 31906 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:54.523447 31653 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.524343 31906 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.524850 31906 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.494457 31926 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.525868 31653 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.494655 31781 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.526793 31634 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:54.527199 31781 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:54.528697 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.529635 31834 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.529671 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.530750 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:54.531629 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.532356 30705 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:54.533388 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.516710 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.534476 30707 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:54.535043 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.535241 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.516096 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.536206 30702 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:54.536358 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.510916 31695 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.537200 31396 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.515156 30706 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:54.537597 31695 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.539644 31934 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.474186 31772 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.540190 31695 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.540482 31772 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.535471 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:54.542526 31937 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.496840 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:54.543131 31937 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.538183 31834 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:54.543967 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47247.5us
I20260709 13:13:54.544158 31929 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:54.522954 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800040400 after lost signal to thread 30658
I20260709 13:13:54.547636 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:54.547959 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800381540 after lost signal to thread 30731
W20260709 13:13:54.548388 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800074860 after lost signal to thread 30732
I20260709 13:13:54.548455 30627 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:54.507254 31789 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:13:54.465948 30699 rpcz_store.cc:276] 0709 13:13:52.349052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.350867 (+  1815us) service_pool.cc:224] Handling call
0709 13:13:52.831322 (+480455us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:52.831597 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:52.831864 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:52.832297 (+   433us) log.cc:844] Serialized 14 byte log entry
0709 13:13:54.464847 (+1632550us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:54.464922 (+    75us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:54.464978 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.465070 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.465246 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428157,"prepare.queue_time_us":185,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":8999680,"thread_start_us":1203879,"threads_started":1,"wal-append.queue_time_us":358398,"wal-append.run_cpu_time_us":1536,"wal-append.run_wall_time_us":1001548}
I20260709 13:13:54.517905 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:54.551815 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.492120 31950 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.550447 31564 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.550582 31712 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.552983 31564 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.554750 31926 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.555323 31926 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.555436 31564 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.555802 31926 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.556612 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:54.557083 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.558331 30557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.755s	user 0.034s	sys 0.027s
I20260709 13:13:54.542387 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:54.559032 30705 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.559212 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 130) took 1764 ms. Trace:
I20260709 13:13:54.495730 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:54.561053 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65423.1us
I20260709 13:13:54.561128 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:13:54.567833 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:54.550287 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08000855e0 after lost signal to thread 31426
I20260709 13:13:54.571005 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.571329 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.571938 31383 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.572160 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.572834 30559 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:54.574980 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.592669 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.592537 30628 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:54.559722 30557 rpcz_store.cc:276] 0709 13:13:52.794653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.800802 (+  6149us) service_pool.cc:224] Handling call
0709 13:13:52.801235 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:52.801671 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:52.801704 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:52.802486 (+   782us) log.cc:844] Serialized 26 byte log entry
0709 13:13:54.558860 (+1756374us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:54.558905 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:54.558977 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:54.559020 (+    43us) raft_consensus.cc:1685] finished
0709 13:13:54.559030 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:54.559154 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2924,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":144,"thread_start_us":1755918,"threads_started":1,"wal-append.queue_time_us":459315,"wal-append.run_cpu_time_us":2786,"wal-append.run_wall_time_us":1015366}
I20260709 13:13:54.598063 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.603338 31772 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.605726 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.601405 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:54.610999 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.612061 31361 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:54.613304 31772 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:54.614025 31772 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.613742 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.614565 31772 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:54.614970 31367 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:54.622385 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.622445 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:54.627058 30699 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.630473 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.505103 31842 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:54.636746 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.637856 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.638557 31361 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.558264 31948 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.639213 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.639523 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.640334 31378 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 2: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:54.640475 31886 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.640975 31948 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.641218 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.593536 31564 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.503822 31790 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.642551 31564 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.642975 31790 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.645584 31790 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.571794 31652 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:54.645980 31564 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.647042 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.647271 31391 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.648079 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.628839 31950 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.592082 31929 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.648795 31907 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.526472 31653 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:13:54.526825 31726 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:54.591488 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800284200 after lost signal to thread 31427
I20260709 13:13:54.642568 31942 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:54.647328 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.652263 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.591665 31934 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.661058 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.651571 31834 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.672210 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.662364 30555 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:54.672220 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.674043 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.675984 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.662439 31950 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.678325 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.684657 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.685389 30559 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:54.688094 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:54.689550 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:13:54.641040 31886 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.690196 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.687170 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:54.667471 31951 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.669981 31652 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.696573 30704 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:54.675307 31949 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:13:54.723517 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.709816 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:54.678020 31867 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.676242 31834 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.723558 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.678324 31950 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.690394 31907 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.726924 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.686477 31938 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:54.730691 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.747257 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.689254 31847 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:54.694787 31781 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.672914 31929 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.726733 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:54.728865 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08003e62a0 after lost signal to thread 31876
W20260709 13:13:54.758908 30413 debug-util.cc:398] Leaking SignalData structure 0x7b080038c520 after lost signal to thread 31881
I20260709 13:13:54.760105 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.761289 31712 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.761575 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.762940 31367 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:54.763777 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.766036 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.704770 31951 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.725576 31714 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.746248 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.675690 31934 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.710639 31920 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.738832 31959 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:54.754104 31890 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.754253 31781 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.740500 31834 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.737105 31867 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.772325 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.773380 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:54.723161 31652 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.778965 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:54.758916 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.754142 31836 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:54.780727 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.822993 30705 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:54.823868 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.770885 31951 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.746594 31907 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.774087 31714 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.781111 31867 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.837574 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.804175 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.839654 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.840718 31714 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.841394 31714 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.841773 31714 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.842272 31714 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.780148 31959 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:54.844085 31786 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.844178 31959 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.844800 31786 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.827512 31836 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:54.767606 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.847282 31836 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.849150 31836 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:54.806063 31394 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:54.841641 31928 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.804816 30556 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:54.830051 31653 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.851511 31653 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.852023 31653 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.854040 31907 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:54.854943 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.808195 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.855602 31906 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.806917 31738 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.856635 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.857730 31786 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.857918 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.808755 31781 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.858635 31738 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:54.809695 31954 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.058s	user 0.000s	sys 0.002s
I20260709 13:13:54.810310 31652 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.812806 31934 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.816654 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.850899 31928 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:54.860751 31928 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.816900 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.862257 31364 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:54.863222 31634 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:54.863826 31634 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.810868 31886 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.816977 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.818326 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:54.867756 30704 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:54.777910 31920 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.782449 30703 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:54.816227 31961 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.871553 31961 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:54.872009 31961 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.872426 31920 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.873791 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.874104 31389 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:54.874645 31361 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:54.875981 31634 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.827349 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.828557 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:54.817970 30699 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:54.869524 31926 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:54.854772 30705 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:54.887524 30559 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:54.877797 31929 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.878001 31890 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:54.889824 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.890941 31938 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:54.891214 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.891495 31938 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:54.892195 31389 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:54.892525 31938 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.895568 31926 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:54.820791 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:54.878897 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:54.899865 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:54.901201 31890 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:54.901671 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.902707 31890 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.903087 31890 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.903473 31890 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:13:54.822038 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 163704us
I20260709 13:13:54.804185 31726 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.897753 31928 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.914832 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.915287 31928 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.916046 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.917062 31779 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.916630 31890 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.916731 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.917495 31779 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.917883 31779 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.921243 31890 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.922938 31696 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.923593 31696 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.924067 31696 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.900913 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.926328 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.927487 31847 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:54.927662 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:54.928256 31847 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:54.928534 31936 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.928637 31847 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:54.929126 31936 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.929286 31847 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.929602 31936 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.928648 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.933483 31949 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:54.933887 31949 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:54.934283 31949 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.939963 31936 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.940958 31847 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.943436 31847 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.943286 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.945117 31936 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.946996 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.955265 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.956634 31890 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:54.957072 31890 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:54.957443 31890 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:54.957940 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.958679 31383 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:54.960520 31867 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:54.824936 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46072.1us
I20260709 13:13:54.967296 31383 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:54.967023 31772 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:54.966831 31890 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.968746 30700 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 3. Ops: []
I20260709 13:13:54.970852 31772 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:54.970860 31890 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:54.971634 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:54.975593 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:54.975749 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.978245 30630 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:54.980690 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 74076.7us
I20260709 13:13:54.982025 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:54.982788 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:13:54.984630 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.985314 31394 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:54.989348 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:54.989537 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.993669 31890 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:54.993907 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.994454 31890 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:54.994510 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:54.995818 31388 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:54.995615 31890 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:54.994724 31361 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.009177 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:54.993207 31383 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:54.903824 31937 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:54.835085 31911 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:55.030642 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.008564 31634 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:55.032841 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.033386 31634 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.036286 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:55.007586 31726 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:55.037701 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.018837 31772 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.021508 31950 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.058774 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.066426 31391 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:55.023634 31948 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:13:55.071134 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 228129us
I20260709 13:13:55.072518 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:55.074136 30632 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:55.079007 31394 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:55.040095 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.070222 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:55.035531 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.035137 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:55.093443 30632 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:55.086374 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.023962 31911 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:55.044234 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.060736 31960 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.079842 31883 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.072202 31948 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:55.109126 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.037161 31834 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.085153 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:55.056669 31959 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:13:55.047295 31907 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 13:13:55.096102 31702 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:55.111389 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.085479 31963 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:55.067651 31726 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.072867 31772 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.116600 30556 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:13:55.081779 31653 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.107136 31389 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:55.103701 31928 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:13:55.136456 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75368.9us
I20260709 13:13:55.137985 30559 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:55.107348 31926 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:55.110522 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.102270 30625 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:55.121652 31738 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:55.184021 31772 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:55.184659 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.162243 31949 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.185487 31928 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.140065 31378 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:55.137763 31702 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:55.186596 31714 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.187332 31702 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.187475 31949 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.187939 31928 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:55.189618 31361 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:55.189905 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:55.123370 31948 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:55.155177 31911 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.106364 31964 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:13:55.158416 31950 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.148660 31967 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.159621 31867 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.160682 31937 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:55.121055 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.120360 31779 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.171157 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:55.182662 31926 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:55.142323 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.196781 31383 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:55.197535 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71540.3us
I20260709 13:13:55.141803 31963 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:55.162161 31696 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.189162 31714 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.216905 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.201521 31652 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:13:55.224288 30630 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:13:55.239979 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 168240us
I20260709 13:13:55.185516 30698 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.244364 31772 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:55.249606 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.138433 31961 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.243287 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.240212 31911 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.272168 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:55.245251 31960 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:13:55.258944 30556 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:55.244592 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.268236 31772 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.249434 31963 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.253988 30551 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.265149 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.212778 31967 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.256183 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:55.274289 30706 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:55.274958 31738 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.294829 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.281477 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:55.281970 30633 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:55.296535 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:55.261350 31867 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:55.251307 31951 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.265376 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:55.272764 31937 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:13:55.270061 30626 raft_consensus.cc:2376] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:13:55.299388 30633 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:55.279651 31653 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.314719 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.316202 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.317152 30553 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:55.289173 31772 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:55.309991 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.298529 31973 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:55.327203 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.289601 31960 consensus_queue.cc:1243] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:55.331940 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.307118 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.321135 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.321163 31856 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:55.321134 31836 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:55.323088 31779 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.323649 31834 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:55.325021 31696 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.324797 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.311443 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.334801 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.297891 31842 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:55.336925 30557 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:55.342345 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.351416 30559 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.345026 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.311872 31937 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:55.334527 31856 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.354429 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.359383 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.336779 31836 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.308352 31867 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.361794 30707 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:55.365224 30704 raft_consensus.cc:2368] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5.
I20260709 13:13:55.367079 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.371527 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.372381 30627 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:55.372596 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.373528 30554 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:55.366998 31836 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.347834 31916 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:13:55.379382 31916 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:55.379823 31916 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:55.380630 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:55.381928 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:55.383874 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.358469 31842 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.362905 31937 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:55.390870 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.361795 31378 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.388680 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.368144 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.365805 31856 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.391396 31916 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.408331 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.320694 31926 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.392014 30630 raft_consensus.cc:2368] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:55.367141 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.387933 31959 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:55.395574 31391 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:55.404382 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.425861 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.418270 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.445681 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.447481 30707 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.370078 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.395762 31961 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.407820 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.416676 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.493010 30624 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:55.394678 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.495078 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.431962 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.501418 30698 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
W20260709 13:13:55.502486 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70627.7us
I20260709 13:13:55.502786 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:55.502931 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 312518us
I20260709 13:13:55.503429 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.504170 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.504352 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:55.436700 30633 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:55.532809 31712 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:55.533478 31712 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:55.436682 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.447770 30706 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:55.533954 31712 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.448814 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:55.407888 31842 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.534907 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:55.535276 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 86570.2us
I20260709 13:13:55.451149 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.535751 30706 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:55.535540 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:55.452500 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.536759 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.537180 30631 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.537434 31714 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:55.405128 30559 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:55.537942 31714 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:55.411597 31364 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:55.454421 30553 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.538703 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.456140 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.418186 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.433805 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.540223 31779 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:55.464088 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.540122 31936 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:55.540990 31779 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.541419 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.541741 31936 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:55.540220 31367 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:55.411482 31702 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.542865 31936 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.399523 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.465615 31772 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.544515 30627 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:55.545444 31702 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.467499 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.473843 31920 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:55.480314 31726 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.546087 30556 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:55.470969 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.483270 31867 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.473124 31836 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.486377 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.481823 31695 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:55.446110 31949 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:55.546267 31920 consensus_queue.cc:1243] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:13:55.547894 31949 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:55.548650 31934 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:55.549206 31934 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:55.549288 31836 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:55.548924 31949 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.546164 30558 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.505723 30703 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:55.546427 31726 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.550143 31934 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.488408 31948 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.547107 30699 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:55.553455 31934 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.553732 31695 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:55.472962 31634 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.554793 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.556100 30701 raft_consensus.cc:2368] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:55.556344 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.557816 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.559059 30628 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:55.559696 31695 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.506659 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.561509 31695 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.468384 31906 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:55.565337 31906 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:55.562906 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.566632 31906 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.567921 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.509981 31738 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:55.512320 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.514693 31967 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:55.525641 31951 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.509037 31963 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.539361 31929 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:13:55.571122 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.540831 30555 raft_consensus.cc:2368] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:13:55.542433 31907 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:55.573374 31963 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.538359 31714 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.445324 31847 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:55.558167 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.493608 31916 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:55.574892 31847 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.575335 30701 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:55.454080 31928 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:55.540342 31673 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.548565 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.576540 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.577597 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.506351 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.537309 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:55.578596 30557 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:55.573573 31907 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:55.576071 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:55.583856 31907 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:55.571913 31929 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:55.584532 31929 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:55.584990 31907 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.585458 31929 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.405908 30551 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:55.586673 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.587767 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.588992 30557 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:13:55.591625 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.561518 31948 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.571900 30699 raft_consensus.cc:2376] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:13:55.568365 31906 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:55.602836 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.604727 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.578380 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.611335 31937 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.511240 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.614235 31673 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:55.617622 31847 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.618584 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.620078 31907 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:55.620540 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.621199 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.621923 30628 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:55.622545 30624 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
W20260709 13:13:55.622822 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 86242us
I20260709 13:13:55.624532 31928 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:55.626313 31779 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:55.627076 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.628232 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.630142 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.630972 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.631768 30559 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:55.632543 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:55.633162 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:55.633457 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 96224.6us
I20260709 13:13:55.634191 30627 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:55.634745 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:55.635535 31383 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:13:55.636408 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.636705 31972 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:55.637974 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.638857 30554 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.638882 30627 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:55.641114 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:55.643651 31847 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:13:55.644968 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.646343 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:55.647514 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 136382us
I20260709 13:13:55.608352 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.624660 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.656491 30599 leader_election.cc:400] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:13:55.658017 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:55.648967 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:55.668231 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.669600 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.686592 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:13:55.701110 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76534.1us
I20260709 13:13:55.702152 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:55.703279 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.711128 30707 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:13:55.654759 31929 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.757085 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.703859 31972 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:55.699857 31948 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.579056 30698 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:55.734305 31738 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.818048 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
W20260709 13:13:55.811647 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 125192us
I20260709 13:13:55.715472 31937 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:55.817554 31963 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:55.836004 30700 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:55.837407 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.839619 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:55.840472 30555 raft_consensus.cc:2368] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:13:55.839701 30700 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:55.822898 31781 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:13:55.841815 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.660202 31847 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:55.845252 31847 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.814325 30703 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:55.820644 30629 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:55.822724 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.850276 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.851017 30707 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:13:55.851312 31367 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:55.852095 30707 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:55.853760 30700 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:55.850276 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.855405 30707 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:55.855823 30703 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:55.857007 30707 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:55.857455 31856 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.858707 30550 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:55.861338 31856 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.862334 30558 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:55.863436 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:55.864593 31847 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:55.866601 30700 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:55.884256 30378 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.757535 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:55.887720 30627 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:55.886920 30378 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:55.889237 30378 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.891033 30378 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:13:55.892634 30378 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 2 to 4, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:55.824914 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:55.812054 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.819211 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.827315 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:55.929536 30551 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:55.827972 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:55.829756 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.830346 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.831075 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.832243 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.823262 31920 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:55.832635 31378 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:13:55.816869 31959 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:55.951351 31972 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:55.951835 31959 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.947018 30382 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 153) took 5328 ms. Trace:
I20260709 13:13:55.715473 31950 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
W20260709 13:13:55.812975 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 164115us
I20260709 13:13:55.959303 31950 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:55.959774 31950 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:55.960717 31950 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.962052 31950 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:55.749104 31772 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:55.963150 31967 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:55.955807 30382 rpcz_store.cc:276] 0709 13:13:50.618959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:50.619390 (+   431us) service_pool.cc:224] Handling call
0709 13:13:55.946927 (+5327537us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:54.755844 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:54.756465 (+   621us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:54.756497 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:13:54.756515 (+    18us) tablet.cc:662] Decoding operations
0709 13:13:54.757358 (+   843us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:54.757439 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:54.757471 (+    32us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:13:54.758063 (+   592us) tablet.cc:701] Row locks acquired
0709 13:13:54.758090 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:13:54.758243 (+   153us) write_op.cc:270] Start()
0709 13:13:54.758367 (+   124us) write_op.cc:276] Timestamp: P: 1783602834758199 usec, L: 0
0709 13:13:54.758400 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:13:54.792537 (+ 34137us) log.cc:844] Serialized 2848 byte log entry
0709 13:13:55.275018 (+482481us) op_driver.cc:464] REPLICATION: finished
0709 13:13:55.486921 (+211903us) write_op.cc:301] APPLY: starting
0709 13:13:55.487042 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:55.487339 (+   297us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:55.487518 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:55.488770 (+  1252us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:55.489087 (+   317us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:13:55.489159 (+    72us) write_op.cc:312] APPLY: finished
0709 13:13:55.489578 (+   419us) log.cc:844] Serialized 58 byte log entry
0709 13:13:55.591138 (+101560us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:55.591275 (+   137us) write_op.cc:454] Released schema lock
0709 13:13:55.591327 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:55.597800 (+  6473us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:55.597850 (+    50us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2004366,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":179353,"apply.run_cpu_time_us":3543,"apply.run_wall_time_us":104859,"num_ops":7,"prepare.queue_time_us":334314,"prepare.run_cpu_time_us":18836,"prepare.run_wall_time_us":262603,"raft.queue_time_us":132588,"raft.run_cpu_time_us":71266,"raft.run_wall_time_us":84681,"replication_time_us":516433,"thread_start_us":637671,"threads_started":4,"wal-append.queue_time_us":249868}]]}
I20260709 13:13:55.964395 31967 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:55.964761 31967 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:55.812929 31911 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:55.965772 31967 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:55.895061 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.896963 31781 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:55.825088 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:13:55.977985 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 153044us
I20260709 13:13:55.981647 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:55.898375 31948 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:13:55.813216 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 110956us
I20260709 13:13:55.900424 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:55.910548 31934 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:55.923533 31842 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:55.924223 30554 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:55.935779 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:55.937103 30631 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:55.939913 31726 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:55.991568 30628 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:13:55.940140 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 108007us
W20260709 13:13:55.942574 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:55.905934 31714 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:55.994545 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.995615 30705 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:55.842973 30704 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 2.
I20260709 13:13:55.996222 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.997691 30599 leader_election.cc:400] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:13:55.814595 31951 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:55.967069 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:55.975373 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:55.988317 31652 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:55.999328 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:55.989455 31916 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:56.001402 30706 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:56.004312 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:56.017205 30556 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:56.024513 30703 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:56.026510 31383 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:56.038895 30523 leader_election.cc:400] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5.
I20260709 13:13:56.039758 30523 leader_election.cc:403] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:13:56.047885 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48658.4us
I20260709 13:13:56.047979 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:56.048967 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:56.050796 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56332.1us
I20260709 13:13:55.997283 31836 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:56.006807 31652 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.019511 30700 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:56.018572 31738 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:56.013599 31867 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.009326 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:56.025836 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:55.995191 31712 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.047049 31916 consensus_queue.cc:1243] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:13:56.065871 31726 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:56.063862 31906 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.066452 31634 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.066063 31786 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5.
I20260709 13:13:56.062489 30554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:56.074946 31652 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.096284 31967 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:56.094879 31847 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:56.115432 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.155355 30698 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:56.146338 31726 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.117964 31964 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 13:13:56.153616 30554 raft_consensus.cc:606] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:56.162483 30701 raft_consensus.cc:2368] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:56.175034 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:56.182670 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:56.203164 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:13:56.226174 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 522942us
I20260709 13:13:56.228506 31292 leader_election.cc:400] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:56.229033 31292 leader_election.cc:403] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:56.199088 31916 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.199054 31906 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.199725 31929 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:56.225708 31928 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:56.161208 31847 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:56.255766 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.204329 31634 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:13:56.201046 31920 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:13:56.284941 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 233351us
I20260709 13:13:56.242656 30627 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:13:56.288717 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 237575us
W20260709 13:13:56.273468 31695 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.709s	user 0.005s	sys 0.011s
I20260709 13:13:56.250052 31842 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.269786 30625 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:56.237537 31911 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:13:56.288038 31986 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:13:56.271008 31929 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:56.295096 31983 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:13:56.292382 31906 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:56.286841 31714 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:56.306695 31987 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:13:56.308422 31726 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.323737 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:56.324813 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.316510 31992 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:56.328409 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:56.347711 31383 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.326715 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:56.352447 30628 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:13:56.310082 31842 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:56.311156 31906 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.312090 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:56.318361 31712 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:56.309846 31988 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
W20260709 13:13:56.377825 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 149541us
I20260709 13:13:56.365422 31842 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:56.391002 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.393668 31383 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:56.395874 30628 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:13:56.396387 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:56.396416 31394 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:56.373836 30554 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:56.374423 31738 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:56.372323 31991 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:13:56.369351 31920 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.420779 32001 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.423079 32001 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.425940 32001 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:56.427395 32002 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:56.428231 31842 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:56.428959 31842 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:56.429747 31936 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:56.430639 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:56.429497 31842 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.430373 31936 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.415122 31928 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:56.437708 31929 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.436385 31934 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:56.436821 31936 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:56.434878 31847 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:56.460558 31652 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:56.460956 31982 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.441s	user 0.003s	sys 0.003s
I20260709 13:13:56.482888 31738 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:56.487659 31994 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:13:56.412238 31906 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.476850 31936 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:56.492095 31936 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.495067 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:56.496086 30627 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:13:56.496254 31950 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.534s	user 0.016s	sys 0.002s
I20260709 13:13:56.497974 31950 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:56.498803 31950 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.502410 31950 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:56.504030 31383 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:56.505712 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.507856 30628 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:56.508054 30631 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.509138 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:56.507655 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:56.515522 30703 raft_consensus.cc:2368] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6.
W20260709 13:13:56.515125 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 438065us
I20260709 13:13:56.433410 31842 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.522502 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:56.528115 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.538028 30597 leader_election.cc:400] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6.
I20260709 13:13:56.538542 30597 leader_election.cc:403] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:56.540905 30553 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:56.544963 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.549791 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.552071 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.514974 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:56.561124 30631 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:56.575114 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.577704 30558 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:56.580410 30523 leader_election.cc:400] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:56.580979 30523 leader_election.cc:403] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:13:56.537488 31906 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.559428 31985 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6.
I20260709 13:13:56.559881 31948 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.583237 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.567800 31950 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 13:13:56.582273 31847 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.595480 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.595891 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.582811 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:56.595849 31948 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.599596 30556 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:56.600909 31652 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:56.591336 31928 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.597839 31856 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3.
I20260709 13:13:56.596185 30554 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:13:56.623677 31786 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:56.584702 30699 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:56.593441 31996 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.591321 32005 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:13:56.605854 31936 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:56.585286 32002 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:56.603799 30551 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:56.611392 31966 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:13:56.637760 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:56.637429 31996 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:56.649910 31907 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:56.650105 31856 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:56.650985 31772 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:13:56.640142 31652 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:13:56.651630 31772 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.652552 31772 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.653528 31772 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:56.638242 31936 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.598131 31847 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.655946 31394 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:56.665624 30631 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:56.667614 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:56.670192 30628 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.648306 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 618) took 1030 ms. Trace:
I20260709 13:13:56.650113 31996 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:13:56.683842 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 292925us
I20260709 13:13:56.650949 31856 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:56.708931 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.706269 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.682904 31936 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:13:56.721834 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 100711us
W20260709 13:13:56.721791 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 83748.8us
I20260709 13:13:56.720005 30628 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:13:56.699405 31907 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.726194 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:56.706936 31292 connection.cc:256] RPC call timeout handler was delayed by -1.2025177089999488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 13:13:56.728430 31292 connection.cc:256] RPC call timeout handler was delayed by -1.1705992610000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 13:13:56.729358 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:56.730176 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.731788 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:56.729732 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0711177089999637s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 13:13:56.785780 30628 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:56.786311 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:56.722306 31926 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:13:56.789148 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0297177089998968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1771s for a response
I20260709 13:13:56.790606 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.788623 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:56.792655 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0027177090000805s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 13:13:56.727159 32011 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:13:56.755005 31967 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.008s
W20260709 13:13:56.801162 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69430.7us
I20260709 13:13:56.804898 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.806452 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.753998 31712 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:56.749204 30624 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:13:56.753400 31948 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:56.746802 31989 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:56.748472 31847 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:56.810456 31292 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:56.740643 32012 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.742055 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 899) took 1262 ms. Trace:
I20260709 13:13:56.729514 31726 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 13:13:56.682515 30625 rpcz_store.cc:276] 0709 13:13:55.617797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.617992 (+   195us) service_pool.cc:224] Handling call
0709 13:13:55.618416 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:56.307227 (+688811us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.647992 (+340765us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54000712d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:56.648231 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437894016}
I20260709 13:13:56.800107 32007 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:56.806553 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:56.804205 31779 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.828948 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.822050 31989 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:56.807463 31702 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.750975 31856 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:56.815613 31991 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:56.811154 31926 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:56.812461 31934 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.837656 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.821817 31712 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.821669 31985 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.809510 32005 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:56.821606 32009 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:13:56.809324 32022 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:56.811614 32011 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:56.832206 30630 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:56.825903 30701 rpcz_store.cc:276] 0709 13:13:55.479369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:55.578044 (+ 98675us) service_pool.cc:224] Handling call
0709 13:13:56.741815 (+1163771us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b540008cad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:13:56.741987 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214702464}
I20260709 13:13:56.849069 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.867851 30703 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:56.868629 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.869582 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.869989 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.841781 31967 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.850459 31934 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.838450 31928 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.873634 31967 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.875706 32001 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.875515 32011 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.753646 31996 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:56.878991 31959 consensus_queue.cc:1175] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6b393e79ded4461ea5da7b6b6063305f" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:13:56.838860 31738 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.854699 32025 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:56.836804 31966 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:56.872233 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.872828 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:56.873106 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:56.883455 31712 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.883953 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.884707 30698 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:56.873978 31949 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:56.885468 31949 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:56.842089 31786 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:13:56.886669 31786 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:56.886495 31949 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.887130 31786 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:56.887362 31714 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:56.887812 31714 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:56.887877 31949 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:56.867878 32022 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.888373 31934 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.888779 31714 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:56.890041 31929 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.890179 31738 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.892954 30378 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 131) took 5489 ms. Trace:
I20260709 13:13:56.837198 32007 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.875993 31967 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.882761 30703 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:56.883072 32006 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:56.852013 31926 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.869639 31991 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:56.845515 31702 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:56.892580 31966 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:56.885370 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:56.896361 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:56.923792 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 67756.1us
I20260709 13:13:56.924654 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.895995 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.924600 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.928275 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.930653 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.928326 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:56.931823 30555 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:56.933245 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.934101 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.936553 30597 leader_election.cc:400] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:56.936918 30597 leader_election.cc:403] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:56.938041 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.893390 30378 rpcz_store.cc:276] 0709 13:13:51.406507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:51.408789 (+  2282us) service_pool.cc:224] Handling call
0709 13:13:56.892880 (+5484091us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:56.360157 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:56.420585 (+ 60428us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:56.420643 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:13:56.420670 (+    27us) tablet.cc:662] Decoding operations
0709 13:13:56.421601 (+   931us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:56.421687 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:56.421716 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:13:56.422375 (+   659us) tablet.cc:701] Row locks acquired
0709 13:13:56.422401 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:13:56.422578 (+   177us) write_op.cc:270] Start()
0709 13:13:56.422753 (+   175us) write_op.cc:276] Timestamp: P: 1783602836422525 usec, L: 0
0709 13:13:56.422780 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:13:56.423790 (+  1010us) log.cc:844] Serialized 2078 byte log entry
0709 13:13:56.500189 (+ 76399us) op_driver.cc:464] REPLICATION: finished
0709 13:13:56.535763 (+ 35574us) write_op.cc:301] APPLY: starting
0709 13:13:56.535891 (+   128us) tablet.cc:1367] starting BulkCheckPresence
0709 13:13:56.536160 (+   269us) tablet.cc:1370] finished BulkCheckPresence
0709 13:13:56.560955 (+ 24795us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:13:56.569288 (+  8333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:13:56.569548 (+   260us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:13:56.569595 (+    47us) write_op.cc:312] APPLY: finished
0709 13:13:56.569919 (+   324us) log.cc:844] Serialized 46 byte log entry
0709 13:13:56.570160 (+   241us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:56.570295 (+   135us) write_op.cc:454] Released schema lock
0709 13:13:56.570351 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:13:56.570841 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:13:56.570873 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":187734,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":35233,"apply.run_cpu_time_us":3027,"apply.run_wall_time_us":35054,"num_ops":5,"prepare.queue_time_us":380980,"prepare.run_cpu_time_us":5103,"prepare.run_wall_time_us":101293,"replication_time_us":77202,"thread_start_us":296633,"threads_started":2}]]}
I20260709 13:13:56.933589 31781 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:56.894843 32025 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.942643 32032 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:56.943406 32032 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:56.897727 31936 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:56.860838 31985 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:56.929309 31934 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:56.898910 31928 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:56.941923 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:56.932581 32004 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:56.933018 31992 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:56.940532 31781 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:56.894991 31738 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.956732 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:56.944222 31950 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:56.929718 31964 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:56.962203 32025 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.963142 32002 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:13:56.963286 32034 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:13:56.963609 32027 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:56.963025 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:56.964182 31967 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:56.953047 31998 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:13:56.941001 31867 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.967933 32004 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:56.968417 31985 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:56.969398 32009 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:57.019768 32009 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:57.020320 32009 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.969915 32007 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:56.888033 31786 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.012231 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.023804 31786 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:56.971947 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:56.982120 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:56.945822 31991 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:57.024914 30555 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:57.025019 31991 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:56.985589 31928 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.027513 31991 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:56.995827 30559 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
W20260709 13:13:57.029495 32031 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:13:56.947072 32033 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:57.030259 32014 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:13:57.031461 31702 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:57.032493 31972 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:57.005275 31394 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:13:57.012682 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79525.6us
I20260709 13:13:56.996531 31950 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:57.035527 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:56.997148 31781 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.040267 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:56.965349 32032 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.002221 31696 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:13:57.036827 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.038950 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:57.050746 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78949.6us
I20260709 13:13:57.001770 32002 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:57.005298 31842 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.058581 32011 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.060675 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:57.031571 32014 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:57.063485 30559 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:13:57.053416 31991 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:57.056926 31998 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:57.050813 31950 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.027544 31928 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.053036 31695 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:57.070052 30555 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:57.036087 32022 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.052299 32004 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:57.006740 31964 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.054260 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1119) took 1028 ms. Trace:
I20260709 13:13:56.965957 31989 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.087769 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.074427 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.073076 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.065574 32014 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:13:57.141651 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101748us
I20260709 13:13:57.146459 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.146001 30559 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:57.089144 32007 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:13:57.144539 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 105637us
W20260709 13:13:57.087697 32039 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 13:13:57.150275 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75949.8us
I20260709 13:13:57.120863 31695 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:57.154795 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.157423 30559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:57.118997 31929 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:57.072052 31998 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:57.139324 31738 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:57.166561 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.150704 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.064759 31842 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.221475 31842 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.170001 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:13:57.223094 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67930.1us
W20260709 13:13:57.203244 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.072096 32041 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.073858 31991 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:57.226964 31991 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.227608 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:57.116897 31934 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:13:57.219408 31998 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.160427 32022 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:57.172065 31989 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.215004 31847 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:57.236781 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.175068 31772 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 13:13:57.234833 32041 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 13:13:57.263078 31997 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:13:57.264389 31856 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:13:57.169548 31928 consensus_queue.cc:1175] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6b393e79ded4461ea5da7b6b6063305f" responder_term: 3 status { last_received { term: 1 index: 1 } last_committed_idx: 1 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:13:57.267601 32038 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 13:13:57.267966 32004 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.270051 32008 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.269852 31695 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.270694 32008 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.271523 31695 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.273967 32049 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:13:57.275346 32008 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:13:57.276042 32011 consensus_queue.cc:1175] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 3 status { last_received { term: 3 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 2 } }
I20260709 13:13:57.276248 32014 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.279057 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.279918 30628 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:13:57.280704 31963 consensus_queue.cc:1175] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 13:13:57.284991 30625 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:57.286245 32001 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.286902 32001 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.287773 31702 raft_consensus.cc:993] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:57.289325 32001 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.289923 32001 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.242184 31966 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:13:57.291280 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:57.292651 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.295758 31985 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:57.296413 31985 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.300818 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.298305 31991 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:57.317548 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 150225us
I20260709 13:13:57.320273 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.311741 31964 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.303030 31975 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:57.262075 31847 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.303874 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.274029 32025 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:13:57.321777 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:57.166435 32002 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.316324 31996 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:57.330669 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:57.307130 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.236123 31738 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.305743 31985 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.360316 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.336634 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.283397 32014 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:57.341107 30631 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:13:57.331552 32022 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.342331 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:57.374688 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:57.330273 31975 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:13:57.329135 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 105038us
I20260709 13:13:57.307107 31779 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:57.345254 30629 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:57.302553 32006 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.290370 32001 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.330474 31937 raft_consensus.cc:1081] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:13:57.355788 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:57.334774 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.363492 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:57.364081 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:57.399426 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43774.5us
I20260709 13:13:57.366089 32042 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:13:57.367568 31378 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:57.362277 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.361364 30557 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:57.391772 30553 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
W20260709 13:13:57.392663 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 222716us
I20260709 13:13:57.338220 32025 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:57.346983 31972 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:57.405902 31391 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:57.395550 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:57.410590 30559 raft_consensus.cc:606] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Received request to transfer leadership to TS 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.411003 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.392799 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.395099 31975 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.418370 32045 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:57.419025 32045 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:57.419440 32045 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.421540 31867 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:13:57.423801 30632 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:57.426833 32052 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.427284 32001 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:57.427591 32052 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.142165 30550 rpcz_store.cc:276] 0709 13:13:56.023756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.025554 (+  1798us) service_pool.cc:224] Handling call
0709 13:13:56.612544 (+586990us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:56.612807 (+   263us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:56.625103 (+ 12296us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:56.724368 (+ 99265us) log.cc:844] Serialized 14 byte log entry
0709 13:13:56.881331 (+156963us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:56.946257 (+ 64926us) log.cc:844] Serialized 26 byte log entry
0709 13:13:56.955394 (+  9137us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:56.960613 (+  5219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:56.996271 (+ 35658us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:57.003829 (+  7558us) raft_consensus.cc:1685] finished
0709 13:13:57.014523 (+ 10694us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:57.052203 (+ 37680us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22781,"prepare.queue_time_us":200985,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":15381,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":8288,"spinlock_wait_cycles":46689152,"thread_start_us":141831,"threads_started":2,"wal-append.queue_time_us":203141}
I20260709 13:13:57.428277 31907 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.429771 31907 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:57.430576 31991 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.433241 32053 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:13:57.415081 30700 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.563s	user 0.013s	sys 0.000s
I20260709 13:13:57.434461 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.436789 30700 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:13:57.407902 32037 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:57.438710 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.435786 32025 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.439770 32025 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:57.414863 31972 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.433897 32053 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.393791 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.450404 31396 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:57.405627 31772 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:57.454150 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:13:57.476528 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.461298 31673 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.436067 31847 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:57.477455 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:57.459064 32006 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:57.465135 31772 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.458408 32053 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.481451 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.513586 30703 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:57.487347 31963 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:57.502487 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.503175 31396 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:57.484251 31388 raft_consensus.cc:2368] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
W20260709 13:13:57.551951 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.519702 30629 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:57.515141 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:57.468150 31992 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.559293 31383 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:57.484308 31967 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:57.572595 31367 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.573993 31847 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:57.574579 31847 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.488484 31673 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.575170 31847 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.575251 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.577052 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.577438 31673 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.577602 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.578387 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.521510 31772 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.578714 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.578945 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.579682 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.579862 31367 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:13:57.572738 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57673.8us
I20260709 13:13:57.571393 30625 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:57.587772 31378 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:13:57.589659 31972 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:57.570801 31786 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.546s	user 0.014s	sys 0.000s
I20260709 13:13:57.579805 31364 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:13:57.592084 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.590535 31972 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:13:57.595307 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:57.599720 32055 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:13:57.503021 32006 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.606966 32006 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:57.623018 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.512831 31712 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:57.614323 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:57.679971 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65761.6us
I20260709 13:13:57.681653 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:13:57.687475 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 292005us
I20260709 13:13:57.688951 31992 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:57.689579 31992 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:57.691917 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:57.570148 30554 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:57.559448 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.513438 32009 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.714561 31394 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:57.715257 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.690555 31992 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.686501 32011 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.694701 31948 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:57.561100 31963 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:57.753302 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 290766us
I20260709 13:13:57.705632 31991 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:13:57.755257 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 194099us
I20260709 13:13:57.684780 32053 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:13:57.760485 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:57.686098 31972 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:13:57.673496 32045 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.762665 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.735214 30630 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:57.759542 30698 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:13:57.736021 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:57.765255 31998 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.002s
I20260709 13:13:57.773327 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:57.756822 31948 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.773013 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.758425 31991 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:57.762197 30700 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:57.762003 32052 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:57.785727 31391 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:57.785032 30701 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:57.798063 32060 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 13:13:57.781816 31695 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.003s
I20260709 13:13:57.790271 31991 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.792151 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:57.792834 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.796162 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.754791 32011 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.778980 31966 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:57.840552 32009 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:57.816962 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:13:57.841645 31361 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:13:57.842064 31361 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.819622 31364 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:57.820329 31388 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:13:57.794885 31948 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.855183 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:57.822871 32029 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.876s	user 0.008s	sys 0.000s
W20260709 13:13:57.812855 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.807675 31695 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:57.783434 31738 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.828265 31963 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.836445 32011 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.766986 31673 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:13:57.784149 31998 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.838040 32006 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.844564 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:57.846115 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:57.848469 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:57.856694 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:13:57.849737 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 164977us
W20260709 13:13:57.863409 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.875439 30703 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:57.796011 31959 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.877003 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.879015 30631 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:57.881039 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.882066 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.865469 31361 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:57.849694 32014 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:57.885414 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.856377 32000 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.809934 31964 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 13:13:57.876354 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:57.886658 30628 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:57.875430 31394 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:13:57.885002 31738 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:57.888641 31959 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:57.902751 30633 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:57.803448 32037 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 13:13:57.904826 31847 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.906468 31975 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.907697 32045 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.908223 31772 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:57.902840 31856 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.002s
I20260709 13:13:57.888080 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.910602 30627 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:57.911175 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:57.911799 31781 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.854s	user 0.012s	sys 0.001s
I20260709 13:13:57.884744 31867 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:57.912843 32000 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:57.914960 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:57.920272 30630 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
W20260709 13:13:57.923187 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46933.4us
W20260709 13:13:57.891279 31950 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.004s
I20260709 13:13:57.993017 31948 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.993753 31950 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.996836 31950 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.997645 31959 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:57.976171 31702 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:57.998998 31702 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.001565 32000 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:57.925025 31926 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.002527 30624 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:13:57.928555 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:57.929222 30701 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:13:57.935014 31998 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:57.936272 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:57.987143 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:57.936728 32006 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.009383 30699 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:57.884685 31985 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:13:57.938705 31991 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:57.986743 31781 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.013703 31991 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.942304 31695 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:58.015261 31781 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:57.950449 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.016976 32049 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:57.889914 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:13:58.018601 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128860us
I20260709 13:13:58.020011 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:58.020874 30704 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:57.960391 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:57.960855 30629 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:57.964293 31972 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 13:13:57.961077 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:57.974828 30553 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:13:58.017488 31975 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:57.956579 31738 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:57.968477 31726 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:58.027208 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.035296 30703 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:58.028518 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:58.037781 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:57.998206 31950 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:13:58.045445 31972 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:58.046391 31963 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:58.046741 31972 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:58.047108 31963 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.049612 30698 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:58.051478 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.052872 31396 raft_consensus.cc:2368] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
W20260709 13:13:58.053579 31989 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.002s
I20260709 13:13:58.054181 31911 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:58.063447 30522 leader_election.cc:400] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:13:58.064158 30522 leader_election.cc:403] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:13:58.065475 30704 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:58.066886 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.068512 30559 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 2: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:58.070446 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:58.049826 31963 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.035218 31907 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:58.086134 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.041344 31738 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.099864 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.095363 31937 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:58.104579 31937 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.076450 31985 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:58.078505 31847 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.078012 31836 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.082490 31781 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:58.099519 32003 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:58.083096 31963 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:58.065915 31726 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:58.133255 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 141451us
I20260709 13:13:58.090018 31907 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.089314 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.096994 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:13:58.126869 32025 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.008s
I20260709 13:13:58.121692 32000 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.149358 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:58.129809 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.144198 31907 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:13:58.162807 32073 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:13:58.163118 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:58.165024 31867 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:58.165858 31726 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.166056 31847 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:58.167393 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:58.168409 31959 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:58.168215 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.171656 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:58.172317 31702 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.175468 31714 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.177297 31959 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:13:58.178108 32061 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.401s	user 0.000s	sys 0.001s
I20260709 13:13:58.184923 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.183142 31772 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.197340 31772 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.198073 31772 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.201812 32051 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.201645 30701 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:58.138568 31836 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.205332 31836 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.141659 31989 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:13:58.206651 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.206082 31989 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.208477 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.208925 31989 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.209825 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.210006 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:58.210757 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.211225 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.158944 31937 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:58.141839 31781 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:58.216717 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.135214 31985 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:58.187755 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:58.224229 31394 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:58.159379 30554 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:13:58.179847 31948 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.183310 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.218304 30559 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:58.226505 31772 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.202436 32051 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.230600 31702 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:13:58.237571 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.215127 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.160465 31964 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.165869 31867 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:58.240263 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:58.251466 30554 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:58.247285 31959 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:58.145084 31963 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.260360 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:58.266124 31394 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 4. Ops: []
I20260709 13:13:58.267627 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.269394 30632 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:13:58.272735 31772 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:58.251086 30550 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:58.279570 31772 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.293638 31772 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.263255 31959 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.275835 31696 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:58.274906 32002 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:13:58.303673 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:58.304440 30705 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:58.310750 30703 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:58.304330 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:58.269908 31964 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.336858 31964 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.301251 31959 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:58.300666 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.308748 31696 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.254583 31985 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.320434 31998 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:58.348630 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.340246 31949 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:58.347265 30700 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:58.345234 31696 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:58.357234 32081 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:13:58.358765 32048 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:58.356462 31998 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.326658 31781 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.353399 32025 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:13:58.360133 30556 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:13:58.365559 32037 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:13:58.368822 31949 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.391515 31949 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.391335 31772 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:58.393421 30627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 377) took 1913 ms (client timeout 1597 ms). Trace:
W20260709 13:13:58.393937 30627 rpcz_store.cc:269] 0709 13:13:56.506502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.512185 (+  5683us) service_pool.cc:224] Handling call
0709 13:13:57.227385 (+715200us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:13:58.393180 (+1165795us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400052b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:13:58.393319 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913179136}
I20260709 13:13:58.382716 30556 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:13:58.427004 30556 raft_consensus.cc:614] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:13:58.432348 31998 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.394527 31772 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.374991 31738 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.392341 32003 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:13:58.436359 31926 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:58.459458 32051 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.491086 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.447357 31907 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.464267 30699 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:58.592614 30558 raft_consensus.cc:2368] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:58.617841 30632 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:58.605857 31966 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:58.597540 31975 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.598165 31991 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.594880 31738 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.598929 30624 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:58.602804 31949 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.464201 31911 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.648092 31911 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.650960 30550 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:58.606523 32082 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.613906 31696 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.618224 30555 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:58.405392 32049 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.604496 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.614130 30552 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:13:58.619817 30556 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.626479 31991 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.673326 31991 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:58.635540 30704 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:13:58.636781 31907 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.626151 31975 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.675274 31696 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.675719 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.676717 30551 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:58.602473 31867 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:58.643534 31673 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:58.677852 31673 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:58.679116 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:58.679533 31967 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:58.679970 31967 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:58.646327 31361 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.651702 31383 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:58.681046 31383 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:58.680817 31967 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.681875 31383 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.682884 31383 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:58.644544 32000 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:58.654589 30631 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
W20260709 13:13:58.653873 30633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 386) took 1499 ms (client timeout 1609 ms). Trace:
W20260709 13:13:58.684835 30633 rpcz_store.cc:269] 0709 13:13:57.154327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.154461 (+   134us) service_pool.cc:224] Handling call
0709 13:13:58.653588 (+1499127us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400569ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:13:58.653794 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668590464}
I20260709 13:13:58.688267 31967 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.692073 31996 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:13:58.695595 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:58.696913 30633 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:58.698662 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.663827 32082 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.665170 32049 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.669308 30630 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:58.677376 31975 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.679342 31673 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.695870 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.660970 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:58.700646 30633 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:13:58.655134 31911 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:58.599565 31959 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.701844 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.705467 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:58.710278 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:58.711267 31383 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:13:58.727679 31383 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:13:58.727964 31383 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:13:58.728242 31383 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:13:58.725023 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:58.711876 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 355651us
I20260709 13:13:58.714711 30553 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
W20260709 13:13:58.718066 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.717196 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.720817 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.721426 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.721522 32082 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:58.721822 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:58.731916 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:58.624336 31966 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.737382 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.704653 32050 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.739467 30629 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.739255 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.662786 31949 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.744130 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:13:58.706063 31712 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.741746 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:58.746297 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.749508 30633 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.750344 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.748170 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:13:58.716907 32014 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.753599 30704 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:58.731832 31673 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.723768 30624 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:13:58.738607 31936 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:58.760243 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:58.762715 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:58.766105 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:13:58.769415 32062 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.973s	user 0.000s	sys 0.012s
I20260709 13:13:58.771060 30704 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:13:58.772019 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:58.772756 31963 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.738343 32070 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:58.806058 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.806874 31383 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.807472 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:58.808198 31394 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:58.807545 30629 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:58.759732 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.780869 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.787559 31907 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:58.740835 31714 raft_consensus.cc:2764] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:13:58.740257 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.778595 30633 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:58.752000 31989 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:58.757843 31934 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:58.812639 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.793715 32005 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:58.779343 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:58.744421 32050 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:58.793782 31696 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:13:58.756579 31712 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.796571 32051 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:58.799943 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:58.784610 32014 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:58.803815 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:13:58.898654 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98760.8us
I20260709 13:13:58.805799 31950 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:58.885170 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:58.817586 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.815701 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:13:58.852039 32087 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.223s	user 0.004s	sys 0.000s
I20260709 13:13:58.813225 32091 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.106s
I20260709 13:13:58.912053 32091 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:58.912503 32091 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:58.820012 32089 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:13:58.913492 32091 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.917096 32091 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:58.804862 31738 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:13:58.815807 31985 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:58.852309 31966 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:13:58.897163 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84624.2us
I20260709 13:13:58.831983 31991 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:58.862931 31959 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:58.804469 31936 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:58.812369 31702 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:58.857009 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.928004 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:58.941736 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 129771us
I20260709 13:13:58.909164 30699 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.908655 31950 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:58.883793 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:58.903502 32002 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.928910 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:58.818396 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:58.850800 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:58.904349 32051 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.928120 31985 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:58.904637 31367 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.906031 31963 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:58.831063 32070 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:58.896266 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:58.943025 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.000960 30629 raft_consensus.cc:2368] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
W20260709 13:13:58.817041 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57383us
I20260709 13:13:58.904557 31696 consensus_queue.cc:1243] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:13:58.942664 30553 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:58.883627 31949 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:58.943346 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:58.862594 32049 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:58.953110 31383 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:58.819721 31907 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:58.909626 30556 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:58.930826 32089 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:58.929499 31934 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:58.904106 32005 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:58.859122 32022 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:13:58.967963 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 660) took 1021 ms. Trace:
W20260709 13:13:58.985219 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:58.988888 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 170608us
W20260709 13:13:58.985311 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56498us
I20260709 13:13:58.999837 31963 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:58.965858 31950 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.963485 31389 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:58.966596 31937 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:13:59.019184 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75998.1us
I20260709 13:13:58.983697 30698 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:13:58.988369 31702 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:58.984205 32093 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:58.995880 31738 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:13:59.022071 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.027776 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:58.947404 31964 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:13:59.034452 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.028949 30628 rpcz_store.cc:276] 0709 13:13:57.946535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.974143 (+ 27608us) service_pool.cc:224] Handling call
0709 13:13:57.974545 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:57.975060 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:13:57.975096 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:57.976774 (+  1678us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.184637 (+207863us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b1800448300. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:13:58.323028 (+138391us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:58.323078 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:58.323163 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:58.851577 (+528414us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b24001629f8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:13:58.967506 (+115929us) raft_consensus.cc:1685] finished
0709 13:13:58.967544 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:58.967793 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6726,"spinlock_wait_cycles":73107072,"thread_start_us":137716,"threads_started":1,"wal-append.queue_time_us":651473}
I20260709 13:13:59.045849 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.023275 32084 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:59.022184 32100 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.014952 31991 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:13:59.026500 31992 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.035015 32022 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.055452 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:59.020005 31989 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:13:59.056310 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:59.077257 30705 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.056355 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.058341 30702 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:59.021534 31928 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
W20260709 13:13:59.079974 32099 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:13:59.053382 31949 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.049098 31738 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.054111 31673 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.080930 31738 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.081357 31673 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.083227 31989 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:59.083731 31989 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:59.083760 32050 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.053547 32049 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:13:59.084547 32049 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.084995 32049 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.084708 31989 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.085384 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.086127 31989 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.084156 31673 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:59.087167 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 388) took 1748 ms. Trace:
I20260709 13:13:59.040129 31963 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.087987 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:59.088127 32005 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.049216 31964 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.046481 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:59.061811 32100 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.036139 32089 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.067404 31991 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:59.069036 31985 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.071789 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.087277 31928 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:13:59.089789 32082 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:59.089635 32070 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.130926 31991 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:59.041868 31937 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:59.053143 32093 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.128831 32037 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:59.097725 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.103127 31712 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.104393 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.059985 30671 leader_election.cc:400] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
I20260709 13:13:59.163949 30671 leader_election.cc:403] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:59.164908 31367 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:59.089169 30701 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:13:59.112749 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66351.6us
I20260709 13:13:59.062464 32051 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:59.187098 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.136809 31928 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:59.032728 31936 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.033619 31934 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.128800 30705 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:59.141276 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.142199 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.200259 30558 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.075237 32084 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:59.147774 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.148259 31950 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.157022 31911 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:13:59.092307 32101 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:13:59.115095 31964 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.160946 31991 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.120394 30698 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:59.107074 32022 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:13:59.153803 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44874us
I20260709 13:13:59.189154 32037 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:59.210661 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.215003 31937 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.218382 31950 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.206986 30551 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:13:59.218904 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.219839 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.087875 30626 rpcz_store.cc:276] 0709 13:13:57.339128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:57.342449 (+  3321us) service_pool.cc:224] Handling call
0709 13:13:58.375759 (+1033310us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:58.382360 (+  6601us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.599677 (+217317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.599864 (+   187us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.603871 (+  4007us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.819069 (+215198us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:59.025949 (+206880us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.038430 (+ 12481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.050286 (+ 11856us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.054900 (+  4614us) raft_consensus.cc:1685] finished
0709 13:13:59.059357 (+  4457us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.087054 (+ 27697us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1724,"prepare.queue_time_us":141549,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":18275,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6561,"spinlock_wait_cycles":34901120,"thread_start_us":252215,"threads_started":2,"wal-append.queue_time_us":229486}
I20260709 13:13:59.230604 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:59.199069 31959 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:13:59.227872 31937 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.239529 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.240271 31928 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.241720 31928 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:13:59.232609 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.238196 30627 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:59.242954 32084 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.245905 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.233476 30558 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:13:59.240758 32022 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.238011 32037 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.251257 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.252110 31361 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:59.252588 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.253252 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.253417 31367 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:59.253950 31361 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:59.234300 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.255616 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.256364 30700 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:13:59.256726 32089 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.257211 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.258044 31388 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.215159 31911 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.200362 30624 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:13:59.278904 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.281629 31361 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:13:59.216400 32093 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.296310 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.297372 31383 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:13:59.300904 31911 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.301882 31991 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.203969 31963 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.305898 31963 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.306672 31383 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:59.288846 32049 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.289237 31959 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:13:59.289570 32050 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:59.296583 31389 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:59.254602 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.288411 30699 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:13:59.288123 32100 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.293912 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.329828 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.297997 31992 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 13:13:59.333884 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 288115us
W20260709 13:13:59.337985 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44199.3us
I20260709 13:13:59.338860 31294 leader_election.cc:400] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:59.339733 31294 leader_election.cc:403] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:13:59.340212 32100 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:13:59.326428 31911 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:59.342789 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:59.340741 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.333312 32050 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:59.336347 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.354105 31948 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:13:59.337051 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:13:59.360710 32112 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:13:59.370682 32049 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.388306 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.334542 32002 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.336623 32022 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:59.359848 31936 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:13:59.332254 31738 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.352821 31967 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:59.398525 30698 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 13:13:59.392247 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:13:59.403575 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.422035 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.370250 31964 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:13:59.397063 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58352us
I20260709 13:13:59.351226 31911 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.453133 30704 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:13:59.457842 31964 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.434145 31967 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:59.397079 32022 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.450989 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.442687 31364 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:59.491393 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 396) took 1219 ms. Trace:
I20260709 13:13:59.491981 30632 rpcz_store.cc:276] 0709 13:13:58.271822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.271972 (+   150us) service_pool.cc:224] Handling call
0709 13:13:58.272359 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:58.618481 (+346122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.491063 (+872582us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54004ab0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:59.491311 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792145280}
I20260709 13:13:59.460453 32070 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:59.492033 32001 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:13:59.500716 32001 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:13:59.502336 32118 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:13:59.503266 31738 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.504750 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.505182 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:13:59.505559 31394 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.506018 30699 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.482530 32050 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.506985 31378 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:13:59.503299 32001 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.510916 31967 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.515745 32022 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:59.496963 32100 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:59.516059 30699 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:13:59.521755 32070 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.522722 31911 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.523202 32003 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:13:59.524068 31394 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:13:59.493100 31367 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:59.535848 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:59.537063 31673 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:13:59.537660 31673 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:59.539085 31673 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.532912 31391 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:13:59.557475 30633 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:13:59.534726 32003 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:13:59.559851 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.560202 32003 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.561349 32003 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.561403 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:59.562718 32094 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.907s	user 0.000s	sys 0.007s
I20260709 13:13:59.560740 30551 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:13:59.563154 30381 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.569337 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.570068 30381 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:13:59.570941 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.574126 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.568958 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 725) took 1286 ms. Trace:
I20260709 13:13:59.579059 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.579900 30550 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:13:59.579954 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 434980us
I20260709 13:13:59.581035 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.581780 31695 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:13:59.583069 32011 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:59.583725 32011 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.577541 30706 rpcz_store.cc:276] 0709 13:13:58.282062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.282204 (+   142us) service_pool.cc:224] Handling call
0709 13:13:58.798477 (+516273us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:13:58.798732 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.022512 (+223780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.022548 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.022593 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:59.022667 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.568598 (+545931us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54009c8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:13:59.568684 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.568869 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516919936}
I20260709 13:13:59.584132 32051 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
W20260709 13:13:59.583868 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 525261us
I20260709 13:13:59.581789 32082 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:59.589074 32082 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.589097 31388 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:13:59.594317 30551 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:13:59.577301 30381 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 3, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:13:59.597036 30381 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:13:59.598299 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.599884 32100 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.600529 31934 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.602814 32070 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.604187 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.606115 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.606986 30633 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:59.607177 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.608343 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.610262 32014 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.615031 31907 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.610356 31936 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:13:59.619391 31936 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:59.620416 31936 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:13:59.623034 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.624122 30633 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:59.624688 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:59.611635 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:59.634282 30631 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:13:59.634071 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.635651 30633 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:13:59.635839 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.636502 30554 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.637470 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.638484 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.639329 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.640035 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.640266 30552 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.630563 30630 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:13:59.641527 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.663079 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.634481 31712 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:13:59.665316 30552 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:13:59.650527 31781 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:59.642890 31996 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:59.643009 32001 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:59.676201 30556 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:13:59.662781 32116 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.668704 31949 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.680030 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:59.680614 30554 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:59.667562 31712 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.682186 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.686156 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:13:59.686200 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.671710 31781 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.672348 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:13:59.688237 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.688154 30631 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:13:59.688870 32011 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.687489 31907 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:13:59.703471 31907 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:13:59.704572 31907 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.707420 32116 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:59.683347 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:13:59.690331 31673 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:59.694010 31781 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.687986 31712 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:13:59.687883 32091 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.732256 31856 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:59.714367 31673 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.736096 31673 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.735985 31856 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.740326 31856 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:13:59.742889 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.743930 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.714723 31999 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:59.751777 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.761528 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.753041 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.753624 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.754194 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.757085 30702 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:13:59.759382 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.760282 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.765223 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:13:59.765909 30633 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:13:59.766470 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.771646 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.729131 31738 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:13:59.773936 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:13:59.778074 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.785981 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:13:59.762401 32014 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:13:59.782840 31738 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.774091 32012 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.786963 32014 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.791626 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.795907 30633 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:13:59.784019 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.783351 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.786156 31999 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:13:59.813334 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.637s (SENT) [suppressed 8 similar messages]
I20260709 13:13:59.816684 31948 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:13:59.817355 31948 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:13:59.806694 31949 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:13:59.811519 31996 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:13:59.807819 32014 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:13:59.836570 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.818554 31948 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.855978 31996 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:13:59.857666 32091 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:13:59.832309 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.860857 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.861598 30550 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:13:59.862983 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:13:59.858597 31996 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:13:59.864403 31738 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:13:59.883592 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 663) took 1174 ms. Trace:
W20260709 13:13:59.887967 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 457575us
I20260709 13:13:59.884305 31383 rpcz_store.cc:276] 0709 13:13:58.709265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.307765 (+598500us) service_pool.cc:224] Handling call
0709 13:13:59.308216 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:59.308662 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:59.308699 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:59.309283 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:13:59.381214 (+ 71931us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:13:59.421487 (+ 40273us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.601891 (+180404us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18004a6c80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:13:59.882805 (+280914us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b18004a6c80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:13:59.883078 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.883200 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.883268 (+    68us) raft_consensus.cc:1685] finished
0709 13:13:59.883291 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.883499 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4511,"prepare.queue_time_us":43503,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":53,"raft.queue_time_us":9442,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":8741,"spinlock_wait_cycles":29473152,"thread_start_us":71374,"threads_started":1,"wal-append.queue_time_us":146636}
I20260709 13:13:59.870859 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:13:59.862541 31781 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.878559 31949 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:13:59.907820 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 299650us
I20260709 13:13:59.908929 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 522) took 1143 ms. Trace:
I20260709 13:13:59.910825 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.911192 32100 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:13:59.912237 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:13:59.913146 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.914731 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.907823 31714 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.915580 31907 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:13:59.915820 32102 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.662s	user 0.006s	sys 0.000s
I20260709 13:13:59.916102 31714 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:13:59.916515 32121 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:13:59.916851 31714 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:13:59.880069 31999 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.919391 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 654) took 1212 ms. Trace:
I20260709 13:13:59.919893 31396 rpcz_store.cc:276] 0709 13:13:58.706890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.707016 (+   126us) service_pool.cc:224] Handling call
0709 13:13:58.707428 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:13:58.707859 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:13:58.707889 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:13:58.708402 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 13:13:58.886493 (+178091us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b18002e9240. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:13:58.967198 (+ 80705us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:13:59.015536 (+ 48338us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.025093 (+  9557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.025189 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:13:59.253693 (+228504us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b18002e9240. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:13:59.915209 (+661516us) spinlock_profiling.cc:243] Waited 7.56 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:13:59.919097 (+  3888us) raft_consensus.cc:1685] finished
0709 13:13:59.919126 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.919316 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12090,"prepare.queue_time_us":87680,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":9328,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":13580,"spinlock_wait_cycles":41592320,"thread_start_us":128220,"threads_started":2,"wal-append.queue_time_us":350647}
I20260709 13:13:59.922674 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:13:59.923468 30550 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.918360 31714 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:13:59.909664 31389 rpcz_store.cc:276] 0709 13:13:58.765907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:59.345744 (+579837us) service_pool.cc:224] Handling call
0709 13:13:59.381673 (+ 35929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:13:59.397005 (+ 15332us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:13:59.461939 (+ 64934us) log.cc:844] Serialized 14 byte log entry
0709 13:13:59.604763 (+142824us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b1800241a80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:13:59.908307 (+303544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:13:59.908422 (+   115us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:13:59.908500 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:13:59.908630 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:13:59.908838 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":637972,"prepare.queue_time_us":33367,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":1301,"raft.queue_time_us":235594,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":143,"spinlock_wait_cycles":21614336,"thread_start_us":66341,"threads_started":1,"wal-append.queue_time_us":226209}
I20260709 13:13:59.927425 32100 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:13:59.917022 32121 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.916016 31991 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:13:59.941507 32121 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:13:59.954977 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.956158 30551 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:13:59.957341 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.903604 31781 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:13:59.958835 32082 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:13:59.959287 32082 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:13:59.964077 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:13:59.968428 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98351.4us
I20260709 13:13:59.970072 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.971462 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:13:59.973595 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 160306us
I20260709 13:13:59.974656 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.975775 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.976651 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:13:59.932006 31949 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:13:59.987481 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.981663 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.990708 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.991518 31383 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:13:59.992687 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:13:59.994571 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.986522 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:13:59.995474 31396 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.997931 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:13:59.998138 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:13:59.998508 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:13:59.999317 31394 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:00.000069 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:13:59.967044 32082 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:13:59.977943 31907 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:13:59.932212 31673 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:13:59.994125 31949 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:00.023458 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45030.2us
I20260709 13:14:00.025022 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.021080 31394 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:00.034255 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.035172 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.036435 30628 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:14:00.098454 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62813.9us
I20260709 13:13:59.992908 32014 raft_consensus.cc:2764] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:14:00.006946 31936 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:13:59.980310 31934 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:14:00.100150 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:00.103129 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.014915 32025 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:00.115091 31389 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:00.026942 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.022377 31991 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.113104 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:00.113829 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:13:59.986479 31712 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.121577 30704 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:00.123883 30556 raft_consensus.cc:2368] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:14:00.124665 31396 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:14:00.129148 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102294us
I20260709 13:14:00.138340 31391 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:00.095640 31964 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:00.140579 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.093111 32051 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.027264 31999 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.131366 31963 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:00.142292 31907 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:14:00.146273 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 656) took 1438 ms (client timeout 1637 ms). Trace:
W20260709 13:14:00.198814 31378 rpcz_store.cc:269] 0709 13:13:58.707603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:58.713371 (+  5768us) service_pool.cc:224] Handling call
0709 13:13:59.221377 (+508006us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:14:00.145999 (+924622us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540020d354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:00.146173 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590650112}
I20260709 13:14:00.142360 30699 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:14:00.200138 31907 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:00.147845 32136 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 13:14:00.148169 31738 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:00.150297 31712 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:00.207621 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70438.4us
I20260709 13:14:00.152568 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:00.210079 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.108896 30551 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:14:00.133020 31934 consensus_queue.cc:1243] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:00.180372 31391 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:00.182008 32125 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:00.184921 32121 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.191787 31388 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:00.197305 30550 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:00.198087 31948 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:00.200662 31907 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:00.150983 30702 raft_consensus.cc:2368] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7.
W20260709 13:14:00.293440 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 175042us
I20260709 13:14:00.294756 31294 leader_election.cc:400] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
W20260709 13:14:00.270519 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:00.275869 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.280421 30553 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:00.280591 30558 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:00.204510 31856 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:00.238528 31772 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:00.295228 31294 leader_election.cc:403] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:00.303035 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 303107us
I20260709 13:14:00.310115 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:00.247494 32125 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:00.320463 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:00.268895 31695 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:00.337028 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:00.337357 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:00.338055 30633 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.349843 30633 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:00.331540 32112 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.323657 31991 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:00.351262 30628 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:00.318055 31911 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:00.355103 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:00.370445 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47850.3us
I20260709 13:14:00.315435 31856 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:00.354043 31714 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:00.356328 31992 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:14:00.361974 32112 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:00.383941 32153 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.381052 32125 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.445245 32022 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:14:00.501770 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 329454us
I20260709 13:14:00.555282 31714 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:14:00.560739 30702 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:00.543823 32022 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:00.567642 32084 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:00.569072 32084 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:00.571162 32093 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:00.572464 32093 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:00.507318 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 170100us
W20260709 13:14:00.511569 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 270668us
W20260709 13:14:00.574512 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.583024 32025 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.584300 32025 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.586118 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:00.587847 31367 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:00.602492 31966 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:00.603749 31966 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.605659 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:00.608457 32082 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.582s	user 0.013s	sys 0.003s
I20260709 13:14:00.578537 31998 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:14:00.527760 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:00.400781 31963 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:00.541464 32012 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.449239 32121 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:00.438499 32112 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:00.447726 32125 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:00.706959 32012 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:00.385608 31673 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:00.388594 31934 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.711179 30552 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:00.715390 32158 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:00.433033 32143 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.321s	user 0.003s	sys 0.000s
I20260709 13:14:00.716671 32158 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:00.650079 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 332582us
W20260709 13:14:00.721560 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 203156us
I20260709 13:14:00.609205 31991 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:00.722873 31991 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:00.724574 31991 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.725558 32149 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:00.580817 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:00.577958 31781 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.566025 32022 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.541703 31985 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.137s
I20260709 13:14:00.691280 31364 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:00.609212 32082 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.694856 31856 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:00.748737 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:00.696848 30702 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:00.752082 32082 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.753865 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:00.726481 32149 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:00.756287 32149 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.761171 32022 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:00.727396 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:00.742240 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:00.738790 32025 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:00.726382 31991 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:00.737207 31966 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.737017 32084 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:00.764864 32093 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:00.764462 32012 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.781883 31985 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:00.732723 31673 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:14:00.796748 30630 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:00.811681 31781 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.816248 31391 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:00.807252 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 307628us
I20260709 13:14:00.818935 31394 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:14:00.822247 31394 pending_rounds.cc:85] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 1
I20260709 13:14:00.802774 31911 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
W20260709 13:14:00.825812 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 245387us
I20260709 13:14:00.826498 32150 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.807138 32093 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:00.740058 31963 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.829768 32158 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.822595 31394 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:14:00.839937 31394 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:00.803347 31966 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.841090 31966 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:00.844388 31966 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.830947 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:00.849437 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:00.850739 30628 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:14:00.851042 30550 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:00.857892 31378 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:00.862959 32159 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:00.864398 31673 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:00.864225 32159 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.871502 32150 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.872977 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:00.849402 32151 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:14:00.874207 30558 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:00.876652 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:00.821307 31781 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:00.886688 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.903285 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.887843 32159 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:00.918509 32165 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.088s	user 0.003s	sys 0.001s
I20260709 13:14:00.919807 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:00.920881 31391 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:00.938846 30628 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:00.940668 32159 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.941536 32159 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:00.946632 30552 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:00.923066 30550 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:00.914065 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:00.955266 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.955505 31856 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:00.957870 32150 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:00.957638 32096 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:00.958689 32096 consensus_queue.cc:1243] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:14:00.962929 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:00.968009 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.967970 31966 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:00.970209 31966 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:00.970718 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:00.972950 31966 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:00.973631 32012 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:14:00.973965 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 232026us
I20260709 13:14:00.974241 32012 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:00.974850 32012 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:00.976437 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:00.976840 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:00.978856 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90691.9us
W20260709 13:14:00.983173 31907 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.597s	user 0.022s	sys 0.000s
I20260709 13:14:00.983034 31781 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:00.983505 30630 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:00.983835 31907 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:00.993278 32158 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:00.993953 32125 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:00.994149 32158 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:00.995546 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:00.996505 31911 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:00.996965 32161 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:00.997542 32161 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:00.998139 32161 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:00.997484 31911 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.003679 31911 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:00.993338 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44839.6us
I20260709 13:14:01.008317 31292 leader_election.cc:400] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7.
I20260709 13:14:01.008713 31292 leader_election.cc:403] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:01.006798 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.018445 32125 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7.
I20260709 13:14:00.947191 32159 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:01.022797 31391 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:00.986536 31907 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:01.028505 32151 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:01.029578 32150 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:01.034883 31934 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:01.036103 31949 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.806s	user 0.017s	sys 0.007s
I20260709 13:14:01.036702 31949 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:00.997732 30558 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:14:01.044241 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.020309 32158 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:01.051481 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.057567 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:01.043815 32093 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:14:01.064863 32002 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:01.067621 31967 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:01.056164 31856 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.084407 31856 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.065094 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.080547 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:01.089740 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:01.125164 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44784.1us
I20260709 13:14:01.061911 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.128924 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:01.087687 32002 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:01.065615 30630 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:01.139887 31949 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:01.148921 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.152874 30558 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:14:01.129750 32025 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:01.143669 32002 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:01.142930 32082 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:01.179955 30628 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:01.180955 31391 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:01.187789 31856 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:01.142944 31378 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:01.151700 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:01.209960 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.217602 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.221230 30628 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:01.192461 31934 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.216671 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:01.234120 32169 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:14:01.235364 30555 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:01.236865 32168 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:14:01.237057 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:01.237721 32049 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:14:01.238951 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:01.239976 30554 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:01.243033 32155 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:01.243925 32155 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.220940 32151 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:01.245352 32151 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.188704 31906 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:14:01.248135 32151 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.189004 32082 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.250943 32091 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:01.251657 32091 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:01.253008 32082 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.252812 32091 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.219817 31911 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 13:14:01.254743 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.255600 30631 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:01.180294 32012 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.258810 31364 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:01.259409 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:01.261106 32159 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:01.261551 32159 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.261926 32159 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:01.262027 31963 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.262756 31963 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:01.263062 31934 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:01.234221 30550 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:01.265750 31934 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:01.265898 31963 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:01.258924 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:01.268137 32140 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:01.268817 32140 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.271935 31383 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:01.272394 32084 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:14:01.273077 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:01.275372 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.276183 30698 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:01.258253 32012 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:01.234213 30559 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:01.226653 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:01.277848 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:01.281538 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:01.283762 31394 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:01.281538 31391 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:01.240172 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.286420 30556 pending_rounds.cc:85] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 1
I20260709 13:14:01.287005 30556 pending_rounds.cc:107] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:14:01.283541 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:01.287237 30556 raft_consensus.cc:2889] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.298391 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:01.285974 31378 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:01.306993 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:01.272089 32096 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:14:01.289498 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.352180 31396 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:01.361912 30554 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:01.368556 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.278357 32155 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.323951 32161 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.350404 32150 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:01.393325 30698 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:01.394356 32147 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:01.395124 32147 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:01.396838 31367 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:01.383996 32150 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:01.397410 32011 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:14:01.398128 32150 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:01.398774 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.378238 30700 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:01.401099 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.404682 32161 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.414875 32140 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.415297 31388 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 13:14:01.419409 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:01.433452 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48010.2us
I20260709 13:14:01.440951 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:01.429412 32012 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:01.425028 32125 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.427153 31991 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.467005 31991 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:01.469556 31991 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:01.475308 32025 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.476101 31966 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.476229 32025 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.476927 31966 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.437539 31673 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.461997 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.465143 30703 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:14:01.483063 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:01.377868 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:01.466126 32125 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.511906 30699 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:01.519752 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.520680 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:01.515573 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:01.520004 30704 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:01.529373 30631 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:01.519403 31781 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.511117 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.535557 30698 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:01.521193 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:01.534710 30703 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:01.516817 31367 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:14:01.519295 31364 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:01.578528 31396 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:01.549793 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.551767 30558 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:01.553190 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:01.571414 30381 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 67) took 9644 ms. Trace:
I20260709 13:14:01.532553 31781 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:01.592788 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67294.3us
I20260709 13:14:01.594909 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:01.500084 31673 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.538502 31911 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:01.543527 31367 pending_rounds.cc:85] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:01.594720 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:01.605906 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.606184 32014 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:01.607858 30705 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:14:01.609344 32161 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.610075 32161 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:01.610894 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 80172.1us
I20260709 13:14:01.612306 32161 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:01.612469 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.659533 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.659658 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:01.661967 31391 raft_consensus.cc:2376] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3. Re-sending same reply.
I20260709 13:14:01.616291 31367 pending_rounds.cc:107] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:01.601169 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:01.669281 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.599208 32049 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:01.665571 32186 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:14:01.682957 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.682176 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:01.678905 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66947.3us
I20260709 13:14:01.684803 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:01.686434 32022 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:01.687003 32022 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:01.687438 32022 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:01.691269 31934 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:01.681584 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72597.3us
I20260709 13:14:01.696940 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:01.720093 32201 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.674364 31367 raft_consensus.cc:2889] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:01.652395 32159 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:01.742076 32201 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.744843 32159 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:01.677572 32014 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:01.747134 32014 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:01.714300 31966 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:01.764951 32200 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.765714 32200 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.766045 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:01.766390 32149 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:01.767033 32149 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:01.768316 32200 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:01.768028 32149 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:01.727406 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:01.768057 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:01.775457 32179 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:14:01.776988 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:01.778028 32012 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:01.778442 32200 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:01.780452 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:01.781565 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:01.797621 32012 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:01.783289 31391 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:01.775457 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:01.785454 32201 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:01.810249 30705 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:01.779711 32082 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.722574 31911 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:01.812165 32191 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 13:14:01.827039 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 232425us
I20260709 13:14:01.844097 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:01.838096 32147 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:01.704252 32049 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:01.782217 32188 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:14:01.891039 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 306509us
I20260709 13:14:01.859114 31991 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:01.893877 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:01.863591 32125 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:01.923826 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51509.1us
I20260709 13:14:01.911495 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:01.853013 32082 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:01.871531 32147 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.864562 32151 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 13:14:01.945113 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:01.946182 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 77602.5us
I20260709 13:14:01.961229 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:01.961928 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:01.933437 32201 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:01.968271 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:01.584525 30381 rpcz_store.cc:276] 0709 13:13:51.945300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:52.042570 (+ 97270us) service_pool.cc:224] Handling call
0709 13:14:01.571095 (+9528525us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:13:59.722979 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:13:59.723855 (+   876us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:13:59.723906 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:13:59.723935 (+    29us) tablet.cc:662] Decoding operations
0709 13:13:59.731147 (+  7212us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:13:59.731274 (+   127us) write_op.cc:641] Partition lock acquired for write op
0709 13:13:59.731318 (+    44us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:13:59.731877 (+   559us) tablet.cc:701] Row locks acquired
0709 13:13:59.731896 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:13:59.732064 (+   168us) write_op.cc:270] Start()
0709 13:13:59.732259 (+   195us) write_op.cc:276] Timestamp: P: 1783602839732012 usec, L: 0
0709 13:13:59.732292 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:13:59.781665 (+ 49373us) log.cc:844] Serialized 1693 byte log entry
0709 13:14:00.450382 (+668717us) op_driver.cc:464] REPLICATION: finished
0709 13:14:00.454440 (+  4058us) write_op.cc:301] APPLY: starting
0709 13:14:00.454592 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:00.454839 (+   247us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:00.687324 (+232485us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:00.950003 (+262679us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:00.959424 (+  9421us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:14:00.959515 (+    91us) write_op.cc:312] APPLY: finished
0709 13:14:00.960069 (+   554us) log.cc:844] Serialized 40 byte log entry
0709 13:14:00.960356 (+   287us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:00.960579 (+   223us) write_op.cc:454] Released schema lock
0709 13:14:00.960653 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:00.961281 (+   628us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:00.961315 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":111761,"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":3624,"apply.run_cpu_time_us":5576,"apply.run_wall_time_us":506857,"num_ops":4,"prepare.queue_time_us":121172,"prepare.run_cpu_time_us":52102,"prepare.run_wall_time_us":153623,"raft.queue_time_us":248158,"raft.run_cpu_time_us":3316,"raft.run_wall_time_us":3654,"replication_time_us":717863,"thread_start_us":427923,"threads_started":4,"wal-append.queue_time_us":185266}]]}
I20260709 13:14:01.991307 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:01.941185 31991 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:01.971150 32201 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.004380 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:01.976863 31906 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:01.977818 32197 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:01.983318 32155 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:01.957269 32200 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:01.954171 32147 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:02.003424 32002 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.817s	user 0.017s	sys 0.000s
I20260709 13:14:01.988433 32199 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:02.018270 32201 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.020931 31781 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.085441 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.109948 31391 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:02.072732 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:02.144052 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 330880us
W20260709 13:14:02.091797 32172 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:14:02.040587 31911 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.084182 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:02.091095 32192 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.326s	user 0.004s	sys 0.000s
I20260709 13:14:02.090900 32199 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.038888 31673 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.037988 32198 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:02.091643 32185 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.214s	user 0.001s	sys 0.001s
I20260709 13:14:02.091337 32121 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.037927 32206 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:02.086715 32147 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.156533 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:02.149511 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76917.4us
I20260709 13:14:02.049109 32155 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:02.091977 32204 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:02.158926 32014 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:02.109514 32149 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:02.037518 32197 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.055516 32002 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.089730 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:02.178628 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 178991us
I20260709 13:14:02.179145 32206 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.182288 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.164947 32189 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:02.192814 32189 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:02.193470 32189 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.195358 32125 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.207523 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.208495 30555 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:02.213088 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.217396 30632 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:02.238884 30554 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:02.220115 32207 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.255s
I20260709 13:14:02.171409 32121 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.241544 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:02.211202 32093 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.266785 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 467755us
I20260709 13:14:02.267428 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:02.212430 32203 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:02.240362 32208 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:14:02.284221 32194 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.338s	user 0.002s	sys 0.000s
I20260709 13:14:02.285458 32197 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.286115 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 683) took 1036 ms. Trace:
W20260709 13:14:02.287644 32205 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:14:02.286721 30556 rpcz_store.cc:276] 0709 13:14:01.249861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.278514 (+ 28653us) service_pool.cc:224] Handling call
0709 13:14:01.286181 (+  7667us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:01.539627 (+253446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.539663 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:01.549007 (+  9344us) log.cc:844] Serialized 52 byte log entry
0709 13:14:01.741710 (+192703us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:01.741754 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.741833 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.285776 (+543943us) raft_consensus.cc:1685] finished
0709 13:14:02.285804 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.286027 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9584,"raft.queue_time_us":6492,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":17497,"thread_start_us":390346,"threads_started":2,"wal-append.queue_time_us":303036}
I20260709 13:14:02.168839 32199 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:02.290796 32211 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.317s	user 0.001s	sys 0.000s
I20260709 13:14:02.290985 32199 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.272805 32093 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:02.294909 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48150.4us
I20260709 13:14:02.295713 32002 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.206856 32204 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.303882 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.305652 30558 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:02.302380 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.271332 32200 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:02.297427 32221 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:14:02.314317 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.321185 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.322701 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.298767 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.325587 32206 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.277930 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:02.329290 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.330235 30553 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:02.330381 32204 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.331808 32155 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.335052 30701 raft_consensus.cc:606] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.298121 32121 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.338384 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:02.340466 32183 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.731s	user 0.002s	sys 0.000s
I20260709 13:14:02.342236 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 553) took 1483 ms. Trace:
I20260709 13:14:02.242820 31781 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:14:02.318709 32229 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:02.309056 32150 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.005s
I20260709 13:14:02.348068 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.280337 32082 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.298535 32125 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:02.299067 31991 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.308477 32093 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.366165 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:02.363505 30705 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.546s	user 0.014s	sys 0.000s
I20260709 13:14:02.373096 30705 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:02.367304 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.366514 32125 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.380920 31991 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.376338 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.381366 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:02.382159 30553 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
W20260709 13:14:02.383090 32022 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.004s
W20260709 13:14:02.383661 32218 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:14:02.298309 32147 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:14:02.342723 31361 rpcz_store.cc:276] 0709 13:14:00.858565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.886578 (+ 28013us) service_pool.cc:224] Handling call
0709 13:14:00.902843 (+ 16265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:00.982763 (+ 79920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:00.982808 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.195380 (+212572us) spinlock_profiling.cc:243] Waited 6.2 ms on lock 0x7b2c00056c38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:01.228984 (+ 33604us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.429113 (+200129us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:01.429177 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.429411 (+   234us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.339539 (+910128us) spinlock_profiling.cc:243] Waited 6.32 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:02.341781 (+  2242us) raft_consensus.cc:1685] finished
0709 13:14:02.341810 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.341993 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4120,"raft.queue_time_us":45229,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":131,"spinlock_wait_cycles":28788736,"thread_start_us":186174,"threads_started":1,"wal-append.queue_time_us":301178}
I20260709 13:14:02.381117 32182 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:02.296535 32012 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.396414 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.308197 32202 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.103s
I20260709 13:14:02.363171 31966 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:02.411278 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.014s	sys 0.026s
I20260709 13:14:02.365123 32121 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.395363 32149 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:02.380975 32125 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:02.422053 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.015s	sys 0.026s
I20260709 13:14:02.374217 32002 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:02.363991 32150 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.406507 32093 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.413976 31781 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:02.378700 32140 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.051s
W20260709 13:14:02.448058 31394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 529) took 1338 ms (client timeout 1729 ms). Trace:
I20260709 13:14:02.383699 32022 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:02.444047 31388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.040s
I20260709 13:14:02.500705 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:02.459398 31394 rpcz_store.cc:269] 0709 13:14:01.094450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.186260 (+ 91810us) service_pool.cc:224] Handling call
0709 13:14:02.425439 (+1239179us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b540020d354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:02.432724 (+  7285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":134248704}
W20260709 13:14:02.506088 31388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.008s	sys 0.040s
I20260709 13:14:02.431910 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.433499 32149 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:02.443225 31966 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.458290 31781 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.394359 32229 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.503919 32204 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:02.529269 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 118759us
I20260709 13:14:02.503144 32206 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:02.453084 32002 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.534943 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.535737 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:02.520776 31364 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:02.522223 31781 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.534297 31378 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:02.458964 32093 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.565583 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:02.569262 30632 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:02.570811 32229 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.556931 32206 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.542409 32204 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.576105 32204 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.559523 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:02.538743 32201 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.555091 32210 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.578821 32140 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:02.602933 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:02.616441 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:02.590008 31967 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:02.602030 32241 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:02.630899 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:02.622742 32147 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:02.620529 32246 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:02.601498 32125 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.602267 32236 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:14:02.628036 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 686) took 1153 ms. Trace:
I20260709 13:14:02.641049 32248 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:02.684311 32248 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.684839 32248 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:02.657655 31391 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:02.686889 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.688360 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.659381 32238 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:02.690449 32238 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.690929 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:02.660370 32239 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 13:14:02.647948 32246 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:02.692754 32246 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.693540 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 146) took 1350 ms. Trace:
W20260709 13:14:02.659114 32225 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.154s	user 0.001s	sys 0.001s
I20260709 13:14:02.695178 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 537) took 1088 ms. Trace:
I20260709 13:14:02.694023 30704 rpcz_store.cc:276] 0709 13:14:01.342903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.529022 (+186119us) service_pool.cc:224] Handling call
0709 13:14:01.576270 (+ 47248us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.676691 (+100421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.676737 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.781192 (+104455us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.006079 (+224887us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:02.006128 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.006213 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.285238 (+279025us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b4800550458. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:14:02.692486 (+407248us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b2400329d38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2efb0 00007f1806aa5488 00007f1806aa5be8 00007f1806a7d398 00007f1806a7876a 00007f1806a77bb6
0709 13:14:02.693264 (+   778us) raft_consensus.cc:1685] finished
0709 13:14:02.693290 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.693450 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60167,"raft.queue_time_us":77947,"raft.run_cpu_time_us":4733,"raft.run_wall_time_us":610883,"spinlock_wait_cycles":153584256,"thread_start_us":144900,"threads_started":1,"wal-append.queue_time_us":327785}
I20260709 13:14:02.695534 31396 rpcz_store.cc:276] 0709 13:14:01.606883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.842090 (+235207us) service_pool.cc:224] Handling call
0709 13:14:01.842537 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.942333 (+ 99796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.942442 (+   109us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.967595 (+ 25153us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.182555 (+214960us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:02.182677 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.182758 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.694962 (+512204us) raft_consensus.cc:1685] finished
0709 13:14:02.694987 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.695116 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19477,"raft.queue_time_us":244586,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":235257,"threads_started":2,"wal-append.queue_time_us":411257}
I20260709 13:14:02.663048 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.665392 31966 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.645526 31967 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:02.667187 32184 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.038s	user 0.004s	sys 0.001s
I20260709 13:14:02.632115 32149 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.668273 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.671401 32002 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.674109 32216 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:02.675599 32188 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:02.676944 32201 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.679345 32093 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.651221 32121 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:02.723152 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.654621 32150 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.646970 32147 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:02.724643 32147 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:02.684381 32195 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.916s	user 0.003s	sys 0.000s
I20260709 13:14:02.692059 32200 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:02.703989 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:02.666463 32049 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.659207 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.723282 32121 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.727126 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:02.728036 31396 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:02.729235 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:02.639528 32241 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.757917 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 531) took 1614 ms. Trace:
I20260709 13:14:02.759686 32219 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:02.758642 31389 rpcz_store.cc:276] 0709 13:14:01.143618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.247777 (+104159us) service_pool.cc:224] Handling call
0709 13:14:01.248197 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.248725 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.248762 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.249414 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.508171 (+258757us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:01.527513 (+ 19342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.576227 (+ 48714us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.527966 (+951739us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:02.757497 (+229531us) raft_consensus.cc:1685] finished
0709 13:14:02.757527 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.757836 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12170,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"spinlock_wait_cycles":113743104,"thread_start_us":252880,"threads_started":1,"wal-append.queue_time_us":334306}
I20260709 13:14:02.730890 32188 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.761370 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.762372 30555 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:02.720950 32216 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:02.773468 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:02.777385 32249 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:14:02.778211 32121 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.778461 31907 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:02.779196 31907 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.779608 32216 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:02.781126 32216 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:02.781381 32241 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.781569 31907 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.721324 32182 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
W20260709 13:14:02.784709 32181 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.133s	user 0.006s	sys 0.006s
I20260709 13:14:02.787465 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 691) took 1366 ms. Trace:
I20260709 13:14:02.787839 32022 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.788364 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.790117 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.745333 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:02.791047 30700 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:02.651597 31367 rpcz_store.cc:276] 0709 13:14:01.474849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.753227 (+278378us) service_pool.cc:224] Handling call
0709 13:14:01.753668 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.871593 (+117925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.876374 (+  4781us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.990418 (+114044us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.424205 (+433787us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:02.570385 (+146180us) log.cc:844] Serialized 14 byte log entry
0709 13:14:02.597736 (+ 27351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.604985 (+  7249us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.611342 (+  6357us) raft_consensus.cc:1685] finished
0709 13:14:02.614162 (+  2820us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.624534 (+ 10372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118152,"prepare.queue_time_us":170822,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":60,"raft.queue_time_us":324226,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":23964,"thread_start_us":266030,"threads_started":2,"wal-append.queue_time_us":498495}
I20260709 13:14:02.795122 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:02.795840 32189 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.602s	user 0.018s	sys 0.001s
I20260709 13:14:02.796311 32189 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.796691 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:02.797449 30698 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:02.801376 32238 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.801715 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:02.802601 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:02.803128 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 210) took 1604 ms. Trace:
I20260709 13:14:02.803422 30555 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
I20260709 13:14:02.804512 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.803558 30557 rpcz_store.cc:276] 0709 13:14:01.198354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.234254 (+ 35900us) service_pool.cc:224] Handling call
0709 13:14:01.235004 (+   750us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:01.235556 (+   552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.235589 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:01.236404 (+   815us) log.cc:844] Serialized 26 byte log entry
0709 13:14:01.427037 (+190633us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:01.427093 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:01.427188 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.783691 (+1356503us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:02.802701 (+ 19010us) raft_consensus.cc:1685] finished
0709 13:14:02.802726 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.802949 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6083,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":140,"spinlock_wait_cycles":16341248,"thread_start_us":185314,"threads_started":1,"wal-append.queue_time_us":288972}
I20260709 13:14:02.805130 31394 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:02.805441 31391 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:14:02.805855 31391 pending_rounds.cc:85] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:02.806089 31391 pending_rounds.cc:107] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:14:02.806318 31391 raft_consensus.cc:2889] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:02.723032 31967 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.788002 30554 rpcz_store.cc:276] 0709 13:14:01.420996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.421331 (+   335us) service_pool.cc:224] Handling call
0709 13:14:01.438863 (+ 17532us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.239524 (+800661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.787153 (+547629us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54002d0ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:02.787367 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839199616}
W20260709 13:14:02.809916 32233 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
I20260709 13:14:02.802451 30702 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:02.815860 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 677) took 1403 ms. Trace:
I20260709 13:14:02.830578 32125 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:02.832042 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.840237 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.852706 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.853338 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:02.854240 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.852874 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.854864 32189 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:02.860280 30673 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
I20260709 13:14:02.862030 30673 leader_election.cc:403] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:14:02.860971 31394 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:02.867347 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.871567 31383 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:02.872371 30558 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:02.876359 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:02.861799 32198 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.684s	user 0.015s	sys 0.000s
I20260709 13:14:02.879617 32198 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.884093 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:02.885015 31367 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:02.886196 32125 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:02.887015 30628 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:02.887099 32125 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.816444 31378 rpcz_store.cc:276] 0709 13:14:01.412117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.422468 (+ 10351us) service_pool.cc:224] Handling call
0709 13:14:02.431750 (+1009282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.559463 (+127713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.725945 (+166482us) spinlock_profiling.cc:243] Waited 71.1 ms on lock 0x7b540020d354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:02.815632 (+ 89687us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b540020d350. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:02.815789 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338488064}
I20260709 13:14:02.855526 31781 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.900628 32248 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.902173 30631 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:02.903399 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 702) took 1135 ms. Trace:
I20260709 13:14:02.904754 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.905133 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.861052 32049 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:02.873692 32155 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:14:02.907009 32155 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:02.907377 32155 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:02.908322 32155 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.909823 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:02.917912 32241 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:02.930452 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 167325us
I20260709 13:14:02.930932 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.931777 30558 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:02.909682 32155 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:02.936667 30702 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:02.859212 32147 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:02.890406 32125 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:02.947669 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.952751 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:02.910852 32188 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:02.955133 31367 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:02.943413 32158 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5.
I20260709 13:14:02.958060 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:02.955128 31394 raft_consensus.cc:2410] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:02.962672 30555 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:02.918704 32241 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.971846 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:02.977931 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:02.966199 32189 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:02.978973 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:02.979256 32189 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:02.979677 30555 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:02.980458 32189 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:02.983611 30699 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:02.978250 32257 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:14:02.990983 32201 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:02.939275 32049 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:14:02.994477 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:02.998260 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66269.1us
I20260709 13:14:02.993279 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:03.069634 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68658us
W20260709 13:14:03.064905 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 70568.8us
I20260709 13:14:03.068445 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:03.070537 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.071146 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:03.071913 32238 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:03.072817 32260 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:14:03.073310 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:03.076671 32188 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:02.903946 30630 rpcz_store.cc:276] 0709 13:14:01.768305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.777231 (+  8926us) service_pool.cc:224] Handling call
0709 13:14:01.777703 (+   472us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:01.778236 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:01.778271 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:01.779061 (+   790us) log.cc:844] Serialized 52 byte log entry
0709 13:14:02.319583 (+540522us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:02.319651 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.319733 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:02.809158 (+489425us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:02.875255 (+ 66097us) raft_consensus.cc:1685] finished
0709 13:14:02.902924 (+ 27669us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:02.903257 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31798,"raft.queue_time_us":15180,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":10988,"spinlock_wait_cycles":11885440,"thread_start_us":499657,"threads_started":1,"wal-append.queue_time_us":667390}
I20260709 13:14:03.078852 30627 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:03.080166 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.081709 32241 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.083684 32093 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:03.084182 32093 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:03.085053 32093 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.088588 32246 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.063354 31781 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.071647 32025 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:03.091591 31394 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:03.080304 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:03.093220 32147 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.093900 32147 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:03.069435 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.094822 32236 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:03.095413 32236 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.095999 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.097714 30626 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:03.097949 32261 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.080269 32248 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.098894 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.098641 32261 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.099646 30632 raft_consensus.cc:2368] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:03.100915 31907 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:03.101233 32261 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.069694 30700 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:03.103590 31907 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:03.104988 31364 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:03.105842 32206 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.108088 30552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:03.108814 30552 raft_consensus.cc:606] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:03.109177 30552 raft_consensus.cc:614] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:03.076483 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.110095 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:03.119818 30555 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:03.124363 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.097532 32149 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:03.144690 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 273507us
I20260709 13:14:03.119994 32238 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:03.094385 32147 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.093308 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:03.152871 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:03.097601 32236 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:03.119994 32091 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:03.120113 32206 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:03.143013 32022 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:14:03.146149 32203 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:03.148334 32255 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 13:14:03.162092 30624 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:03.148703 32238 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.162232 32206 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:03.160198 32091 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:03.159690 30630 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:14:03.189695 32246 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.197194 32238 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:03.282279 30630 pending_rounds.cc:85] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 1
I20260709 13:14:03.283828 32091 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.278266 32206 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:03.407328 32267 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 13:14:03.397634 31673 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:03.409549 31673 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.413938 32125 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.389596 30630 pending_rounds.cc:107] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:14:03.338212 32241 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.339879 32188 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:03.441996 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:03.446756 30631 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:03.446331 32241 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:03.458164 32022 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.458923 32022 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.459302 32241 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:03.461231 32022 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.437508 30630 raft_consensus.cc:2889] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:03.403128 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:03.446960 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:03.465301 32269 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:14:03.414667 32125 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.529273 32125 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.439560 32188 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.532059 32188 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:03.516709 32265 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
I20260709 13:14:03.605988 31673 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:03.622023 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 155) took 1340 ms. Trace:
I20260709 13:14:03.622956 32125 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.623594 32125 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.625986 32125 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.622674 30703 rpcz_store.cc:276] 0709 13:14:02.281606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.281768 (+   162us) service_pool.cc:224] Handling call
0709 13:14:02.315637 (+ 33869us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.361740 (+ 46103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.361793 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:02.406578 (+ 44785us) log.cc:844] Serialized 26 byte log entry
0709 13:14:02.607691 (+201113us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:02.607762 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:02.607881 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:03.618947 (+1011066us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b2400329ca8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2efb0 00007f1806aa5488 00007f1806aa5be8 00007f1806a7d398 00007f1806a7876a 00007f1806a77bb6
0709 13:14:03.621631 (+  2684us) raft_consensus.cc:1685] finished
0709 13:14:03.621655 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:03.621943 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25200,"raft.queue_time_us":42610,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":4059,"spinlock_wait_cycles":97794432,"thread_start_us":143243,"threads_started":1,"wal-append.queue_time_us":221732}
I20260709 13:14:03.630147 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.622685 32091 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.634092 32091 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.651901 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:03.653012 31383 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.659695 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:03.676131 32093 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:03.679396 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:03.683743 32012 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:03.610451 32241 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.704267 32241 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.639468 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:03.715891 32147 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:03.736119 32149 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.002s
I20260709 13:14:03.798108 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 539) took 1158 ms. Trace:
W20260709 13:14:03.739554 32155 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.003s
I20260709 13:14:03.747102 32241 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.760637 30550 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:03.763497 30553 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:03.766117 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:03.767693 30558 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.768110 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:03.768392 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:03.768602 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:03.769121 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:03.770002 31907 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.771114 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:03.776281 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 146244us
W20260709 13:14:03.837363 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.037s	sys 0.060s
W20260709 13:14:03.781424 30671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.044s	sys 0.016s
I20260709 13:14:03.774948 31949 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.840741 30556 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:03.836278 30552 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:14:03.793473 31383 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:03.812628 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:03.792128 32207 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:03.776773 32121 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:03.797497 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 215) took 1818 ms. Trace:
I20260709 13:14:03.798578 31391 rpcz_store.cc:276] 0709 13:14:02.639610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.689891 (+ 50281us) service_pool.cc:224] Handling call
0709 13:14:02.705768 (+ 15877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.806825 (+101057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.797872 (+991047us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b5400522954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:03.798049 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228302592}
W20260709 13:14:03.796033 32219 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.006s	sys 0.053s
I20260709 13:14:03.828236 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:03.835454 30559 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:14:03.837834 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.779570 31396 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:14:03.838945 30671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.044s	sys 0.016s
I20260709 13:14:03.846300 30628 rpcz_store.cc:276] 0709 13:14:01.978992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.979187 (+   195us) service_pool.cc:224] Handling call
0709 13:14:01.996768 (+ 17581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:02.887556 (+890788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:03.797240 (+909684us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54001ea854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:03.797434 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046497024}
W20260709 13:14:03.839534 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.038s	sys 0.060s
I20260709 13:14:03.845043 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:03.858493 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.852982 30554 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:03.855389 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 746985us
I20260709 13:14:03.793253 32198 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:14:03.827001 32261 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.815778 32082 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:03.829504 32084 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:03.830816 32246 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.861963 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:03.872916 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.873301 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:03.851490 32219 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.006s	sys 0.054s
I20260709 13:14:03.888819 32246 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.890801 31389 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:03.871062 32238 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:03.852555 31949 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:03.890324 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:03.894718 31949 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.895377 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.862783 32096 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:03.855423 32207 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:03.897872 30523 leader_election.cc:400] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:03.898204 30523 leader_election.cc:403] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:03.898512 32207 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.898837 32209 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:03.876638 31673 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:03.894722 32255 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:03.899912 31673 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.876012 32266 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 13:14:03.905426 31394 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:03.908099 30559 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:03.851828 32012 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:03.886684 32082 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.830255 32262 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:14:03.838905 32241 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:03.957264 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.867556 32201 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:03.909451 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 3, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:03.863309 32182 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 13:14:03.965849 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:03.893632 32238 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:03.869789 32282 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:03.850916 32121 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.878739 31934 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:03.915403 31966 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:03.967432 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:03.903321 31673 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:03.852422 31907 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:03.967432 30705 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 1.
I20260709 13:14:03.986768 30382 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 4, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:03.988487 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:03.992884 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:03.993773 30707 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:03.997309 32255 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:04.001533 32121 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.001905 31907 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:03.920874 32084 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:04.003450 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 201302us
I20260709 13:14:03.964713 31856 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:03.882045 32261 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:03.961922 32241 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:03.970521 32201 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:03.966696 32262 consensus_queue.cc:1243] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:14:04.135416 32206 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.719s	user 0.013s	sys 0.003s
I20260709 13:14:04.138266 30382 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:04.141281 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
W20260709 13:14:04.141348 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 274510us
I20260709 13:14:04.144224 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.148650 30707 raft_consensus.cc:606] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.145541 32206 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:04.157719 32206 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:04.034520 32147 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.116s
I20260709 13:14:04.161720 32201 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:04.170540 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.174424 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.176909 30558 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:04.167033 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.179136 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:04.179719 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 303102us
I20260709 13:14:04.180565 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:04.171011 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.171236 32084 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:04.172123 30550 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:04.173344 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.183627 30552 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:04.187278 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 167241us
I20260709 13:14:04.193140 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.199126 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.200686 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.205291 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.208403 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.208879 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.209141 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.216387 32202 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.217003 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:04.217492 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:04.217061 32202 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.218561 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.218956 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:04.219611 30632 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:04.219923 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.220346 32202 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.221081 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.222211 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.222961 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:04.223610 30629 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:14:04.225816 32297 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.226020 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:04.226495 32297 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.226949 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:04.228572 32261 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.228952 32297 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.232105 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:04.232919 32246 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.189499 32084 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 13:14:04.213300 32271 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:14:04.210824 32182 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:04.207298 32289 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.232023 32082 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:04.250351 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:04.297910 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47735.7us
W20260709 13:14:04.294467 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 85399us
I20260709 13:14:04.299007 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.296661 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:04.306546 30701 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.307461 30625 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:04.292694 32271 consensus_queue.cc:1243] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:04.311249 32298 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:04.312529 32295 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 13:14:04.312112 32298 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:04.303319 31907 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:04.315464 32291 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:04.210583 31673 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:14:04.316452 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78978.9us
I20260709 13:14:04.318113 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.315785 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.325656 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.319470 32255 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.316013 32291 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:04.327139 32255 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:04.295929 32246 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.037s
I20260709 13:14:04.328074 32291 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.329456 32291 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.323190 32289 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.329684 32255 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.322427 32202 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:14:04.323894 32271 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:14:04.319733 32147 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:04.335152 30704 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:04.337240 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.349565 32084 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:04.345168 32262 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:04.344652 32203 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:14:04.355839 32093 raft_consensus.cc:993] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:14:04.350251 32084 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:04.334522 32241 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.356709 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:14:04.345033 32082 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:14:04.365972 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:04.347261 32297 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.367460 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:04.375641 32202 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:04.376474 32202 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:04.377955 30629 pending_rounds.cc:85] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 2
I20260709 13:14:04.377713 32202 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:04.377419 32084 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.378374 30629 pending_rounds.cc:107] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:04.346276 32246 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:04.381937 30629 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:04.390791 32084 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:04.394598 32014 raft_consensus.cc:1081] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Signalling peer acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:14:04.395267 32298 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.374493 32147 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:14:04.437878 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56542.4us
I20260709 13:14:04.434175 30632 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:04.446319 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.448904 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.453437 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.453641 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:04.383481 32246 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.446152 32093 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:04.472609 32203 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:04.465270 32246 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:04.480669 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.482721 32198 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:04.474035 31966 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:04.489318 32198 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:04.489550 31966 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:04.398212 32297 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.490403 32198 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.490831 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.491909 32198 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.492794 32297 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.495157 32093 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.496080 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.496845 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.497519 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.497722 32093 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:04.498118 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.499265 31966 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:04.503170 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.504734 30698 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:04.492947 32201 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:04.475062 30632 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:46186
I20260709 13:14:04.463749 32298 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:04.492368 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:04.492113 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.511392 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.511467 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.545998 30698 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:04.550402 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.548298 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:04.553053 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.484786 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.503746 32203 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:04.521970 32298 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:04.592226 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 78961.5us
W20260709 13:14:04.595434 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.568966 31856 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:04.594241 32298 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:04.640223 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 62756.3us
I20260709 13:14:04.570583 32294 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:14:04.602240 31856 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.651693 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:04.653438 31856 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:04.662405 31673 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:04.663394 31949 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:04.721048 32306 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:14:04.747802 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.727239 30701 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:14:04.759943 32305 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
I20260709 13:14:04.751089 32307 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:04.774955 32320 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:04.776772 32312 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:14:04.780230 32310 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:14:04.781544 31673 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:04.726307 31949 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:04.807117 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47628.6us
I20260709 13:14:04.804409 32307 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:04.813035 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.820878 30705 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:04.809892 31673 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:04.825955 30703 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:04.825259 31949 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.844820 31856 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:04.849412 31949 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:04.825294 32307 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.849535 32291 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:14:04.850891 31856 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.852910 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:04.862650 30704 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:14:04.863940 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:04.864593 31673 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.868175 32284 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:04.883044 32284 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:04.873854 32155 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:04.885397 32155 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:04.886472 32284 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.892416 31673 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:04.880756 32014 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.896528 32014 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:04.873831 32268 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:04.899672 32268 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:04.915313 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.915127 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:04.923259 32155 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:04.926887 32140 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:04.927702 32140 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:04.928969 32140 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:04.880240 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:04.939291 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.940691 30557 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:04.938252 32277 raft_consensus.cc:993] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower acf852f85f47483d99cbe4020baee627 to start an election
W20260709 13:14:04.960347 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45165.9us
I20260709 13:14:04.963562 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:04.966351 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:04.969169 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:04.973547 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.970712 30557 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:14:05.007350 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43894.5us
I20260709 13:14:05.010103 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:04.966058 32155 raft_consensus.cc:1081] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Signalling peer acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:14:04.964709 32140 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:04.959914 32207 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:05.028671 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 884478us
I20260709 13:14:04.959570 32093 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:04.969645 32307 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:04.964501 32268 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.052920 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.005920 32315 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:04.970305 32255 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 13:14:05.070168 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:05.023938 32219 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.028268 32282 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:05.032998 32206 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:05.072038 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.051401 32121 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.081063 32121 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.039074 31949 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.051754 32125 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:05.039793 32304 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.035485 32325 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:05.064199 32014 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.054570 32293 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.054385 32241 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.035s
W20260709 13:14:05.093921 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119433us
W20260709 13:14:05.095338 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 54463.6us
I20260709 13:14:05.072332 30627 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:05.075879 32315 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:05.096848 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56012us
I20260709 13:14:05.110018 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.063134 32093 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:05.093238 32246 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.544s	user 0.010s	sys 0.006s
I20260709 13:14:05.053613 32207 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.110971 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.108202 32022 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:05.077919 32219 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.173314 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.091732 32125 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:05.122973 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:05.124305 32340 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:05.121498 31673 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:05.114689 32206 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:05.113971 32304 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.137781 32289 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.138811 32207 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.119206 32246 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.082439 32282 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.140358 32121 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.141495 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 696) took 2724 ms. Trace:
I20260709 13:14:05.199891 32340 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:14:05.205312 32219 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.206262 32304 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.196161 32290 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.206837 32330 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:05.207477 32290 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.209990 32282 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.143553 32203 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.148635 30702 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:05.114008 32293 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.150079 32286 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.152822 32093 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.155813 32315 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.220003 30625 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:05.221261 32246 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.221946 30632 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:05.222136 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:05.222424 30632 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:05.227126 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.228070 32292 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:05.228606 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.228787 32292 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.204435 31388 rpcz_store.cc:276] 0709 13:14:02.397173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:02.527452 (+130279us) service_pool.cc:224] Handling call
0709 13:14:02.585789 (+ 58337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:02.614985 (+ 29196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:02.617804 (+  2819us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:03.666296 (+1048492us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b2c00056c38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:03.667077 (+   781us) log.cc:844] Serialized 26 byte log entry
0709 13:14:03.906782 (+239705us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:03.907345 (+   563us) log.cc:844] Serialized 14 byte log entry
0709 13:14:04.860819 (+953474us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b18001ae780. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:04.944167 (+ 83348us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18001ae780. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:14:05.090525 (+146358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.108570 (+ 18045us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.112333 (+  3763us) raft_consensus.cc:1685] finished
0709 13:14:05.113646 (+  1313us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.117887 (+  4241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58454,"prepare.queue_time_us":4124,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1099829,"raft.run_cpu_time_us":2892,"raft.run_wall_time_us":191535,"spinlock_wait_cycles":84233216,"thread_start_us":334214,"threads_started":2,"wal-append.queue_time_us":296915,"wal-append.run_wall_time_us":1038146}
I20260709 13:14:05.171972 30558 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:05.094923 31949 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.176638 32336 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:05.236692 30632 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.198818 30633 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:05.193996 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:05.239128 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:05.239646 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45769.5us
W20260709 13:14:05.201722 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 129783us
I20260709 13:14:05.242839 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:05.195235 32334 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.207839 32330 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:05.121968 32335 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:05.216017 32286 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.228132 31966 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:05.183204 32238 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:05.214780 32293 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.240302 31396 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:05.248930 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.243791 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.209990 32290 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.262944 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.265554 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:05.238976 32282 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:05.267910 31394 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:05.268189 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:05.269526 31391 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:05.270526 32290 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:05.274488 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.258165 32262 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:05.275684 30703 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:05.252018 32330 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:05.249815 32335 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:05.280313 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:05.263230 31966 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:05.271178 31949 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.283670 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.286959 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.286690 31966 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.287842 32294 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:05.288339 32294 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.288511 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.288789 32294 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.289263 31394 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:05.262409 32209 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:14:05.290360 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:05.279824 32298 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.626s	user 0.015s	sys 0.000s
I20260709 13:14:05.294054 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.275835 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.262681 32304 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 13:14:05.276304 32293 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:05.300395 31391 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:05.283520 32303 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:05.304507 32298 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.281850 32335 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.307075 30701 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:05.254984 32125 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.263015 32238 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.291317 32149 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:05.278981 32262 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.256546 32334 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.292563 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.331945 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.315052 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.332721 32334 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.333710 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.332721 31364 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:05.335307 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.335811 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.336354 32334 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:05.337361 32334 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:05.338029 32286 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.338485 32334 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.338853 32330 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.340378 32330 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:05.324410 32149 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.343434 32149 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:05.312170 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:05.310918 32331 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.259s	user 0.004s	sys 0.000s
I20260709 13:14:05.300947 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.356362 30550 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:05.281420 32282 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.305037 32261 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:05.328689 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.330080 32125 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.332414 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:05.358496 32261 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.368273 30557 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:05.346478 30559 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:05.348930 32188 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.371546 32282 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.371876 32188 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.373412 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.374184 31394 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:14:05.375133 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.374293 32188 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.376299 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.352617 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:05.390336 31394 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:05.393519 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.398224 30627 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:05.398582 32238 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.317910 32293 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.395382 32324 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:05.334587 32344 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:14:05.336535 32262 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.398432 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.400085 32298 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.400955 31394 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:05.401618 31391 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:05.396155 32207 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:05.401981 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:05.400866 32293 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.402384 32207 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.402474 32292 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.402896 31394 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:05.403012 32207 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.400004 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.336601 31364 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:05.405838 31364 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:05.406185 31364 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:05.406447 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.407055 31364 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.409171 31364 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.416209 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 997) took 1204 ms. Trace:
W20260709 13:14:05.416808 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 303454us
I20260709 13:14:05.416707 30702 rpcz_store.cc:276] 0709 13:14:04.211419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.293218 (+ 81799us) service_pool.cc:224] Handling call
0709 13:14:05.148309 (+855091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.212004 (+ 63695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.415965 (+203961us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54002c4cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:05.416142 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1929906048}
I20260709 13:14:05.421672 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.426959 31394 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:05.429108 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.430215 30701 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:05.420992 32049 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.431380 32049 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.433714 32049 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.429157 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:05.437533 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82466.5us
I20260709 13:14:05.447906 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:05.463099 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 68200.5us
I20260709 13:14:05.463399 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.464200 30707 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:05.471926 31394 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:14:05.475003 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.475255 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:05.475660 31394 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:05.479072 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:05.484508 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62429.9us
I20260709 13:14:05.480475 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.490659 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:05.469185 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.763s (SENT) [suppressed 2 similar messages]
I20260709 13:14:05.496500 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:05.496781 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 62213.9us
I20260709 13:14:05.496554 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.507984 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.415410 32294 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.514328 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.523126 32091 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:05.523613 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.523765 32091 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.526224 32091 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.530902 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.531636 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.532825 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:05.536691 32084 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.537420 32084 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.538102 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.517313 32207 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.541844 32207 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.511233 32096 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:05.543087 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.544852 32262 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.545867 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.546296 31389 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:05.492919 32286 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:05.548643 31364 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:05.549280 32335 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.551506 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.555866 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.556648 30630 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:05.556877 32294 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.557075 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.533003 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.546733 31383 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:05.518407 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.550698 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:05.550949 32343 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.164s	user 0.003s	sys 0.000s
I20260709 13:14:05.534340 30625 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:46186
I20260709 13:14:05.561271 30625 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:05.561875 30625 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.561896 30627 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:05.521229 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:05.520288 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.567273 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.530256 32261 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.567260 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.572376 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:05.573665 32084 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.621975 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.573917 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 726) took 1217 ms. Trace:
I20260709 13:14:05.530489 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.550477 31389 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:05.574532 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.575022 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.577435 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.575189 32337 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:14:05.575521 32279 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:14:05.578943 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:05.637048 31364 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.639530 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.640556 30624 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:05.641847 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.633971 30630 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:05.646663 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.371357 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:05.647328 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.647732 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.648607 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:05.648694 30626 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:05.649961 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.604154 32268 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:05.651270 32049 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:05.651662 32049 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:05.652113 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.652894 31378 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:05.652591 32049 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.653723 32049 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:05.607861 30550 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:14:05.607950 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:05.622768 32084 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:05.655318 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47465.4us
I20260709 13:14:05.655946 32084 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:05.656345 32084 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:05.656731 32084 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.608930 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:05.610718 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 110022us
I20260709 13:14:05.620357 32293 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:05.622277 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101197us
I20260709 13:14:05.627516 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.634575 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:05.623049 30629 rpcz_store.cc:276] 0709 13:14:04.356614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.363198 (+  6584us) service_pool.cc:224] Handling call
0709 13:14:04.377701 (+ 14503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:04.946520 (+568819us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b18002f8480. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ed5bfb 00007f1806ee064b
0709 13:14:04.946773 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:04.946816 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:04.947794 (+   978us) log.cc:844] Serialized 52 byte log entry
0709 13:14:05.250217 (+302423us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:05.298815 (+ 48598us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.303362 (+  4547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.303479 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:05.420648 (+117169us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b18002f8480. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:05.550239 (+129591us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:05.573635 (+ 23396us) raft_consensus.cc:1685] finished
0709 13:14:05.573667 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:05.573837 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16359,"prepare.queue_time_us":39489,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":111304,"raft.run_cpu_time_us":558,"raft.run_wall_time_us":32184,"spinlock_wait_cycles":120047232,"thread_start_us":461947,"threads_started":3,"wal-append.queue_time_us":362484}
I20260709 13:14:05.633734 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.664045 30702 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.585796 31383 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:14:05.667405 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.671522 32337 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.672304 32337 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.673822 32293 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.676265 32293 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.677539 32279 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.678193 32279 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:05.680598 32279 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.676280 32337 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:05.650199 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.684296 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.685673 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.687422 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:05.688499 31949 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:05.688946 31949 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:05.689895 31949 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.690843 32093 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:05.700096 32262 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.700443 32093 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.700907 32093 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:05.703151 32330 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:05.704128 32330 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.706169 30625 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.706735 32149 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:05.707805 32337 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.708158 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:05.708452 32337 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.710913 32149 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:05.706933 31949 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:05.699088 31966 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:05.720338 30625 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.693498 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.663671 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:05.731070 30702 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:05.736786 30625 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:05.740922 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:05.751050 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.756196 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.758595 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:05.763623 30559 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:05.764813 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:05.767843 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104315us
I20260709 13:14:05.769629 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.779592 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:05.781041 30555 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:05.798759 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:05.800236 30559 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:05.805963 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:05.807315 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:05.809763 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:05.808468 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49968.1us
I20260709 13:14:05.811290 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.806411 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:05.815124 30700 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:05.815469 30700 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:05.819252 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49743.8us
I20260709 13:14:05.820453 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:05.816236 30700 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.825393 30700 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:05.738363 32337 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.826151 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.826447 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:05.827859 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:05.831383 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 362244us
I20260709 13:14:05.832999 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:05.788897 31389 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:05.818751 30626 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:05.837872 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:05.821998 32323 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:05.836278 32188 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:05.824476 32155 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:05.825651 32330 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:05.828274 32158 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:05.858515 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:05.832533 32335 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:05.839246 32337 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:05.872208 30704 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:05.878087 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:05.857497 32323 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.886900 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:05.895749 30700 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:05.898775 30703 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:05.901415 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.869483 32158 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:05.878355 32289 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:05.855350 32093 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:05.880057 32337 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:05.887939 32149 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:05.906919 32246 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:05.903384 32155 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:05.906154 32323 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:05.917268 32158 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:05.946923 31673 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:05.950934 32274 consensus_queue.cc:1175] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acf852f85f47483d99cbe4020baee627" responder_term: 6 status { last_received { term: 5 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:05.952318 32276 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:14:05.953989 32349 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.619s	user 0.001s	sys 0.003s
I20260709 13:14:05.935722 32337 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:05.935297 32289 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:05.972227 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70925.4us
I20260709 13:14:05.976029 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.956040 32149 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:05.977613 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:05.975081 32262 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.001989 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:05.957120 32155 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:05.978919 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:05.980682 32261 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:06.015252 32246 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:14:06.029069 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.003484 32317 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:06.027679 32261 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:06.021562 32014 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:14:06.043313 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 271892us
I20260709 13:14:06.004052 32084 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.048983 32261 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:06.019176 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:06.052361 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 232028us
I20260709 13:14:06.059537 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.062278 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:06.054519 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 198508us
I20260709 13:14:06.030778 32246 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:06.063952 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:06.065593 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46581.8us
I20260709 13:14:06.066579 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:06.071702 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.104017 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:06.109846 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.105471 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.070461 32246 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.073553 32268 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:06.070183 32206 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.068691 32188 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.143499 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:06.143653 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.104590 32317 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.149032 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.149958 30698 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:06.106027 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:06.152726 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.051498 32289 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.115391 32291 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:06.151592 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:06.148515 32355 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:14:06.142127 32049 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:06.110831 32207 raft_consensus.cc:2764] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:14:06.158558 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52618.4us
I20260709 13:14:06.146780 32268 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:06.055791 32149 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.146586 32022 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:06.149215 32093 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:06.144797 32334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.011s	sys 0.030s
W20260709 13:14:06.189633 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.185766 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:06.163005 32236 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:14:06.197638 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54058.9us
I20260709 13:14:06.198383 30704 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:14:06.229429 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.246306 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.185799 32049 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:06.198832 32268 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.190944 31934 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:14:06.208830 32334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.012s	sys 0.030s
W20260709 13:14:06.263516 32359 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.210s	user 0.003s	sys 0.000s
I20260709 13:14:06.249549 32377 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:06.265897 32365 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
W20260709 13:14:06.276767 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 124925us
W20260709 13:14:06.254127 31364 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.616s	user 0.020s	sys 0.000s
W20260709 13:14:06.255909 32333 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:14:06.205824 32304 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:14:06.256239 32049 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:06.270334 32277 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:14:06.308547 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60768.3us
I20260709 13:14:06.310599 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.305804 32277 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:06.314589 32277 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.308516 32155 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:06.330451 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.295830 32206 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:06.338748 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.327656 32303 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.340986 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.296195 31364 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:06.338922 32377 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:06.365422 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:06.348706 32357 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:14:06.341100 32293 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:06.350310 32360 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.327s	user 0.002s	sys 0.000s
I20260709 13:14:06.351243 32236 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:06.363652 31396 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:06.355489 32084 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.341548 31934 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:06.358414 32303 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:06.375332 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:06.377187 32377 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:06.379112 32367 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:14:06.379504 32236 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:06.384038 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.385936 32268 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.388607 32268 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.389523 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.386155 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.395188 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.377310 32255 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:06.399308 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.401823 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.401823 31934 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:06.403439 32362 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.361s	user 0.003s	sys 0.000s
I20260709 13:14:06.402829 32377 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.404281 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.394094 31673 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.401811 32303 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:06.405755 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.399745 32293 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.408875 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.415009 32337 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.417116 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.420598 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.422711 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.419996 32255 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:06.425287 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.407743 32377 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:06.435436 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.424607 32323 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:06.428341 31949 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.717s	user 0.014s	sys 0.000s
I20260709 13:14:06.428495 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 714) took 1146 ms. Trace:
I20260709 13:14:06.421926 32382 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.437903 31391 rpcz_store.cc:276] 0709 13:14:05.281953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.537460 (+255507us) service_pool.cc:224] Handling call
0709 13:14:05.537805 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:05.538189 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.538216 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:05.566046 (+ 27830us) log.cc:844] Serialized 52 byte log entry
0709 13:14:05.959183 (+393137us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:05.978043 (+ 18860us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.978939 (+   896us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.979253 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.979399 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.349659 (+370260us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:06.428221 (+ 78562us) raft_consensus.cc:1685] finished
0709 13:14:06.428249 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.428421 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4811,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":3104,"raft.queue_time_us":9464,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":19104,"spinlock_wait_cycles":506696832,"thread_start_us":379229,"threads_started":1,"wal-append.queue_time_us":443707}
I20260709 13:14:06.449620 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:06.450596 32375 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.190s	user 0.003s	sys 0.000s
I20260709 13:14:06.450790 31949 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.427101 32206 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:06.434149 32293 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.419342 32378 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.460220 32255 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.425778 32022 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.462148 32369 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:06.462787 32369 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:06.464315 32337 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.464936 32369 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:06.465946 32384 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.466485 32384 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.468441 32209 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:06.468925 32209 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:06.469132 32384 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.470854 32380 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:06.471526 32380 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.417119 31673 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.431921 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.452632 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.491804 31394 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:06.476996 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:06.457041 30557 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:06.457803 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.459601 31856 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:06.404306 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.479655 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:06.527357 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.496371 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.540665 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:06.541603 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45384.2us
W20260709 13:14:06.542603 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63120.9us
I20260709 13:14:06.474972 31389 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:06.447548 32376 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:06.551564 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.469916 32209 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.474180 32380 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.546757 32298 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.580890 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 429353us
I20260709 13:14:06.548699 32317 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:06.548075 32206 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:06.552884 32207 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:06.549032 31856 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.608045 31394 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:06.609668 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.610026 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.611393 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.612756 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.613790 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.614864 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.586282 32298 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:06.616107 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 459451us
I20260709 13:14:06.617743 32298 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:06.615550 31394 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:06.623607 32378 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:06.624482 31394 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:06.624634 32378 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:06.625270 32378 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:06.622391 32277 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.627009 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.635208 30632 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.635941 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46186 (request call id 728) took 2191 ms. Trace:
I20260709 13:14:06.636334 30632 rpcz_store.cc:276] 0709 13:14:04.444636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:04.467028 (+ 22392us) service_pool.cc:224] Handling call
0709 13:14:05.885660 (+1418632us) spinlock_profiling.cc:243] Waited 712 ms on lock 0x7b540007d5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed4a3d 00007f180bea4cd0 00007f1805429254 00007f18054291e2 00007f1805429111 00007f180542908c 00007f1805428342 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:06.634525 (+748865us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4c00106480. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18051a9a20 00007f18051baa29 00007f18051bb13f 00007f180517f2ab 00007f18051acf3c 00007f18051ad3fe 00007f180543a88b 00007f1806e39495 00007f1806e8a30f 00007f1806ed4a55 00007f180bea4cd0
0709 13:14:06.635843 (+  1318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1674530816}
I20260709 13:14:06.626256 32378 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.641245 31856 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.641940 31856 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.642174 32378 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:06.607945 32207 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:06.643864 32207 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.647294 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 170) took 1160 ms. Trace:
W20260709 13:14:06.647871 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.572307 32261 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:06.648765 32374 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.485s	user 0.002s	sys 0.000s
I20260709 13:14:06.650935 32261 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.653291 30627 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:06.647246 32277 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:06.638842 32364 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.601s	user 0.004s	sys 0.000s
I20260709 13:14:06.654248 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:06.638520 32158 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.671s	user 0.015s	sys 0.000s
I20260709 13:14:06.638095 31389 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:14:06.655803 32246 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.507s	user 0.014s	sys 0.000s
I20260709 13:14:06.655475 32158 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.656311 32246 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:06.658061 32158 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:06.603806 32206 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.635753 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:06.653692 31673 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.640812 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 571) took 1023 ms. Trace:
I20260709 13:14:06.662091 30704 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:06.676215 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:06.690745 31396 rpcz_store.cc:276] 0709 13:14:05.617109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.686509 (+ 69400us) service_pool.cc:224] Handling call
0709 13:14:06.363408 (+676899us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:06.373431 (+ 10023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.640584 (+267153us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b54002b0f50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:06.640741 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1513177856}
I20260709 13:14:06.691917 30552 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:06.659004 30707 raft_consensus.cc:2368] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:14:06.683012 30558 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:06.647974 30703 rpcz_store.cc:276] 0709 13:14:05.487005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.487304 (+   299us) service_pool.cc:224] Handling call
0709 13:14:05.835120 (+347816us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:05.835364 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:05.900780 (+ 65416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.646993 (+746213us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b5400342454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:06.647224 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126004864}
I20260709 13:14:06.659008 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.693840 30700 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:06.688477 30629 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
W20260709 13:14:06.590468 32340 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.011s	sys 0.030s
I20260709 13:14:06.659013 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 604) took 1011 ms. Trace:
I20260709 13:14:06.698056 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:06.696089 31361 rpcz_store.cc:276] 0709 13:14:05.647531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.704234 (+ 56703us) service_pool.cc:224] Handling call
0709 13:14:05.704746 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.705209 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.705236 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.705757 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.839040 (+133283us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:05.839088 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.839168 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.638176 (+799008us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:06.658699 (+ 20523us) raft_consensus.cc:1685] finished
0709 13:14:06.658724 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.658946 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1208,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":96,"spinlock_wait_cycles":454742400,"thread_start_us":132915,"threads_started":1,"wal-append.queue_time_us":329581}
I20260709 13:14:06.704730 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.709631 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:06.711942 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:06.712925 30706 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:06.717123 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.717819 31391 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:06.718196 31396 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:06.655797 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:06.721203 31378 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:06.658502 32246 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:06.722563 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:06.698783 32340 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.011s	sys 0.031s
I20260709 13:14:06.642462 31856 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.731003 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:06.731261 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:06.731662 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:06.741859 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.742383 30559 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.734411 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:06.627048 32014 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:06.737880 30631 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:06.738320 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:06.739217 30703 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:06.746667 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:06.716403 32188 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:06.740281 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:06.743506 30555 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:06.750797 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:06.751935 30632 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:06.760002 30628 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:06.759629 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:06.752069 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.771714 30704 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:06.778576 32206 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:06.779585 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.768703 32149 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:06.782299 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:06.784121 32340 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:06.745450 32014 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:06.786217 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.739333 32294 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:14:06.766321 32380 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.791954 31361 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:06.792042 30558 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:06.744894 32304 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:14:06.816277 30559 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:06.784608 32340 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:06.821237 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:06.789336 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:06.788767 32206 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:06.789723 32323 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:06.830850 32340 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:06.834085 32049 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.513s	user 0.013s	sys 0.000s
I20260709 13:14:06.836563 32385 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.878131 32385 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:06.879231 32385 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:06.880121 31856 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.834899 32049 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.883600 32049 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:06.891410 30701 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:06.893064 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 137351us
I20260709 13:14:06.840278 32206 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:06.912495 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 128956us
I20260709 13:14:06.912526 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:06.910763 31396 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:06.934646 30380 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:06.915361 32198 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.945204 32353 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.924s	user 0.013s	sys 0.035s
W20260709 13:14:06.946453 32363 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.796s	user 0.009s	sys 0.003s
I20260709 13:14:06.948138 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 446) took 1645 ms. Trace:
I20260709 13:14:06.949644 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 563) took 1681 ms. Trace:
I20260709 13:14:06.948477 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:06.950166 31367 rpcz_store.cc:276] 0709 13:14:05.268139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.410267 (+142128us) service_pool.cc:224] Handling call
0709 13:14:05.423668 (+ 13401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.435648 (+ 11980us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.435685 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.892881 (+457196us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b18002d4780. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:14:06.751334 (+858453us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b2c00056c38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:06.794509 (+ 43175us) log.cc:844] Serialized 26 byte log entry
0709 13:14:06.938831 (+144322us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:06.938890 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:06.939136 (+   246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.949367 (+ 10231us) raft_consensus.cc:1685] finished
0709 13:14:06.949392 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.949553 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":32644,"spinlock_wait_cycles":80386048,"thread_start_us":55837,"threads_started":1,"wal-append.queue_time_us":116243}
I20260709 13:14:06.956490 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.959472 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:06.959800 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:06.960376 30630 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:06.964248 30624 raft_consensus.cc:1217] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-6.4]   Dedup: 6.4->[]
I20260709 13:14:06.965454 30624 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:06.938827 31856 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:14:06.966985 32246 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:06.967607 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:06.968698 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:06.969600 32282 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:14:06.943485 32334 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:06.957159 30698 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:06.948848 30633 rpcz_store.cc:276] 0709 13:14:05.302300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.302893 (+   593us) service_pool.cc:224] Handling call
0709 13:14:05.312903 (+ 10010us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:05.313386 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:14:05.314976 (+  1590us) log.cc:844] Serialized 14 byte log entry
0709 13:14:05.749107 (+434131us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:05.821471 (+ 72364us) log.cc:844] Serialized 52 byte log entry
0709 13:14:05.834837 (+ 13366us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:05.834881 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.834953 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:06.889871 (+1054918us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:06.947644 (+ 57773us) raft_consensus.cc:1685] finished
0709 13:14:06.947773 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:06.948046 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19870,"prepare.queue_time_us":5909,"prepare.run_cpu_time_us":5306,"prepare.run_wall_time_us":64737,"raft.queue_time_us":12124,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":24451,"spinlock_wait_cycles":471023872,"thread_start_us":419566,"threads_started":1,"wal-append.queue_time_us":656680}
I20260709 13:14:06.971205 32292 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:06.977902 32292 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.995012 31949 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:06.995560 31949 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:06.996568 31949 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:06.997025 32292 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.972497 32397 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:06.997949 31949 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:06.987187 32206 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:06.967710 32246 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.001511 32340 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:06.944658 32198 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.004055 32246 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:07.004925 32361 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.966s	user 0.005s	sys 0.000s
I20260709 13:14:07.011471 32198 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.006878 30706 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:06.993736 30630 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:07.013270 32340 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:07.016331 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.023671 32014 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.025262 30703 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:07.029996 32014 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:07.031282 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:07.035853 32324 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:07.038985 32324 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.041651 32324 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:07.043969 32203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.011s	sys 0.015s
W20260709 13:14:07.044464 32203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.011s	sys 0.015s
W20260709 13:14:07.044548 32387 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.230s	user 0.000s	sys 0.002s
I20260709 13:14:07.046726 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:07.048741 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.049697 32246 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.051329 32198 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.051119 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:07.052075 30627 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:07.052420 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.053941 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:07.054581 30630 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:07.054921 30633 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:07.055109 32246 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:07.056990 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.056854 32198 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:07.057749 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:07.060454 32304 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:07.062466 31934 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:07.062975 31934 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:07.063378 31934 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.070528 32198 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:07.071847 32198 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.075268 32198 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.080868 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 248) took 1489 ms. Trace:
I20260709 13:14:07.081441 30624 rpcz_store.cc:276] 0709 13:14:05.591694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.729973 (+138279us) service_pool.cc:224] Handling call
0709 13:14:06.952443 (+1222470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:06.966160 (+ 13717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.080599 (+114439us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001ea850. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:07.080782 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666249984}
I20260709 13:14:07.081619 31388 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:07.087848 32206 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:07.093343 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1255) took 1499 ms. Trace:
I20260709 13:14:07.094483 31934 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:07.096411 31383 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:07.097137 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.099607 31934 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:07.100517 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 157472us
I20260709 13:14:07.103103 32188 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:07.103829 32188 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.104315 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.105149 30624 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.105736 32397 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:07.106890 32397 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.110041 30550 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:07.094081 30551 rpcz_store.cc:276] 0709 13:14:05.594433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.594696 (+   263us) service_pool.cc:224] Handling call
0709 13:14:05.595215 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:05.595776 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:05.595824 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:05.596536 (+   712us) log.cc:844] Serialized 26 byte log entry
0709 13:14:05.783554 (+187018us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:05.791136 (+  7582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:05.818638 (+ 27502us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.004016 (+1185378us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:07.092847 (+ 88831us) raft_consensus.cc:1685] finished
0709 13:14:07.092896 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:07.093249 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11447,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":112,"spinlock_wait_cycles":459586304,"thread_start_us":129973,"threads_started":1,"wal-append.queue_time_us":397085}
I20260709 13:14:07.113520 32340 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.114821 32255 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:07.115183 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.117460 32014 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.091163 32198 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:07.118697 32014 consensus_queue.cc:1243] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:07.119305 32014 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:07.119745 32014 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:07.121054 32014 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.122156 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.106370 32188 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:07.122473 32014 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.124557 32198 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:07.128176 32198 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:07.128890 32209 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.129578 32198 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.131544 32255 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:07.131999 32255 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:07.133783 32380 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:07.135167 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.139513 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.139779 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.140367 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:07.141233 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.142666 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.141341 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.144074 30704 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:07.147442 30556 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:07.152072 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.132915 32255 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.152858 30704 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:07.161325 30704 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:07.154898 32279 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:07.155658 32049 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:07.166230 32049 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.167842 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:07.168570 30700 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:07.168933 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.170580 31673 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.170642 32380 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:07.171115 31673 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.171260 32380 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.171655 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.171905 32380 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.171273 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.173061 32279 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:07.173650 32279 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.175153 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.175850 30626 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:07.175951 32279 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.176231 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.177037 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:07.161624 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.188827 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.156934 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.193459 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.195086 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.146080 32188 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.196776 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.197067 30698 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:07.197124 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.191555 32049 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:07.198650 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.179052 32396 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:07.205875 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.171619 31673 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:07.204566 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:07.151222 32096 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.202390 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.207649 30630 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:07.141839 32091 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:14:07.160022 32385 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:07.220429 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.224053 32317 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.217403 32315 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:07.227102 32315 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:07.195721 32158 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 13:14:07.228183 30624 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:07.228506 32158 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:07.228983 32158 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:07.229288 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.206043 32255 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.196013 32277 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:07.231050 32277 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:07.228092 32315 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.206434 32188 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.232012 32277 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:07.235374 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.240911 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:07.242352 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73494.9us
I20260709 13:14:07.206267 32396 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.246251 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.242658 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.256261 32382 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:14:07.270293 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.271783 32158 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:07.273278 32158 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:07.275259 32294 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:07.275734 32294 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:07.276115 32294 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:07.276731 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:07.278520 31391 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:07.278884 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.197525 31949 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 13:14:07.286842 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.214265 32096 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.243774 32206 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:14:07.290330 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 126160us
I20260709 13:14:07.257098 32396 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:07.294348 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51815.8us
I20260709 13:14:07.284934 32381 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.298084 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.239149 32385 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.306046 30704 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:14:07.276796 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:07.320741 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106461us
I20260709 13:14:07.294039 32337 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:07.298702 32381 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.335474 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:07.300848 32335 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:07.341110 30552 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:07.349402 31388 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:07.327312 31934 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:07.297145 32210 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:14:07.332542 32381 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:07.374918 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53850us
I20260709 13:14:07.404429 32304 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.411811 32396 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:07.415190 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1036) took 1582 ms (client timeout 1974 ms). Trace:
W20260709 13:14:07.415876 30707 rpcz_store.cc:269] 0709 13:14:05.805594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:05.805752 (+   158us) service_pool.cc:224] Handling call
0709 13:14:07.384937 (+1579185us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b540062ff54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:07.388314 (+  3377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904799616}
I20260709 13:14:07.421267 32140 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:07.424888 30557 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:07.425344 32149 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:07.434739 32149 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.422199 32140 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.398140 32303 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:07.447381 32380 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.381575 32324 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:07.450524 32324 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.451843 31673 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.453902 32324 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.461251 30626 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
W20260709 13:14:07.486177 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 188197us
I20260709 13:14:07.480605 32140 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.455381 32396 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.490820 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:07.492017 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 112397us
I20260709 13:14:07.495595 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.497711 32209 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:07.502416 32294 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:07.503625 32394 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.620s	user 0.003s	sys 0.000s
I20260709 13:14:07.505098 32294 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:07.505467 30553 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:07.505187 31673 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:07.539832 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.540074 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:07.541204 30630 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:07.541760 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:07.542727 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:07.542966 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52298us
I20260709 13:14:07.545222 31396 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:07.549499 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:07.550366 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.550673 31361 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.553006 30707 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:07.536240 32380 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:07.557184 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:07.558113 30553 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.561767 30631 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:07.564427 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 232121us
I20260709 13:14:07.572839 32149 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.575975 32416 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.565487 32335 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:14:07.584975 32149 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.585701 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:07.581600 30553 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:07.554203 32427 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:07.593485 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.595012 32425 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:14:07.595227 32381 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.596254 32140 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:07.597146 32140 consensus_queue.cc:1243] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:07.597829 32324 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:07.598840 32324 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:07.600179 32427 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:07.599866 32324 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.601516 32381 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:07.601212 32427 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:07.609059 32294 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:07.598304 30555 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:07.611581 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:07.614226 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:07.622992 32373 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:07.623704 32373 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:07.624231 32373 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.627468 30553 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:07.641909 31361 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:07.615980 32337 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:07.645448 30705 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:07.645666 32337 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:07.651582 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68879.4us
I20260709 13:14:07.653019 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.655110 32377 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:07.629722 32246 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:07.653525 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.658468 30704 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.663303 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.667970 32246 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:07.669957 32246 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:07.674350 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 107893us
I20260709 13:14:07.676353 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.677311 30705 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.675683 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.680687 32383 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:07.681083 30631 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:07.683035 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.684412 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.690483 30704 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:07.699947 30701 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:07.701155 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:07.705919 32434 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:07.709183 30597 leader_election.cc:400] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:14:07.709581 30597 leader_election.cc:403] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:14:07.716228 32209 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:07.686677 32289 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:07.686319 32428 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:07.699399 32158 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:07.716856 32209 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:07.724409 32140 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4.
I20260709 13:14:07.724499 32377 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:07.725221 32377 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.724972 32209 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.722062 30705 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:07.727913 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:07.731798 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.732416 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.733050 30701 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.734882 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:07.735538 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:07.744841 30704 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:07.727250 32397 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:14:07.747614 32274 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:07.748248 32274 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:07.755007 30630 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:07.755187 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:07.758910 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:07.713541 32413 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:14:07.715523 32279 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:14:07.768682 30704 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:14:07.770982 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58456.8us
I20260709 13:14:07.753322 32428 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.773717 30553 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:07.772655 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:07.745066 32158 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.767822 32315 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:07.775638 32371 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.516s	user 0.003s	sys 0.003s
I20260709 13:14:07.757299 32380 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 13:14:07.774376 30626 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:07.767297 32149 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:07.793663 30701 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:07.783120 32140 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:07.749195 32274 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:07.800807 30705 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:14:07.802742 32444 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:07.800249 32404 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:14:07.815120 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:07.818310 32436 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:14:07.819746 32158 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:07.820878 32337 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.797693 32335 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:07.795454 32315 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:07.783272 32428 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.828037 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:07.829424 32377 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:07.855222 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:07.832414 32315 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:07.834786 32158 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 13:14:07.843070 32335 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:07.888437 32428 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:07.890910 32335 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:07.874123 32337 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:07.899966 32140 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:07.907961 32440 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.143s	user 0.002s	sys 0.000s
I20260709 13:14:07.899045 32373 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:07.922588 32140 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:07.949414 32442 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.132s	user 0.000s	sys 0.003s
I20260709 13:14:08.001820 32289 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:08.002560 32289 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:08.003783 32289 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.005373 32289 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:08.351315 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 513290us
I20260709 13:14:08.374264 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.375162 30559 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:14:08.389227 30630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.022s	sys 0.018s
W20260709 13:14:08.389988 30630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.023s	sys 0.018s
I20260709 13:14:08.417495 32455 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:08.474438 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 256) took 1522 ms. Trace:
I20260709 13:14:08.490883 30557 rpcz_store.cc:276] 0709 13:14:06.951569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.951708 (+   139us) service_pool.cc:224] Handling call
0709 13:14:06.957838 (+  6130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.440257 (+482419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.473942 (+1033685us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:08.474331 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1073429760}
I20260709 13:14:08.478978 30380 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 5, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:08.480623 32455 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.479071 32456 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:08.500432 30557 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:08.499158 32456 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.506515 32456 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.508666 32455 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.509219 32455 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.509651 32455 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.512348 32455 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.513114 32282 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:08.513588 32282 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:08.514515 32282 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.520195 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 618) took 2222 ms. Trace:
I20260709 13:14:08.531612 31394 rpcz_store.cc:276] 0709 13:14:06.298260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.625439 (+327179us) service_pool.cc:224] Handling call
0709 13:14:06.625770 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.904362 (+1278592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.904411 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:07.904496 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:07.904546 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.519934 (+615388us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540031d8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:08.519990 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.520133 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2939831680}
I20260709 13:14:08.516242 32282 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:08.543464 32277 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.544186 32277 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.548584 32277 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:08.562249 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.573379 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 754) took 1027 ms. Trace:
I20260709 13:14:08.574901 31396 rpcz_store.cc:276] 0709 13:14:07.545473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.563896 (+ 18423us) service_pool.cc:224] Handling call
0709 13:14:08.521942 (+958046us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:08.522172 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:08.522422 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:08.564205 (+ 41783us) log.cc:844] Serialized 14 byte log entry
0709 13:14:08.572909 (+  8704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.572987 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:08.573039 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.573118 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.573304 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50333,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":144,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":3387136,"thread_start_us":49543,"threads_started":2}
I20260709 13:14:08.577530 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1281) took 1083 ms. Trace:
I20260709 13:14:08.578096 30555 rpcz_store.cc:276] 0709 13:14:07.493799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.494051 (+   252us) service_pool.cc:224] Handling call
0709 13:14:07.494578 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.615619 (+121041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.577259 (+961640us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540052add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:08.577474 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":237709184}
I20260709 13:14:08.613323 32373 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.632995 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:08.634899 32315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.014s	sys 0.000s
W20260709 13:14:08.635389 32315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.014s	sys 0.000s
I20260709 13:14:08.635437 30628 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:08.640859 32337 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.742s	user 0.007s	sys 0.007s
I20260709 13:14:08.648849 32277 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.649641 32277 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:08.651872 32315 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.652570 32315 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.657280 32373 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.658190 32373 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.663875 32369 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.667899 32369 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.668593 32014 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:08.669173 32014 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.672348 32277 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.681770 32277 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.684038 32277 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:08.672871 32369 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.673834 32315 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.797740 32277 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:08.801399 32140 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.865s	user 0.012s	sys 0.016s
I20260709 13:14:08.680505 32294 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.686796 32373 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.665064 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1274) took 1525 ms. Trace:
I20260709 13:14:08.696189 30624 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:08.699843 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 823) took 1499 ms. Trace:
W20260709 13:14:08.703524 32392 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.821s	user 0.009s	sys 0.002s
W20260709 13:14:08.711634 32452 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.815s	user 0.002s	sys 0.005s
I20260709 13:14:08.729828 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:08.807191 30702 rpcz_store.cc:276] 0709 13:14:07.200630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.217051 (+ 16421us) service_pool.cc:224] Handling call
0709 13:14:07.652139 (+435088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.657735 (+  5596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.657779 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.658335 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:14:07.680133 (+ 21798us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:07.680177 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:07.680256 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:07.819668 (+139412us) raft_consensus.cc:1685] finished
0709 13:14:07.819691 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.699370 (+879679us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540037d2d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:08.699727 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5146,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":2239,"spinlock_wait_cycles":994512768,"thread_start_us":21455,"threads_started":1,"wal-append.queue_time_us":59597}
W20260709 13:14:08.809082 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79382us
I20260709 13:14:08.809303 32294 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:08.811317 32315 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:08.811652 32294 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:08.812054 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.732477 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 586) took 1658 ms. Trace:
I20260709 13:14:08.812638 31383 rpcz_store.cc:276] 0709 13:14:07.074271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.074427 (+   156us) service_pool.cc:224] Handling call
0709 13:14:07.074887 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:07.096914 (+ 22027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.732039 (+1635125us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b5400522954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:08.732372 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49008896}
I20260709 13:14:08.734365 32380 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:08.741837 32022 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:08.744671 32378 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:08.815140 32378 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:08.814963 32380 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.816143 32378 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.809631 32385 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:08.817240 32378 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.817601 32380 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.814981 32022 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.821319 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.654019 32096 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.760013 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.760373 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.766130 32147 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:08.824000 32147 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.768519 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.829324 32022 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:08.770222 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.771955 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:08.830948 31396 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.771214 32297 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:08.835068 32297 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.837682 32297 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:08.777200 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:08.777410 32014 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.777769 32282 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:08.839629 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.779946 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:08.840533 30559 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:08.840600 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60753.5us
I20260709 13:14:08.841427 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.841553 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.778551 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.842242 30556 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:08.780613 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1062) took 1046 ms. Trace:
I20260709 13:14:08.843207 30705 rpcz_store.cc:276] 0709 13:14:07.734376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.734525 (+   149us) service_pool.cc:224] Handling call
0709 13:14:08.780402 (+1045877us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:08.780546 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130138368}
I20260709 13:14:08.843743 30551 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:14:08.803043 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:08.826329 32147 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.847200 32147 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:08.847585 32147 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:08.847927 32147 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.849349 32246 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:08.842231 30559 raft_consensus.cc:2410] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:14:08.806844 30554 rpcz_store.cc:276] 0709 13:14:07.139724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.142204 (+  2480us) service_pool.cc:224] Handling call
0709 13:14:08.664089 (+1521885us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:08.664347 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:08.664601 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.664639 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.664702 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:08.664746 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.664822 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.664983 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4463488}
I20260709 13:14:08.851280 32445 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:08.851689 32445 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:08.803318 30703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.011s	sys 0.023s
W20260709 13:14:08.852665 30703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.012s	sys 0.024s
I20260709 13:14:08.852614 32445 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:08.853586 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1064) took 1117 ms. Trace:
I20260709 13:14:08.854018 32445 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.853938 30703 rpcz_store.cc:276] 0709 13:14:07.735738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.735894 (+   156us) service_pool.cc:224] Handling call
0709 13:14:07.736365 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:07.746041 (+  9676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:07.746073 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:07.746582 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:14:08.853206 (+1106624us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:08.853257 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.853342 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:08.853397 (+    55us) raft_consensus.cc:1685] finished
0709 13:14:08.853408 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.853516 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9161,"raft.queue_time_us":11450,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":1073866,"threads_started":1,"wal-append.queue_time_us":159613}
I20260709 13:14:08.855155 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.804939 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.855921 30628 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:08.857056 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.861774 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 735) took 2364 ms. Trace:
I20260709 13:14:08.862255 31389 rpcz_store.cc:276] 0709 13:14:06.497469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:06.656251 (+158782us) service_pool.cc:224] Handling call
0709 13:14:06.656708 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:08.798905 (+2142197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:08.798958 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:08.799027 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:08.799078 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:08.861523 (+ 62445us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54004b05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:08.861576 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:08.861702 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4925984640}
I20260709 13:14:08.865414 30631 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:08.865986 30380 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 6 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:08.867807 30380 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:08.869464 30380 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:08.621076 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:08.873730 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:08.874652 32381 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:08.875094 32381 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:08.875483 32381 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.879871 32294 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:08.880687 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.010s	sys 0.013s
W20260709 13:14:08.881103 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.010s	sys 0.013s
I20260709 13:14:08.880618 32294 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.883440 32188 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:08.884332 32093 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:08.884780 32093 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:08.782785 32382 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:08.885608 32093 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.886051 32382 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.887100 32093 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.888459 32382 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.675990 32324 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:08.839386 32282 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.891237 32294 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.892450 32282 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:08.894428 32274 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:08.895095 32274 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.895394 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.896359 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 467) took 1573 ms. Trace:
I20260709 13:14:08.896800 30624 rpcz_store.cc:276] 0709 13:14:07.323089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.325671 (+  2582us) service_pool.cc:224] Handling call
0709 13:14:07.805460 (+479789us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:14:08.896160 (+1090700us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b54006effd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:08.896271 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96027904}
I20260709 13:14:08.897826 32274 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.898437 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:08.898653 32274 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.899155 30630 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:08.899227 32274 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:08.865563 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 140) took 12665 ms (client timeout 15000 ms). Trace:
I20260709 13:14:08.902189 32274 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.904338 30380 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:08.932096 30380 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:08.907964 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.933804 31383 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:14:08.934008 30380 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:08.806416 32140 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.012s	sys 0.017s
I20260709 13:14:08.913756 32207 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.946127 30626 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:08.914358 31949 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.946895 31949 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:08.949239 31949 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:08.917467 32396 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.917876 32435 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:08.925501 31394 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:08.950845 32396 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.951749 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:08.945968 32207 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.956674 31394 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:14:08.957687 32396 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:08.959787 30580 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc700 after lost signal to thread 21529
I20260709 13:14:08.960004 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.960743 31391 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
W20260709 13:14:08.961288 30580 debug-util.cc:398] Leaking SignalData structure 0x7b0800106720 after lost signal to thread 30416
I20260709 13:14:08.961741 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.962966 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:08.963228 30580 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac7a0 after lost signal to thread 30583
I20260709 13:14:08.963586 31391 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:08.964128 32435 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:08.964500 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.965133 31391 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:08.927479 32014 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:08.967346 32014 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:08.967773 32014 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:08.952615 30559 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:08.968688 32014 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.969832 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:08.970031 32458 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:08.928654 30555 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:08.970817 32396 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:08.971357 32396 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:08.901240 30382 rpcz_store.cc:269] 0709 13:13:56.199757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:13:56.207101 (+  7344us) service_pool.cc:224] Handling call
0709 13:14:08.865477 (+12658376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:04.373174 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:04.479332 (+106158us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:04.479434 (+   102us) write_op.cc:435] Acquired schema lock
0709 13:14:04.479469 (+    35us) tablet.cc:662] Decoding operations
0709 13:14:04.480422 (+   953us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:04.480595 (+   173us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:04.480640 (+    45us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:14:04.481223 (+   583us) tablet.cc:701] Row locks acquired
0709 13:14:04.481291 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:14:04.481535 (+   244us) write_op.cc:270] Start()
0709 13:14:04.481761 (+   226us) write_op.cc:276] Timestamp: P: 1783602844481447 usec, L: 0
0709 13:14:04.481794 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:14:04.558743 (+ 76949us) log.cc:844] Serialized 1303 byte log entry
0709 13:14:04.815539 (+256796us) op_driver.cc:464] REPLICATION: finished
0709 13:14:04.867052 (+ 51513us) write_op.cc:301] APPLY: starting
0709 13:14:04.882108 (+ 15056us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:04.970791 (+ 88683us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:05.049593 (+ 78802us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:05.050339 (+   746us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:05.050642 (+   303us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:14:05.050711 (+    69us) write_op.cc:312] APPLY: finished
0709 13:14:05.051056 (+   345us) log.cc:844] Serialized 34 byte log entry
0709 13:14:05.051317 (+   261us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:05.051463 (+   146us) write_op.cc:454] Released schema lock
0709 13:14:05.051508 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:05.051961 (+   453us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:05.051998 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1545741,"reported_tablets":16,"spinlock_wait_cycles":12032,"child_traces":[["op",{"apply.queue_time_us":23709,"apply.run_cpu_time_us":4262,"apply.run_wall_time_us":203817,"num_ops":3,"prepare.queue_time_us":41797,"prepare.run_cpu_time_us":19580,"prepare.run_wall_time_us":353873,"raft.queue_time_us":35406,"raft.run_cpu_time_us":19842,"raft.run_wall_time_us":24149,"replication_time_us":333465,"thread_start_us":207680,"threads_started":4,"wal-append.queue_time_us":191531,"wal-append.run_cpu_time_us":14532,"wal-append.run_wall_time_us":1285610}]]}
I20260709 13:14:08.866005 30631 pending_rounds.cc:85] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 3
I20260709 13:14:08.939429 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:08.940804 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.941920 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.943184 32282 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:08.979036 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.979782 32382 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:08.979910 31378 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:08.980261 32382 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:08.981194 32188 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:08.981357 32382 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:08.981860 32188 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:08.982276 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.983124 32382 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:08.984390 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.985848 32207 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.986425 32188 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:08.909965 30558 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
W20260709 13:14:08.963582 30580 debug-util.cc:398] Leaking SignalData structure 0x7b080014ab60 after lost signal to thread 30584
W20260709 13:14:08.989141 30558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 745) took 1880 ms (client timeout 1684 ms). Trace:
W20260709 13:14:08.989715 30580 debug-util.cc:398] Leaking SignalData structure 0x7b08003de300 after lost signal to thread 30657
W20260709 13:14:08.990149 30580 debug-util.cc:398] Leaking SignalData structure 0x7b080010c980 after lost signal to thread 30658
I20260709 13:14:08.927062 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:08.970059 32014 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:08.971719 32396 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:08.929387 32096 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:08.977048 30631 pending_rounds.cc:107] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:08.966348 32435 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:08.978752 31388 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:08.995643 30631 raft_consensus.cc:2889] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:08.992901 32207 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:08.944083 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:08.998754 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54779.5us
I20260709 13:14:08.949479 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:14:09.002269 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52882.7us
I20260709 13:14:09.003069 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:09.003904 30553 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
W20260709 13:14:08.989589 30558 rpcz_store.cc:269] 0709 13:14:07.108699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:07.854764 (+746065us) service_pool.cc:224] Handling call
0709 13:14:08.988928 (+1134164us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b540028a5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:08.989041 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806174464}
I20260709 13:14:09.004372 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:08.947178 30380 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.023274 31396 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:09.040370 30701 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:14:09.011564 30380 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.048707 30380 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:09.015727 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.062752 30702 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:09.063951 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49967.9us
W20260709 13:14:09.007579 30580 debug-util.cc:398] Leaking SignalData structure 0x7b08003dd9c0 after lost signal to thread 30731
I20260709 13:14:08.998422 32022 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:09.078334 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62680.7us
I20260709 13:14:09.079444 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:09.078507 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:09.076493 30380 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 4, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:09.012463 32294 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:09.080413 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:09.084132 30380 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:09.067448 30580 debug-util.cc:398] Leaking SignalData structure 0x7b080039c920 after lost signal to thread 30732
I20260709 13:14:09.089416 30380 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:09.013339 32293 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:09.091141 30380 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:09.094663 30703 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:09.095350 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.066851 32458 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:09.097352 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.091065 32380 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:09.099221 32380 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.099416 32210 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:09.099681 32380 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:09.099906 32210 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.100363 32210 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:09.080904 32373 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.103930 32373 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:09.104372 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.103957 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.110400 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.110716 32458 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.081321 32337 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:09.111491 32315 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:09.111944 32337 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:09.112061 32315 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.093519 32293 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:09.112757 32315 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:09.112908 32293 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:09.111508 31949 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:09.115422 31949 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:09.118992 31949 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:09.121140 31364 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:14:09.121661 31364 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:09.121928 31364 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:09.122175 31364 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:09.087116 32294 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.123155 32294 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:09.095618 30380 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 2 to 4, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:09.131687 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.111232 32458 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:09.132722 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.143476 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.147107 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:09.148414 30558 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:09.153717 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.106372 32373 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:09.155776 30553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:09.076617 32188 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:14:09.097833 32445 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:09.174573 30701 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:14:09.179136 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 427071us
I20260709 13:14:09.181906 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.167011 30706 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:09.160797 32209 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:14:09.173213 30580 debug-util.cc:398] Leaking SignalData structure 0x7b08000ff160 after lost signal to thread 31426
I20260709 13:14:09.174149 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.143962 32378 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.185483 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.194558 30631 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:09.156077 32324 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.156533 30553 raft_consensus.cc:606] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
W20260709 13:14:09.187611 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.188122 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.188966 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.198094 31391 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:14:09.189045 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.097503 32298 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.077s
I20260709 13:14:09.187188 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.194639 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.174974 32445 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:09.211732 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.213950 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.193145 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.231635 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.230283 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:09.230865 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:09.234114 30556 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:09.233920 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.231323 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.231014 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:09.253619 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.258299 30703 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:09.206995 32209 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:09.261126 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.213858 32445 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:09.260008 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:09.229957 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:09.218158 32274 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:09.205646 32324 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.203933 32378 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:09.222431 32480 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.391858 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:09.399606 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139677us
I20260709 13:14:09.228952 30699 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:14:09.258870 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:09.417122 32476 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:14:09.359045 32261 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.365509 32400 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:09.261761 32462 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:09.217736 32381 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:09.390389 32093 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
W20260709 13:14:09.398474 32147 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.000s
W20260709 13:14:09.463655 32481 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:14:09.464478 32478 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:14:09.438675 32484 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:09.471293 32484 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.408960 32236 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:09.473053 32236 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:09.418486 32274 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:09.475054 32274 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:09.476132 32274 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.417845 32486 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:09.451529 32477 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
W20260709 13:14:09.518713 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:09.450026 32487 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
W20260709 13:14:09.450659 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.500910 32147 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.453509 32261 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:09.444890 32489 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.515148 32479 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:09.567111 32298 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:09.455432 32400 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.510279 32236 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.508033 32282 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.436606 32480 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:09.599050 31361 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.069s	sys 0.142s
W20260709 13:14:09.599809 31361 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.069s	sys 0.143s
I20260709 13:14:09.574795 32022 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:09.607514 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 377179us
I20260709 13:14:09.616632 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.625882 31396 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:09.523505 32380 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.590057 32503 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.570360 32293 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.634577 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.628608 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:09.613669 32382 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.630s	user 0.008s	sys 0.004s
I20260709 13:14:09.529407 32486 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:09.518469 32209 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:09.630188 32396 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 13:14:09.664849 31396 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:09.617472 32188 consensus_queue.cc:1243] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:09.600142 32282 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:09.672015 32482 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:14:09.624477 32455 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:14:09.674283 32282 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.611902 32493 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:09.675652 32282 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:09.679008 32493 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.677543 32486 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.696175 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.670866 31364 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:09.703987 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:09.704775 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.704917 31361 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:09.706931 30701 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:09.707851 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.708597 32391 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.709245 32391 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:09.712147 32391 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.713213 32384 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:09.717875 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.718079 32207 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:09.720485 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.720945 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.721282 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:09.721400 30624 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:09.729492 30558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.009s	sys 0.019s
I20260709 13:14:09.592463 32479 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.590111 32489 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.732209 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.738062 31391 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:09.736801 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:09.742660 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 181311us
I20260709 13:14:09.748386 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:09.749591 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 526609us
I20260709 13:14:09.744297 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.752368 30629 raft_consensus.cc:1217] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:14:09.752537 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.753118 30629 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:09.753518 30558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.011s	sys 0.019s
I20260709 13:14:09.754220 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.755003 31361 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:09.754444 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:09.750540 31391 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:14:09.763230 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:09.765653 31391 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:09.769603 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:09.736764 32294 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.613s	user 0.010s	sys 0.003s
I20260709 13:14:09.770478 31394 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:09.777346 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.780957 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.728914 32484 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:09.785543 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:09.787657 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:09.729566 32418 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.098s
I20260709 13:14:09.794857 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:09.762960 32465 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:09.687428 32396 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.761972 32140 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.662957 32503 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.764781 32378 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.821303 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.820132 32140 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:09.828435 32465 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:09.829002 32465 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.829348 32465 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:09.800534 32297 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:09.755949 32391 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:09.783335 32294 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.837052 32324 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:09.854847 32445 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:09.837502 32400 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:09.869206 30627 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 2: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:09.804796 32262 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:14:09.871541 32315 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.758s	user 0.015s	sys 0.000s
W20260709 13:14:09.834434 32014 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.838s	user 0.007s	sys 0.003s
W20260709 13:14:09.874755 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80750.9us
W20260709 13:14:09.903679 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:09.926625 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:09.894135 32458 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.733s	user 0.011s	sys 0.003s
I20260709 13:14:09.911168 32262 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.935223 32262 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:09.902392 32391 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.937918 32391 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:09.924571 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:09.934383 32492 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:09.938891 32492 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:09.939380 32492 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:09.943171 32435 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:09.943641 32435 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:09.925235 32315 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.944078 32435 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:09.902033 32380 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:09.946161 32315 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:09.922969 30624 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:14:09.946023 32210 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.845s	user 0.010s	sys 0.001s
I20260709 13:14:09.947387 32210 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:09.948635 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:09.949887 32210 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:09.950049 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:09.950326 32400 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:09.950929 32400 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:09.951344 32400 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:09.951902 32445 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.954001 32140 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:09.956168 32294 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:09.962765 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:09.962499 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:09.930158 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 260000us
I20260709 13:14:09.903055 32484 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:09.989429 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 233375us
W20260709 13:14:09.997718 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 235286us
I20260709 13:14:10.000857 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:09.988906 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.928373 32147 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.008105 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.012049 30630 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:10.012621 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:09.987797 32324 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:10.011510 32500 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:09.986841 32261 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:10.028700 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 278231us
I20260709 13:14:10.039836 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.041014 31364 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:09.964412 32458 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.014413 32293 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.047746 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.053891 31364 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:10.018774 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:10.015977 32324 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:10.004896 32484 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:09.989889 32465 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.049985 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.076825 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:10.058209 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50165.2us
I20260709 13:14:10.027572 32274 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.084378 32396 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.065276 32512 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:10.085744 32484 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.087666 32458 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.064383 32324 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:10.057101 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.092186 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.093143 31378 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:10.093204 32381 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.095732 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.102380 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:10.103658 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 241234us
I20260709 13:14:10.103220 31367 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.119270 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.128062 31364 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
W20260709 13:14:10.076241 32490 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:14:10.157109 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.162770 30553 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:10.163265 30553 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:10.164196 30553 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:10.174041 30553 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:10.061180 32391 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.170601 32463 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.182443 30552 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:10.101807 31388 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:10.178655 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:10.186661 30629 raft_consensus.cc:2368] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6.
I20260709 13:14:10.081453 32385 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:14:10.194556 31367 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:10.211589 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 200293us
I20260709 13:14:10.211923 30553 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:14:10.212442 30553 pending_rounds.cc:85] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 2
I20260709 13:14:10.187794 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:10.213357 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 94220.3us
I20260709 13:14:10.188449 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.188642 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.189388 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.189468 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.190850 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.217561 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.195523 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.184888 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:10.162295 32479 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.200991 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.271763 32277 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:10.272248 32503 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.272455 30699 raft_consensus.cc:2368] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:14:10.202286 30706 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:10.173357 32279 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:10.274451 32279 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:10.275570 32262 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.277014 32279 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.161485 32484 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:14:10.207726 32465 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.160566 32293 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.282871 32293 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.283931 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.285537 32293 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.153164 32435 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.287941 32435 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.212878 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:10.212827 30553 pending_rounds.cc:107] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:10.288642 30553 raft_consensus.cc:2889] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:10.289083 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76289.6us
I20260709 13:14:10.289767 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:10.290493 30550 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:10.293367 32510 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:14:10.296329 32495 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.224794 30707 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:10.298831 32464 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.299808 32210 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:10.300369 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:10.269523 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50160.5us
I20260709 13:14:10.269650 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.270558 31388 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:10.183703 32515 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:10.301287 30554 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:10.205791 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.277825 32262 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.279263 30552 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 1.
I20260709 13:14:10.280834 32400 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:10.266682 32490 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:14:10.303866 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1307) took 1118 ms. Trace:
I20260709 13:14:10.304268 30558 rpcz_store.cc:276] 0709 13:14:09.185921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.186469 (+   548us) service_pool.cc:224] Handling call
0709 13:14:09.186905 (+   436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:09.204156 (+ 17251us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:14:09.216692 (+ 12536us) log.cc:844] Serialized 14 byte log entry
0709 13:14:09.769801 (+553109us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:09.796551 (+ 26750us) log.cc:844] Serialized 26 byte log entry
0709 13:14:09.796910 (+   359us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:09.796974 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:09.797089 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:10.270482 (+473393us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b18000c4900. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:10.303543 (+ 33061us) raft_consensus.cc:1685] finished
0709 13:14:10.303658 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.303804 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24774,"prepare.queue_time_us":13538,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":90906,"raft.queue_time_us":193873,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":15711,"spinlock_wait_cycles":19382656,"thread_start_us":536961,"threads_started":1,"wal-append.queue_time_us":601144}
I20260709 13:14:10.183275 32489 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.300696 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.309083 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.314577 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:10.315302 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 101158us
I20260709 13:14:10.299553 32464 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.298951 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:10.316463 30628 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:10.225888 30702 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:10.317842 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:10.217250 32385 consensus_queue.cc:1243] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:10.320103 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:10.195217 32463 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.218685 32508 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.323822 32210 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:10.317399 32012 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:14:10.328472 32210 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.328694 32492 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.331001 32492 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.321470 30701 raft_consensus.cc:606] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:10.331471 32480 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.312983 32512 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.332444 32512 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.335546 32512 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.319785 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:10.323074 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.325610 31949 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.321573 31378 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:10.331573 30701 raft_consensus.cc:614] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:10.323088 32489 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:10.340801 31949 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.341435 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.342447 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.343818 31292 leader_election.cc:400] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:14:10.344280 31292 leader_election.cc:403] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:10.320708 32479 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:10.347062 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:10.351785 32298 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:10.352476 32298 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:10.347144 32489 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:10.352912 32149 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:10.353451 32298 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.345575 32340 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:10.354593 32340 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:10.363873 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:10.364250 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.364910 30707 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:10.364785 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.372490 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.373495 31391 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:10.383201 30550 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:10.387287 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:10.388224 30631 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:10.348683 32480 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:10.375074 32489 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:10.364387 32209 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.402908 32209 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.405429 32209 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.408346 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.413154 32463 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.414124 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:10.416409 30558 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:10.421366 30702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:10.425525 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.425932 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:10.357697 32494 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:14:10.422122 30702 raft_consensus.cc:606] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.353505 32149 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.429387 32463 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:10.449705 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:10.463945 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.469815 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.470860 30703 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 2.
I20260709 13:14:10.471722 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.358960 32324 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.500630 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:10.440793 32508 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.367060 32340 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.431906 30551 raft_consensus.cc:2376] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6. Re-sending same reply.
I20260709 13:14:10.509239 32400 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.514107 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.504204 32209 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.497550 32207 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:10.505560 32385 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:10.529477 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.510656 31949 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.520947 32495 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:10.521270 32418 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.499217 32463 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.538196 31396 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:10.538861 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:10.563105 32529 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:14:10.564945 32493 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.516096 32508 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:10.535470 30551 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:10.567411 30551 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:10.567946 32508 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.566598 32149 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.530082 32209 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.571301 30550 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:10.573374 32209 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.574811 32465 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:10.531345 32274 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.591979 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.577798 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.593346 31396 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:10.580909 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.584510 32495 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:10.585251 32465 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:10.586164 32463 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.587164 32210 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 13:14:10.588790 32489 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.586227 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.594959 30700 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:10.562333 32418 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.602214 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:10.531277 32207 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.611436 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.534906 32531 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:10.631340 31856 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:10.616842 32324 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.619853 32207 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.629887 32385 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:10.632614 32385 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.632915 32465 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.637015 32385 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.639454 32480 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:14:10.640352 32458 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:10.640878 32458 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.641218 32458 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:10.643152 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:10.606878 32495 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.645313 32464 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.645500 32096 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:10.646144 32096 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:10.646736 32096 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:10.631997 31856 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:10.651587 31856 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:10.653620 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44676.5us
I20260709 13:14:10.655230 30556 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:10.655253 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.657934 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:10.657974 30556 raft_consensus.cc:606] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:10.662220 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.663376 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.663667 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.668040 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.669456 31361 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:10.670560 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:10.672360 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.679188 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.683419 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.684798 31396 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:10.685694 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.686996 31361 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:10.689103 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.691341 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.693667 31378 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:10.698913 31367 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:10.646457 32495 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:10.705276 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.707546 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.710958 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.712074 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:10.718143 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.719236 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:10.706113 32506 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:10.699564 32480 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:10.700596 32489 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:10.728363 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:14:10.712219 32541 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 13:14:10.731289 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:10.732003 32501 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:10.752409 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.753785 31391 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:10.755393 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 763) took 1129 ms. Trace:
I20260709 13:14:10.756091 31383 rpcz_store.cc:276] 0709 13:14:09.625962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:09.627888 (+  1926us) service_pool.cc:224] Handling call
0709 13:14:09.663071 (+ 35183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:09.775728 (+112657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:09.775782 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:09.775856 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:09.775908 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:10.755014 (+979106us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002b1454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:10.755106 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:10.755303 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258313088}
I20260709 13:14:10.763412 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.725731 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.767226 31391 pending_rounds.cc:85] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:10.767738 31391 pending_rounds.cc:107] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:10.768123 31391 raft_consensus.cc:2889] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:10.766726 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:10.783950 31396 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:10.786108 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.793404 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:10.793869 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:10.741020 32391 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.764964 32515 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.804844 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:10.740342 32480 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.807309 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.737123 32489 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:10.739359 32506 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.809841 31378 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:10.814735 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:10.799361 32464 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.810870 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:10.819124 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:10.808440 31364 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:10.820026 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:10.799978 32274 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.806205 32503 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:10.806787 32531 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:10.814550 32493 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:10.825989 32494 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:10.813598 32515 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.828613 32492 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:10.846247 32547 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:14:10.848089 32506 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.849999 32489 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:10.822432 32529 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:14:10.851372 32529 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:10.851794 32529 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:10.858903 32515 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.856690 32529 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.860513 32529 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:10.899024 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:10.829162 32274 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.933248 30552 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:10.946949 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:10.951535 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 129830us
I20260709 13:14:10.954547 32274 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.955919 32385 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:10.837064 31367 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:10.968178 32458 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:10.972182 32458 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.825690 32464 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:10.976500 32489 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:10.978483 32464 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:10.980836 32480 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:10.842777 32494 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:10.985896 32494 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:10.993212 32548 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:14:10.995875 32554 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:14:11.011155 31364 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:11.015594 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.016397 32418 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.018242 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.023005 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.024161 31364 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:11.030954 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76747.4us
I20260709 13:14:11.016149 31292 leader_election.cc:400] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6.
I20260709 13:14:11.039162 31292 leader_election.cc:403] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:14:11.045023 32479 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6.
I20260709 13:14:11.049897 32462 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:14:11.061264 32501 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:11.062060 32501 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.070286 32480 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:11.071043 32480 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:11.072543 32480 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:11.079917 32522 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.767s	user 0.003s	sys 0.003s
I20260709 13:14:11.082301 32523 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:11.082770 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.084487 30700 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:11.083676 32523 consensus_queue.cc:1243] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:11.090260 32465 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:11.090845 32465 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:11.087807 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.100703 32523 raft_consensus.cc:993] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower acf852f85f47483d99cbe4020baee627 to start an election
W20260709 13:14:11.115698 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 487470us
I20260709 13:14:11.115072 32465 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.047466 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:11.049860 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:11.135887 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68521.1us
I20260709 13:14:11.122648 32480 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:11.143483 32516 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:11.145002 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:11.148692 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:11.150288 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 100715us
I20260709 13:14:11.151659 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.153280 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:11.153339 32294 raft_consensus.cc:993] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:11.151448 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.155539 32294 raft_consensus.cc:1081] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:11.156427 30624 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:11.156914 32462 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:11.154735 32499 raft_consensus.cc:993] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:11.158397 32499 raft_consensus.cc:1081] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:11.159794 32384 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:11.118026 32464 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:14:11.161173 32464 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:11.151363 32500 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:11.161731 32464 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:11.161842 32500 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.162315 32500 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.163249 32464 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.164832 32464 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.166381 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:11.166918 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.151551 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:11.156229 32556 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:11.163777 32494 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.160765 32384 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.064308 32096 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.195477 31389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:11.196606 31389 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:11.197180 31389 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:11.199606 31856 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.532s	user 0.019s	sys 0.019s
I20260709 13:14:11.200225 31856 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.203907 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.204968 31394 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:11.206827 32445 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:11.207515 32445 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:11.208017 32445 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:11.199299 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.215606 30626 raft_consensus.cc:2410] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:14:11.217836 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:11.219174 31361 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:11.220350 32261 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:11.221318 32261 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:11.222738 32261 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.223734 31361 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:11.226555 32418 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:11.227952 32096 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:11.230118 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:11.231581 30553 raft_consensus.cc:2368] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:11.232726 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.232918 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.235369 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.236472 30631 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:11.236845 30673 leader_election.cc:400] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:11.237407 30673 leader_election.cc:403] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:14:11.191684 31383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:11.239236 31383 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:11.239436 32523 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:11.239730 31383 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:11.248028 32014 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:11.248445 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:11.250567 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.257932 32465 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:11.258771 32503 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.259539 32503 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.262318 32503 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.262883 32479 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.262349 32556 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.264356 32533 raft_consensus.cc:1081] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Signalling peer acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:14:11.266364 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:11.256352 32014 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.268052 31856 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.270193 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:11.276527 32529 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:11.277626 32479 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.279934 32556 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.288117 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.303964 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:11.309028 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 261785us
I20260709 13:14:11.305526 30700 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:11.315312 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.316586 31391 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:11.319010 30705 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:14:11.321640 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55358.7us
I20260709 13:14:11.322942 32495 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:11.323736 32495 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:11.324615 32479 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.326236 31383 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:11.341954 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 83535.7us
I20260709 13:14:11.342756 32014 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:11.343918 32384 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.345542 32014 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.346678 32297 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:11.347231 32297 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:11.347594 32297 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:11.350996 32140 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:11.350203 32384 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:11.353441 32396 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:11.353950 32396 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:11.355639 31396 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:11.355835 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:11.357034 32400 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:11.354144 32384 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:14:11.357436 32396 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.361936 31383 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:11.362830 32499 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:11.364292 32499 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:11.364714 32499 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:11.367856 32494 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.370873 32556 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:11.372781 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.373821 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.374897 30700 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:11.376410 32564 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.074s	user 0.000s	sys 0.002s
W20260709 13:14:11.335182 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80116.6us
I20260709 13:14:11.379235 32396 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.381467 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.370658 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:11.365803 32499 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.275770 32501 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.392489 32140 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:11.393637 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:11.406876 31394 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:11.409583 30553 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:11.325003 32495 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.410856 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:11.423094 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.423399 32294 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:11.393687 32140 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:11.420186 30552 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:11.385035 30551 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:11.413935 30380 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 132) took 10166 ms. Trace:
W20260709 13:14:11.423584 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 139409us
I20260709 13:14:11.390262 30626 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:11.410441 32499 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:11.436751 31391 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:11.454833 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.461314 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:11.435664 32294 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:11.441972 32523 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:11.443025 32384 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:11.490854 32524 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:11.491998 32524 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:11.479039 32418 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:11.500746 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.501087 32418 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:11.501881 30702 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:11.505820 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48112.5us
I20260709 13:14:11.453358 32400 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:11.455888 32014 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:11.521957 32492 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.527788 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.525346 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.442884 32140 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.458128 32445 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.491499 32207 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:11.520457 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 127020us
I20260709 13:14:11.475014 32500 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.548933 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:14:11.551007 32559 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:14:11.562186 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.553228 30378 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.553361 32524 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:11.570863 32207 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:11.578655 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 52507.8us
I20260709 13:14:11.532677 32418 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.541075 32400 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.562462 32501 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.549115 32492 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.602212 30378 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:11.603224 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.591784 32523 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:11.605474 32297 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:11.609009 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:11.609910 30700 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:11.610677 30707 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:11.614054 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.618410 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.620070 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.620333 32400 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:11.621574 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:11.637593 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.622119 32207 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:11.622937 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.623638 32455 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:11.641714 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.631157 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.642701 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:11.634693 30702 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:11.648337 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.648738 32149 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:11.641162 32523 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:11.649482 32149 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.651441 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.651746 32523 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.656481 30703 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:11.642946 32533 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:14:11.673171 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 134723us
I20260709 13:14:11.651317 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.678766 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.588050 32294 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.432170 30380 rpcz_store.cc:276] 0709 13:14:01.247665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:01.266760 (+ 19095us) service_pool.cc:224] Handling call
0709 13:14:11.413695 (+10146935us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:09.509805 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:09.510717 (+   912us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:09.510773 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:14:09.510812 (+    39us) tablet.cc:662] Decoding operations
0709 13:14:09.963468 (+452656us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:09.990163 (+ 26695us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:09.997450 (+  7287us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:14:10.087497 (+ 90047us) tablet.cc:701] Row locks acquired
0709 13:14:10.087520 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:14:10.087705 (+   185us) write_op.cc:270] Start()
0709 13:14:10.087847 (+   142us) write_op.cc:276] Timestamp: P: 1783602850087648 usec, L: 0
0709 13:14:10.087869 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:10.088712 (+   843us) log.cc:844] Serialized 6309 byte log entry
0709 13:14:10.718013 (+629301us) op_driver.cc:464] REPLICATION: finished
0709 13:14:10.841332 (+123319us) write_op.cc:301] APPLY: starting
0709 13:14:10.842987 (+  1655us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:10.847678 (+  4691us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:10.847815 (+   137us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:10.849572 (+  1757us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:10.849853 (+   281us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:14:10.849901 (+    48us) write_op.cc:312] APPLY: finished
0709 13:14:10.850303 (+   402us) log.cc:844] Serialized 112 byte log entry
0709 13:14:10.850557 (+   254us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:10.850791 (+   234us) write_op.cc:454] Released schema lock
0709 13:14:10.850861 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:10.851742 (+   881us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:10.851778 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3287815,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":104966,"apply.run_cpu_time_us":4264,"apply.run_wall_time_us":12964,"num_ops":16,"prepare.queue_time_us":134537,"prepare.run_cpu_time_us":17841,"prepare.run_wall_time_us":650481,"raft.queue_time_us":365751,"raft.run_cpu_time_us":16146,"raft.run_wall_time_us":100676,"replication_time_us":629926,"thread_start_us":525394,"threads_started":4,"wal-append.queue_time_us":99133}]]}
I20260709 13:14:11.659469 32149 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.693179 31394 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:11.670667 32533 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.680778 32445 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.697767 32337 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.698531 32337 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:11.699084 32563 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 13:14:11.697438 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.698259 32455 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.700708 32533 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:11.709368 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.710356 32493 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.707605 31388 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:11.703236 32455 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:11.718400 30626 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:11.717077 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:11.722990 32315 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.723819 32315 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:11.715381 30378 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.718449 32529 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:11.734552 32529 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:11.716840 32500 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:11.733299 32493 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:11.737502 32207 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:11.737726 30627 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:11.738162 32207 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:11.738649 32207 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:11.739498 32493 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:11.743050 32140 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:11.749399 32565 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.180s	user 0.000s	sys 0.002s
I20260709 13:14:11.741235 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:11.746392 32480 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.564s	user 0.015s	sys 0.000s
I20260709 13:14:11.759889 30378 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:11.752576 32567 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:14:11.760828 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:11.768998 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:11.773335 32568 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:14:11.773876 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.773365 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:11.775210 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.777988 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:11.776629 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:11.777602 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:11.781877 31367 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:11.794831 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:11.798496 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 103953us
I20260709 13:14:11.800420 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:11.807371 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.808786 32495 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.809692 32495 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:11.812903 32495 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.813165 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.813549 30628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:46186
I20260709 13:14:11.814116 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:11.815369 30552 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:11.815994 30624 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:11.821321 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:11.823624 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:11.825047 32495 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:11.825661 32495 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:11.826368 32495 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:11.828423 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.830360 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:11.838543 32480 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:11.839148 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:11.839776 32480 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:11.833470 30378 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 4 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 5 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:11.853281 32480 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.854025 32384 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:11.856123 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.857023 30378 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 2, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:11.859948 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.862147 32464 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:11.864835 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.854429 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:11.868245 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:11.879889 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:11.775532 32529 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.870541 31388 raft_consensus.cc:2368] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
W20260709 13:14:11.865722 30523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.602s (SENT) [suppressed 11 similar messages]
I20260709 13:14:11.910840 31396 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:11.919839 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:11.872730 32492 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:11.911180 32149 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:11.941807 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:11.946998 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84268.4us
I20260709 13:14:11.948436 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:11.953931 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 99600.5us
W20260709 13:14:11.956223 31389 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.759s	user 0.018s	sys 0.001s
I20260709 13:14:11.958770 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:11.964744 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:11.957669 31389 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:11.965112 30627 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:11.966645 31391 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:11.983095 32384 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:11.986047 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:11.990521 32562 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.459s	user 0.004s	sys 0.002s
I20260709 13:14:11.945019 32523 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:14:12.007496 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.000043 31389 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:12.010011 31396 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:14:12.012024 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53375.9us
I20260709 13:14:11.969799 32575 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.019404 32575 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:12.016690 32570 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:12.013047 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.025842 30627 raft_consensus.cc:1217] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Deduplicated request from leader. Original: 3.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:14:12.026702 30627 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:14:12.012578 31388 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:12.028450 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 179971us
I20260709 13:14:12.031733 31383 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:12.032023 32418 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:12.032683 32418 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:12.033784 32396 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.648s	user 0.014s	sys 0.001s
I20260709 13:14:12.034709 32207 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.035655 32418 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.035854 30628 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:12.036307 30628 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:12.037873 32207 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.044705 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:12.025843 32384 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:12.048281 32315 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.053310 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.054388 30626 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:12.055115 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.055598 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.055231 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.056296 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.055281 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.058784 30629 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:12.063212 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.065164 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:12.066577 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:12.019605 32455 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.078243 30629 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:12.083427 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.085278 30633 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:12.091195 31396 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:12.086336 32384 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.081259 30628 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.097086 30631 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 3. Ops: []
W20260709 13:14:12.098805 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:12.098939 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97975.3us
W20260709 13:14:12.100391 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 93013.7us
W20260709 13:14:12.093782 32574 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:14:12.067550 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:12.098179 32455 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:12.104857 30625 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:12.107237 32337 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:12.111100 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:12.112311 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:12.118804 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.133778 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.137889 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.068765 32533 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.155498 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.153687 30670 leader_election.cc:400] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:12.165563 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:12.165921 31389 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:12.131872 31388 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:12.132001 32294 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:12.170773 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.166721 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:12.180656 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 125748us
W20260709 13:14:12.169704 32499 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.723s	user 0.013s	sys 0.003s
I20260709 13:14:12.193063 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.212399 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.220537 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.221709 30633 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:12.222733 32579 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:14:12.223274 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:12.193580 32294 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:12.225284 30550 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:12.161549 32455 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:12.226601 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:12.232201 32207 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:12.228114 31856 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:12.233706 31856 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.234228 32396 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:12.190929 32495 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:12.220619 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.237287 32495 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:12.237663 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.238308 32297 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:12.230509 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.240036 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.231752 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:12.231315 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49629.6us
I20260709 13:14:12.231727 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.248504 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.251631 30627 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:12.242956 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.252991 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.243377 30630 raft_consensus.cc:2376] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5. Re-sending same reply.
I20260709 13:14:12.255635 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.253645 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.259691 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:12.260892 30700 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:12.272110 31856 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.272681 32274 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:12.273188 32274 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:12.274091 32506 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:12.275987 32576 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:14:12.277245 32506 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:12.278843 32500 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:12.279310 32500 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:12.280241 32500 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.281572 32500 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:12.283205 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.228574 32455 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:12.285802 30698 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:12.286579 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 184818us
I20260709 13:14:12.260839 32495 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:14:12.293792 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50943.8us
I20260709 13:14:12.257364 32294 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:12.296324 32573 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.411s	user 0.006s	sys 0.000s
I20260709 13:14:12.299902 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.253378 32418 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:14:12.309701 30553 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:12.285261 32400 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:14:12.291147 32524 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:14:12.309964 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 171081us
I20260709 13:14:12.311015 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:12.316813 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 141269us
I20260709 13:14:12.298497 32529 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:12.307498 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.315339 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.352145 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.299227 32495 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:12.357321 32298 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.358129 32495 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:12.359622 32495 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:12.311231 32516 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:14:12.293504 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.311930 32533 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:12.367569 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52351.4us
I20260709 13:14:12.315513 32207 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.392438 32516 consensus_queue.cc:1243] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:12.387691 32298 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.395535 31364 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:12.399576 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54909.5us
I20260709 13:14:12.380570 32523 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.401437 32523 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:12.365638 32586 consensus_queue.cc:1175] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 5 status { last_received { term: 3 index: 2 } last_committed_idx: 2 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: [4.3-4.3]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:12.403666 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:12.380390 32400 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.402458 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:12.408317 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 123756us
I20260709 13:14:12.410300 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:12.410539 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 136017us
W20260709 13:14:12.408566 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 97655.1us
I20260709 13:14:12.347381 32579 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:14:12.417582 30633 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:12.419629 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 154481us
I20260709 13:14:12.433660 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.426355 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.435145 30629 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:12.417744 32480 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:12.432653 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:12.437393 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.437105 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:12.438242 30631 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:12.406908 32274 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:12.441090 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 133738us
I20260709 13:14:12.425786 32297 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.449296 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:12.450825 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.459007 30551 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:14:12.460886 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70238.4us
I20260709 13:14:12.462136 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.462316 32464 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:12.462728 32400 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:12.463943 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.437970 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:12.429956 32533 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:14:12.467806 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:12.456579 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.454322 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:12.470221 31391 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:12.479853 32480 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:12.484802 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:12.451979 32207 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:12.485680 30698 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:12.455546 32140 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:12.455869 32149 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:12.502172 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45701.3us
I20260709 13:14:12.502331 30628 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:12.470364 32556 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:12.474893 32400 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:12.470674 32384 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:14:12.484913 31388 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:12.507202 32499 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.509711 32014 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:12.519906 32014 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:12.521322 32014 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:12.523115 32014 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:12.527093 32499 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:12.531045 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:12.531231 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:12.467396 32298 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:14:12.511811 32587 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:12.508172 32590 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:12.495923 32261 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:14:12.548094 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93843.2us
I20260709 13:14:12.575011 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:12.577181 31396 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:12.511953 32583 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
W20260709 13:14:12.583918 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 69749.7us
I20260709 13:14:12.545207 32587 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:12.584641 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:12.589808 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.591138 32584 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
W20260709 13:14:12.592051 32595 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:14:12.593854 32597 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:14:12.594769 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 130919us
I20260709 13:14:12.621227 32592 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:14:12.629215 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 247831us
I20260709 13:14:12.580895 32140 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:14:12.596041 32506 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:14:12.677001 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:12.678139 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:12.687920 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 219220us
I20260709 13:14:12.660789 32455 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:12.693168 30703 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:12.694988 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.610929 32400 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:12.705307 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:12.701050 32464 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.706338 30700 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:12.713122 30700 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:12.681707 32603 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:12.713634 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:12.718500 32601 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:12.719105 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:12.719822 32600 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:14:12.721458 32596 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 13:14:12.724407 32588 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:14:12.729427 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 277) took 1111 ms. Trace:
I20260709 13:14:12.729986 30553 rpcz_store.cc:276] 0709 13:14:11.618288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:11.628455 (+ 10167us) service_pool.cc:224] Handling call
0709 13:14:12.729193 (+1100738us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540067b6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:12.729321 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1292042112}
I20260709 13:14:12.736198 30558 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:12.736819 32396 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:12.737398 32396 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:12.700028 32014 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:14:12.749679 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:12.750000 30698 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:12.749213 30700 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.696998 32584 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:14:12.791337 30700 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:12.743788 32603 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:12.676204 32587 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:12.834110 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 238460us
I20260709 13:14:12.822671 32274 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:12.738365 32396 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:12.780591 32592 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:12.850284 32616 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:12.856050 32529 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.500s	user 0.011s	sys 0.001s
I20260709 13:14:12.859474 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:12.862597 32590 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:12.863332 32590 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:12.863708 32556 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:12.864751 31367 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:12.840296 31856 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:12.864594 32590 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:12.876238 30700 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:14:12.879200 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.873472 31856 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:12.885998 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 301393us
I20260709 13:14:12.887300 32533 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:12.892381 32533 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:12.886966 32592 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:12.888254 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.889302 32501 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:12.890846 31856 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:12.905282 32533 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:12.894490 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.907117 32533 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:12.905335 31856 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:12.908967 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:12.909828 30630 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:12.901315 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:12.900903 32501 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:12.899488 32495 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.539s	user 0.015s	sys 0.002s
W20260709 13:14:12.916347 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88251.5us
I20260709 13:14:12.922539 31396 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:12.923579 32618 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:12.925403 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:12.923285 32315 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:12.924365 32501 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:12.936020 32501 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:12.932441 30558 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
W20260709 13:14:12.971674 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76921us
I20260709 13:14:12.995646 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.000193 32604 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:13.001003 32604 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:13.029323 32604 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.043342 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.048170 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:13.025831 32294 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:13.061213 32294 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:13.062695 32294 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:13.070124 32096 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
W20260709 13:14:13.074009 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 204261us
I20260709 13:14:13.025884 32575 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:13.078563 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:13.102545 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 120572us
I20260709 13:14:13.105194 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.106400 32556 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:13.110790 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.115157 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.080013 32575 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.116760 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.118125 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:13.118911 32575 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.099961 32618 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:13.120573 32618 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:13.120981 32618 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:13.121120 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.122988 32501 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:13.123698 32501 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:13.124507 30700 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:13.125720 32556 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:13.126255 32556 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.128042 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.128696 31367 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:13.128044 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.129848 31378 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:13.130896 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.131297 31396 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:13.132568 32556 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:13.133143 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.136354 30700 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:13.138918 30629 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: [3.3-5.4]
W20260709 13:14:13.151324 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:13.164557 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.131238 32501 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:13.174897 32579 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:13.175778 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.177105 32274 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.177640 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.177992 32583 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.177835 32274 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.179869 32495 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:13.180564 32495 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.181035 32495 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:13.184932 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.185209 32274 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.175781 32579 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:13.200372 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 95268.4us
I20260709 13:14:13.201309 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.205986 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.212186 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.212347 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.214161 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:13.230984 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:13.211509 32480 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.211350 32274 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:13.201943 32575 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:13.220104 32529 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:13.220715 32556 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.239652 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.216547 32315 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:13.178574 32583 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.262418 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.238605 32579 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:13.251592 32529 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:13.269964 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 112709us
I20260709 13:14:13.255594 32315 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:13.279734 32315 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:13.275933 32445 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.251151 32274 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.292910 32274 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.296629 32603 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.292914 32583 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.299804 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.300611 30626 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:13.261503 32586 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:13.274150 32529 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:13.268968 32480 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.309589 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.281592 32445 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:13.294471 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 93267.9us
I20260709 13:14:13.254356 32575 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.276996 32633 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:13.249387 32602 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:13.313165 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69106.1us
I20260709 13:14:13.319027 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.317409 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.322602 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.328747 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.334089 30700 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.339320 32644 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:13.339520 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.340023 32644 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.342185 32644 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:13.320449 32634 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.343518 32634 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:13.327538 32337 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:14:13.332036 32014 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.346342 32014 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.347180 32575 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.348541 32014 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.350370 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.328511 32640 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:13.352533 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.356153 32445 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:13.325523 32516 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.371415 30707 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:13.375514 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:13.379136 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 166745us
I20260709 13:14:13.342756 32644 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:13.381850 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48232.8us
I20260709 13:14:13.384068 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.312548 32586 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.385109 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.387086 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.388522 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.326292 32618 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:13.383785 32445 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:13.392518 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.387317 32516 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.399736 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:13.407078 32533 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.402740 32640 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:14:13.407868 32533 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.405783 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.397962 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.410239 32533 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.382396 32480 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.401007 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:13.410931 32480 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.411619 32445 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:13.405608 30558 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:13.413017 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.413174 31367 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:13.413892 32445 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:13.399359 32618 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:13.393401 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.415088 32495 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.416929 32609 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.418201 32337 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:13.412508 32586 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.418550 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.420686 32586 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:13.421720 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.426198 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.426518 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.426766 30554 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:13.427019 30555 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:13.421667 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.432935 32584 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:13.433142 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.433424 32584 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.434481 32584 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.413317 32480 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.439639 32618 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:13.440945 32495 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.425408 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.442504 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.442682 32615 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:13.443732 32615 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:13.398254 30557 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 1. Current term is 3. Ops: []
I20260709 13:14:13.444046 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.448041 32294 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.448828 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.449383 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.449793 32014 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.449947 32587 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.451022 32633 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.451601 32633 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.452165 30707 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:13.454078 32633 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.454273 32590 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:13.455194 32294 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.458913 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.458988 32634 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:13.458953 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:13.459647 30554 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:13.465785 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.467268 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:13.469231 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:13.474222 32618 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:13.474860 32618 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:13.475857 32618 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.476303 31389 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:13.478662 32464 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.490707 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.492311 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.494887 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.511919 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.512387 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.513505 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.514947 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.514221 30625 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:13.479344 32464 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.518121 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.519479 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.519857 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:13.528259 30628 raft_consensus.cc:2376] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6. Re-sending same reply.
I20260709 13:14:13.519862 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.529529 30557 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:13.530510 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:13.531318 31367 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.533382 31389 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:13.535223 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.538798 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.537367 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:13.538868 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:13.542769 31378 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:13.387766 32644 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:13.517305 32586 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:13.516903 32533 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:13.531585 32556 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.518769 32140 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:13.527204 32495 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:13.538676 32602 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:13.580582 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.571276 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:13.595978 32495 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.595539 32608 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:14:13.558980 32384 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:13.611522 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.613370 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.558861 32480 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:13.619154 32464 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.619841 32140 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:13.624648 32396 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:13.628535 32644 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.575927 32642 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.630319 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:13.630802 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:13.631120 31388 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:13.607622 32495 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:13.635954 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.636976 32529 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.608791 30624 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.639555 32529 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.563407 32400 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:14:13.630129 32642 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.648406 31367 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:13.650971 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.622392 32140 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:13.606788 32384 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.675498 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.680649 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:13.676510 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:13.575496 32592 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:14:13.690462 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.662627 32609 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:13.684442 32014 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:13.708228 30627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:13.709174 30627 raft_consensus.cc:606] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.722235 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.683717 32384 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:13.724953 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.679564 32523 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.684580 32583 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:13.723048 30701 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:13.701809 32609 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:13.731941 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:13.757000 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:13.759545 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:13.753621 30625 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:13.765988 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:13.770797 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.746987 32609 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:13.791769 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:13.793221 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:13.793952 30704 raft_consensus.cc:606] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.761592 32583 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:13.776467 32602 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:13.740623 32523 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.772379 32556 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:13.774668 32651 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:13.817615 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48967.8us
I20260709 13:14:13.820015 32583 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:13.793262 32644 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:13.822885 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:13.800074 32615 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:13.853096 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:13.827518 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67051.1us
I20260709 13:14:13.929499 31388 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:13.900573 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.909857 32207 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.930910 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:13.829981 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:13.967620 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72181014679a4c538f96f2d305a5474f"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:13.968667 30704 raft_consensus.cc:606] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:13.976416 31378 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
W20260709 13:14:13.844189 32646 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:14:13.899271 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58734.8us
W20260709 13:14:13.985797 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55026.1us
I20260709 13:14:13.934561 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:13.914654 30624 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
W20260709 13:14:13.997735 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 253105us
W20260709 13:14:13.998584 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64159.8us
I20260709 13:14:14.001803 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:14.001995 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172138us
I20260709 13:14:13.964466 32533 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.004586 30551 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:14.007068 31367 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:13.973843 32298 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:14.008627 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:13.976303 32556 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:13.863097 32642 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.998811 32529 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:14.046763 32618 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:14.047211 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44374us
I20260709 13:14:14.041693 32556 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.049652 32480 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:14.050550 32480 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:14.051841 32647 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:14:14.053109 32480 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:13.976871 32609 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:14.053838 32400 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:14.055362 30698 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:14.055658 32618 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:14.056165 32609 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:13.977535 32207 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.056763 32618 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.058604 32207 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:13.848568 32615 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.064862 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:14.074424 31378 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:14.076442 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:14.077252 30552 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.003809 32506 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:14.006484 32644 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:14.082792 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:14.083544 31388 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.050311 32556 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:14.015650 32584 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.003s
I20260709 13:14:14.010442 32298 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:14.092300 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49401us
I20260709 13:14:14.062743 32529 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.063143 32587 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.099438 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.098440 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.099781 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.100505 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.100733 30633 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:14.082093 32506 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:14.115754 32523 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.118959 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.119971 30631 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:14.120174 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.120893 31389 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:14.066670 32495 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:14.122776 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:14.123889 32523 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:14.124334 32523 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.124704 32523 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.124779 32495 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:14.112118 32499 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:14:14.126796 32499 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:14.127178 32499 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:14.127529 32615 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.127962 32384 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.128803 32656 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:14.099126 32529 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.130077 32652 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:14.128392 30701 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:14.097395 32584 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.134193 32584 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.134078 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:14.095001 32298 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.135912 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:14.136234 32579 raft_consensus.cc:993] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:14.128031 32499 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.151031 31388 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
W20260709 13:14:14.152635 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59042.2us
I20260709 13:14:14.156047 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:14.013957 32533 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:14.149856 30552 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:14.162108 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:14.097469 32643 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:14.189823 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:14.200541 30631 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:14.191257 30558 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.151743 32506 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:14.210393 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48424.8us
I20260709 13:14:14.212105 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:14.152871 32384 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.164575 32586 raft_consensus.cc:1081] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:14.223322 30700 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:14.224026 30700 pending_rounds.cc:85] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 3
I20260709 13:14:14.182654 32495 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:14.230947 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:14.224350 30700 pending_rounds.cc:107] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:14.243978 30700 raft_consensus.cc:2889] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:14.107735 32587 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.218196 32634 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:14.165464 32656 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 13:14:14.229905 32602 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:14.225037 32499 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.272789 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.248585 32644 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:14.203490 32643 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.253665 32495 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:14.286573 30558 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:14.294840 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.260474 32658 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.297210 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.301528 30552 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:14.298502 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:14.298955 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88953us
I20260709 13:14:14.265789 32400 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.271090 32602 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.309156 31388 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:14.303691 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:14.306437 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:14.307260 32586 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:14.313217 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50024
I20260709 13:14:14.375180 31361 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:14.375007 32586 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.375758 31361 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.311141 32602 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.315325 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.378207 30631 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:14.378341 32664 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:14:14.315168 32533 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:14.323154 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:14.324432 32643 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.325148 32495 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:14.310848 32400 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.358966 32644 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:14.362468 32384 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:14.364650 32587 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:14.362190 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55839.4us
I20260709 13:14:14.382753 30704 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:14.349823 32523 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.396615 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.404524 31361 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.412652 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.413841 31394 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
W20260709 13:14:14.451427 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 147712us
I20260709 13:14:14.451905 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:14.454313 31361 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.394375 32644 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:14.308835 32658 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.459242 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:14.421582 32384 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:14.453840 32586 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.475271 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.461688 32644 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.463176 32464 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:14.448909 32529 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.478999 32014 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:14.507416 30703 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:14.516474 32529 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.519699 32400 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:14.489781 32464 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.522992 32464 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.526207 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:14.527915 32667 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:14:14.501339 32014 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.535141 32014 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.497444 32666 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:14:14.498312 32598 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.541937 32598 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.542228 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:14.542824 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:14.543268 31361 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.543550 30701 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:14.544410 32598 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.544718 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:14.545051 32598 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:14.545380 30705 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.547487 30699 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:14.549005 30702 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:14.526894 32298 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.551762 32586 raft_consensus.cc:993] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:14.552393 32586 raft_consensus.cc:1081] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:14.553227 30551 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:14.522993 32384 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.561057 32610 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:14.560889 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.561645 32610 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:14.568831 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.548552 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.581460 30700 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:14.590854 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.594213 30551 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:14.601261 31367 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "72181014679a4c538f96f2d305a5474f"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50024
W20260709 13:14:14.601707 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 113764us
I20260709 13:14:14.602157 31367 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:14.602751 31367 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.617476 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:14.590044 32337 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:14.619284 32337 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.619530 32602 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.616430 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:14.567889 32610 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.628865 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:14.634585 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.644824 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.645104 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.640838 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:14.645610 30633 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:14.638993 31367 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.641904 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:14.650035 30551 raft_consensus.cc:2368] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:14.647464 30553 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:14.622480 32337 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.655663 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:14.656143 31389 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:14.657552 32670 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:14.650784 32261 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:14.659015 32261 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.659147 32658 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.659482 32261 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:14.660207 32602 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.635831 32501 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:14.661127 32501 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:14.661593 32501 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.662149 32610 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.635007 32298 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.647282 32669 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:14.675074 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.678989 30673 leader_election.cc:400] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:14.679477 30673 leader_election.cc:403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:14.687018 31367 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.689317 32140 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:14:14.693493 32593 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:14.694054 32593 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:14.646863 32672 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:14:14.716302 30705 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
W20260709 13:14:14.714700 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62064us
I20260709 13:14:14.676340 32495 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:14.718808 32495 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.721146 32495 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:14.685076 32669 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.724792 32669 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.726457 32464 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4.
I20260709 13:14:14.731048 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.733064 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:14.734028 30700 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:14.739349 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:14.744273 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:14.745464 32400 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:14.746018 32400 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:14.746376 32400 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:14.747845 31383 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:14:14.752735 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97121.3us
I20260709 13:14:14.787622 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:14.739017 32261 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:14.797329 32506 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.572s	user 0.004s	sys 0.010s
I20260709 13:14:14.856437 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.856760 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.858705 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.850360 32516 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:14.863806 30633 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:14:14.863709 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.864053 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.864403 30627 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:14.864755 30624 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:14.867132 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.867749 30624 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.848027 32657 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:14.857856 32500 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:14.874909 32657 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.875039 32500 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.876057 30625 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:14.876495 32261 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.877410 32500 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.879029 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.879418 32014 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:14.853034 32506 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.882910 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:14.883332 32506 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:14.887924 32657 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.889110 32516 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:14.889277 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.889812 32516 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:14.890374 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.890789 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.891119 32516 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.893775 30625 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:14.851888 32615 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.089s
I20260709 13:14:14.953918 30633 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:14.901705 32644 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.948166 30552 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:14.950059 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:14.963562 30630 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:14.964584 32583 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:14.965977 32464 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:14.966753 32464 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:14.966835 32602 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:14.967738 32464 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:14.968698 30628 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:14.968835 32633 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:14.969465 32633 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:14.969683 32464 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:14.970844 32516 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:14.907059 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119527us
I20260709 13:14:14.970389 32633 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.942678 32523 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:14.944751 32445 raft_consensus.cc:993] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:14.917618 32014 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:14.925016 32400 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:14.940248 30382 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 6, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 6 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:14.905750 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:14.909289 32653 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:14:14.974079 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.910380 30627 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:14.908586 32501 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:15.007329 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.009282 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.009541 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.010272 31383 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:14.917263 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.011271 31394 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:15.012207 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 95051.2us
I20260709 13:14:15.014683 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:15.015497 31389 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:14:15.015664 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.018448 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:15.019421 31364 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.020766 32400 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:15.023000 32644 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:15.024516 30557 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:14.974124 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 154) took 14966 ms (client timeout 15000 ms). Trace:
W20260709 13:14:15.028594 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:14.976854 32556 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:14.979477 32593 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:14.990924 30630 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:14.997694 32633 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:15.051998 32593 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:14.998751 30658 heartbeater.cc:646] Failed to heartbeat to 127.21.5.190:45121 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.5.190:45121 timed out after 15.000s (SENT)
I20260709 13:14:15.032487 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.055342 32662 raft_consensus.cc:2764] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:14:14.999856 30382 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 3 to 6, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 6 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:15.056916 30382 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 3 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:15.003273 30632 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:14.983810 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:15.000185 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 356618us
I20260709 13:14:15.061487 32668 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.062675 32669 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:15.062935 32556 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:15.063081 32669 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:15.063308 31361 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:15.063699 32556 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.064214 32669 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.036516 32688 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.005154 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:14.975932 32445 raft_consensus.cc:1081] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:15.003587 32653 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:14.973362 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.038657 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.028851 31364 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:15.039598 32501 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:15.039664 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:15.040990 32583 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:15.041010 30631 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:15.042078 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:15.028851 30624 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:15.058540 30382 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:14.997296 32523 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.066547 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:15.091316 31394 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:15.091192 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:15.067278 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:15.092332 30552 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.067281 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:15.068457 32670 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.037720 31292 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:14:15.085453 30702 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:15.094889 31292 leader_election.cc:403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:15.095569 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58023.4us
I20260709 13:14:15.078037 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.096305 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.097601 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.071889 31389 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:14:15.086925 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 113681us
I20260709 13:14:15.039327 31396 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [4.3-5.4]
I20260709 13:14:15.087845 32650 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:15.097635 30382 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:14:15.049991 30378 rpcz_store.cc:269] 0709 13:14:00.025913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:00.094401 (+ 68488us) service_pool.cc:224] Handling call
0709 13:14:14.973982 (+14879581us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:12.101088 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:12.223169 (+122081us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:12.223261 (+    92us) write_op.cc:435] Acquired schema lock
0709 13:14:12.223281 (+    20us) tablet.cc:662] Decoding operations
0709 13:14:12.224258 (+   977us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:12.224363 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:12.224399 (+    36us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:14:12.225120 (+   721us) tablet.cc:701] Row locks acquired
0709 13:14:12.225176 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:14:12.225390 (+   214us) write_op.cc:270] Start()
0709 13:14:12.225567 (+   177us) write_op.cc:276] Timestamp: P: 1783602852225311 usec, L: 0
0709 13:14:12.225604 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:14:12.278994 (+ 53390us) log.cc:844] Serialized 2463 byte log entry
0709 13:14:12.675585 (+396591us) op_driver.cc:464] REPLICATION: finished
0709 13:14:12.965480 (+289895us) write_op.cc:301] APPLY: starting
0709 13:14:12.965635 (+   155us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:12.965967 (+   332us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:12.966181 (+   214us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:12.967491 (+  1310us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:12.967955 (+   464us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:14:12.968029 (+    74us) write_op.cc:312] APPLY: finished
0709 13:14:12.968534 (+   505us) log.cc:844] Serialized 52 byte log entry
0709 13:14:12.997100 (+ 28566us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:13.012618 (+ 15518us) write_op.cc:454] Released schema lock
0709 13:14:13.024893 (+ 12275us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:13.058994 (+ 34101us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:13.059028 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":24574,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":268657,"apply.run_cpu_time_us":7721,"apply.run_wall_time_us":93525,"num_ops":6,"prepare.queue_time_us":60398,"prepare.run_cpu_time_us":26027,"prepare.run_wall_time_us":397275,"raft.queue_time_us":140896,"raft.run_cpu_time_us":54860,"raft.run_wall_time_us":292365,"replication_time_us":426718,"thread_start_us":442587,"threads_started":4,"wal-append.queue_time_us":190858,"wal-append.run_cpu_time_us":64913,"wal-append.run_wall_time_us":1563631}]]}
I20260709 13:14:15.103502 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.105104 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:15.105777 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.109248 32687 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:14:15.109862 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:15.088459 32653 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:15.111053 32670 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.111734 30382 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:15.113488 32670 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:15.114271 30626 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:15.114463 32495 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:15.105667 32523 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:15.114391 32014 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:15.118827 32644 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:15.119061 32014 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:15.119499 32644 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:15.072407 32657 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
I20260709 13:14:15.120594 32644 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.120997 32529 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:15.121917 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.122413 32480 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:15.122601 32529 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.122648 32634 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:15.123442 32480 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:15.123847 32480 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:15.124183 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:15.125535 31361 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:15.126163 31389 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:15.124730 32480 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.130931 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:15.131892 30554 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:15.132543 30702 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:15.133457 32583 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:15.133649 32602 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:15.134708 32495 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:15.135183 32495 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:15.135594 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.136179 32495 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.137701 32014 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.082904 32688 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.142167 32480 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:15.143477 32688 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:15.144510 32556 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:15.159996 30552 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:15.108979 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.115274 30382 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 7, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:15.165390 31394 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
W20260709 13:14:15.167923 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 80489.4us
I20260709 13:14:15.172005 32464 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:15.174263 32523 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:15.176656 32583 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.180591 32669 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:15.181483 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:15.232795 30705 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:14:15.234966 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 126063us
I20260709 13:14:15.235924 31292 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:15.236240 31292 leader_election.cc:403] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:15.237396 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.238247 30705 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:15.238988 32643 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.238823 32602 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:15.243122 31367 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50010
I20260709 13:14:15.243752 31367 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:15.244150 31367 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:15.238534 32644 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:15.246832 32644 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:15.245774 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.249655 31394 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:14:15.220914 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 85465.6us
I20260709 13:14:15.224323 30382 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:15.229154 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.209537 30701 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:15.263438 32688 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.264190 32688 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.266816 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:15.267026 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:15.268085 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:15.258807 30382 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 5 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:15.270279 30382 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 6, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 6 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:15.271497 32523 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:15.272130 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:15.272951 32523 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.273742 30382 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:15.275605 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.278417 30382 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 3, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:14:15.280169 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51272.5us
I20260709 13:14:15.282639 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.283408 30632 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:15.284775 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.285375 30632 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:15.286891 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.287727 30632 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:15.261737 31367 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.289245 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:15.290709 30382 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:15.292987 30382 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:15.295110 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.278417 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:15.295437 30382 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:15.297256 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.311543 30382 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:15.310346 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:15.323086 31367 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:15.318854 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:15.326349 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:15.326226 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.250159 32464 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:15.266542 32688 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:15.310145 32633 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:15.318966 32501 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:15.319366 32593 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:15.353232 30551 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:14:15.384114 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51691.8us
I20260709 13:14:15.309638 32660 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:15.274597 32523 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:14:15.372290 30557 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:15.379452 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 255360us
I20260709 13:14:15.327917 32602 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:15.324182 32618 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:15.382863 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113938us
I20260709 13:14:15.297812 32500 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:14:15.388008 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:15.390115 30701 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:15.394542 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:15.390964 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.349735 32608 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:15.349195 32681 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:15.399264 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:15.377942 32633 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:15.395797 32602 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.366230 32464 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.396332 32618 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.397621 32495 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:15.396095 32501 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:15.335974 32644 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.408632 32633 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:15.439343 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51446.2us
I20260709 13:14:15.448931 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:15.405534 32660 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:15.451434 30632 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:15.388387 32653 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.410346 32688 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:15.409518 32593 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:15.418773 32602 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:15.457352 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62914.7us
I20260709 13:14:15.428054 32618 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:15.470158 31394 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:15.446337 32529 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:15.445876 32668 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:15.447813 32014 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:15.409812 32692 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:15.473034 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:15.486805 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.478417 32583 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:15.490213 32501 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:15.492163 32653 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:15.494271 32014 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:15.497602 32014 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:15.500699 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.458801 32608 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:15.503726 32014 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:15.469655 32445 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:14:15.514883 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:15.477174 32688 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:15.476938 32691 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:15.553610 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 162748us
I20260709 13:14:15.555624 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:15.500003 32529 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:15.505725 32692 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.576839 32701 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:15.577898 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:15.552588 32691 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:15.579381 32688 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.618221 32691 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:15.591112 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50920.8us
I20260709 13:14:15.597384 32701 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:15.620364 32701 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:15.556334 32662 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260709 13:14:15.632575 32652 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:15.633380 32652 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:15.590245 32618 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.638572 30632 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:14:15.640201 31396 raft_consensus.cc:2368] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7.
I20260709 13:14:15.644300 30700 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:15.647570 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:15.648401 30700 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
I20260709 13:14:15.542897 32660 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.617033 30699 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
W20260709 13:14:15.591295 32685 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 13:14:15.658514 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.664618 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.665400 30701 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:15.665726 30701 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:15.666605 30701 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.669087 30701 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:15.660034 32695 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.275s	user 0.005s	sys 0.000s
I20260709 13:14:15.674386 30705 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:15.658051 30699 raft_consensus.cc:614] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:15.684645 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:15.680375 30701 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
W20260709 13:14:15.693418 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 166187us
I20260709 13:14:15.706642 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:15.717041 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:15.567232 32593 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.714937 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:15.642452 32652 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:15.679116 32657 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:15.735663 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:15.736919 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:15.750128 30702 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:15.674912 32480 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:14:15.715551 32556 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:15.760568 32687 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.021s
I20260709 13:14:15.736285 31856 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:15.787500 30698 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:14:15.798043 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48948.9us
I20260709 13:14:15.805945 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:15.764263 32556 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:15.814747 30557 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:15.826676 30631 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:15.830250 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:15.774238 32657 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:15.834316 30700 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:15.822891 32699 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:15.828565 32464 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:14:15.852192 32701 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:15.822103 32692 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:15.860123 30702 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
W20260709 13:14:15.875969 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.878007 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:15.893464 30705 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:15.896878 30704 raft_consensus.cc:606] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:15.897493 30704 raft_consensus.cc:614] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:15.878281 32529 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:15.905045 32699 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:15.914048 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 154741us
I20260709 13:14:15.919960 31292 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5.
W20260709 13:14:15.924840 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 188838us
I20260709 13:14:15.925967 30522 leader_election.cc:400] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7.
I20260709 13:14:15.926280 30522 leader_election.cc:403] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:14:15.928699 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.928846 32708 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:15.929420 31389 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
W20260709 13:14:15.912146 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:15.875515 32556 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:15.947436 32556 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:15.938206 31856 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:15.953150 32693 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:14:15.936195 32680 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:15.932713 32699 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:15.954339 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:15.954767 31856 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:16.122931 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 191978us
I20260709 13:14:16.093907 32717 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:16.123667 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.157768 31396 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:16.159031 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:15.986716 32680 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:16.166993 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 70338.2us
I20260709 13:14:16.184397 31361 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:14:16.194787 31361 pending_rounds.cc:85] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:16.195039 31361 pending_rounds.cc:107] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:14:16.195304 31361 raft_consensus.cc:2889] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:16.169976 32701 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:16.182504 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:16.198741 32602 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.721s	user 0.014s	sys 0.001s
W20260709 13:14:16.214640 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84999us
I20260709 13:14:16.217022 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:16.147710 32717 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:16.200816 32713 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:14:16.223321 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172265us
W20260709 13:14:16.281891 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75167.6us
I20260709 13:14:16.283308 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:16.224326 32701 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:16.291183 32657 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.272608 32464 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.299968 30630 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:16.219529 32602 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:16.310714 32721 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:14:16.311733 32720 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
W20260709 13:14:16.312548 32726 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:14:16.345620 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252108us
I20260709 13:14:16.352025 30624 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:16.352584 30624 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:16.359601 31367 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:16.302744 32657 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:16.364159 32602 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.364996 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.309451 32464 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.375977 32717 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.377197 32669 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:16.377707 32669 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:16.378074 32669 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:16.366995 32732 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:14:16.380548 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.322155 32701 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.361853 30624 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.388917 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.389813 31378 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:16.392868 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:16.422705 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 149390us
W20260709 13:14:16.381054 32729 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.096s	user 0.000s	sys 0.002s
I20260709 13:14:16.389627 32699 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.381825 32400 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.449985 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:16.391893 32602 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:16.451771 32749 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:14:16.443892 32680 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:16.457309 32556 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.005s
I20260709 13:14:16.383092 32657 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:16.437765 32717 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:16.406446 32701 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.430784 32736 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:16.478385 31388 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:14:16.482488 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:16.451129 32747 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.477813 32717 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.466970 32400 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.471479 32602 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.559151 31391 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:16.519212 32556 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.541767 32722 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.582159 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.558027 32717 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:16.591619 31383 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:16.606771 32753 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:16.543644 32736 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:16.542797 32747 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.592726 32644 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:16.602789 32633 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.127s	user 0.018s	sys 0.000s
I20260709 13:14:16.617393 30703 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:14:16.607704 32708 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.004s
I20260709 13:14:16.606799 32668 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.607549 32753 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.591430 32556 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.626187 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:16.631058 32740 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:14:16.631489 32668 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:16.598114 32722 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.644505 32722 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.652787 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:16.662446 32753 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.662216 32744 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:16.646447 32464 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.634752 32708 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7.
I20260709 13:14:16.641592 32602 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.628086 32633 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.625248 32644 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:16.683162 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86124.9us
I20260709 13:14:16.684453 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.685287 31389 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:16.688041 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:16.689270 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:16.689998 30628 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.693768 30631 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:16.694216 30631 pending_rounds.cc:85] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 3
I20260709 13:14:16.694438 30631 pending_rounds.cc:107] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:16.694304 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:16.695003 30631 raft_consensus.cc:2889] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:16.694832 30625 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:16.699800 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:16.679593 32657 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.701082 31394 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:16.704439 32618 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.691469 32464 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:14:16.715440 32400 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.721175 32747 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.722208 32701 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.724411 32644 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.726751 32633 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.734252 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:16.733495 30631 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:16.736469 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:16.705268 32495 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:16.705420 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:16.706267 32736 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.707911 32668 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:16.751314 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46035.3us
I20260709 13:14:16.697286 32699 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:14:16.733754 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:16.691732 32602 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:16.754602 31394 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:16.770978 30626 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:16.735497 32704 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.783860 32704 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.785632 32669 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.786221 32464 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.786732 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1405) took 1111 ms. Trace:
I20260709 13:14:16.787106 32464 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:16.787214 30553 rpcz_store.cc:276] 0709 13:14:15.660002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.664165 (+  4163us) service_pool.cc:224] Handling call
0709 13:14:15.664630 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:15.665173 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.665211 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:15.671605 (+  6394us) log.cc:844] Serialized 26 byte log entry
0709 13:14:16.458586 (+786981us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b180019dc80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:14:16.670240 (+211654us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:16.670279 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:16.670360 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:16.746054 (+ 75694us) raft_consensus.cc:1685] finished
0709 13:14:16.746072 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:16.771495 (+ 25423us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4114,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":7118,"spinlock_wait_cycles":3917056,"thread_start_us":211339,"threads_started":1,"wal-append.queue_time_us":1020204}
I20260709 13:14:16.789903 32464 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.743427 32495 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.790779 32495 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:16.792090 30703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:16.792897 30703 raft_consensus.cc:606] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:16.793896 32669 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.794796 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:16.806394 30628 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:16.807780 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:16.763473 32618 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.786522 32704 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:16.835799 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 83571.8us
I20260709 13:14:16.804431 32750 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:16.812960 32602 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:16.845232 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:16.839740 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:16.848623 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:16.850786 30554 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:16.852682 32704 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:16.858111 32687 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:16.835851 32618 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.858834 32687 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.861228 32687 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:16.861622 32618 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.863003 32727 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.863673 32727 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.864020 32556 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.839332 32658 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:16.865212 32658 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.865849 32727 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:16.867807 32658 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:16.868824 32400 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.869463 32400 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.871402 32602 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.875180 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:16.879113 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:16.880501 30703 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:16.835474 32736 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:16.882078 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.881790 32736 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:16.882927 31383 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:16.884932 32736 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:16.886134 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:16.888414 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:16.847072 32750 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:16.889518 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:16.889350 32724 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:16.890249 30628 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:16.890552 32724 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:16.890971 32724 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:16.891543 32750 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:16.892129 30628 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:16.900220 32608 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:16.900990 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:16.901041 32608 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:16.904397 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:16.905478 30553 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:16.913370 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:16.915632 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 262968us
I20260709 13:14:16.926141 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:16.927178 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:16.928290 30625 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:16.931710 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:16.932469 30625 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:16.927392 30631 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
W20260709 13:14:17.011454 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77076.9us
I20260709 13:14:17.012252 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.013067 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:17.013118 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:17.013558 31292 leader_election.cc:403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:14:17.013345 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.016618 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:17.017967 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:16.847522 32464 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:14:16.892771 32701 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:14:17.037746 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.142432 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:17.143716 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 676) took 1553 ms. Trace:
I20260709 13:14:16.916448 32658 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:17.146080 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 128227us
I20260709 13:14:17.161052 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:16.915563 32750 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:17.151005 30628 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:17.170086 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.038s	sys 0.065s
W20260709 13:14:17.170514 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.038s	sys 0.065s
I20260709 13:14:17.151382 31367 rpcz_store.cc:276] 0709 13:14:15.463448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.464066 (+   618us) service_pool.cc:224] Handling call
0709 13:14:15.478091 (+ 14025us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.360248 (+882157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.016450 (+656202us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b540020d354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:17.016685 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026618112}
I20260709 13:14:16.918677 32506 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.167968 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:17.173532 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 149606us
I20260709 13:14:17.175447 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.176360 30705 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:17.179728 32506 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.188248 32644 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:17.188429 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:17.188685 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:17.189057 32644 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.189531 31367 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:17.189716 32747 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.189873 32633 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.190292 32747 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.190460 32633 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.183682 32750 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.192209 32750 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.192843 32747 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.193571 32747 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:17.194270 32767 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:17.192881 32633 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.194970 32767 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.195452 32767 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:17.198141 32633 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.198827 32633 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.194731 32750 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.200559 32617 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.201285 32633 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:17.202713 30628 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:17.201251 32617 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:17.173184 32658 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.209604 32658 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.213831 32617 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:17.226840 32725 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:17.227867 32725 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.232098 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:17.173004 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.233454 30626 raft_consensus.cc:2410] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:16.915757 32717 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.235149 32725 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.236461 32717 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.237500 30551 raft_consensus.cc:2368] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:14:17.170284 32687 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:17.238356 32687 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.238814 32687 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:17.237505 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:17.249629 31361 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:17.015460 32736 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:17.259116 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 76133.4us
I20260709 13:14:17.163028 32618 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:17.189596 32644 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:17.278859 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 431109us
I20260709 13:14:17.033478 32643 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.229s
W20260709 13:14:17.261087 32721 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:14:17.310647 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 565) took 1623 ms. Trace:
I20260709 13:14:17.315974 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:17.315327 30632 rpcz_store.cc:276] 0709 13:14:15.687680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.687873 (+   193us) service_pool.cc:224] Handling call
0709 13:14:15.688326 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:16.455988 (+767662us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:16.456027 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:16.456074 (+    47us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:16.456113 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.310308 (+854195us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b540007d0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:17.310376 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.310552 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764839808}
I20260709 13:14:17.330961 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.182492 32495 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.342839 31361 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:17.347466 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:17.275295 32736 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.338253 32608 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.358891 32615 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:17.346139 32701 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:14:17.382697 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 952) took 1560 ms. Trace:
I20260709 13:14:17.418372 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:17.385558 32400 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:17.424520 32758 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.100s	sys 0.135s
I20260709 13:14:17.425382   303 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:17.428910 30554 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:17.409986 32615 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:17.437361 32758 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.100s	sys 0.136s
I20260709 13:14:17.437738   303 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.457738   303 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:17.466342 32669 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:17.466780 32687 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.467099 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:17.468055 32400 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:17.469281 32687 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:17.473376 32767 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.474817 32602 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.475728 32602 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.481474 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.482342 32400 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.483181 32400 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.485353 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.455974 32681 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:17.485785 32400 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:17.492944 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:17.498853 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:17.504493 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 171172us
I20260709 13:14:17.505541 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:17.510736 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 321013us
I20260709 13:14:17.512617 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
W20260709 13:14:17.515202 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.503973 32608 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:17.566305   304 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.566533   310 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.503628 32602 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:17.575500 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.514186 32669 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:17.568302 32480 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.577191 32701 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.576911 32643 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.580394 32644 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.580828 30628 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:17.582108 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:17.582743 32718 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.582826 32556 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:17.585074 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.592396 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.595460 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:17.563930 32495 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:17.423120 30699 rpcz_store.cc:276] 0709 13:14:15.819545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.820606 (+  1061us) service_pool.cc:224] Handling call
0709 13:14:15.846017 (+ 25411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:15.870040 (+ 24023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:15.872636 (+  2596us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:15.881790 (+  9154us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:15.893082 (+ 11292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.356928 (+1463846us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b54001f22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:17.360131 (+  3203us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:17.379376 (+ 19245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8008576}
I20260709 13:14:17.600621 30705 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:17.502584 32681 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.602005 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.590049   310 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.541168 32618 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:14:17.611920 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.618180 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.634680 30700 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:17.634886 30698 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:14:17.618736 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:17.620908 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:17.621162 30702 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:17.623241 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117833us
I20260709 13:14:17.639175 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.607739 32736 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.642199 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.615725 32602 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:17.603570 32480 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.615064   306 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:17.616053 32706 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:17.654520 30625 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:17.544194 32615 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.608377 32643 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.659575   311 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.661067 32767 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:17.622407 32556 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.663086 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.643853 32495 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:17.619024 32718 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.667863 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.587998   304 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:17.670010 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:17.673190 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:17.675179 32681 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.625752 32699 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:17.675179 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.685813 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.687264 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:17.690078 31388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 687) took 1305 ms (client timeout 1709 ms). Trace:
I20260709 13:14:17.843688 30632 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:17.606817 32701 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:17.691005 32739 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:17.682816 32602 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:17.681860 32680 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:17.794162 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 171367us
I20260709 13:14:17.794457 32633 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:17.802625 31292 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.900s (SENT) [suppressed 9 similar messages]
W20260709 13:14:17.814982 32760 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.134s	user 0.003s	sys 0.007s
I20260709 13:14:17.842299 30625 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:17.842654 30699 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
W20260709 13:14:17.843604 31388 rpcz_store.cc:269] 0709 13:14:16.379403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.380221 (+   818us) service_pool.cc:224] Handling call
0709 13:14:17.678237 (+1298016us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:17.684361 (+  6124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222286720}
I20260709 13:14:17.689682 32687 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.128s
I20260709 13:14:17.862134 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:17.868249 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.875530 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:17.876748 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82956.1us
I20260709 13:14:17.877018 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:17.866847 32724 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.975s	user 0.012s	sys 0.004s
I20260709 13:14:17.880609 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:17.887147 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.888394 30701 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:17.690640 32706 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:17.854359 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:17.851048 32681 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:17.730271   311 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.687532   306 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:17.898058 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54786.3us
I20260709 13:14:17.873715 32687 consensus_queue.cc:1243] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:17.903390 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.904217 30701 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:17.849905 32699 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.921305 32506 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.932770 31367 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:17.933305 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1417) took 1088 ms. Trace:
I20260709 13:14:17.933836 30551 rpcz_store.cc:276] 0709 13:14:16.844667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.844878 (+   211us) service_pool.cc:224] Handling call
0709 13:14:17.933093 (+1088215us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:17.933234 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899528448}
I20260709 13:14:17.870281 32680 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.908320 30626 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:14:17.951630 32602 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:17.952239 32739 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:17.952989 32739 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:17.954069 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:17.959672 32643 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.962107 30626 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:17.894599   322 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:17.873080 32633 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:17.968081 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:17.969605 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:17.974659 30700 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:17.993624 30701 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:17.988253 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:17.995990 30699 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:17.871691   303 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:17.961297   315 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
I20260709 13:14:17.902644 32681 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:18.084295   321 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 13:14:18.084453 32687 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.085850 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:17.965373 32644 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:17.893945 32724 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.104578 31389 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:18.002053 32750 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.121855 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:18.122776 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:18.123692 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 242813us
W20260709 13:14:18.126551 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 323982us
I20260709 13:14:18.132645 30553 raft_consensus.cc:2368] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7.
I20260709 13:14:18.133494 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.110658 32692 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.134445 30701 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:18.107368 32691 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:18.116724   312 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 13:14:18.129086 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.084442   320 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:14:18.106839 32480 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.000957   322 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.143893 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.146158 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.140813 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.147257 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.151582 30625 pending_rounds.cc:85] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 2
I20260709 13:14:18.151978 30625 pending_rounds.cc:107] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:18.152217 30625 raft_consensus.cc:2889] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:18.144611 32656 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.152612 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:18.157799 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.158237 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:18.154562 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70282.2us
I20260709 13:14:18.161947 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:18.114053 32687 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.163125 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:18.167152 30628 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:18.167333 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 291648us
I20260709 13:14:18.141511 32495 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:18.145534 32644 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.172591 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:18.144367 32692 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.127278 32750 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.160462 32767 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:18.165886 32400 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:18.164566 32718 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.166203 32618 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:18.159868 32643 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:18.150820 32691 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.159451 32656 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.239494 32400 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:18.174052 32556 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.174279 32701 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.263342 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 325) took 1340 ms. Trace:
I20260709 13:14:18.208194   310 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:18.244658 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 72161.4us
W20260709 13:14:18.266381 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 91348.6us
I20260709 13:14:18.177625 32644 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.250061 32618 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.274993 30630 rpcz_store.cc:276] 0709 13:14:16.923384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.923609 (+   225us) service_pool.cc:224] Handling call
0709 13:14:17.031419 (+107810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:17.180038 (+148619us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.180088 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.266576 (+ 86488us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.569147 (+302571us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:17.569217 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.569322 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.263003 (+693681us) raft_consensus.cc:1685] finished
0709 13:14:18.263038 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.263256 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148039,"raft.queue_time_us":152963,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":106,"thread_start_us":232110,"threads_started":1,"wal-append.queue_time_us":322572}
W20260709 13:14:18.302484 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 198428us
I20260709 13:14:18.280895 30628 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:18.257354   304 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:18.309136 30524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.003s	sys 0.012s
W20260709 13:14:18.310878 30524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.003s	sys 0.013s
I20260709 13:14:18.262565 32706 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.287160 32643 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:18.291976 32736 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.269714 32400 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:18.287319   342 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:18.319052 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.324229   342 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.301805 32618 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:18.306113 32633 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.308362 32680 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.327692 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.301028   311 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.342702 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.323067 32767 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.320134   303 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.349850 30599 leader_election.cc:400] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:14:18.355126 30599 leader_election.cc:403] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:18.355207 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.330966 32681 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.341698   320 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.329965   306 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.307978 32495 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.345767 32724 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.354389   343 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:18.344950   334 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:18.331624 32736 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:18.362659 32608 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.005s
I20260709 13:14:18.372087 32687 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.396967   337 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.397670   337 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.401052 32692 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.357880 32767 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.403697 32767 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.405453 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.409013 32736 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.416250   337 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.384130   331 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5.
I20260709 13:14:18.422600 32656 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.360448   305 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:14:18.423894   329 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:14:18.424151 32767 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.424782 32644 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:18.425483   313 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:14:18.425077 32767 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.426538 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:18.385113   343 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.427350 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 679) took 2272 ms. Trace:
I20260709 13:14:18.427802   343 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:18.423816 32608 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:18.427841 31364 rpcz_store.cc:276] 0709 13:14:16.154982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.164383 (+  9401us) service_pool.cc:224] Handling call
0709 13:14:16.188311 (+ 23928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:16.188875 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:16.901784 (+712909us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b1800419d40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:14:17.371740 (+469956us) log.cc:844] Serialized 14 byte log entry
0709 13:14:17.631526 (+259786us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:17.635294 (+  3768us) log.cc:844] Serialized 26 byte log entry
0709 13:14:17.635480 (+   186us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:17.635515 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:17.635605 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:18.426968 (+791363us) raft_consensus.cc:1685] finished
0709 13:14:18.427100 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.427251 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22771,"prepare.queue_time_us":44045,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":23126,"raft.run_cpu_time_us":417,"raft.run_wall_time_us":43641,"spinlock_wait_cycles":13167360,"thread_start_us":724870,"threads_started":2,"wal-append.queue_time_us":562534}
I20260709 13:14:18.430001 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.427959 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.395401 32643 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.384792   334 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:18.434881   326 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:14:18.435161   334 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:18.437865 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.438835 30698 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
W20260709 13:14:18.439437   327 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:14:18.427946 32767 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:18.508210 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.396325 32754 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:18.514526 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 145260us
I20260709 13:14:18.380143   320 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.524334 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.528785 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:18.524212 32739 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.571s	user 0.015s	sys 0.000s
I20260709 13:14:18.531219 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.520588   322 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:18.567646 31394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 683) took 2396 ms (client timeout 3124 ms). Trace:
W20260709 13:14:18.568269 31394 rpcz_store.cc:269] 0709 13:14:16.170810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:16.171004 (+   194us) service_pool.cc:224] Handling call
0709 13:14:18.567371 (+2396367us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54002b1e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:18.567525 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147372416}
I20260709 13:14:18.533767 32739 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:18.535269   333 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.246s	user 0.000s	sys 0.002s
I20260709 13:14:18.539741 30559 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:18.540367   320 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.514146 32754 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.576154 32754 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.543740 32750 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.545661   352 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.576865 32766 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:18.577239   352 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:18.578001 32766 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:18.570482 32499 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:18.579630 32499 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:18.580096   352 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.549935 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:18.580721 32499 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.582274 32499 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:18.583159 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:18.549752 32400 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.554419 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.585187 30627 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:18.586238 32400 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.586898 30633 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:18.587129 32634 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:18.584810   334 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.587647 32634 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.588071 32634 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:18.588505 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:18.590967   357 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:18.591650   357 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.592487   357 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:18.578425 32766 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:18.554811 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:18.595691 30629 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:18.595911   334 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.596869 32639 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:18.597472 32639 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.597899 32639 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:18.600796 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:18.601564 31364 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:18.603251 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.603416 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.604079 31367 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:18.557116 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.606782 32766 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.558410 32677 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.559818   324 raft_consensus.cc:993] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:14:18.608444   324 raft_consensus.cc:1081] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Signalling peer 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:14:18.609880 32766 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:18.614650 32756 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:18.620019 32691 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.621112 30699 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:18.621373 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.622040 31383 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:18.622164 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:18.624255 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.624982   348 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:18.625610   348 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.626917 31391 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:18.627422 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.627976   348 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.628543   348 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.628599 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.629315 32767 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.630209 31394 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:18.631201 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.608233 32677 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.527031 32699 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.629887 32767 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.584146 32643 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:18.606213 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.572963 32739 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:17.816285 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:18.710281 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.733068 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.751170 32643 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.740715   305 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.677551 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:18.692517 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.754527   332 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:18.754683 30705 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:14:18.655638 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.755055   332 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:18.755757   332 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:18.680631   342 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.736076 32718 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:18.756922 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:18.757277 32718 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.757774 32767 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.739758 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:18.687917 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.739827 32644 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:14:18.674340 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:18.628007 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.755105 32643 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.694435 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.727896 32660 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.761701 32660 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.659582 32634 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.762183   317 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:18.763247 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.643150 31378 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:18.764710 32634 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.765679 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.709488 30699 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:14:18.761924 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74095.7us
I20260709 13:14:18.764103 32660 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.684882 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.764183   317 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:18.765576   357 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.765996 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.699057 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:18.766744 31383 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:18.766811   332 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.941831 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.938073 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.934551   317 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:18.942862   332 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:18.753775   305 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:18.944584 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:18.943904 32639 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:18.945250   305 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:18.769773 32677 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:18.945511   357 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.946198 31391 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:18.946486 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.947844 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.949138 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.669516 32691 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 13:14:18.945895 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.950990 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:18.951730 30703 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:18.682601 30629 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:18.648489 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1427) took 1318 ms. Trace:
I20260709 13:14:18.652109 32652 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:18.735158 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.949771 31391 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:18.772184 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.629079   348 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.647473 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 972) took 1034 ms. Trace:
I20260709 13:14:18.645313   343 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:18.774106 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.962265 30704 rpcz_store.cc:276] 0709 13:14:17.612578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.612732 (+   154us) service_pool.cc:224] Handling call
0709 13:14:17.622044 (+  9312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:17.681508 (+ 59464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:17.681542 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:17.681587 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:17.681621 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:18.647126 (+965505us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b54000c9754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:18.647200 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:18.647387 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136229248}
I20260709 13:14:18.751745 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.775754 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:18.965627 30698 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:18.965713 31394 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:18.776710 32718 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:18.758193 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:18.965132 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 191096us
I20260709 13:14:18.967224 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.689039 30624 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:18.760324 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 86082us
I20260709 13:14:18.788633 31388 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:18.968674 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.969525 31388 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:14:18.790926 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 448807us
W20260709 13:14:18.754774 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77368.9us
I20260709 13:14:18.643615 31364 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:18.650871 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:18.928835 30707 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:18.935683 30700 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:18.939284 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.943203   317 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:18.933401 32699 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.986951 32699 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:18.703557 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:18.990913 32650 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:18.991425 32650 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:18.952186 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.991978 32650 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:18.993212 31383 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:18.993547 32766 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:18.935945 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:18.994910 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:18.995327 32718 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:18.996155 32718 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:18.997824 32677 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:19.005400 32677 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:19.005868 32677 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:19.006752 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.008262 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.009532 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.010030   343 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.011960   359 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.013738 31378 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:19.013866   332 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:19.014319   332 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.014730   332 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.017513 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:19.018201 32743 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:19.019174 32495 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:19.019995 32495 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:19.020511 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.021204 32495 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.026376 30625 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:19.034597 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.035816 32660 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.006793 32677 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.041282 32677 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.050397 30625 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:19.052636 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:19.053720 30625 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
W20260709 13:14:19.055330 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 60491us
I20260709 13:14:19.056002 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:19.057057 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:19.057523 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.056416 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:18.955374 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.961752 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.060226 30702 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:19.062767 31378 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:18.962404 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:18.975081 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:18.976266 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:18.977046 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 330957us
I20260709 13:14:19.067763 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:18.983364 30624 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:18.984676 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:18.958377 32618 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.626s	user 0.016s	sys 0.003s
W20260709 13:14:18.991345 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 287946us
I20260709 13:14:19.070955 32618 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.072520 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:19.073563 32618 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.074276 32669 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:18.952625 30553 rpcz_store.cc:276] 0709 13:14:17.330206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:17.330545 (+   339us) service_pool.cc:224] Handling call
0709 13:14:18.648191 (+1317646us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54000f0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:18.648406 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840458496}
I20260709 13:14:19.075057 31389 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:18.998701 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62847.8us
W20260709 13:14:19.076253 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:18.957005 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:18.961752 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.064508 31396 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:19.069902 30624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 319) took 3744 ms (client timeout 1900 ms). Trace:
I20260709 13:14:18.997499 32718 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.074792 32750 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.082881 32717 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:19.073590 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.089278 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.094364 31364 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:19.089387 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.095309 30553 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:19.090154 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:19.080314 30624 rpcz_store.cc:269] 0709 13:14:15.325774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.325996 (+   222us) service_pool.cc:224] Handling call
0709 13:14:16.351875 (+1025879us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b54002669a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:14:17.233762 (+881887us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b18002857e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806f18646 00007f1806e5258e 00007f1806ed3f9e 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:14:18.688464 (+1454702us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b58004052d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:14:19.069815 (+381351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4266401280}
I20260709 13:14:19.094125 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.098030 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:19.098999 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112190us
I20260709 13:14:19.099148 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:19.100121 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.102636 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.102931 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.103407 30699 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.103991 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.104861 30702 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:19.105038 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.109418 30700 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:19.113709 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.123958 31367 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:19.128751 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.135197 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:19.135749 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.120332 30380 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 5, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.141028 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.141732 30707 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:19.142684 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:19.154336 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:19.184602 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53468.5us
I20260709 13:14:19.187960 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.138942 30380 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.191354 30380 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 4, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.083367 32691 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:19.196745 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106668us
W20260709 13:14:19.201148 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101196us
I20260709 13:14:19.214454 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.216632 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.199978 30698 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:19.220116 30701 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:19.210837 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79268.4us
I20260709 13:14:19.222075 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:19.123982   357 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.229064 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.083817 32669 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.234324 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.198805 32634 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.194136 32717 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:19.222352   313 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.101s	user 0.004s	sys 0.001s
I20260709 13:14:19.241704 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.210037 32593 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:19.245901 30702 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:19.109514 32750 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.235720 30554 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:19.250602 32643 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.238827   357 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.222745   300 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.274363 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.280480 30626 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:19.224551 32500 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:14:19.214605 32725 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:14:19.252646 32593 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:19.266755 32699 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:19.270419 32464 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.308632 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:19.307658 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 247656us
I20260709 13:14:19.246058 32652 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:14:19.239377 32691 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.257257 32634 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.335220 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.295354   348 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.278798   300 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.335214 30554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:49258
I20260709 13:14:19.307157 32593 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.331293   317 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:19.476234 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 372682us
I20260709 13:14:19.521373 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:19.507997 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 434393us
W20260709 13:14:19.509622 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 268415us
W20260709 13:14:19.510807   360 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.295s	user 0.005s	sys 0.006s
I20260709 13:14:19.336508 32717 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:19.509961   367 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.299s	user 0.000s	sys 0.009s
I20260709 13:14:19.519008 30554 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:19.539733 30554 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.522883 30559 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:19.528607 30553 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:19.533181 30703 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.581s	user 0.016s	sys 0.002s
I20260709 13:14:19.532954 30699 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:19.541786 30703 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:19.535123 30557 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:19.537878 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.543697 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.544430 30707 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
W20260709 13:14:19.515865 32650 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.523s	user 0.028s	sys 0.008s
I20260709 13:14:19.524298 32652 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:19.532178   359 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.520s	user 0.002s	sys 0.018s
I20260709 13:14:19.553335 32652 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.554064   359 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.542335 32693 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
W20260709 13:14:19.560755 32677 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.517s	user 0.016s	sys 0.000s
I20260709 13:14:19.553380   317 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:19.558490 32650 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.521778   330 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:14:19.564677   357 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.565207   377 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:19.572605   377 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.576316 32650 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:19.576990   332 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.009s
I20260709 13:14:19.577098 32652 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.513628 32691 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.563159 32593 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.578969   332 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.579921 32634 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:19.567646 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.581661 32750 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.582566 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.581359   377 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:19.583889 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.585703 32593 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:19.586558 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.587442 30559 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:19.586695 32593 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:19.588593 32639 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:19.569880 32615 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:19.589336 32615 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:19.590471 32615 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.591645 32615 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.567953 32767 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.595291 32767 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:19.598286 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.598516 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:19.598891 32741 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.599289 31389 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:19.599619 32741 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.566395 32693 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:19.600924 32693 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:19.601754 32693 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.602592 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:19.603308 32693 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:19.603612 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:19.604247 30629 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.595148 32708 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:19.606859 32708 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:19.587669 30380 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 5, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:19.607745 32708 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.609212 30380 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:19.609845 32660 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:19.609869 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 395469us
I20260709 13:14:19.611133 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.611724 30630 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:19.612819 32651 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.613588 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.614888 32651 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.615345 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:19.582747 30555 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:19.617501 30550 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:19.618040 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.618772 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.602591 30624 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:19.619328   322 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:19.619745   322 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:19.620034 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.620657   322 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.621696   322 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.623600 30632 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:19.624094 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:19.626654 32767 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.626801 32593 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:19.627496 32593 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.629834 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.630267 32669 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:19.631834 32691 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:19.633237   300 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:19.633926   342 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:19.634577   342 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:19.635607   342 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.609236 32708 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:19.640414 30382 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 68) took 8743 ms. Trace:
W20260709 13:14:19.645452 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.645354 30629 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:19.629529 30380 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.653573 32709 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:19.654935 32660 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:19.655370 32660 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:19.655828   330 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:19.656281 32660 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:19.598054 32658 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:19.657435 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:19.657645 32660 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.657732 32658 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.658651 31367 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:19.660123 32658 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:19.639627 32741 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.660878 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:19.661185 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.661913 31378 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:19.663352 32593 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:19.664141 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.664968 30553 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:19.668470 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.549675 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:19.668743 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 130959us
I20260709 13:14:19.661960   336 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:19.677884 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.678516 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 337) took 1566 ms. Trace:
I20260709 13:14:19.677129 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.679286 30553 raft_consensus.cc:2376] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7. Re-sending same reply.
I20260709 13:14:19.670403 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.671026 30380 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.581429   332 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.673568 32709 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:19.675853 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.669595 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:19.679010 30626 rpcz_store.cc:276] 0709 13:14:18.112351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:18.112583 (+   232us) service_pool.cc:224] Handling call
0709 13:14:19.069621 (+957038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:19.281194 (+211573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:19.639347 (+358153us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54001fbb54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:19.678266 (+ 38919us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b54001fbb50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:19.678438 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2664604416}
I20260709 13:14:19.685150 30559 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:19.684368   330 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:19.681559 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 112803us
I20260709 13:14:19.686771 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.682049 30380 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:19.688828 32691 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.691424 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.691640 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.691708 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.694824 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:19.695784 30672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.008s	sys 0.011s
I20260709 13:14:19.650249 30554 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.708673 30558 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:19.706032 30672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.008s	sys 0.011s
I20260709 13:14:19.682540 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.701854 31389 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:19.640867 30382 rpcz_store.cc:276] 0709 13:14:10.897647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:10.899505 (+  1858us) service_pool.cc:224] Handling call
0709 13:14:19.578602 (+8679097us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:15.635129 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:15.635782 (+   653us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:15.635831 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:14:15.635853 (+    22us) tablet.cc:662] Decoding operations
0709 13:14:15.637367 (+  1514us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:15.637455 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:15.637484 (+    29us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:14:15.814894 (+177410us) tablet.cc:701] Row locks acquired
0709 13:14:15.814919 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:14:15.815103 (+   184us) write_op.cc:270] Start()
0709 13:14:15.815247 (+   144us) write_op.cc:276] Timestamp: P: 1783602855815048 usec, L: 0
0709 13:14:15.815270 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:14:15.816157 (+   887us) log.cc:844] Serialized 6698 byte log entry
0709 13:14:16.803129 (+986972us) op_driver.cc:464] REPLICATION: finished
0709 13:14:17.560192 (+757063us) write_op.cc:301] APPLY: starting
0709 13:14:17.568573 (+  8381us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:17.580973 (+ 12400us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:17.581117 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:17.583069 (+  1952us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:17.583411 (+   342us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=17
0709 13:14:17.583487 (+    76us) write_op.cc:312] APPLY: finished
0709 13:14:17.583902 (+   415us) log.cc:844] Serialized 118 byte log entry
0709 13:14:17.584099 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:17.584330 (+   231us) write_op.cc:454] Released schema lock
0709 13:14:17.584403 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:17.621548 (+ 37145us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:17.627597 (+  6049us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1079460,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":703567,"apply.run_cpu_time_us":5013,"apply.run_wall_time_us":61498,"num_ops":17,"prepare.queue_time_us":164327,"prepare.run_cpu_time_us":61997,"prepare.run_wall_time_us":523512,"raft.queue_time_us":196132,"raft.run_cpu_time_us":237997,"raft.run_wall_time_us":797229,"replication_time_us":920738,"thread_start_us":1099781,"threads_started":4,"wal-append.queue_time_us":392167,"wal-append.run_cpu_time_us":38131,"wal-append.run_wall_time_us":2371646}]]}
I20260709 13:14:19.708151   377 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:19.719201 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.720073 30554 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.712589 30557 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8. Re-sending same reply.
I20260709 13:14:19.698122 30380 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:19.722744 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.722900 30380 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 1 to 3, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:19.726246 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.728772 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.729156 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:19.727916 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 486390us
I20260709 13:14:19.730418 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.731690 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.715940 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.717150 30699 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:14:19.712435 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:19.687784 32608 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:14:19.695741 32765 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:19.747282 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.736651 30699 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:19.733392 30380 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.739252 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.740573 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.741904 30704 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:19.749552 30555 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:19.741806 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.749569 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.732450 30703 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.750672 30704 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:19.750907 30699 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:19.705049   300 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:19.754720   300 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:19.755182   300 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:19.756446 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.706307   330 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:19.760054   330 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:19.760571   330 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:19.763463 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.765664 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.708388 32651 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.767079 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.768875 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.770813   343 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:19.770961   305 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.771349 32767 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:19.771654   343 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:19.771852   305 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:19.772178   343 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:19.772662   343 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:19.772626   305 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:19.773454   305 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:19.775218   317 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:19.777807 32691 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:19.695355   336 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:19.778350 32691 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.778836 32691 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:19.779222 32608 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:19.779647 32608 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:19.780495 32608 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.781806 32608 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.774749 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.784564 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.785569   317 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.786249   317 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:19.787639 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.791409 30380 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 6, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.792708 30559 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:19.798530 30380 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:19.808568 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.804680 30380 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 4, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.821069 30380 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 4, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.819633 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.822309 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.832351 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.791062 32615 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:19.835475 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.836324 30701 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:19.836791 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:19.838114 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:19.822715   377 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:19.839550 32495 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:19.837364 32618 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:19.778817   305 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.787775   343 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.747555 32634 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:14:19.854501 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.853734 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:19.829072 32556 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.847893 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.842123 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:19.851584 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.842103 32618 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:19.826318 32765 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:14:19.874075 32618 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:19.818862   357 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:19.879573 31367 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:19.856535 30558 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:19.884101 30558 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:19.840531 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.841012 32743 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:19.876497 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.854552   343 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:19.898099 31383 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:19.840844 30703 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:19.899962 30700 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:19.855572 30380 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 5, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:19.840245   374 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:19.875442 32658 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:19.921972 32658 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:19.918907 32743 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:19.920168 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:19.936936 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.897837   317 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:14:19.850803   332 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:19.866202 32615 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:19.956307 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:19.843122   305 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:19.922689 32658 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:19.939316 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:19.945972 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.960604 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.853543 32691 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:19.779260   336 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:19.958163 30551 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:19.971041 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.975135 30701 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:19.970739 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.970510 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.957763 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:19.965289 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.971426 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:19.978204 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.979626 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.948634 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:19.980168 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.985364 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.987636 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:19.991259 30699 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:19.962280   332 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:19.992707 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:19.993253   332 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:19.993971 32687 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:19.957401 32495 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.963645 32765 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:19.966495 32718 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:14:20.009449 32687 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:19.998132 32643 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:20.017973 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:19.995316   348 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:19.990429 32644 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:20.011586 32608 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:19.983083 32741 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:19.998735 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:20.039186 30705 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:19.990640 32556 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:19.984791 32766 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:20.042917 32615 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:20.033070 30703 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
W20260709 13:14:20.033066 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.044495 30704 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:20.027463 32644 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:20.045137 30704 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:20.045310 32644 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:20.034252   380 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:20.035708 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56219.6us
I20260709 13:14:20.052842 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:20.046335 30704 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:20.062300 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:20.067660 30703 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:20.020012   357 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:20.018420 32643 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:20.072220 30704 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.028107 32593 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.069793 32651 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:20.063935 32741 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:20.072463 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 378811us
I20260709 13:14:20.142283 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:20.144203 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:20.146056 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.149302 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.149485 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.070036 32708 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:20.139424   348 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.151427   348 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.152259 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.072883   374 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:20.147586   342 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:20.154294   348 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:20.058223 30707 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:20.151957   378 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:20.156134   378 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:20.156567   342 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:20.157263   378 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.158092   342 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:20.150570 32677 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:20.135000 32593 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:20.161943 30704 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:14:20.162954 32766 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:14:20.165086 32593 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:20.115077 32643 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:20.171634 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 479977us
I20260709 13:14:20.179306 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:20.062991 32650 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 13:14:20.142153 32741 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.015182 32743 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.124759   376 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:20.192018   342 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:20.073923   357 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:20.195591 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:20.191334 32725 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:20.275779 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:20.344897 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:20.269783 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 321249us
I20260709 13:14:20.347291 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:20.210927 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:20.212460 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.209607 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57430.4us
W20260709 13:14:20.211243 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.366515 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:20.301424   381 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:14:20.366351 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 378781us
I20260709 13:14:20.377986 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:20.380219 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:20.382763 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.382267 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:20.327922   378 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:20.323046 32669 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:20.326848   357 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.416240 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:20.420892 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 271515us
W20260709 13:14:20.407595 32618 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.000s
I20260709 13:14:20.444288 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:20.479995 30706 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:20.398272 32718 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:20.482566 30702 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:20.291859 32658 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:20.290417   376 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.396329 32608 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:20.486521 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:20.407058 32556 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:20.419384   343 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:20.410861   348 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:20.410560 32765 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:20.494644 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47078.6us
I20260709 13:14:20.394907 32717 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 13:14:20.440201 32669 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:20.407364 32687 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 13:14:20.439635 32691 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:20.420863 32725 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
W20260709 13:14:20.513180 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 116760us
I20260709 13:14:20.497860   343 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:20.507325   300 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.752s	user 0.014s	sys 0.000s
I20260709 13:14:20.489360   392 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:20.509615   385 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:14:20.519389 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.487095 32618 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:20.525688 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:20.541066 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.542678 32718 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:20.547447 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.549453 32643 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:20.551771 31367 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:20.559047 30626 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:20.553772 30630 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:20.512584 32717 consensus_queue.cc:1243] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:20.554679 30624 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:20.508913   348 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:20.565330 30633 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:20.555642 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60142.9us
I20260709 13:14:20.567752 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:20.568636 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:20.568642 31364 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:20.570008 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:20.571218 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:20.576804 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.570719 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:20.523715   343 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:20.593026 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.595165 30624 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:20.545877 32765 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:20.557399 32725 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:20.603271 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:20.612020 32658 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:20.552598   300 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:20.630259 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:20.615489   330 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.854s	user 0.015s	sys 0.000s
I20260709 13:14:20.644412 30558 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:20.662007 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:20.697026   382 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.330s	user 0.002s	sys 0.000s
W20260709 13:14:20.664161 32644 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.618s	user 0.011s	sys 0.003s
W20260709 13:14:20.743038 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101026us
I20260709 13:14:20.743818 32618 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:20.623795 32717 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:20.604226 32725 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:20.747403 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:20.748272 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.748530 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:20.749040 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.750604 30624 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:20.593231 32615 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:14:20.783167 30633 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:20.750451 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:20.751693 30702 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:20.753943   417 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:20.754427   403 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:20.793141 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:20.619601 32556 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:20.761952 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:20.606267 32669 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.646402 30558 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.828119 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.587946   406 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
W20260709 13:14:20.824187 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75771us
I20260709 13:14:20.831977 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:20.764165   376 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:20.766765 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:20.841009 30625 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.769850   400 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:20.770877 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:20.772926 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:20.773171 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:20.773630 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:20.821836   401 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:14:20.780514 30626 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:20.782066 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:20.850075 30633 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:14:20.787290   397 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 13:14:20.732079 32643 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:20.797013 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:20.784502   330 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.799333 31391 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:20.800359   342 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:20.802414 32615 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:20.804483 31389 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:20.805469 31367 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:20.806073   300 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:20.776868 32718 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:20.810767 32765 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.811966   417 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:20.797011 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:20.550803 32608 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:20.821398   389 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
I20260709 13:14:20.614569 32650 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
W20260709 13:14:20.848121 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 75247.4us
I20260709 13:14:20.902602 30698 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:20.851904 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 282356us
I20260709 13:14:20.852403   376 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:20.853917   336 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:20.855466 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:20.830896 32766 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:20.864485 30624 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:20.821084 32717 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:20.785207 32644 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:20.886435 30558 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:20.735486   348 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:20.912334 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 139276us
I20260709 13:14:20.882746   342 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:20.911022 30630 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:20.915870 30558 pending_rounds.cc:85] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 3
W20260709 13:14:20.900528   305 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.924s	user 0.013s	sys 0.002s
I20260709 13:14:20.933060 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:20.883224   415 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:20.824116   403 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:20.920917 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:20.883104 32615 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:20.929733 30558 pending_rounds.cc:107] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:20.941215 30558 raft_consensus.cc:2889] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:20.942448 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 849) took 1088 ms. Trace:
W20260709 13:14:20.921463 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90004.9us
I20260709 13:14:20.936285 30630 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:20.944411 30630 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:20.945669 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:20.943132 30558 rpcz_store.cc:276] 0709 13:14:19.854404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.855950 (+  1546us) service_pool.cc:224] Handling call
0709 13:14:19.856366 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.153916 (+297550us) spinlock_profiling.cc:243] Waited 38.7 ms on lock 0x7b4c0034ec80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18051a9a20 00007f18051baa29 00007f1805180644 00007f18051a7f2d 00007f18051a7b93 00007f1806ed818b 00007f1806ed8cb5 00007f1806ed3f30 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416
0709 13:14:20.643843 (+489927us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54006efb20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5b06f 00007f1806ed3f60 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62
0709 13:14:20.942010 (+298167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.942353 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350387968}
I20260709 13:14:20.859203   400 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:20.949036 30625 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:20.955814 30626 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:20.945315 30630 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:20.909787 32725 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:20.911535 32766 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:20.961418 30633 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:20.964730 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 889) took 1297 ms. Trace:
I20260709 13:14:20.930828 32677 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:20.965382 30629 rpcz_store.cc:276] 0709 13:14:19.666867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.667195 (+   328us) service_pool.cc:224] Handling call
0709 13:14:19.667601 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:19.691097 (+ 23496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:19.692010 (+   913us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:19.706719 (+ 14709us) log.cc:844] Serialized 26 byte log entry
0709 13:14:20.184511 (+477792us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:20.184561 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:20.184755 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:20.964450 (+779695us) raft_consensus.cc:1685] finished
0709 13:14:20.964493 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:20.964654 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20013,"raft.queue_time_us":21585,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":12900,"thread_start_us":405165,"threads_started":1,"wal-append.queue_time_us":599951}
I20260709 13:14:20.973551   417 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:20.963133   359 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:20.957621 32639 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:20.967350 32766 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:20.975287   336 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:20.965654   402 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:21.034317 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54022.3us
W20260709 13:14:21.048722 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 115765us
W20260709 13:14:21.084123 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 129866us
I20260709 13:14:20.924108 32556 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.085093 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:21.037642 32639 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:21.104382 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:21.110072   359 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:21.083958   336 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:21.117105   422 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 13:14:21.117740   418 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:14:21.043044 32717 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.120191   359 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:21.137038   359 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.138710   359 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:21.139950   336 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.104056 32593 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:21.154027 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1222) took 1428 ms. Trace:
I20260709 13:14:21.049239 32615 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.161957 30698 rpcz_store.cc:276] 0709 13:14:19.725477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:19.725625 (+   148us) service_pool.cc:224] Handling call
0709 13:14:19.725954 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.903256 (+1177302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.153621 (+250365us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540078bc54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:21.153929 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705571584}
I20260709 13:14:21.165781   330 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.112165   432 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:14:21.173142 32644 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.179263   403 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.124966   425 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:21.183071   378 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.756s	user 0.008s	sys 0.006s
I20260709 13:14:21.201406   400 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.154415 32639 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.203171 32639 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.204095 32634 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:21.204746 32634 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:21.208245   343 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.005s
I20260709 13:14:21.208822   343 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:21.193356   357 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.004s
I20260709 13:14:21.211479   343 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.211671   415 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:21.211539   357 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.206590   342 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.212472   415 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:21.214010   342 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.214010   357 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.205986 32634 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.223824 32634 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:21.224421   376 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:21.224897   376 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:21.225876   376 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.233467 32766 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.236145 32725 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:21.237210 32766 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.239135   376 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:21.283077 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 331559us
W20260709 13:14:21.287840 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 182706us
I20260709 13:14:21.289968 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:21.291400 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.299857 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:21.303232 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:21.309729 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:14:21.317616 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 213146us
I20260709 13:14:21.320972 32650 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:21.323318 32652 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:21.327180 32677 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.328336 32677 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.320571 32499 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:14:21.331090   375 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.332284   375 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.332701 32499 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:21.335325 32500 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:21.336304   336 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:21.337476   336 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:21.338948 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.340049   403 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:21.341881 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:21.343353 31388 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.343784 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.346292   300 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:21.347159   300 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:21.347220   336 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.348289   336 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.349423   336 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.349509   300 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:21.350109   438 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:21.352317   438 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:21.355436 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.357729 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.360539 32615 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:21.376731 32687 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.378023 32687 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.390431   441 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.391701 32501 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.391829   441 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.393101 32501 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:21.395027 32499 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.395911 32499 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.396744   441 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.395931 30628 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:21.416607 31396 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:21.390463 32687 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.398583 32499 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.437896 32501 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.441861 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:21.400306   330 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.444104 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:21.459786 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.461166 31367 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:21.461522   429 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:21.402285 32693 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 13:14:21.479319 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.475303   336 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.497345 31391 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:21.478107   375 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.522704 32750 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.523872 32750 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.492293 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:21.526757   336 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.533968   336 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:21.526993 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 212825us
I20260709 13:14:21.527151 32660 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:21.537343 30558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:21.538232 32660 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.538765 30558 raft_consensus.cc:606] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:21.530074 32693 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:21.530717   330 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:21.549769 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47371.9us
I20260709 13:14:21.562280 30631 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 2. Current term is 3. Ops: []
I20260709 13:14:21.566977   438 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.571337 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.578876 32608 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:21.583359   432 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.584304   432 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.584147 32608 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.580155 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:21.580771 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:21.590155 30699 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 3: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:21.591269 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 360) took 1069 ms. Trace:
I20260709 13:14:21.591472 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:21.592135 30628 rpcz_store.cc:276] 0709 13:14:20.521574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.521738 (+   164us) service_pool.cc:224] Handling call
0709 13:14:20.547015 (+ 25277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.431438 (+884423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.590654 (+159216us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b54000712d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:21.591068 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950983936}
I20260709 13:14:21.598121   330 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.601323 32608 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.502907 32677 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.602628 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.577883 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.605068 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.560359 31388 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:21.612794 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.613909 31394 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:21.614451 30557 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:21.603335 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 361) took 1081 ms. Trace:
I20260709 13:14:21.612985 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.645821 32750 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.659977   330 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:14:21.675895 30705 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:14:21.659862 32608 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:21.681448 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101559us
W20260709 13:14:21.679975 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.678081 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:21.690383 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:21.694054 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 582) took 1123 ms. Trace:
I20260709 13:14:21.686861 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:21.694640 30631 rpcz_store.cc:276] 0709 13:14:20.570278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.571341 (+  1063us) service_pool.cc:224] Handling call
0709 13:14:20.571767 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.563106 (+991339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.678254 (+115148us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b540007d0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:21.693962 (+ 15708us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2250463616}
I20260709 13:14:21.686542 32750 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.663269 32556 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:21.688210 32660 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:21.708503 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:21.721066 30705 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
W20260709 13:14:21.722029   456 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:21.723728 32750 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.722229 30698 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:21.622191 30632 rpcz_store.cc:276] 0709 13:14:20.521989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.522173 (+   184us) service_pool.cc:224] Handling call
0709 13:14:20.522559 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:20.522880 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:20.834420 (+311540us) log.cc:844] Serialized 14 byte log entry
0709 13:14:21.336858 (+502438us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b180016ac20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:14:21.470499 (+133641us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:21.480905 (+ 10406us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:21.487424 (+  6519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.502453 (+ 15029us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:21.603111 (+100658us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107750,"prepare.queue_time_us":385598,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":3544,"raft.queue_time_us":121861,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":296,"spinlock_wait_cycles":18729856,"thread_start_us":377862,"threads_started":2,"wal-append.queue_time_us":199290}
I20260709 13:14:21.727963 30627 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:21.731288 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:21.732715   380 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:21.733130   378 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.734028   378 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:21.737329   378 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:21.708274   432 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.717751 30699 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:21.740319 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45110us
W20260709 13:14:21.750754 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.733209   380 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:21.761811 32660 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:21.717748 32608 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.772528 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:21.775146 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:21.776350   300 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:21.776891   300 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.776608 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:21.743310   378 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:21.768493 32691 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.784539 30628 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:21.786198 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:21.798866 32660 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.799197 32691 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.780983   300 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:21.756135 32556 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.802487 30630 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:21.809458 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:21.805059 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:21.812592   432 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:21.787809   330 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:21.816660 32634 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.515s	user 0.016s	sys 0.005s
I20260709 13:14:21.831648 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:21.834494 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 100733us
W20260709 13:14:21.839156 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:21.826380   455 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.072s	user 0.003s	sys 0.000s
W20260709 13:14:21.845211 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76965.6us
I20260709 13:14:21.817512   432 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:21.857801 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.861850 30630 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 13:14:21.815867   378 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:21.858876 30631 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:21.864295 32766 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:21.862576 30630 pending_rounds.cc:85] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 3
W20260709 13:14:21.873678 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.874635 32464 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:21.875494 32464 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.868438 30630 pending_rounds.cc:107] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 13:14:21.853896 32750 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.882072 30630 raft_consensus.cc:2889] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:21.886193 30699 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:21.849352   330 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:21.870860 32766 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:21.896996 30700 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:21.897825 32691 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.854661   380 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:21.899320 30705 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:14:21.900693   330 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:21.902807 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.908763 32660 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.909215 32464 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:21.910444 32608 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:21.912030   438 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:21.912489   438 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:21.913021 32650 raft_consensus.cc:993] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:21.913642 32650 raft_consensus.cc:1081] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:21.913410   438 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.927762 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:21.928902 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:21.922322 31391 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:21.919129 32766 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.943645 31396 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:21.955192 32750 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:21.951712   438 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:21.889818 32634 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:21.954037   460 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.229s	user 0.002s	sys 0.000s
I20260709 13:14:21.929572 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:21.972899 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91145.8us
I20260709 13:14:21.974215 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:21.961002 30698 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:21.964986   432 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:21.965759 30633 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:21.958413 31396 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:21.979007 31396 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:21.980466   377 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:21.988083 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 601) took 1048 ms. Trace:
I20260709 13:14:21.994340 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.998667   377 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:21.999266   377 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:21.999676   377 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:22.000648 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.987764 32634 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.002784 32693 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:22.003736 32464 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:22.004494   415 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.791s	user 0.015s	sys 0.010s
I20260709 13:14:22.004436 32464 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.005057   415 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.007030   380 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:22.008100 32634 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.003492 32693 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.011626   330 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.012301   330 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.014652 32691 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.015463 32691 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.017716 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.018801 31391 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:22.019738 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:21.994176 30630 rpcz_store.cc:276] 0709 13:14:20.935554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:20.935755 (+   201us) service_pool.cc:224] Handling call
0709 13:14:20.936133 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:21.333006 (+396873us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b18000c8dc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806f18646 00007f1806e5258e 00007f1806ed3f9e 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4
0709 13:14:21.799739 (+466733us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5800712dd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4
0709 13:14:21.945963 (+146224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:21.983709 (+ 37746us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426745472}
I20260709 13:14:22.027848 30557 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:22.032244 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:21.982496 32750 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.063501 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.059136 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.070796 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.100374 32556 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:22.102150   461 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 13:14:22.121541 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.124486 32750 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.057014   336 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.127163   336 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.127980   378 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.127838   336 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.134193 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.134438 32687 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.135046 32687 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.136670 32556 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.137475 32687 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.137331 32556 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.138741 32687 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.139740 32556 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:22.141602   459 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.314s	user 0.001s	sys 0.001s
I20260709 13:14:22.139762 32687 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.145620 32687 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.147787 32693 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.153640 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:22.154771 30698 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.154024 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.153678 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.157871 31396 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.159354 30700 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:22.159833   464 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.169308 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.171110 32691 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.172525   330 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.172664   336 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:22.174194 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.132645 31394 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:22.071600   415 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.183895 31396 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:22.181281 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.189023 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.198539 31383 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:22.206549 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.207574 30550 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:22.200723 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.204533 32677 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.221637 32677 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.227698   429 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.228838   429 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:22.234156   376 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.994s	user 0.007s	sys 0.014s
I20260709 13:14:22.244165 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:22.190981 32639 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.982s	user 0.017s	sys 0.006s
I20260709 13:14:22.225118 32677 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.250705 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.246506 30698 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:22.263974 31394 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:22.272992 30630 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "536e99f894744c35a0490416a32a875e"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:22.273720 30702 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:22.269692 30559 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:22.276703 30630 raft_consensus.cc:606] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:22.277287 30630 raft_consensus.cc:614] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:22.255190   415 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.189172   464 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.312093 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:22.285841 32660 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.308542   376 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.339641 30700 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:22.307490 32608 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.307111 32556 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.365239 31383 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:22.316772   336 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.377383 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.369730 32634 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.385032 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:22.386360 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.336843   415 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.337807   377 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.397032 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.405840 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.426656 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:22.428462 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 372303us
I20260709 13:14:22.370172 32660 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.431680 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.432616 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:22.423986 32615 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.059s	user 0.021s	sys 0.001s
I20260709 13:14:22.433234 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.378973 32608 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.399425   343 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.399670   453 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:22.445521   429 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.442998   348 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.467537   348 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.388542 32556 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.444597 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.388931   376 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:22.450519   300 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.628s	user 0.015s	sys 0.003s
I20260709 13:14:22.431911   330 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:22.399657 32634 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.450659 32693 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:22.402971   336 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:22.498818 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.484532 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.479038   348 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.492175 30631 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:22.496193 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.487895   330 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.506831 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.530591 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.495258 32693 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.549404 30554 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:22.507438   348 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.563076 30633 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:22.501396   464 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.479873   453 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.498479   429 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:22.487552   300 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.564323 32693 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:22.490631   343 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.589020 32691 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:22.616856 32691 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:22.550099   330 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:22.594810 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:22.594810 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 165062us
I20260709 13:14:22.574203 32651 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.506996   438 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.592336   348 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.609112 31383 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:22.609340   359 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.368s	user 0.015s	sys 0.011s
I20260709 13:14:22.611697 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.612470   480 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:22.612485 32766 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.630s	user 0.015s	sys 0.000s
I20260709 13:14:22.614192   376 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.547958   378 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:22.622484 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:22.624876 30632 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:22.626283 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.627347 32660 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:22.627120 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 184428us
I20260709 13:14:22.628543 30558 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:22.629197 32608 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.674672   486 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.675246   486 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.675693 31383 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:22.674355 30555 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
W20260709 13:14:22.631438   380 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.624s	user 0.014s	sys 0.002s
I20260709 13:14:22.635653 32556 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.639343   377 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.607234   429 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.642120   336 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.645274 32687 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.671020   480 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:22.678037 32687 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.679507 32608 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:22.679847   429 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.680361 32687 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.648509   343 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.678115 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.681263   348 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.652722 32464 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.658915 32651 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:22.662631 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.684413 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.676918 30555 pending_rounds.cc:85] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 3
I20260709 13:14:22.677259 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:22.681200 32652 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:22.651652 32634 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.682345   415 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.688097 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.677552 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:22.685734 30555 pending_rounds.cc:107] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:22.688963 30555 raft_consensus.cc:2889] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:22.688772 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66416us
I20260709 13:14:22.666883 30633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "907e974a334846208c70f0b813199df0"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:22.690840 30633 raft_consensus.cc:606] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:22.691331 30633 raft_consensus.cc:614] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:22.687597 31389 raft_consensus.cc:2410] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:14:22.693918 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:22.663779   453 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.698971 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.700661 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:22.697072 30631 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
W20260709 13:14:22.703351 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.707854 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.708192   375 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.708772   375 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.709519 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.709697 32651 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.710044 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:22.710239 31367 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:22.710685 30551 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:22.711318 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.703555 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.621256 32691 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.716742 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.708688 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.718113 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.718947 31396 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:22.719580 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:22.687668   376 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.720323 31391 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:22.720698   376 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.667663   438 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.701581 32652 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.703958 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:22.701093 31378 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:22.682835   429 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:14:22.697150   377 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:22.705546 32750 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:22.722549 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 853) took 1331 ms. Trace:
I20260709 13:14:22.731175 30557 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:22.731547 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.732733 30555 rpcz_store.cc:276] 0709 13:14:21.399489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.400267 (+   778us) service_pool.cc:224] Handling call
0709 13:14:21.400846 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.703784 (+1302938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.722256 (+ 18472us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54002d0ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:22.722458 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2914848000}
I20260709 13:14:22.735975 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.736318 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.736656 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:22.736690 30550 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:22.739197 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:22.731880 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:22.671770 32766 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.741931   472 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:22.742455   432 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.740425   336 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:22.739897 30557 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:22.743579 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.748226 30556 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:22.746738 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.697625   378 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:22.743410   453 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:22.749053   472 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.751147   336 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:22.789714 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46221.2us
I20260709 13:14:22.797348   485 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.801118 32750 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:22.801757 32750 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.801944   480 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.802129 32750 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:22.806833 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 129852us
I20260709 13:14:22.763957   432 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.782083   472 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:22.812081 32766 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.789561   336 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:22.776451   376 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:22.822789   377 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.826632 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:22.827764 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 81144.5us
I20260709 13:14:22.832588 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.793249 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:22.834354 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47660.2us
I20260709 13:14:22.845667 30554 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:22.830425 30551 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:22.837695 31396 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:22.838258   469 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.381s	user 0.000s	sys 0.004s
I20260709 13:14:22.859175 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.860683 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.861711 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.861950 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:22.863934   471 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.303s	user 0.002s	sys 0.001s
I20260709 13:14:22.867151 31391 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:14:22.867789 31391 pending_rounds.cc:85] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:22.868088 31391 pending_rounds.cc:107] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:14:22.868369 31391 raft_consensus.cc:2889] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:22.869778 31367 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:22.835412 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.830264 32651 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:22.831995 32464 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:22.849253   376 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:22.849668 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.798171   485 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.876431 31364 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:22.849345 30550 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:14:22.877318 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 84141.5us
I20260709 13:14:22.877568 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.853118   378 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.855299 32652 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.856969   486 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.873672 32766 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:22.907444 30553 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:14:22.907928 32766 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:22.908479 32766 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:22.908740 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.839397   300 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:22.880578   376 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.880606 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.880908 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.880620 31361 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:22.881765 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.881945 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:22.882411 32651 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:22.884200 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1478) took 1027 ms. Trace:
I20260709 13:14:22.875934 32464 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:22.889899   375 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:22.891732 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.892642 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.894745   480 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:22.964170 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.964504 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.965090 30556 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:22.966219 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:22.895831 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.979785 30552 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:22.980952 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.982191 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.897385   485 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:22.984349 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.984396 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.985374 32464 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:22.900164 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 861) took 1057 ms. Trace:
I20260709 13:14:22.986644 30559 rpcz_store.cc:276] 0709 13:14:21.842532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.842740 (+   208us) service_pool.cc:224] Handling call
0709 13:14:22.899933 (+1057193us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:22.900055 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856516864}
I20260709 13:14:22.988525 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.989372 30559 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:22.961513 30558 rpcz_store.cc:276] 0709 13:14:21.856567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:21.861454 (+  4887us) service_pool.cc:224] Handling call
0709 13:14:21.888882 (+ 27428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:22.674828 (+785946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:22.883899 (+209071us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54001e8cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:22.884126 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695739264}
I20260709 13:14:22.900507 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:22.991962 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:22.992381 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91958.5us
I20260709 13:14:22.992774 30551 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:22.986240 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.966296 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.986356 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:22.991225 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.995003 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.001331 30702 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:22.931324 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 83805.8us
I20260709 13:14:22.963025 31396 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:22.895210 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:22.967804 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58435.6us
I20260709 13:14:22.970227   480 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:22.974164 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:22.975019 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:22.978780 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.012284 31378 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.013559 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.961086   378 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:22.900507   377 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:22.983017 30556 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:22.897116   490 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:23.000455 30628 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:23.010072 30557 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:22.907603 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:22.978191   336 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.018771 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:23.012364 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:23.023387 30556 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:23.028726 30552 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:23.032157 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:23.017076 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:22.967816   486 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.040750 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.005781   485 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:23.044261 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:23.052559   464 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:23.053208   464 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:23.054734 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.055780   464 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:23.055869 32725 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:23.056449 32725 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.056845 32725 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:23.058593   473 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:14:23.062258   468 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:23.062407 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50129.2us
I20260709 13:14:23.063616   488 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:14:23.064896 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:23.066287 32651 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:23.066598   468 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.069825 31364 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:23.051606   378 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.066377 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.073738 30551 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:23.074684 30554 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: []
W20260709 13:14:23.043617 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.679s (SENT) [suppressed 7 similar messages]
W20260709 13:14:23.077240 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 191103us
I20260709 13:14:23.011410   480 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:23.075577 32634 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:14:23.079011   492 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:14:23.045476   490 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.049192 32693 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.049647   485 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:23.084424   489 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:23.086503 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.096113 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.096308   376 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.092352 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.098176   348 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:23.098765   348 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:23.079564   468 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:23.099495   468 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.099808   348 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.093964   489 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.103008 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.103240 32634 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:23.085948 32464 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.104785 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.103055   489 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.092289 32691 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:23.099885   468 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:23.107321 32691 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.111091 30698 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:23.111791 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 201410us
I20260709 13:14:23.036564   429 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 13:14:23.114856   479 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:23.115476   476 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:23.107743   432 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.116010 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:23.116001   476 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:23.085245 30556 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.082206   473 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:23.125237 30556 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.130873 30700 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:23.089054   336 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.091741   485 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:23.127326 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:23.134282 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90727.4us
I20260709 13:14:23.115010 32634 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.136263 30702 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:23.072429   330 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.140905 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:23.076395   486 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:23.123041 32464 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.155957 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.150161 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.076056   472 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.155531 30633 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:23.156091 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.127710   376 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:23.132412   473 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.146721   336 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:23.180364   472 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.126838   479 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:23.181092   479 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:23.181648   479 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:23.160918 30381 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 7, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.182809   473 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:23.184540 30381 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 8, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:23.163573 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:23.126489 32501 consensus_queue.cc:1175] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 7 status { last_received { term: 6 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:23.163741 30380 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 141) took 9550 ms. Trace:
I20260709 13:14:23.151582   432 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:14:23.165939   486 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.123773   429 raft_consensus.cc:476] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Not starting pre-election -- already a leader
I20260709 13:14:23.166693 30629 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:23.170594 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:23.225455 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55003.6us
I20260709 13:14:23.162249   490 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.156800 31396 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:23.145736   480 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.180003   336 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.117168 32652 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 13:14:23.187314   496 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:14:23.166574 32693 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:23.231128 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76474.1us
W20260709 13:14:23.233333 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.130709   488 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:14:23.237509 31378 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:23.236689 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:23.246204 30699 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:23.212147 32691 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.191138 32725 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.249086 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.161515   478 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.161s
W20260709 13:14:23.257673 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 141784us
I20260709 13:14:23.257966 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:14:23.258656 30699 raft_consensus.cc:606] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:23.259130 30699 raft_consensus.cc:614] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:23.261068 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.265611 32766 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.195456   376 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.266914 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:23.268621   376 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.270149 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.271036   489 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.271531 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.272274 30554 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:23.272861 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.273468 30558 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:23.275738   343 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:23.278601   377 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.279366   480 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:23.165436   476 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.283871   476 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:23.286819   378 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:23.167867 32750 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.289052   468 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.289558 32750 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.266294 30556 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:23.292024   489 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.293162   485 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:23.296748 30699 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:23.297004 32725 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.249739 32634 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:14:23.298008 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.298272 32634 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:23.298769 32634 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:23.299273 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.250200   359 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.303050 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.303792 30699 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
W20260709 13:14:23.304487 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.307758 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.308449 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:23.310429 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:23.258922 31396 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:23.313926 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.312551 32651 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:23.318267 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.318068 32651 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.319270   330 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.320333   438 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.321007 32651 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.320889 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.322871   336 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.324224 31361 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:23.324395 30556 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:23.325191 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.328739 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:23.314162 30381 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:23.329530 30632 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:23.353958 30700 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:23.356153 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:23.356362 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.361634 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.377876 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.381919 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.383066 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.383317 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.386685 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.387383 32651 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:23.387820 32651 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:23.387957 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.388217 32651 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.388379 32766 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:23.388589 30555 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:23.389278 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.390771   376 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:23.390985 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.391657 30702 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:23.391360   376 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:23.323638   343 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:23.386550   492 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:23.407200 30558 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:23.411325 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.416021   343 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:23.420459 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:23.421480 30705 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:23.425266 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.426604 30706 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:23.427937 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:23.429226 30702 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:23.430733 30703 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:23.432870 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 53647.4us
I20260709 13:14:23.434983 32766 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:23.436879 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.438088 32766 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.438721 32634 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.442339 32634 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.395136   499 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.446079 30555 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:23.463030   484 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:23.463099 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.464198   484 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.464820   484 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.465104 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.466459 32766 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:23.467170 32766 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.467507   376 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:23.468433   376 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:23.469091   376 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.468461 32766 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.432894 30381 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:23.481050 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.497761 30706 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:23.499915 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:23.500931   494 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.501489 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:23.501953   468 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:23.499711 32660 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:23.188696 30380 rpcz_store.cc:276] 0709 13:14:13.612652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:13.612853 (+   201us) service_pool.cc:224] Handling call
0709 13:14:23.163406 (+9550553us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:20.749618 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:20.751126 (+  1508us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:20.751171 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:14:20.751185 (+    14us) tablet.cc:662] Decoding operations
0709 13:14:20.752930 (+  1745us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:20.753027 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:20.753077 (+    50us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:14:20.807418 (+ 54341us) tablet.cc:701] Row locks acquired
0709 13:14:20.807438 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:14:20.807627 (+   189us) write_op.cc:270] Start()
0709 13:14:20.807765 (+   138us) write_op.cc:276] Timestamp: P: 1783602860807566 usec, L: 0
0709 13:14:20.807788 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:14:20.808618 (+   830us) log.cc:844] Serialized 6313 byte log entry
0709 13:14:20.921630 (+113012us) op_driver.cc:464] REPLICATION: finished
0709 13:14:21.185876 (+264246us) write_op.cc:301] APPLY: starting
0709 13:14:21.186023 (+   147us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:21.186507 (+   484us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:21.186764 (+   257us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:21.189077 (+  2313us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:21.189526 (+   449us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:14:21.189601 (+    75us) write_op.cc:312] APPLY: finished
0709 13:14:21.190187 (+   586us) log.cc:844] Serialized 112 byte log entry
0709 13:14:21.199769 (+  9582us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:21.228324 (+ 28555us) write_op.cc:454] Released schema lock
0709 13:14:21.228431 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:21.229017 (+   586us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:21.229073 (+    56us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1800836,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":256247,"apply.run_cpu_time_us":5764,"apply.run_wall_time_us":43126,"num_ops":16,"prepare.queue_time_us":714190,"prepare.run_cpu_time_us":34931,"prepare.run_wall_time_us":103046,"raft.queue_time_us":8174,"raft.run_cpu_time_us":32345,"raft.run_wall_time_us":326902,"replication_time_us":82201,"spinlock_wait_cycles":145280,"thread_start_us":674301,"threads_started":4,"wal-append.queue_time_us":43867,"wal-append.run_cpu_time_us":7489,"wal-append.run_wall_time_us":1354162}]]}
I20260709 13:14:23.502568   494 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:23.504398 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.505326   494 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:23.505506 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:23.505805   494 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.506184   494 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.443615   499 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.503051 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:23.507407 30703 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:23.508739   481 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.509075   499 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.509600 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.503191 32660 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:23.503818   453 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:14:23.511009 32660 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:23.512095 32660 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.513720 32660 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.527171 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.528483 31396 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 4: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:23.515041   486 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:23.533210 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52795.7us
I20260709 13:14:23.539064 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:23.536059 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 213000us
W20260709 13:14:23.537784 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 230923us
W20260709 13:14:23.544200 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 406647us
I20260709 13:14:23.570528 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.552778 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.551929 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 367) took 1241 ms. Trace:
I20260709 13:14:23.538501   486 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.536990   438 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:23.598117 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.596464 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.616622 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:23.617957 30555 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:23.617014 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.647207 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.591042   330 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:23.654064 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57683.6us
I20260709 13:14:23.470429   376 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.595697   453 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:14:23.652844 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 231648us
I20260709 13:14:23.618929   486 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:23.665689   438 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.654595   468 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.674126   499 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:23.674836   485 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:23.680419 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63559.2us
W20260709 13:14:23.681875 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 325530us
W20260709 13:14:23.686934 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 166740us
I20260709 13:14:23.584666 30627 rpcz_store.cc:276] 0709 13:14:22.310149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:22.310369 (+   220us) service_pool.cc:224] Handling call
0709 13:14:22.335846 (+ 25477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:22.348638 (+ 12792us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:22.348693 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:22.374363 (+ 25670us) log.cc:844] Serialized 26 byte log entry
0709 13:14:22.585869 (+211506us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:22.607600 (+ 21731us) log.cc:844] Serialized 14 byte log entry
0709 13:14:22.885755 (+278155us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b18000228a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:23.146744 (+260989us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18000228a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:14:23.515446 (+368702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:23.515546 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:23.515609 (+    63us) raft_consensus.cc:1685] finished
0709 13:14:23.515630 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:23.551819 (+ 36189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5958,"prepare.queue_time_us":21727,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":79951,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":9749,"spinlock_wait_cycles":81515136,"thread_start_us":510646,"threads_started":2,"wal-append.queue_time_us":239554}
I20260709 13:14:23.591081   500 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.134s
I20260709 13:14:23.693406 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.701102 30558 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:23.703024 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.712067 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.708141 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.716958 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.718591   359 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:23.719923   359 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:23.702509 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.678716 32464 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:23.728292 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:23.679503   453 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:23.663182   481 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 13:14:23.700423   499 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:23.749634 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:23.734989 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:23.756659 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:23.758222 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54304.5us
I20260709 13:14:23.759598 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:23.750324   472 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:23.750022   478 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.755777   377 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.755136   485 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:23.756527   499 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.757783 32766 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.761646   359 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:23.784662 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:23.793175 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:23.805195 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48644.2us
I20260709 13:14:23.806684 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:23.809897 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.757563   468 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:14:23.771903 32501 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.823313   518 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.755934   506 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:14:23.825173 32691 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.773262   472 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.840057 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:23.823371 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 120937us
W20260709 13:14:23.840103   473 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.003s
I20260709 13:14:23.795681   359 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:23.783885   507 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:23.876492 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.783919 32464 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:23.879172 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:23.849740   464 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:23.838019   432 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:23.879773   330 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:23.880626   473 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:23.884815   330 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:23.885656   330 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.877905   359 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:23.887835 30705 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:23.890777   507 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:23.891917   507 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.894315   520 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:23.839000   485 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.897742   473 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:23.908659   518 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:23.909406   518 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:23.910511   518 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:23.915700   438 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:23.923462   472 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.926429   503 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:23.927314   503 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:23.927857   503 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:23.907374 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 147916us
I20260709 13:14:23.873442   513 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
W20260709 13:14:23.932482 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94185.3us
I20260709 13:14:23.935504   485 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:23.816128   453 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.875217   478 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:23.943962   478 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:23.945636   514 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:14:23.945336   478 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:23.931080   330 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:23.957741   453 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:23.900799 32464 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:23.895103   520 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:23.969125 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 231494us
W20260709 13:14:23.967015 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 255094us
W20260709 13:14:23.965485 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 272137us
I20260709 13:14:23.983512 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.984957 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:23.985855 30555 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:23.983548 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.986300 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.988916 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:23.993415 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:23.993690 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:23.994159 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:24.006181 31394 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:23.994347 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:23.997036 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:24.003336 31388 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:24.007216 30554 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:24.009128 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:23.995875 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:24.022203 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.030175 30555 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:24.134177 31361 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.031193 30550 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:24.033003 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:23.966514   484 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:24.035470   476 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.004s
W20260709 13:14:24.130266 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 200165us
I20260709 13:14:23.983840 32650 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:14:24.013978   438 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:24.001130 32766 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:23.995788   300 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:23.995744 32750 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:24.135632 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 102752us
W20260709 13:14:24.138948 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 129912us
I20260709 13:14:24.012531 32725 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 13:14:24.135418 32660 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:24.147099 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 235241us
I20260709 13:14:24.146731 31383 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:24.188958 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:24.146972 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:24.160266 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 172623us
I20260709 13:14:24.197515 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:24.147097   343 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:24.200139 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48857us
I20260709 13:14:24.203562 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.204440 30629 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:24.159430 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.206390 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.206715 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.207614 30628 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:24.208663 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:24.210171 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.210717 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.027508   520 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.211305 30698 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:24.198356   480 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.919s	user 0.010s	sys 0.005s
I20260709 13:14:24.144630   525 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:24.218725 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.216171 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:24.219910 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:24.142930   479 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.961s	user 0.013s	sys 0.012s
I20260709 13:14:24.222254 30630 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:24.232167 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.226524 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:24.232641 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.238266 30627 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
W20260709 13:14:24.210577   495 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:14:24.242998 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.230973 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.151789 32691 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:24.247830 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.215286   476 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:24.230645 32750 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:24.267876 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:24.274595 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.268967 30700 raft_consensus.cc:2368] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:24.273169 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.273885 30630 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:14:24.277154 30632 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:24.273165 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.276485 30703 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:24.278666 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.279644 32464 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:24.280550   516 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:14:24.282531 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.283555 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.263855 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:24.214318   530 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:24.232921   525 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.207697 32660 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.214381   438 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:14:24.252030   479 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.278031 31364 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:24.255584 32691 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:24.287866 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:24.286835 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.300408 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.212982   343 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.266410   429 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:24.269414   476 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:24.312243 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.320274 31396 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:24.291886 32650 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:24.336448 31383 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:24.336477 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.326725 30633 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:24.297448 32750 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:24.341332 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53572.2us
I20260709 13:14:24.291729   525 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:24.306229 32691 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.337513 31364 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
W20260709 13:14:24.312089   494 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.805s	user 0.014s	sys 0.003s
I20260709 13:14:24.343489 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:24.341925 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.347507 30702 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:24.318181   476 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:24.310729 32652 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:24.306766   429 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:24.359395 30629 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:24.361675 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.364274   484 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.366772 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.339046 32634 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.310045   359 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.378887 30705 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:24.379971 30703 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:24.361563   429 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:24.396975 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:24.397210 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 155437us
I20260709 13:14:24.362241 32725 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:24.409415 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:24.411047 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:24.413085 30550 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:24.346089   530 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 13:14:24.413808 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.382436   481 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.417834   438 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:24.419765 30557 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:24.425453 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:24.353494   494 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.440501 30551 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:24.419716   486 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.000s
I20260709 13:14:24.431345 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:24.422509 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:24.444801 30555 raft_consensus.cc:2368] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4.
I20260709 13:14:24.436282 30558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:24.438577 30553 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:24.439103 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:24.382040 32634 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:24.431357 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 175909us
I20260709 13:14:24.413753   479 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:24.414940   539 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:24.456897 32634 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:24.457268   502 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:24.457924   502 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:24.458303   502 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:24.458432   538 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:14:24.462759   531 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.463068 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.463426   531 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.410792 32464 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 13:14:24.478276 30550 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:14:24.479096 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:24.457276   536 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:14:24.490803 30559 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:24.490130 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.497355 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:24.498173 30555 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:24.465845   531 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.495528 30706 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:24.439812 32750 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:24.531762   522 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:14:24.530747   547 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:14:24.557793   359 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:24.481032   539 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.436883   481 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.450652   486 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:24.585613 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 163195us
I20260709 13:14:24.476572   330 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:24.586378 31394 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.572s	user 0.020s	sys 0.000s
I20260709 13:14:24.733470 31394 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
W20260709 13:14:24.585224 32651 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.196s	user 0.015s	sys 0.001s
I20260709 13:14:24.736997 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:24.736842 32651 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.738539 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:24.739974 32651 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.585806 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
W20260709 13:14:24.600564   499 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.000s
I20260709 13:14:24.600137 32691 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:24.744110   486 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.745038 30627 raft_consensus.cc:2368] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7.
I20260709 13:14:24.745774 32691 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:24.601717   453 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.626s	user 0.011s	sys 0.000s
I20260709 13:14:24.602739 32660 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.603757 30701 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:24.751362   429 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.754079   429 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.610368   343 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:24.612776 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.641523 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:24.741945   499 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.742159 30706 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:14:24.757805 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 116354us
I20260709 13:14:24.759148 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:24.759650   499 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.760187   527 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:24.760442   499 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:24.646970 30707 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:14:24.761044   527 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:24.649253   476 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.657997 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:24.702524   472 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:24.608606   494 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.736068 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:24.756186 30626 raft_consensus.cc:2410] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:24.781216   330 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:24.782496   330 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.783519   476 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.783867 30558 raft_consensus.cc:606] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:24.784348 30558 raft_consensus.cc:614] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:14:24.786160 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50186.2us
I20260709 13:14:24.790828 31394 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:24.791254 31394 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:24.766995 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:24.795635 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:24.801998 31367 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:24.802086   438 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:24.802790 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:24.818789 31389 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:24.803267 31394 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.813190   525 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.816864 31367 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:24.821224 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 355430us
I20260709 13:14:24.819365 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:24.823565 31394 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:24.828652 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.830348 30629 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:24.814997 32766 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:24.832535 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.825778 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:24.837606 31394 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:14:24.838328 31394 pending_rounds.cc:85] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:24.838591 31394 pending_rounds.cc:107] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:24.839061 31394 raft_consensus.cc:2889] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.839869 30699 raft_consensus.cc:2368] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4.
I20260709 13:14:24.843119 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:24.820591   473 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:24.829727   503 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.007s
W20260709 13:14:24.867455 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.847193 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:24.869634 30629 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:24.870203 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:24.871187 31391 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:24.851042 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.851981 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.852677 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.854945 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.858975   481 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:24.861155 32766 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:24.861984   539 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.862102   438 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:24.862752 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.879124 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:24.864809   525 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.876634 30624 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:24.842934 31367 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.881820 31367 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:24.836936   330 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:24.885830 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:24.848287   530 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:24.888566 30632 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:24.889768 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.880697   561 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:24.894845   499 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.895531   499 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.896091 30629 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:24.898087   499 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:24.898969 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:24.880396   473 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:24.900923   473 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.883996   432 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:24.907352 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:24.905205 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:24.907968 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:24.889040   539 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:24.886158   330 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:24.919348 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.918453 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.918823 31389 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:24.903492   438 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:24.922408 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:24.878082   503 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.908164   561 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:24.925810   556 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:24.912400   520 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:14:24.926749 30703 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:24.927333 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.903162   432 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.883606   502 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.918354   525 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:24.928700 30624 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 3.
I20260709 13:14:24.929190 30629 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:24.920619   330 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:24.924953   547 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:24.943856 30631 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:24.911803   499 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:14:24.947741 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:24.948050 31367 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:24.940325 31391 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:24.948565 31367 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 3
I20260709 13:14:24.906993   530 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:24.949054 31367 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:24.951772 31367 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:24.952092 32766 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:24.956053 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:24.950269   518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.032s	user 0.012s	sys 0.003s
W20260709 13:14:24.958890   518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.013s	sys 0.004s
I20260709 13:14:24.976462   530 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:24.925832   438 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:24.947046   525 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:24.977447 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:24.997840 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.017443 30557 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:25.022352 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:25.021188   501 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.023332 30705 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:14:25.023298 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:14:25.025209 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:25.027091   564 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:14:25.031234   438 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:25.033957   525 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.038829   503 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.042871   432 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:25.043605   503 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:25.046662 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.020952   468 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:14:25.048198   468 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:25.048626   468 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:25.021193 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:25.054260   562 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:14:25.054908 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.059116 31294 leader_election.cc:400] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4.
I20260709 13:14:25.059607 31294 leader_election.cc:403] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:14:25.060446   343 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4.
I20260709 13:14:25.061254 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:25.062193   343 raft_consensus.cc:2764] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:14:25.063493   343 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:25.064845   432 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.064989 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:25.066478 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 953) took 1110 ms. Trace:
I20260709 13:14:25.067153 31396 rpcz_store.cc:276] 0709 13:14:23.956067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.958689 (+  2622us) service_pool.cc:224] Handling call
0709 13:14:23.958988 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:24.320856 (+361868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:25.065895 (+745039us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002b07d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:25.066320 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":830389504}
I20260709 13:14:25.067685 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.065867   432 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.073261 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:25.075527 30557 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:25.024101   501 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.079147   476 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:25.077713   504 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:25.080696   504 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:25.081140   504 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:25.081411 30699 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.084654   432 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.085477 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.085922   429 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:25.086942 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:25.087761   502 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.091993   501 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.092924   561 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:25.087782 30630 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:25.067677   468 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:25.094530 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.098996 30554 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:25.103046 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:25.087790   478 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.115811   501 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:25.116993 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.117985 30628 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:25.117993 30704 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:25.122092 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.122997 30705 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:14:25.127650 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65393.1us
I20260709 13:14:25.122474   468 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.133059   300 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.140208   300 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.143460   300 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.148938   546 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:25.149608   546 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:25.151072   576 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:25.151654   576 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:25.154199 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.159436   546 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:25.165467   519 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:25.166110   519 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:25.166637   546 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:25.128397   476 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:14:25.167949   476 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:25.168363   476 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:25.169168   343 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:25.169289   476 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.167222   519 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.171491   476 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.171746   519 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:25.175504   523 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:25.176338   523 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:25.180182 30632 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:25.181605   330 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.191164 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.191841 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.192850 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.193670 31389 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:14:25.199121 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.199936 31383 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:14:25.201793 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.210943 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:25.213593 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:25.213867 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.214289 30559 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:25.214824 30551 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:25.215828 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:25.216850 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:25.217489 30553 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:25.212772 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.220379   464 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:25.220949   464 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:25.221849   464 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.223373 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.224411 32693 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:25.224927 32693 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:25.233366 30523 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:14:25.234087 30523 leader_election.cc:403] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:25.225915 32693 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.246652 30551 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:25.233451 30559 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:25.236879 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:25.238468 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.239658 30707 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:25.256044 30700 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:14:25.248661 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.240867 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:25.256920 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 126932us
I20260709 13:14:25.257421 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.177675   523 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.262419 31378 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:25.264108 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.271873 30554 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:25.281241 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.280843 31394 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:25.273615 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.275447 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.257275 30557 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:25.282683 31388 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:25.282796 31361 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:25.286559 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:25.247936 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:25.292198 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:25.297011 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.297765 31367 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:25.293763 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.279522   514 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:25.280772   503 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.325266 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 73301.6us
I20260709 13:14:25.285475 32501 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:25.343263 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:25.343981 30628 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:14:25.327603 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78243.8us
I20260709 13:14:25.328827 30553 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:25.288790   501 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:25.246529   377 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 13:14:25.356875 30671 leader_election.cc:400] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7.
W20260709 13:14:25.359097 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 65459.6us
I20260709 13:14:25.358122 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:25.365581 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:25.367106 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 81681.4us
I20260709 13:14:25.368191 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:25.370693 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:25.328838   514 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:25.358776 32501 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.364065   503 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.356585   543 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:14:25.370836   523 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:25.304478   473 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 13:14:25.375926   527 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:25.383123   552 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:14:25.370152   472 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:25.412766   547 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:25.426203   577 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:14:25.426893   547 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:25.427567   547 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:25.430159   580 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:25.428938   547 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.400511   514 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.401520 32501 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.447973   547 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.302155   576 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.407501   578 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:25.448949 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 156986us
I20260709 13:14:25.403518   438 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:25.405000 32766 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:25.410213   561 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:25.420300   472 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.475764 31391 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:14:25.481509 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.411087   504 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:25.487073 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 118980us
I20260709 13:14:25.487661 30381 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 133) took 10340 ms. Trace:
I20260709 13:14:25.473516   472 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:25.453138   518 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:14:25.485965   468 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.536851 30550 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:25.494451   330 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.473922   438 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.468537   578 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.542385 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:25.542937 30551 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:25.556391   578 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:25.543447 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 174398us
I20260709 13:14:25.562778 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:25.536831 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.491044 30381 rpcz_store.cc:276] 0709 13:14:15.146893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:15.147187 (+   294us) service_pool.cc:224] Handling call
0709 13:14:25.487578 (+10340391us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:23.899377 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:23.935892 (+ 36515us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:23.935978 (+    86us) write_op.cc:435] Acquired schema lock
0709 13:14:23.936012 (+    34us) tablet.cc:662] Decoding operations
0709 13:14:23.937025 (+  1013us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:23.937132 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:23.937175 (+    43us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:14:23.937898 (+   723us) tablet.cc:701] Row locks acquired
0709 13:14:23.937968 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:14:23.938220 (+   252us) write_op.cc:270] Start()
0709 13:14:23.938436 (+   216us) write_op.cc:276] Timestamp: P: 1783602863938126 usec, L: 0
0709 13:14:23.938469 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:14:24.143897 (+205428us) log.cc:844] Serialized 1693 byte log entry
0709 13:14:24.641372 (+497475us) op_driver.cc:464] REPLICATION: finished
0709 13:14:24.696127 (+ 54755us) write_op.cc:301] APPLY: starting
0709 13:14:24.696244 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:24.696423 (+   179us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:24.696588 (+   165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:24.697347 (+   759us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:24.697648 (+   301us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:14:24.697696 (+    48us) write_op.cc:312] APPLY: finished
0709 13:14:24.698076 (+   380us) log.cc:844] Serialized 40 byte log entry
0709 13:14:24.699081 (+  1005us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:24.699195 (+   114us) write_op.cc:454] Released schema lock
0709 13:14:24.699260 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:24.699728 (+   468us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:24.699761 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":451389,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":54421,"apply.run_cpu_time_us":2815,"apply.run_wall_time_us":3577,"num_ops":4,"prepare.queue_time_us":172763,"prepare.run_cpu_time_us":35454,"prepare.run_wall_time_us":481724,"raft.queue_time_us":99599,"raft.run_cpu_time_us":9747,"raft.run_wall_time_us":54711,"replication_time_us":702662,"thread_start_us":342785,"threads_started":4,"wal-append.queue_time_us":285234}]]}
I20260709 13:14:25.480616   561 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:25.569412   501 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:25.569612   561 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:25.569940   501 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:25.576077 30625 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:25.576167 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.578382 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.579603 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:25.580940 30624 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:25.584149   525 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:25.587929 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.590492 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.591761   542 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.592370   542 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.594535   542 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.595561 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.596010   503 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.596563 30630 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:25.600687 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.601634 31391 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:25.603181 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.610237 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.511880 32464 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:14:25.617682 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.624168 30624 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:25.628275 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.628734 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.630142 30625 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:25.630786 30625 pending_rounds.cc:85] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 5
I20260709 13:14:25.631086 30625 pending_rounds.cc:107] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:25.631383 30625 raft_consensus.cc:2889] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.629432 30633 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:25.639480 30630 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:25.570025   504 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:25.643678 30700 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:25.608775   525 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:25.652956 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.653157 30628 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:25.545934   468 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:25.648072   481 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.665434   481 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.668502   438 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:25.669742   468 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:25.680487 30558 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:25.684952 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.611637   472 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.697728 30629 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:25.702260 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:25.717756 31391 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:25.719436 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.717787 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.721050 31378 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:25.725011 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:25.725523   476 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.554s	user 0.015s	sys 0.000s
I20260709 13:14:25.725768 31367 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:25.726302 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.727175   525 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.728449 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.738643 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.740213   468 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:25.740959   468 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.743463   468 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:25.745401 31367 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:25.750193 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.757726 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.750319 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:25.754225 30555 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:25.760293 31367 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:25.762486 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.763322 31292 leader_election.cc:400] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:25.764149 31292 leader_election.cc:403] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:25.763196 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 179789us
I20260709 13:14:25.765513 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:25.766765 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.767692 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:25.773873 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.777191   343 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:25.773764   580 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:25.770174   593 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:25.774320   589 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:25.780412 30627 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:14:25.780969 30627 pending_rounds.cc:85] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:14:25.781213 30627 pending_rounds.cc:107] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:14:25.781469 30627 raft_consensus.cc:2889] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.783851 30626 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:14:25.784335 30626 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 2
I20260709 13:14:25.769304   377 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:14:25.784574 30626 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:14:25.784843 30626 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:25.721964   513 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
W20260709 13:14:25.762432   478 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 13:14:25.762787   468 raft_consensus.cc:2764] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 13:14:25.797736 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:25.790799 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:25.827324 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 52164.3us
I20260709 13:14:25.796850 32634 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:25.835865 31292 leader_election.cc:400] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 3. Current term is 4.
I20260709 13:14:25.836979 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:25.807005   523 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:25.843361 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:25.852313 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:25.842100   593 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:25.840170 32634 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:25.832852   478 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.021s
W20260709 13:14:25.860155   601 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:14:25.842813   578 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:14:25.855818 32501 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.873214 32501 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:25.897003 31396 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:25.899101   472 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:25.853618   604 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:25.861070   343 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:14:25.860736   593 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:25.903796   593 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:25.904824   583 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.366s	user 0.000s	sys 0.003s
I20260709 13:14:25.907074 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:25.911044 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44608.8us
I20260709 13:14:25.915385 30559 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:25.915578 30556 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:25.917001 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.918926 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:25.919555 30699 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:25.914078 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:25.925050 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 462819us
I20260709 13:14:25.938491 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:25.942034 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:25.943238 30704 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:25.956652 31383 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:14:25.926038   527 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.506s	user 0.008s	sys 0.004s
I20260709 13:14:25.912652   576 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:25.965960 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:25.960269 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.989581 31383 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:25.963577 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:25.996655 30522 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:14:25.998476 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.000588 31361 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:25.932674   578 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:25.870335   501 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:25.898464   514 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:25.925384 32693 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.074s
W20260709 13:14:26.027002 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 150822us
I20260709 13:14:25.986637   527 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:25.930732   602 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 13:14:26.042351 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 121727us
I20260709 13:14:25.999192   542 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:14:26.043183 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44809.6us
I20260709 13:14:26.002689 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:25.959308   343 consensus_queue.cc:1243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:26.043288 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.046747 30550 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:26.047151 30632 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:26.003649   478 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:26.049556 30557 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:26.030998   504 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:26.054471 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:26.059412 31361 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:26.049458 30699 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:14:26.053534 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 201277us
I20260709 13:14:26.067718 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:26.061301   547 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.005s
W20260709 13:14:26.078339 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 156303us
I20260709 13:14:26.032788   585 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:14:26.077512 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.060241   504 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:26.045403   578 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:14:26.075433   376 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:26.071221   502 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:14:26.101253 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46893.8us
W20260709 13:14:26.103230 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58347.1us
I20260709 13:14:26.101686 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.062536   478 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:26.067665   330 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:14:26.098631   504 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:26.197185 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:26.235320 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.072397   580 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:14:26.202869   608 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:26.203785 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 102445us
W20260709 13:14:26.207569   607 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:14:26.093972 32464 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:14:26.211701   343 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:26.094022   525 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:14:26.089218   600 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:14:26.239611 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.120075   502 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:26.241659 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:26.113221   359 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:26.242381 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115666us
W20260709 13:14:26.245430 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 219950us
I20260709 13:14:26.123063   377 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
W20260709 13:14:26.251533 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:26.252526 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:26.253333   501 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:26.253875 32634 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:26.262539 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:26.258139 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 137393us
I20260709 13:14:26.251049   432 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:26.245558   578 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:26.196434   336 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:14:26.256551 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.352837 31396 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:26.256264   343 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:26.248466   602 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:14:26.286821   359 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:26.379254 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 122832us
I20260709 13:14:26.388058 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:26.387054 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:26.390318 30557 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:26.312971   551 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:14:26.354362   519 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.383571   376 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:26.407392 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:26.383704 32634 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:26.408396   377 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.356493   609 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:26.356351   592 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.539s	user 0.003s	sys 0.001s
I20260709 13:14:26.356290   578 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:26.356352 32691 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:14:26.387152   514 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:26.395488   359 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:26.455798 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97934.5us
W20260709 13:14:26.445050   589 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.005s
W20260709 13:14:26.443539   468 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.015s	sys 0.043s
I20260709 13:14:26.433820   519 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:26.411747   593 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:26.472932 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:14:26.471624 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83666.8us
I20260709 13:14:26.490149 30559 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:26.484762 32634 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:26.437294   376 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:26.497682   376 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:26.449286   377 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:26.500881   377 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:26.494350 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:26.501538   377 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:14:26.501917 30700 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:26.503376   494 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.503988   494 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:26.506291   494 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:26.512369   578 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.513839   578 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:26.506361 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:26.515214   478 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:26.519438   611 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.396s	user 0.001s	sys 0.000s
I20260709 13:14:26.476744   589 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:26.507146   623 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:26.426908   502 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:26.525462 31383 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:26.525954 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53134.9us
I20260709 13:14:26.526518   300 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:26.526818 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:26.527185   519 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:26.528651 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:26.461483   481 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:26.462409   609 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:26.522661   623 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:26.527031   300 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:26.508042 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:26.511266   626 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:26.531196 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:26.481904   468 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.016s	sys 0.043s
I20260709 13:14:26.581935   609 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:26.583081   590 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:26.583884   590 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:26.584928   590 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:26.585573   593 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:26.570475 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:26.596783   576 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.004s
I20260709 13:14:26.560479 31383 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:26.556442 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:26.568979 30700 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:26.552673 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:26.629846 30559 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:26.575739   620 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:26.555258   501 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:26.567888   300 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:26.479178   438 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.570464 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:26.665073 30555 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:26.555120 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:26.666679 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 111699us
I20260709 13:14:26.667618 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:26.601035 30596 leader_election.cc:336] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.545s (SENT)
W20260709 13:14:26.668704 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67765.2us
W20260709 13:14:26.631345 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75036.7us
I20260709 13:14:26.632885 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:26.597995   617 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:26.551889 32634 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 13:14:26.554580   623 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:26.627533   608 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.630515   590 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:26.680050 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.683996 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51243.1us
I20260709 13:14:26.657415   620 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:26.696139   618 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.253s	user 0.003s	sys 0.001s
I20260709 13:14:26.696630 32634 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:26.697132 32634 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:26.706686 31391 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:14:26.708146 31367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.007s	sys 0.032s
W20260709 13:14:26.708624 31367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.032s
I20260709 13:14:26.708907 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.710383 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:26.676751   617 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:26.688642   501 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:26.727885 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.690882   527 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:26.736423 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:26.693856   580 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:26.770408 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:26.811522 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75226.9us
I20260709 13:14:26.694195   620 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:26.678248   438 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:26.842469 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 184425us
W20260709 13:14:26.877286 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 311324us
I20260709 13:14:26.698108 32634 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:26.692497   608 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:26.896549 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 168791us
I20260709 13:14:26.734936   377 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:26.841737 32750 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:26.727146   343 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:26.750486   527 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:26.810772   481 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.875938   591 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.915706 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:26.720131   600 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.175s
I20260709 13:14:26.811868   589 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
I20260709 13:14:26.965406   438 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:26.934213   527 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:26.961899   501 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.975801   625 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.977149   625 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.979035   615 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:26.962299   359 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.982797   599 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.285s
I20260709 13:14:26.986716   628 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:26.987486   628 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:26.987982   628 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:26.990818   622 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:26.991766   622 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.923674   377 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.933715   343 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:26.943714   481 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.008601   481 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.011144   343 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.013386   625 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:26.996049 32634 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:27.062986   633 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:26.959864   595 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
W20260709 13:14:27.015393 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 128997us
I20260709 13:14:27.059628   630 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:27.046303   504 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.797s	user 0.013s	sys 0.000s
I20260709 13:14:27.072151   377 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.072540   504 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.074883 30550 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:27.076944   504 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.072192   630 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:27.078805   630 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:27.069574 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:26.918895 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 318333us
I20260709 13:14:27.088788 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:26.927871   617 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:26.959355   591 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.079982   359 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.082017   630 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:27.098188 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 428886us
W20260709 13:14:27.152499 30596 leader_election.cc:336] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.620s (SENT)
I20260709 13:14:27.153141 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:27.156584 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:27.123993   638 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:14:27.319168 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:27.244486 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.001s	sys 0.014s
W20260709 13:14:27.126869   578 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.613s	user 0.000s	sys 0.015s
I20260709 13:14:27.120954   502 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:27.101372   591 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.281178   580 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:27.247157 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 316959us
I20260709 13:14:27.280351   599 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:27.245592 30554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.034s	sys 0.056s
W20260709 13:14:27.286456 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.136293   608 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.102973   646 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:27.362851   646 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.363009 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.363361   646 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:27.108314   622 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:27.332818 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 178548us
I20260709 13:14:27.363934 30552 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:27.365947 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:27.340325   523 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:27.332002 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 243771us
I20260709 13:14:27.367296   527 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.344951   600 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.369710   527 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.357513 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.369971   600 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.370764   501 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:27.371419   501 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.127743   630 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:27.342242 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:27.334228   596 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:27.342521 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.348348 30698 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:27.335333 31391 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:27.352459 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:27.352705 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.353325 30705 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:27.130406   300 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:27.354661 30525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.008s	sys 0.010s
I20260709 13:14:27.355048 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.372565 30700 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
W20260709 13:14:27.344920   590 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.012s
I20260709 13:14:27.363490 30704 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:27.339458 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.003s	sys 0.015s
W20260709 13:14:27.535699 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:27.560492 30525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.020s	sys 0.026s
W20260709 13:14:27.562834 30554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.041s	sys 0.062s
W20260709 13:14:27.350950   623 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.657s	user 0.014s	sys 0.004s
I20260709 13:14:27.539062 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 778) took 1602 ms. Trace:
I20260709 13:14:27.563481 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:27.433056 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67167.7us
W20260709 13:14:27.539682 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.566468 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:27.570808 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.567916 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.616794 30559 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:27.617575 30706 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:27.626255 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.553171   501 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.491072   579 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:27.439150   652 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:14:27.640061 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.640388   579 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.641113 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:27.380601   622 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:27.443480   525 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:27.643086 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.643457   579 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.643652   525 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.618597 31367 rpcz_store.cc:276] 0709 13:14:25.936448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.936606 (+   158us) service_pool.cc:224] Handling call
0709 13:14:25.937111 (+   505us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:25.952086 (+ 14975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.952118 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:25.966445 (+ 14327us) log.cc:844] Serialized 52 byte log entry
0709 13:14:26.709051 (+742606us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.040643 (+331592us) log.cc:844] Serialized 14 byte log entry
0709 13:14:27.148318 (+107675us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18000b47c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:27.525293 (+376975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.538632 (+ 13339us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:27.538689 (+    57us) raft_consensus.cc:1685] finished
0709 13:14:27.538801 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:27.538995 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42881,"prepare.queue_time_us":377411,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":76965,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":14478,"spinlock_wait_cycles":8744832,"thread_start_us":1287401,"threads_started":3,"wal-append.queue_time_us":949061}
I20260709 13:14:27.643119   622 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.645403 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.645522 31361 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:27.646319 31367 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:27.647387 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.648200 31367 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:27.648440   472 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.649020   472 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.650765   658 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:27.651678   658 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.656427 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 782) took 1656 ms. Trace:
I20260709 13:14:27.656968 31383 rpcz_store.cc:276] 0709 13:14:26.000395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.002370 (+  1975us) service_pool.cc:224] Handling call
0709 13:14:27.656135 (+1653765us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b54008b2754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:27.656278 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106578176}
I20260709 13:14:27.666842   654 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:27.667769   654 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.668473   378 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:27.668947   378 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:27.671113 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.672142 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.673643 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.674314 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:27.675156 30625 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:27.675320 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:27.676250 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.676461 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:27.676797 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.677062 30630 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:27.677280 30556 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:14:27.670358   654 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.676280 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:27.436167   620 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.541s	user 0.014s	sys 0.003s
I20260709 13:14:27.658422 31383 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:27.446835   561 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:27.436684   650 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:14:27.685322 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:27.747478 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62263.5us
I20260709 13:14:27.680423 30551 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:27.372601   600 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.499800   639 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:27.444284 32691 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.761894 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.669979   378 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.710944 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.525579   468 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:27.448366 32693 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.448347   547 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:27.704713   614 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.490s	user 0.002s	sys 0.011s
W20260709 13:14:27.701712 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.750025 30630 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:14:27.738850 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:27.448753   628 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.490859   641 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.619470   591 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:27.761240 30626 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:27.763156 30559 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:27.763378 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.723348 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.764251 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.802032 30628 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:27.765409 30553 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:27.536887   542 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:27.768005 30703 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:27.768916 30698 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:27.758178 31361 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:27.761729   596 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.726145   523 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:27.779815 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.726565   481 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:27.783550 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:27.787889 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49205us
I20260709 13:14:27.785670 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.765455 32464 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.761619   609 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:27.814420 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:27.815461 30633 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:27.777990   619 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.380093   580 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.737703   620 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.760090 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.720082   623 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.805754 31388 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:27.815994 30631 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:27.777031   531 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:27.771721 32691 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.615700   599 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.750232   561 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.777978   649 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:14:27.805068   542 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:27.799950   514 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:27.785567 32693 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.842567 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 791) took 1145 ms. Trace:
I20260709 13:14:27.771934   639 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.810644 30382 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:27.831142 30551 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:27.817827   378 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:27.854557 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 776) took 1927 ms. Trace:
I20260709 13:14:27.818753   609 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:27.795151 30556 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:27.860321 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.862757 31367 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:27.861650 31396 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:27.859584 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:27.865387 31389 rpcz_store.cc:276] 0709 13:14:26.697032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.698938 (+  1906us) service_pool.cc:224] Handling call
0709 13:14:26.699314 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:26.699731 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:26.699780 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:26.737041 (+ 37261us) log.cc:844] Serialized 26 byte log entry
0709 13:14:27.086244 (+349203us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.086293 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.086372 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:27.705728 (+619356us) spinlock_profiling.cc:243] Waited 6.74 ms on lock 0x7b48000f3fd8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:14:27.829457 (+123729us) raft_consensus.cc:1685] finished
0709 13:14:27.829485 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:27.842505 (+ 13020us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":10277,"raft.run_wall_time_us":997866,"spinlock_wait_cycles":15511296,"thread_start_us":143390,"threads_started":1,"wal-append.queue_time_us":208075}
I20260709 13:14:27.833351   661 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:27.884464 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.885370 31389 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:27.885509 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:27.873445 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:27.812304   523 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.886643 30627 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:27.828095   525 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.874384 30382 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 1 to 3, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:27.841511   531 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.820552   481 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.822363 32464 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:27.876510 31364 rpcz_store.cc:276] 0709 13:14:25.927417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:25.927572 (+   155us) service_pool.cc:224] Handling call
0709 13:14:25.927941 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:25.951362 (+ 23421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:25.951395 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:25.968549 (+ 17154us) log.cc:844] Serialized 26 byte log entry
0709 13:14:26.066685 (+ 98136us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:26.066749 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:26.066851 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:27.703895 (+1637044us) spinlock_profiling.cc:243] Waited 3.96 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:27.837499 (+133604us) raft_consensus.cc:1685] finished
0709 13:14:27.837522 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:27.854470 (+ 16948us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22972,"raft.queue_time_us":24361,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":21236,"spinlock_wait_cycles":9105280,"thread_start_us":97645,"threads_started":1,"wal-append.queue_time_us":126425}
W20260709 13:14:27.879498   502 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.520s	user 0.006s	sys 0.007s
I20260709 13:14:27.856918   668 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:27.891539 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.807818   591 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.807541   641 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:27.904471 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 190168us
I20260709 13:14:27.889006   609 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:27.861060   494 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:27.912899 30382 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 8 to 9, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.812422   596 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.924602 31364 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:14:27.936085 31364 pending_rounds.cc:85] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 3
I20260709 13:14:27.936434 31364 pending_rounds.cc:107] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:14:27.936720 31364 raft_consensus.cc:2889] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:27.936832 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.938202 30382 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:27.937539 30698 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:27.835377   619 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:27.878288   547 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 13:14:27.868149   646 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:27.923960 31389 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:27.872864   514 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:27.940774 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:27.942032 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:27.942775 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:27.943579 30382 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:27.961062 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:27.966498 30382 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 6 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:27.978308   561 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.980243   591 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.981751   639 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.982708   620 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.983506   531 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.984809   523 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.985507   619 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:27.986191 32691 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.987569   599 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.988150   514 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.989065   580 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.942471   494 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:27.990504 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49827.2us
I20260709 13:14:27.990985   641 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:27.994436   667 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:14:27.995143 32464 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.995505 30624 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:27.997251   596 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:27.998757   623 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:27.984894   646 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.003985 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.007156 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.008596 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.011823 30559 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:28.010273 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.010795 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.017653 30551 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:28.019726 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:28.020486   663 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:14:28.021174 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:28.023290 30626 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:28.024487 32693 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.025496 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:28.026923   481 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:27.972114   300 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.029263   628 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.031389 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.032481 30698 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:28.032682 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:28.035012 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.035447   472 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.035741 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:28.036232 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.037384 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.038488 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.040010 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.042058 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.044587 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.008980 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.052407 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:28.057498 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 229254us
I20260709 13:14:28.058792 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.061090 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:28.067605 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 234865us
I20260709 13:14:28.052428 30382 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:28.071714 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:28.072855 30627 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:28.071933 30382 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:27.992611   494 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.069526 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.100577 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.058288   300 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:28.103926 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.104862 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.058121   579 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:28.084334   641 raft_consensus.cc:2764] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:14:28.111254   633 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:28.116979 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.122025   658 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:28.129060 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57403.7us
I20260709 13:14:28.122293   576 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.131681   673 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:28.107968   591 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:28.108384   579 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.137697 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:28.145970 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45542.2us
I20260709 13:14:28.083051   649 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:14:28.079512 30382 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.163177 30551 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:28.156157 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.165481 30382 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 5, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:28.167757 30382 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 4, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 4 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:28.147678   481 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:28.146952 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.145763 30559 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:28.109436   677 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:28.180783 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.169759 30382 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 2 to 3, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:28.142024   673 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.195528 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:28.195586 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.163494   579 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:28.198585 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 790) took 1501 ms. Trace:
I20260709 13:14:28.217608 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.176352   658 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.216583 31394 rpcz_store.cc:276] 0709 13:14:26.696675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.698000 (+  1325us) service_pool.cc:224] Handling call
0709 13:14:26.698369 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.079126 (+380757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.079162 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:27.079212 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.079246 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.198196 (+1118950us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540057b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:28.198271 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.198497 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875018624}
I20260709 13:14:28.130112   633 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.224561 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.189194   531 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.250998   531 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.252210 31388 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:28.253541   531 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.216230   300 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:28.231431   518 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:28.254371   378 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:28.220422   678 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:28.254725   518 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.255034   378 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.255496   678 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:28.217128   646 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:28.236557   464 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.256630   678 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.257221   378 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.257552   464 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.245602   675 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:28.246227   658 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.259899   464 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.260500   609 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:28.261428   658 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.248525   478 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.146341   576 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.262218   478 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.262601 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.192662   591 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:28.251297   596 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:28.193213   519 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:28.264614   576 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.264592   478 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.265679   591 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.219280   673 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.267649 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.268357 30704 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:28.269409 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:28.257222   518 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.276566   633 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.263339 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.284400   518 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.285482   609 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.285295   518 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.288893 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.290904 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:28.292028   531 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:28.292725   531 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:28.294109   531 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:28.296404   518 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.305298   673 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.315625   673 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:28.317736   662 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
W20260709 13:14:28.342933 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94185.5us
I20260709 13:14:28.344142 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.346092   539 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:28.346722   539 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:28.347916   539 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.349336   579 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.352975   678 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:28.349417   539 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.368676 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 311) took 1022 ms. Trace:
I20260709 13:14:28.369429 30700 rpcz_store.cc:276] 0709 13:14:27.345813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.345986 (+   173us) service_pool.cc:224] Handling call
0709 13:14:28.368361 (+1022375us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b5400343354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:28.368569 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4683520}
I20260709 13:14:28.382212   591 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.383076   591 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.385782   591 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.388206   617 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.400496   579 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.403287   531 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.411350   531 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.414244 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:28.419620 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.421578   514 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:28.422896 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.425376 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.426138 31367 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:28.427237 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.427955 31396 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:28.428900 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.429569 31389 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:28.431015 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.432039 31391 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:28.433214 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.433845 31389 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:28.435513 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:28.437345 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.453723 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.455416 31367 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:28.454322 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.461905 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.462194 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.463106 31394 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:28.463568 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.460525 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.465453 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:28.472023 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.474103 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.476403 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.477144 30551 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:28.450770   609 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:28.479270 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.480513 30556 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:28.453723   478 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:28.485179   478 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:28.485687   478 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.472189 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.481685 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.487637 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.488955 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.488945   525 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:28.489945   525 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.491210 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.482426 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.493008 31367 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:28.484443 30551 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:28.483551 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.494871 31394 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:28.458599   576 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.473451 32766 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:28.495370 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.492857 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.464272   658 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:28.494681 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.469079   629 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:28.501083 30555 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.496511 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.498078 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.490356   525 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:28.492498 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.451086   661 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:14:28.475520   378 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:28.490206   679 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:28.494161   502 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:28.528710 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.513175 31396 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:28.504240 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.534685 30700 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:28.535305 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.503750   629 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.499178 32766 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:28.544601 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:28.548007 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.549131   678 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:28.549549 31367 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:28.552496 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.553316 31367 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.523890   531 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:28.563488   531 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:28.527352   679 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:28.564131   531 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:28.564311   679 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.565336   531 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.527249   378 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.567004   378 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:28.569133   670 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.570s	user 0.002s	sys 0.009s
I20260709 13:14:28.569494 30555 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:28.539238   599 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:28.570935   599 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.571297   599 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:28.571089 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.573383 31378 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:28.543629 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.546847   672 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:28.536294   502 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:28.546252 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.589941 31367 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:28.589250 31394 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:28.540302   514 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:28.595847 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49738.7us
I20260709 13:14:28.503357   576 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.515589   658 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.548252   629 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.576321 32766 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:28.599571 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:28.590293   668 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:28.592819   672 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:28.608630   672 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.602738   678 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.613869   478 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:28.616686   658 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.601078   513 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.619052 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:28.618248 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.595609   502 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:28.624675 31292 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.659s (SENT) [suppressed 9 similar messages]
I20260709 13:14:28.628978 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:28.648025 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:28.657713 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:28.655158   641 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:28.667493   641 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:28.668572   641 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.625654   478 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.651141   513 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:28.682986 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1545) took 2574 ms. Trace:
I20260709 13:14:28.671030   639 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:28.683542 30550 rpcz_store.cc:276] 0709 13:14:26.106836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.107389 (+   553us) service_pool.cc:224] Handling call
0709 13:14:26.107876 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.075430 (+967554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.681177 (+1605747us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54002891d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:28.681328 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223280896}
W20260709 13:14:28.691773   666 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.713s	user 0.002s	sys 0.008s
I20260709 13:14:28.696661   576 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.700531   525 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:28.702091 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.702953 30700 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:28.704213   658 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.704818   658 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:28.720755 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.723778   378 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.720037 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 960) took 2178 ms. Trace:
I20260709 13:14:28.725402   525 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.735742 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.729486 30632 rpcz_store.cc:276] 0709 13:14:26.528692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:26.528836 (+   144us) service_pool.cc:224] Handling call
0709 13:14:26.564717 (+ 35881us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:26.632243 (+ 67526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:26.632291 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:26.634702 (+  2411us) log.cc:844] Serialized 26 byte log entry
0709 13:14:27.374302 (+739600us) spinlock_profiling.cc:243] Waited 3.25 ms on lock 0x7b18000f7540. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:14:27.806626 (+432324us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.806673 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:27.806745 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:28.691086 (+884341us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:28.706295 (+ 15209us) raft_consensus.cc:1685] finished
0709 13:14:28.706326 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.706510 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5288,"raft.queue_time_us":31420,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6098,"spinlock_wait_cycles":132688640,"thread_start_us":418939,"threads_started":1,"wal-append.queue_time_us":1280041}
I20260709 13:14:28.736616 31396 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:28.741683 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117044us
I20260709 13:14:28.718564   576 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:28.750735 31391 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:28.751055 31383 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:28.726166   502 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.752151   378 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.723201   639 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:28.755573 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:28.754663   604 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:28.756757   703 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:14:28.758673 30550 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:28.767690 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:28.768798 31391 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:28.759158   604 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.764622   609 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:28.757910   576 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.793017 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 797) took 1081 ms. Trace:
I20260709 13:14:28.793606   576 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.794396   639 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:28.795892   591 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:28.796723   591 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.799593   658 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.775761   609 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:28.801234   609 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.802037 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:28.802290   609 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.802543   542 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:28.804607   599 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:28.793563 31383 rpcz_store.cc:276] 0709 13:14:27.711543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.723066 (+ 11523us) service_pool.cc:224] Handling call
0709 13:14:28.792823 (+1069757us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540057f154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:28.792945 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183265152}
W20260709 13:14:28.810774   674 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.791s	user 0.003s	sys 0.001s
I20260709 13:14:28.799129   591 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.816422   599 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.816874   672 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:28.817663   707 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:14:28.817881 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.818552 31394 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:28.819461 30550 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:28.804756   542 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:28.903481 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:28.904352 31383 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:28.902793   429 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.907711 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1305) took 1281 ms. Trace:
I20260709 13:14:28.908214   429 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:28.908991   622 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.909600 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 377) took 1343 ms. Trace:
I20260709 13:14:28.910054 30552 rpcz_store.cc:276] 0709 13:14:27.565743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.565871 (+   128us) service_pool.cc:224] Handling call
0709 13:14:27.639787 (+ 73916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.642924 (+  3137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.642956 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:27.643004 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:27.643039 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.909296 (+1266257us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b540013a954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:28.909354 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.909529 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6705920}
W20260709 13:14:28.912181 30624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 392) took 1247 ms (client timeout 1659 ms). Trace:
W20260709 13:14:28.912668 30624 rpcz_store.cc:269] 0709 13:14:27.664726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.670771 (+  6045us) service_pool.cc:224] Handling call
0709 13:14:28.911945 (+1241174us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:28.912091 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":742603520}
I20260709 13:14:28.913307   429 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:28.908267 30701 rpcz_store.cc:276] 0709 13:14:27.620648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.641958 (+ 21310us) service_pool.cc:224] Handling call
0709 13:14:27.642266 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.876967 (+234701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.876999 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:27.877046 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:27.877078 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.901376 (+1024298us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:28.901435 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:28.901611 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539183488}
I20260709 13:14:28.919093 31394 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:28.920210   502 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.899817   599 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:28.923060   599 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:28.927135   672 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:28.929047   678 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:28.925740   599 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:28.931869 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.941241 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:28.901388   432 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:28.950664 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:28.996407 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45856.3us
I20260709 13:14:28.999208 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:29.002663 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.003515 31394 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:29.005411 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:28.951038 31383 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:28.920249   438 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 13:14:29.061789 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.052968   432 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.063920   432 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:28.950839   502 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:28.903153 32766 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.073022 31378 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:29.050350   599 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:29.074290 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.079275 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 971) took 1276 ms. Trace:
I20260709 13:14:29.050590   622 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.052575 32691 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:29.081284 30628 rpcz_store.cc:276] 0709 13:14:27.803315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.803695 (+   380us) service_pool.cc:224] Handling call
0709 13:14:27.804100 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:27.909962 (+105862us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:27.938639 (+ 28677us) log.cc:844] Serialized 14 byte log entry
0709 13:14:28.127315 (+188676us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.127402 (+    87us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:28.127490 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.078943 (+951453us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54006f0254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:29.079019 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.079188 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59427,"prepare.queue_time_us":29379,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":33589,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":12138,"spinlock_wait_cycles":239645952,"thread_start_us":154689,"threads_started":1,"wal-append.queue_time_us":224653}
I20260709 13:14:29.085489 31394 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
W20260709 13:14:29.072831 31361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 988) took 1509 ms (client timeout 1883 ms). Trace:
I20260709 13:14:29.054793   501 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.106501 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:29.107825 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.107620 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 967) took 1341 ms. Trace:
W20260709 13:14:29.105119 31361 rpcz_store.cc:269] 0709 13:14:27.563225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.640848 (+ 77623us) service_pool.cc:224] Handling call
0709 13:14:29.072629 (+1431781us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b54002b1454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:29.072759 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8599936}
I20260709 13:14:29.075574   438 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:29.116217   472 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:29.116384 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:29.117503 30552 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:29.108534 30629 rpcz_store.cc:276] 0709 13:14:27.766353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.766479 (+   126us) service_pool.cc:224] Handling call
0709 13:14:27.793764 (+ 27285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:27.891098 (+ 97334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.891146 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:27.899476 (+  8330us) log.cc:844] Serialized 52 byte log entry
0709 13:14:28.143341 (+243865us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:28.143386 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.072026 (+928640us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x7b5400577954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:29.072095 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:29.072157 (+    62us) raft_consensus.cc:1685] finished
0709 13:14:29.072175 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.107528 (+ 35353us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13346,"raft.queue_time_us":43631,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":5068,"spinlock_wait_cycles":194905472,"thread_start_us":199006,"threads_started":1,"wal-append.queue_time_us":301766}
I20260709 13:14:29.080876   599 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:29.108757   679 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.000s
I20260709 13:14:29.122462 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.128479 31361 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.113042 32691 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.091071   502 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.139639 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:29.143675 30702 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:29.145906 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:29.146773 30705 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:29.093415   622 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.148381 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:29.124020 32766 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.112478   501 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.128695   599 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:29.154830 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.156462   542 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:29.153597 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.161943 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.134059   472 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.149660   608 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:29.151844   468 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:29.153043   593 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:29.151906   377 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:29.152514 31361 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:29.166953   502 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.149456   438 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.128510   679 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.151624   658 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:14:29.174690   468 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.197795 30704 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:29.199643 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.179809   593 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:29.176515   608 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.195677   468 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:29.215121 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.179450   377 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.217921 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 933) took 1551 ms. Trace:
I20260709 13:14:29.218804 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.218312 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.221588 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.202145   658 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:29.230785   658 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.222435 30703 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:29.223541 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.223832 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.207212   478 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:29.207538   623 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:29.219159 30557 rpcz_store.cc:276] 0709 13:14:27.666170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.801284 (+135114us) service_pool.cc:224] Handling call
0709 13:14:27.821395 (+ 20111us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:27.821908 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:27.821955 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:28.669206 (+847251us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:28.669722 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 13:14:28.939531 (+269809us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:28.939581 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:28.939687 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:29.213152 (+273465us) raft_consensus.cc:1685] finished
0709 13:14:29.213174 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.217858 (+  4684us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":553,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":10598,"spinlock_wait_cycles":14637312,"thread_start_us":269422,"threads_started":1,"wal-append.queue_time_us":472027}
W20260709 13:14:29.238763 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.229117   608 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.226626   593 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.239602   438 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.234246 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.237334 30705 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:29.241366 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:29.242736 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.244349   679 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.244798   623 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.250765 30706 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:29.188350 32691 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.251542 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.250152 30698 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:29.248641 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:29.251821 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.254426 30707 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:29.254751 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.255813   377 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:29.256327 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:29.256851   623 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:29.257484   716 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.001s
W20260709 13:14:29.257501 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.288816 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:29.257647 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.289816 31361 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:29.290695 31394 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:29.290910 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.260430   626 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:29.260811 32766 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:29.297438 31388 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:29.265141 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:29.257591   679 raft_consensus.cc:2764] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:14:29.293009 31394 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:29.292045 31361 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:29.296623 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.262410 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.301878 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.302897 31389 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:29.312966 31367 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:29.314790 30706 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:29.328193 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:29.304500   629 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.698s	user 0.014s	sys 0.000s
I20260709 13:14:29.314303   377 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:29.235924   472 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.296736   585 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.326406   438 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:29.364485 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 62191.9us
I20260709 13:14:29.370965   429 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.375141   658 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.343421   629 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.358969   377 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.396502   377 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.389066 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.377290   429 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.376927   501 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.408272 30698 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:14:29.360764   585 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.409008   473 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:29.417191   473 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:29.421068 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.410370   658 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:29.441162 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 176119us
W20260709 13:14:29.441401   718 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:14:29.437265   724 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.478403 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:29.440974   658 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.486862 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.495426   468 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.496841   585 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.500588 31388 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:29.479907   542 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:29.502031   542 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.505182   542 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.505885   723 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:14:29.508477   629 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:29.484311 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.511530 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.512951 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:29.513124 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.514421 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.514552 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.513255 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.512578 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.518096 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.524307 31389 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.525770 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.527895 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.528638 30701 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:29.528568 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.516682   478 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:29.531271 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.531770 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.533998 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.534500 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:29.534571 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47817.9us
I20260709 13:14:29.515964   609 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:29.535615   513 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.517055   623 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:29.542681   513 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.543076 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.543318   513 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.543833 30553 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:29.519033   468 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.542861   623 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:29.547575   623 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:29.518957   501 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.535229   639 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:29.555814 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.538408 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:29.558980 30550 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:29.523348   608 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:29.532572   594 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:29.535306   678 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:29.575857 31389 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:29.579527 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 644) took 1560 ms. Trace:
I20260709 13:14:29.580011 30630 rpcz_store.cc:276] 0709 13:14:28.019198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.191698 (+172500us) service_pool.cc:224] Handling call
0709 13:14:28.192181 (+   483us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:28.303889 (+111708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:28.304098 (+   209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:28.304154 (+    56us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:28.304204 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:29.579270 (+1275066us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400865bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:29.579325 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:29.579465 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":253991296}
I20260709 13:14:29.535465   438 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:29.537982   378 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:29.578112   429 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:29.588085   513 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.537590   481 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:14:29.588382   429 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.589020   429 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.589723   513 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:29.590183   513 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:29.591246   429 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:29.584610 30553 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:29.530314   726 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 13:14:29.592617   726 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:29.592955   726 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:29.593770   726 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.556675   542 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:14:29.586759   723 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:29.586532   658 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:29.599535 30552 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:29.595373   726 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:29.575310   594 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.631250   594 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:29.595465   542 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:29.594590   724 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:29.642434 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104992us
I20260709 13:14:29.583704   668 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.609509   723 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:29.647233 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:29.568336   501 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.650027 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.651968 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:29.650763 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.649398   481 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.651157   668 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:29.653563   481 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.655987   481 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.656812 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.651798   723 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:29.658200 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.660902   501 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:29.661566   429 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:29.662148   429 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:29.662721   513 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.663160   429 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.667017 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.675715   604 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.676731 30702 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:29.650188   514 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:29.678681 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.679107   514 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.681226 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.683143   501 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:29.685425   501 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:29.687104   514 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:29.685720   599 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.002s
W20260709 13:14:29.691198 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.692030   599 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.693250 30700 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:29.694262 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:29.694792 30705 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:29.695324   591 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:29.695799   591 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.696098   599 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:29.695359   639 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:29.696890 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:29.698203 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.698954 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:29.696750   591 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.701174   591 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:29.643957   542 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:29.703445   542 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:29.704790   542 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:29.704905   585 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:29.705448   585 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:29.707378 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:29.708851 31383 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:29.723043 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.725518 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.726367 31378 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:29.728662 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.734794 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.735911 30630 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:29.737051 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.737440 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.739428 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.738739 31396 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:29.740566 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.735561 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.743300 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.740147 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:29.724717   481 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.730789 32693 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:29.725137   678 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.762751   438 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:29.763603 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.754271   377 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:29.777947   715 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.621s	user 0.001s	sys 0.002s
I20260709 13:14:29.706344   585 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.780658   502 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:29.781332   502 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.781595   585 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:29.782296   502 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.784478   377 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.785490   377 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.786640   432 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:29.787135   432 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:29.788007   432 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.766724   438 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.809553 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:29.811425 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80143.1us
I20260709 13:14:29.812752 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.813508 30626 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:29.815795 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.769419 32693 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:29.829387 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.764233   678 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.833218 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.833763 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.762238   481 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:29.835573 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:29.836089 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.810878   514 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.842801 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.840046 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.843509 30625 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:29.843930 30630 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:29.838042 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:29.845345 30550 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:29.847016 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.850328 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.836200   735 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:29.853143 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 179480us
I20260709 13:14:29.861344 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:29.856850 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.861347 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:29.862349 30631 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:29.867028 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.827992   639 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:14:29.842555   671 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:29.894527   671 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.852344   514 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.874782 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:29.879163 30630 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:29.859442   735 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.888195   481 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:29.862761   727 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:14:29.890997 32693 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.893736   678 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:29.912909 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.921080 31388 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:29.898805   624 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:29.861840   736 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:14:29.942907 30558 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:29.914647   429 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:29.943547   624 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:29.976657 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 164457us
I20260709 13:14:29.949676 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.982237 30630 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
W20260709 13:14:29.983353 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 123047us
I20260709 13:14:29.911227   594 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:29.871547   438 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:30.066524 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 205248us
I20260709 13:14:30.131356 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.133476 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.134232 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.146282 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:29.978084   624 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.151582 31388 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:30.152537 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:29.923266   734 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.077s
I20260709 13:14:30.178304 30551 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:30.000022   576 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:29.982281   727 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.191s
W20260709 13:14:30.206943 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.212615 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.180267   668 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.009s
I20260709 13:14:30.141271   738 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:30.181566   723 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.003s
I20260709 13:14:30.218310 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:29.987514   623 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:29.978026   429 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:30.211547   713 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:14:30.239013   710 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.464s	user 0.015s	sys 0.025s
I20260709 13:14:30.200951   671 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.201802   576 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:30.270144 31389 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:30.261696 31383 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:30.264245   429 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.243487 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:30.273947 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.270684 30631 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:30.277328 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:14:30.236325   723 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:30.277801 31292 leader_election.cc:403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:14:30.280185 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.277736 30626 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4.
I20260709 13:14:30.282485 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.284632 30626 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:30.281517 30631 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:14:30.201333   742 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:14:30.291636 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.292467 30625 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.221927   678 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:14:30.303293 30625 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:30.300741 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:30.305063 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 128554us
I20260709 13:14:30.308367 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.311538 30626 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:30.314417 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 820) took 1513 ms. Trace:
I20260709 13:14:30.314942 31367 rpcz_store.cc:276] 0709 13:14:28.801532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:28.801722 (+   190us) service_pool.cc:224] Handling call
0709 13:14:30.314228 (+1512506us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54004b05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:30.314350 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173135744}
I20260709 13:14:30.315053 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.237413   668 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.255811   736 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:14:30.321964 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.322330 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 321) took 1105 ms. Trace:
I20260709 13:14:30.322892 30630 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:30.302163   604 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
W20260709 13:14:30.333842 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.320771   542 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.615s	user 0.020s	sys 0.000s
I20260709 13:14:30.322781 30700 rpcz_store.cc:276] 0709 13:14:29.217358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.217760 (+   402us) service_pool.cc:224] Handling call
0709 13:14:29.227039 (+  9279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.693669 (+466630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.322067 (+628398us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54005a32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:30.322267 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071711488}
W20260709 13:14:30.334729 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.267061   576 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.335062   622 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:30.335559   576 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:30.309664 32766 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 13:14:30.341446   726 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.006s
W20260709 13:14:30.343202   740 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 13:14:30.346633   741 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 13:14:30.359198 30633 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:30.314942   514 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:30.369247 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56843.6us
I20260709 13:14:30.370033 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.370990 30630 raft_consensus.cc:2376] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7. Re-sending same reply.
I20260709 13:14:30.313912   631 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:30.359334   735 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.391335 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.392166 30626 raft_consensus.cc:2376] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 4. Re-sending same reply.
I20260709 13:14:30.393762 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.394232   594 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.410643 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.438637 30630 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:30.415140 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.421036   432 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:30.172463 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:30.429291   585 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.647s	user 0.017s	sys 0.003s
I20260709 13:14:30.432431   626 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:14:30.440646 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.408228   712 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.241s	user 0.000s	sys 0.011s
I20260709 13:14:30.449533   626 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:30.452451 30559 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:30.460214 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:30.458169 31292 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.659s (SENT)
I20260709 13:14:30.462551 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.463229 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.467893 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.467768 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.469705 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:14:30.451176   623 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.521485 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.469195   576 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:30.465754   658 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.544211   658 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.546052   514 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.546782   514 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.509667   622 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:30.530804 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61225.1us
I20260709 13:14:30.536195   750 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.549942 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.550524   750 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.551108   749 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.551412 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.538280 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.467275   742 consensus_queue.cc:1243] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:30.538313 32691 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:30.540374 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.541229 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:30.547049   668 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.538992 30700 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:30.551834   749 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.563453 30707 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:30.568367 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.560742 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:30.620954   576 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:30.621608   576 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.622079   576 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:30.562162   671 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:30.534864   723 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:30.626777 32691 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.561533   658 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.629292   671 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.594471   622 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:30.563649   623 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:30.631457 32691 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:30.632308   668 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.632982   750 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.633051   668 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.634112   671 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.636112   749 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:30.629302 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 91125us
I20260709 13:14:30.630544   763 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:30.637344 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.638262   763 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:30.640033   668 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.647220 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:30.648470   763 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:30.648943   763 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:30.649309   763 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.655583 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.658643 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.659747 30551 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:30.661288 32691 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:30.664274 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:30.665078 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 104437us
I20260709 13:14:30.665906 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:30.666801   623 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:30.667260 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:30.667448   623 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:30.668273   514 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.668673 32691 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:30.669319 32691 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.668691   623 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:30.669777 32691 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.593672   744 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 13:14:30.674362 30707 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:30.659816 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.677140 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.679651 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.681321 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:30.692819 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 148212us
I20260709 13:14:30.683864 30552 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:30.660566   763 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:30.707116 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:30.710121 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:30.682122   542 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.684274   622 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:30.707747   623 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.684418   604 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:30.715273   623 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.690935   658 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.716418   763 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.697847   762 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:30.716805   658 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.717362   762 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.719033   658 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.715909   604 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.714426 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.719849 30558 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:30.690465   727 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:30.729612 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.730363 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.736387 31367 raft_consensus.cc:2410] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:30.733130 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:30.716444   542 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.741089 30553 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:30.724599   623 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:30.742820 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:30.722008   749 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:14:30.744029 31391 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:30.745847 31388 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:30.736213 31389 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:30.745723 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:30.719566   658 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.748190   623 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:30.748835   623 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:30.733582   472 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:14:30.749598 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:30.749853   623 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.712937   576 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:30.752377 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.753636   576 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:30.742842   624 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.546s	user 0.011s	sys 0.000s
I20260709 13:14:30.754695   624 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:30.755227 31378 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:30.710633   754 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:14:30.757174   624 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:30.757187 31396 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:30.757692   749 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:30.758184   749 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:30.759374 31396 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:30.759117   749 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.760486   749 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.761446   727 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.763875   727 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:30.735073   622 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:30.765631   622 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:30.748536   658 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.742328   542 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.796110   623 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:30.802507 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81150.9us
I20260709 13:14:30.798700 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.809072 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 979) took 1087 ms. Trace:
I20260709 13:14:30.804955 30631 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:30.809888 30631 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:30.809626 30624 rpcz_store.cc:276] 0709 13:14:29.721292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.721479 (+   187us) service_pool.cc:224] Handling call
0709 13:14:30.798463 (+1076984us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:14:30.808999 (+ 10536us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5266816}
I20260709 13:14:30.804930 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:30.812400 32691 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.814841 32691 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:30.808147 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:30.796505   774 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.822893   774 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.824272 31378 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:30.824436   513 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.829556   576 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.830340   576 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.830863   626 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:30.832254   626 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:30.833217   576 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:30.833833   576 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.834493   639 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:30.834997   639 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:30.834326   576 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.835949   639 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.837693   639 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:30.725652   604 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:30.842159 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 119549us
I20260709 13:14:30.810937 30631 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.854157 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.855170 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.856748 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.857688 30698 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:30.857841 30707 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:30.862440 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.866474 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.868393 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.835103   429 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:30.874178 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.875213 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.880447 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.880733 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 955) took 1032 ms. Trace:
I20260709 13:14:30.875562 30631 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:30.872289 30555 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:30.878763 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:30.893911 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:30.894544 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:30.872267 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:30.796985   623 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:30.846943   774 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:30.899099 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.899973 30631 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:30.912125 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 994) took 1052 ms. Trace:
I20260709 13:14:30.895169 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 984) took 1086 ms. Trace:
I20260709 13:14:30.881635 30550 rpcz_store.cc:276] 0709 13:14:29.848756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.848914 (+   158us) service_pool.cc:224] Handling call
0709 13:14:29.849306 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:29.914739 (+ 65433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.914779 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:29.915374 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:14:30.250908 (+335534us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:30.250972 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.880305 (+629333us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:30.880363 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:30.880424 (+    61us) raft_consensus.cc:1685] finished
0709 13:14:30.880440 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.880670 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15031,"raft.queue_time_us":645811,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":123,"spinlock_wait_cycles":114613504,"thread_start_us":272207,"threads_started":1,"wal-append.queue_time_us":411905}
I20260709 13:14:30.912705 30629 rpcz_store.cc:276] 0709 13:14:29.809161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.809355 (+   194us) service_pool.cc:224] Handling call
0709 13:14:29.828933 (+ 19578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:29.916214 (+ 87281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:29.916266 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:29.916355 (+    89us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:29.916419 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.894935 (+978516us) spinlock_profiling.cc:243] Waited 87 ms on lock 0x7b54005f41d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:30.894986 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:30.895108 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200126336}
I20260709 13:14:30.863437   501 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:30.895149 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.906334 30624 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:30.906499 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.907588 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:30.908548 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.928148 30558 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:30.858814   576 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:30.900439 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.915297 30631 rpcz_store.cc:276] 0709 13:14:29.860001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.300248 (+440247us) service_pool.cc:224] Handling call
0709 13:14:30.300751 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.804841 (+504090us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540064daa0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:14:30.911817 (+106976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:30.912060 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495986560}
I20260709 13:14:30.893033 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:30.916302 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:30.920797 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.922542 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:30.929958 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:30.872174   744 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:30.877009   429 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:30.894433   768 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:30.923312   501 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:30.932492 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.979712 31361 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:30.977342 30552 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:30.978276 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:30.987174 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54830.8us
I20260709 13:14:30.976613 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:30.863508   742 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:14:30.930431   481 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:14:30.974898   762 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:30.983496 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76045.5us
I20260709 13:14:30.987246 30631 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:30.984025   744 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.051052   473 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
W20260709 13:14:31.056341   776 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:14:31.052681 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:30.981047 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 88157.8us
I20260709 13:14:31.060478 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:30.988983 30630 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:30.922003   671 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:30.878803   745 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:14:31.053928 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.064769 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:30.903367   734 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:31.042837 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.042836 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:31.043813 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.044121 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:31.066602 30558 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:31.045050 30552 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:30.992707 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:30.993685 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:30.990983   723 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:31.047384 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.065295 30631 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:31.066366 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:31.069228 30698 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:31.074227 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:30.897873   736 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 13:14:31.077246 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.097071 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.098260 30631 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:31.099762 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.001880   501 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:31.104785 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:31.109409 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 196224us
I20260709 13:14:31.103375 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:31.062997   671 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.102581 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.077279   754 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:31.101727   771 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:31.117600 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 959) took 1264 ms. Trace:
I20260709 13:14:31.117336   429 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.118436   604 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.118968 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:31.075526   723 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.122205   658 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.126224 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.132261 30558 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:31.133759 30559 rpcz_store.cc:276] 0709 13:14:29.853564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.853721 (+   157us) service_pool.cc:224] Handling call
0709 13:14:29.869546 (+ 15825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:30.454098 (+584552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.117285 (+663187us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b5400187254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:31.117530 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1339342080}
I20260709 13:14:31.053212   744 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:31.136348 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.139467 30630 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:31.101648   576 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:31.073343   738 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:31.091760   514 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:31.150087 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.132350   671 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:31.149718 30624 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10. Re-sending same reply.
W20260709 13:14:31.157140 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.063025   775 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:31.157563 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.146004   774 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:31.191792   756 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:14:31.143332 30629 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:31.191581 31378 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:31.229910 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.192049 31367 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:31.101625   773 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:31.190469 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:31.193421 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:31.113508   734 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:31.153088 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:31.144564   726 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:31.146683   754 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:31.095273   742 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.157254   723 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:31.158288   733 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:31.206547 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49092.4us
I20260709 13:14:31.207266   744 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:31.171132 31391 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:31.168895   771 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.044534   768 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.231021 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:31.178633   576 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:31.235926 31394 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:31.237135 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:31.234329 31367 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:31.241655   745 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.244904 31389 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:31.201293   737 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:31.256423 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.204041   769 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:31.242578   594 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:31.242854   429 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:31.245625   658 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:31.302728 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.270805   786 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:31.311973 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49898.3us
I20260709 13:14:31.259997   754 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:31.272387   771 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:31.234759   774 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.295879   576 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:31.316198 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:31.319636 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.271461   742 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.340062 31394 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:31.314989   769 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:31.347741 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 211472us
I20260709 13:14:31.323220   502 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 13:14:31.339234   785 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.221s	user 0.000s	sys 0.002s
I20260709 13:14:31.382696 30631 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:31.309099   745 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.347304   622 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.340701   668 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:31.318974   658 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.315476   594 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.338722   623 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.327915   755 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
W20260709 13:14:31.420821 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69736.3us
I20260709 13:14:31.420703   736 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:31.463773   513 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.002s
W20260709 13:14:31.444061   779 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:14:31.324059   786 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.351239   629 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.125s
W20260709 13:14:31.506243 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 190124us
I20260709 13:14:31.464404   527 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:14:31.508219 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 183376us
I20260709 13:14:31.505661 30632 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:31.528581 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:31.484694   623 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:31.546916 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:31.350312   734 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:31.577847 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65642.7us
I20260709 13:14:31.471688   622 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.556663 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:31.498517   679 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:14:31.475409   668 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.588572 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.531949   733 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.557734 32691 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:31.597918 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:31.507138   736 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.571553   502 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:31.544893   599 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:14:31.608690 30626 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:31.620157 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.620707 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:31.603142   527 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:31.605679   784 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
W20260709 13:14:31.608807   626 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.005s
I20260709 13:14:31.604722   733 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.646956   527 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:31.648407   736 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:31.648303   527 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.602576   780 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:31.650015   502 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:31.649834   780 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.625008 31388 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:31.663715 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:31.664202 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:31.613062 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:31.631847   745 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.677001 30707 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
W20260709 13:14:31.633279   792 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:14:31.672019   737 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:31.679797   769 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.680938   594 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.632129   668 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:31.660771   788 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.351s	user 0.000s	sys 0.002s
I20260709 13:14:31.631991   513 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:31.639087   774 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.619438   795 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:31.633944   742 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:31.675323   726 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.690584   737 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:31.687798   790 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:31.691597   768 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.706037   737 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:31.700659   795 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:31.699028   576 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.715766   795 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:31.709503   790 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.721045   790 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.722887   658 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.723135   786 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:31.722556   749 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.962s	user 0.014s	sys 0.000s
I20260709 13:14:31.727748   780 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:31.728878   754 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:31.729542   723 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:31.731431   754 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.732019   771 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:31.734266   737 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.739810   737 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.739964   723 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:31.743114   789 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.170s	user 0.000s	sys 0.003s
I20260709 13:14:31.743808   795 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:31.746866   795 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:31.749960   794 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.152s	user 0.002s	sys 0.000s
W20260709 13:14:31.787768 30633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.003s	sys 0.000s
W20260709 13:14:31.789031 30633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.003s	sys 0.000s
W20260709 13:14:31.670065 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57146.1us
I20260709 13:14:31.794453 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:31.795938 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:31.799463 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.800935 30628 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:31.803025 30628 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:31.806842 30628 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:14:31.807248 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 472307us
I20260709 13:14:31.833380 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:31.835958 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.013s
W20260709 13:14:31.836619 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.002s	sys 0.013s
W20260709 13:14:31.838811 31383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.016s	sys 0.018s
W20260709 13:14:31.839563 31383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.017s	sys 0.018s
I20260709 13:14:31.841743 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:31.843119 31388 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:31.700345   513 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.701166   502 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:31.855495   786 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:31.856902   786 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:31.861440   639 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.023s	user 0.007s	sys 0.010s
I20260709 13:14:31.863065 30627 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:31.863509   513 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:31.864085 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 627025us
I20260709 13:14:31.865777   786 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:31.868369 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:31.869526 31394 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:31.869575 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.872069 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:31.850725   501 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.695s	user 0.005s	sys 0.011s
I20260709 13:14:31.873394   501 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:31.877408 30554 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:31.877607   795 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:31.877296 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.879695   501 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:31.881162   576 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:31.882151 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.873387 30628 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:31.885787 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.883445 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:31.888902   671 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.680s	user 0.010s	sys 0.006s
I20260709 13:14:31.892158   639 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:31.887089 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.894121   795 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:31.899952   795 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.899952 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:31.901793 31391 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:31.905900 30632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 401) took 2088 ms (client timeout 1687 ms). Trace:
I20260709 13:14:31.908013 30626 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:31.908696   671 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:31.910724 30632 rpcz_store.cc:269] 0709 13:14:29.817633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.817792 (+   159us) service_pool.cc:224] Handling call
0709 13:14:31.905288 (+2087496us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54004aa6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:31.905564 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3835055488}
I20260709 13:14:31.915431 30555 raft_consensus.cc:2368] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:31.916798 31394 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:31.918339 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 405) took 1041 ms. Trace:
I20260709 13:14:31.919097   723 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:31.918960 30555 rpcz_store.cc:276] 0709 13:14:30.876490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.883479 (+  6989us) service_pool.cc:224] Handling call
0709 13:14:31.918045 (+1034566us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b54000f05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:31.918218 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2263697792}
I20260709 13:14:31.923151 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:31.924386   795 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:31.924578   639 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:31.924854   795 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:31.925379   795 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:31.925494   639 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:31.896024 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.928437   750 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:31.929188   639 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:31.931931 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1000) took 1033 ms. Trace:
I20260709 13:14:31.931151 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:31.934909 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1003) took 1033 ms. Trace:
I20260709 13:14:31.936894 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1073) took 1042 ms. Trace:
I20260709 13:14:31.941316 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.944175 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:31.944798 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:31.947216 30633 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:31.948240   599 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:31.949401 30705 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:31.952158   671 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:31.953137 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:31.954371 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 121571us
I20260709 13:14:31.955024 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:31.956429   438 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:31.960209   771 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:31.984648 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:31.983302 30625 rpcz_store.cc:276] 0709 13:14:30.897976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.906498 (+  8522us) service_pool.cc:224] Handling call
0709 13:14:30.906803 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.063799 (+156996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.063835 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:31.063878 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:31.063917 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.931283 (+867366us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54006f0254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:31.931362 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:31.931677 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360517376}
I20260709 13:14:31.973171 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.994346 30626 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:31.983992 30707 rpcz_store.cc:276] 0709 13:14:30.894039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.894260 (+   221us) service_pool.cc:224] Handling call
0709 13:14:31.936397 (+1042137us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540032a354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:31.936616 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716081536}
I20260709 13:14:31.997452 30628 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:31.994946 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.002645   737 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:32.004664 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 401) took 1134 ms. Trace:
I20260709 13:14:32.005247 30554 rpcz_store.cc:276] 0709 13:14:30.870125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.874874 (+  4749us) service_pool.cc:224] Handling call
0709 13:14:32.004352 (+1129478us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:32.004572 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290928512}
I20260709 13:14:32.005952 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:32.006526 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:32.007553   585 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:32.008088   585 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:32.008301 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.983637 30630 rpcz_store.cc:276] 0709 13:14:30.901205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.091696 (+190491us) service_pool.cc:224] Handling call
0709 13:14:31.101386 (+  9690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.160382 (+ 58996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:31.934177 (+773795us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b54005f3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:31.934653 (+   476us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86664576}
I20260709 13:14:32.009847 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:32.011198 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58321.3us
I20260709 13:14:32.011466 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:32.012629 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:32.016105   750 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:32.016472 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:32.018829 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:32.019595   734 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:32.019966 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:32.020598   734 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:32.020991 31394 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:32.022310 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:31.990182   438 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:32.008622   585 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:32.048141 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69903.4us
I20260709 13:14:32.014361 30629 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:32.024819 30624 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:32.025146   678 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:32.023100   767 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:14:32.027199 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 80520.5us
W20260709 13:14:32.024616 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93721.7us
W20260709 13:14:32.039824 31361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1017) took 1309 ms (client timeout 1743 ms). Trace:
I20260709 13:14:32.031504 30632 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:32.026511   438 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:32.008392 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.064087   767 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:32.064818 30625 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:32.065683   767 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.068176 30624 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:32.055382 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:32.073343   481 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:14:32.056085 31361 rpcz_store.cc:269] 0709 13:14:30.730379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.732820 (+  2441us) service_pool.cc:224] Handling call
0709 13:14:32.039255 (+1306435us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400372dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:32.039548 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360017792}
I20260709 13:14:32.068810   724 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:32.082505 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.081322   599 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.104997 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:32.102685 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.110534 30706 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
W20260709 13:14:32.119709 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59766.8us
I20260709 13:14:32.121346   438 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:32.124869 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.126091 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.139163   513 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:32.139714   513 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:32.141135   513 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:32.143674   733 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.148473 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.138937 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:32.157855 31391 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:32.160822 30628 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:32.108810   671 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:32.165655   429 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:32.166715   429 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.169310   429 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:32.022535   734 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.171020   768 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:32.171339 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.171736   768 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:32.177276 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47652.6us
I20260709 13:14:32.180491   768 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:32.186301   599 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.189497   599 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.180567 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.206660 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.205674 31396 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:32.191533   513 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:32.215385 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.216562 31364 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:32.217970 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.216794 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.221299 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.232174 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:32.261615 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.221145   769 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:32.265733 30698 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:32.264664 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.267272 30706 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:32.272162   803 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:32.273448 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82431.5us
I20260709 13:14:32.274055   622 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:32.277081 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 268951us
I20260709 13:14:32.276916   679 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:32.279500 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.299522 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:32.300163 31364 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:32.279605 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 68424.1us
I20260709 13:14:32.283981 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 998) took 1386 ms. Trace:
W20260709 13:14:32.286396 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 183563us
I20260709 13:14:32.287174 30554 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:32.292145   769 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.306742   551 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:32.307361 31396 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:14:32.306380 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54181.4us
I20260709 13:14:32.301865 30627 rpcz_store.cc:276] 0709 13:14:30.897440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:30.899769 (+  2329us) service_pool.cc:224] Handling call
0709 13:14:30.929914 (+ 30145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.863547 (+933633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.283543 (+419996us) spinlock_profiling.cc:243] Waited 932 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:32.283874 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2144149248}
I20260709 13:14:32.312547 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.327627 31394 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:32.318760 31364 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:14:32.333820 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 137460us
I20260709 13:14:32.299542   679 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:32.344331   591 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:32.345337   726 raft_consensus.cc:2764] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:14:32.345892   671 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:32.349041 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.363708 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.359005   585 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:32.366998 30559 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:32.366818   786 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:32.368894 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.329195   734 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.324327   769 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:32.381711   786 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:32.385888 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:32.387497 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58001.3us
I20260709 13:14:32.388085   594 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.388904   726 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:32.389532   815 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:14:32.392970   734 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.396552 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 663) took 1298 ms. Trace:
I20260709 13:14:32.398546 30381 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:32.400329 30554 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:32.410513 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:32.350432   679 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:32.437536 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57466.1us
I20260709 13:14:32.414871 30633 rpcz_store.cc:276] 0709 13:14:31.098506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.796765 (+698259us) service_pool.cc:224] Handling call
0709 13:14:31.910459 (+113694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:31.984466 (+ 74007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.396282 (+411816us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54004aa6d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:32.396471 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259261184}
I20260709 13:14:32.445353 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.451020 31394 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:32.449044 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.405943   671 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:32.416215   725 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:32.473408 31394 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:32.454865 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 847) took 1325 ms. Trace:
I20260709 13:14:32.472995   631 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:32.487804   631 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.488252   631 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:32.494019 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:32.497467   604 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:32.498234   604 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.499415   671 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:32.503103 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43748.4us
I20260709 13:14:32.508112 30554 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:32.459766   771 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.547365 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.564234   770 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.597402 31361 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:32.599140   781 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.601310   604 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.546386   724 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:14:32.608153   724 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:32.608692   724 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:32.590090 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.572163 31364 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:32.598673   770 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:32.622105 30554 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:32.629456 30555 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:32.475828   594 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.643150 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:32.644940   604 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.647531   770 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:32.627259 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:32.587694 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.637192 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:32.660993   604 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.610318   724 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:32.664026   724 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:32.664110   604 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:32.665601 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:32.666920   585 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:32.667721 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.669155 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.623500   591 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:14:32.670035 30553 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
W20260709 13:14:32.628063   795 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.702s	user 0.016s	sys 0.007s
I20260709 13:14:32.671147 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:32.672668 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.482161 31383 rpcz_store.cc:276] 0709 13:14:31.124286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:31.126832 (+  2546us) service_pool.cc:224] Handling call
0709 13:14:31.127233 (+   401us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:31.259981 (+132748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:31.260019 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:31.275452 (+ 15433us) log.cc:844] Serialized 52 byte log entry
0709 13:14:31.840599 (+565147us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:31.840734 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.406351 (+565617us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400522954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:32.406404 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:32.406451 (+    47us) raft_consensus.cc:1685] finished
0709 13:14:32.406465 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:32.449685 (+ 43220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10794,"raft.queue_time_us":41270,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":11231,"spinlock_wait_cycles":279226624,"thread_start_us":511818,"threads_started":1,"wal-append.queue_time_us":403094}
I20260709 13:14:32.671784 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.666960 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.510483   725 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.678217   481 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:32.681471   679 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.672560   604 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.671195 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:32.692227 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128390us
W20260709 13:14:32.691022 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53984.9us
I20260709 13:14:32.558015   771 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.689402 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:32.705077 30633 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:32.700569 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51282.3us
I20260709 13:14:32.699658 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:32.678407   622 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:32.688378   576 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:32.712687 30550 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:32.685104   726 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:32.714641 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 239488us
I20260709 13:14:32.722833   604 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.730670   725 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.699291   585 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:32.731822   622 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.734725   502 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:32.736372 31394 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:14:32.736935 31394 pending_rounds.cc:85] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 3
I20260709 13:14:32.737248   594 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:32.742681 31361 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:32.684692   795 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.723130 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:32.709383   591 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:14:32.752736 30629 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:32.737264 31394 pending_rounds.cc:107] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:32.765453 31394 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:32.775756 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:32.799000 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.753741   604 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:32.804764 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89834.7us
I20260709 13:14:32.756682   481 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.812026   481 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.815655   481 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.818748 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.820581 30627 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:32.773098   591 consensus_queue.cc:1243] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:14:32.837452 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:32.839602 30632 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:32.841373   767 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.774834   737 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:32.843016   737 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:32.843580   737 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:32.779131 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:32.848521 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:32.756232   585 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:32.778255 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:32.850582 30554 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:32.856076 30628 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:32.856287 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:32.774013   631 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.860913   631 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:32.842213   767 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.871766 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:32.879299   585 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:32.880560 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:32.882033 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 166680us
I20260709 13:14:32.883212 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:32.882990 30381 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:32.883950   514 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:32.884236   481 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:32.884878   481 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:32.887269 30381 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:32.868714   750 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:32.898550 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90494us
I20260709 13:14:32.906358 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:32.888918   429 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:32.918280   770 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:32.890798   781 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:32.893568   734 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:32.907823   750 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:32.912235   514 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:14:33.002522   810 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.105s	sys 0.177s
W20260709 13:14:33.003242   810 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.106s	sys 0.177s
W20260709 13:14:33.003353   825 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 13:14:33.004513   824 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
W20260709 13:14:33.007014 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 238042us
I20260709 13:14:33.007838   622 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:33.009732   734 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:32.932749   604 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:32.998347   750 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:33.003269   829 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
W20260709 13:14:33.042570 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 357923us
I20260709 13:14:33.020161   514 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:33.062070 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 179068us
W20260709 13:14:33.072846 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 373317us
W20260709 13:14:33.068181 31391 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.001s	sys 0.012s
W20260709 13:14:33.074327 31391 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.001s	sys 0.012s
I20260709 13:14:33.056109   514 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.061710   481 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:33.091835 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.051445   604 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.073570   767 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.086838   795 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:33.107467 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:33.112140 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:33.112792 30550 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.093559   771 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:33.089560   678 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:14:33.138052 31378 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.003s	sys 0.012s
W20260709 13:14:33.140113 31378 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.004s	sys 0.012s
I20260709 13:14:33.142477 30626 pending_rounds.cc:85] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:14:33.158403 30626 pending_rounds.cc:107] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:14:33.158825 30626 raft_consensus.cc:2889] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.123243   591 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.178771 30633 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:33.182216 31394 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:14:33.183073 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 510513us
I20260709 13:14:33.110252   770 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:14:33.135136   725 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:33.111661   723 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:14:33.187489 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67818.9us
W20260709 13:14:33.178303   671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.031s	sys 0.045s
I20260709 13:14:33.171303   678 consensus_queue.cc:1243] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:33.191589 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1020) took 1015 ms. Trace:
I20260709 13:14:33.144824   734 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 13:14:33.231956 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:33.244699 31364 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.233774 31361 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:33.246855 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:33.240078 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:33.233894   770 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:33.251641   770 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:33.252874 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:33.252759   770 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.254192   770 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.255275 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:33.256984 31388 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:33.257086 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:33.268043   604 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:33.182400   591 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.260921   551 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:33.251699 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.239102 30630 rpcz_store.cc:276] 0709 13:14:32.175861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.181906 (+  6045us) service_pool.cc:224] Handling call
0709 13:14:32.207619 (+ 25713us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:32.280986 (+ 73367us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:32.281402 (+   416us) log.cc:844] Serialized 14 byte log entry
0709 13:14:32.502606 (+221204us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:32.517356 (+ 14750us) log.cc:844] Serialized 26 byte log entry
0709 13:14:32.517569 (+   213us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:32.517613 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:32.517685 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:33.191251 (+673566us) raft_consensus.cc:1685] finished
0709 13:14:33.191284 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.191491 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49883,"prepare.queue_time_us":901,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":50226,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":5412,"thread_start_us":219567,"threads_started":1,"wal-append.queue_time_us":429731}
W20260709 13:14:33.239722   671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.031s	sys 0.046s
I20260709 13:14:33.280023 31364 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:33.279268   591 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.284154   576 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:33.279066 30552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.021s	sys 0.056s
W20260709 13:14:33.287478 30552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.022s	sys 0.057s
I20260709 13:14:33.286854   429 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:33.312184 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 882) took 1033 ms. Trace:
W20260709 13:14:33.310658 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 87697.6us
I20260709 13:14:33.238970   725 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.312817 31391 rpcz_store.cc:276] 0709 13:14:32.279232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.312158 (+ 32926us) service_pool.cc:224] Handling call
0709 13:14:32.335227 (+ 23069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.435942 (+100715us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:33.074841 (+638899us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.299698 (+224857us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.299760 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:33.311820 (+ 12060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.311930 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.312097 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":613702,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2727,"raft.queue_time_us":17412,"spinlock_wait_cycles":188928,"thread_start_us":741072,"threads_started":2}
W20260709 13:14:33.316084 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 219251us
I20260709 13:14:33.316752 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:33.285929   622 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:14:33.329283 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78030.7us
I20260709 13:14:33.316603   429 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:33.341387   551 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.341591   679 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:33.347601 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:33.348481 30633 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:33.355993 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:33.357949   794 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.274s	user 0.002s	sys 0.000s
W20260709 13:14:33.358167   737 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.006s
I20260709 13:14:33.360433   786 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:33.336336   429 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:33.342139   551 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.345150   604 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:33.378856 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:33.338907   590 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.197s
I20260709 13:14:33.358601   737 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.387073   810 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:14:33.428704 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72884.6us
I20260709 13:14:33.429638 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:33.409703   639 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:33.431597   835 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:14:33.439049   551 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:33.424690   843 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:14:33.470511   847 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:33.475041 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:33.475909 30632 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:33.466509   514 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.481724   514 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:33.434768   841 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 13:14:33.425024   599 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:14:33.430459   839 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:14:33.435437   726 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:33.491048   481 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:33.491487   726 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:33.491612   481 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:33.487586   849 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:14:33.490738 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:33.492782   481 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.493974   481 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:33.460104   639 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.500064   639 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:33.500926   771 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.540716 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:33.415099   844 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.223s
I20260709 13:14:33.552404 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:33.501013   863 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:33.558061 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 88398.8us
I20260709 13:14:33.560552 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.501634   771 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:33.560801 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 167926us
I20260709 13:14:33.562068 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 991) took 1192 ms. Trace:
W20260709 13:14:33.568702 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.430529   856 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.171s
I20260709 13:14:33.568361 30698 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
W20260709 13:14:33.572650 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 294170us
I20260709 13:14:33.577133 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:33.581256   725 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:33.492791   726 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.583047   726 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:33.584124   438 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:33.584650   438 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:33.585685   438 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.572167   591 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
W20260709 13:14:33.586740 31294 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.550s (SENT)
I20260709 13:14:33.587462 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:33.587889   855 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:33.588388   839 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:33.588622   843 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:33.590809   724 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:33.591328   724 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:33.592463   858 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.155s	user 0.000s	sys 0.002s
I20260709 13:14:33.592341   724 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:33.592576   852 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
W20260709 13:14:33.588703 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 85284.1us
I20260709 13:14:33.604192 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:33.605057 31391 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:33.607002   724 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:33.595225   514 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:33.568956 31378 rpcz_store.cc:276] 0709 13:14:32.372383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:32.384608 (+ 12225us) service_pool.cc:224] Handling call
0709 13:14:32.384954 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:32.477073 (+ 92119us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:33.167533 (+690460us) log.cc:844] Serialized 14 byte log entry
0709 13:14:33.321957 (+154424us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:33.322035 (+    78us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:33.327074 (+  5039us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:33.561794 (+234720us) spinlock_profiling.cc:243] Waited 64.9 ms on lock 0x7b54003377d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:33.561832 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:33.562010 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18278,"prepare.queue_time_us":599967,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":9664,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":15271,"spinlock_wait_cycles":149321472,"thread_start_us":711435,"threads_started":2,"wal-append.queue_time_us":225786}
I20260709 13:14:33.609287   737 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:33.615067   438 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:33.616190 31389 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:33.620559   771 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:33.621073   750 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.563s	user 0.002s	sys 0.010s
I20260709 13:14:33.621606   750 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.622098   781 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:33.623517 31294 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.614s (SENT)
W20260709 13:14:33.625025 31294 leader_election.cc:336] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.625s (SENT)
I20260709 13:14:33.625486 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:33.631139 31391 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:33.572399   576 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:33.602070 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.630406 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:33.636147 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62691.2us
I20260709 13:14:33.640344   861 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.641109   861 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.588481   855 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:33.639112 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:33.658980 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61835.4us
I20260709 13:14:33.659677   870 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:33.660224   870 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.660329 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.660665   870 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:33.661039 31294 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.662s (SENT)
I20260709 13:14:33.661592 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:33.590904   429 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:33.662737   855 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:33.663625   527 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:33.664093   527 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:33.664458   527 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:33.664601   429 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.666872   576 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:33.671290   671 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.671766   769 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.672022   671 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.672461 30558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:33.672978 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.673314 30558 raft_consensus.cc:606] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Received request to transfer leadership to TS 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:33.675674 30552 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:33.671875   594 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.645972   604 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:33.677924   604 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:33.678395   604 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.638270   514 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:33.680389   514 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.680894   856 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.681704   856 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.637903   737 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:14:33.632486   839 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:33.689922 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51448.3us
I20260709 13:14:33.658856   795 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:33.691880   795 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:33.693238   795 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:33.695604   631 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:33.696112   631 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:33.697031   631 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:33.720734   631 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:33.723693   861 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:33.718194   750 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:33.725777   848 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.287s	user 0.003s	sys 0.000s
W20260709 13:14:33.726850 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 123455us
I20260709 13:14:33.725869 30381 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:33.730583 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:33.734033 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 351260us
I20260709 13:14:33.734730 30557 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:33.735538 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:33.732365 30381 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 6 to 9, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:33.738054 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 68556.7us
I20260709 13:14:33.738235 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:33.741225 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 80210.2us
W20260709 13:14:33.742328 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.667s (SENT) [suppressed 33 similar messages]
I20260709 13:14:33.749949 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.756443 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:33.755276 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.762064 30556 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:33.761926 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:33.750458 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.764165 31391 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:33.757287 30630 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:33.677304   594 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:33.766407 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.770915 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.746091   803 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:33.771886 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.777434 30559 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:33.781703 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.794833 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:33.672516   769 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:33.762423   725 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.793699 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.793694 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.784018   864 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.830206 30556 raft_consensus.cc:2376] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7. Re-sending same reply.
I20260709 13:14:33.830912   864 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:33.775730   781 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:33.777163   861 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:33.858552 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.844632 31391 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:33.841209 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49605.6us
I20260709 13:14:33.836675 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.843698 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.861863 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:33.871449 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.872336 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.867746   726 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:33.776070   639 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 13:14:33.839059   678 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:33.832214   856 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.879776 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.883265 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:33.845279   781 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:33.879801 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:33.887425 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90214.3us
I20260709 13:14:33.859757   861 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:33.894542 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:33.901312 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.880779   786 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:33.906018 30552 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:33.905820 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:33.907761   842 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:33.908790   769 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.892700   781 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:33.857460   725 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:33.927224 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:33.931361 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:33.897899   861 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.898904   839 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:14:33.936539 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:33.936677 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:33.938309   872 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:14:33.955967 30698 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:33.939893 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.941085 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:33.896327   726 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:33.942905   576 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:33.943944 30553 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:33.944293 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.944689 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.945701   594 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.947069 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.970973 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:33.971699 30704 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:33.971881 30552 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:33.952761 31294 leader_election.cc:336] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.990s (SENT)
I20260709 13:14:33.968693 30554 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:33.905719   786 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:33.956136 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.938978 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:33.978732 30559 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:33.928396   514 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:33.983731 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:33.976083 30556 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:33.892343   678 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:33.988727 31367 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:33.989240 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:33.989820 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.977299   840 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:33.990242 30700 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:33.996656 30700 pending_rounds.cc:85] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 3
I20260709 13:14:33.997020 30700 pending_rounds.cc:107] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:33.997346 30700 raft_consensus.cc:2889] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:33.990770 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.983898   786 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:33.990780 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:33.972896   576 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:34.010308 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.012761 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:34.009625 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 129943us
I20260709 13:14:34.017738 30698 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:33.994956   840 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:34.011266   576 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:34.006052   864 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:34.021198   840 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.034921 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.045707 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:34.039796 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.049989 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:34.041414 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88722.7us
I20260709 13:14:34.055119 30704 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:34.057183 30705 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:14:34.041088   874 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:14:34.058791 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.041589   527 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:34.068120   873 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:14:34.068524   882 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:34.070182   867 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:34.038477   604 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.039258   726 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.027071   678 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:34.080022 30699 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:34.083631 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.084030 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.087957   725 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:34.092087 31294 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 2.120s (SENT)
I20260709 13:14:34.100212 30706 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:34.100353   844 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.101765 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.106014 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.105100 30557 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:34.112425 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.108804 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.118052 31361 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:34.120764 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.122010   604 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.122190 30553 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:34.117462   844 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.055222   836 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:14:34.129673 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:34.135805 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 121023us
I20260709 13:14:34.120890 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.134747 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.140489 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.127336 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.148566 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:34.126495   877 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:14:34.154093 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.152870 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.153185 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.105163   839 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:14:34.163621   839 consensus_queue.cc:1243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:14:34.165840   594 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.166780   594 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.169100   594 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.171025 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.172646 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.174099 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.130019   781 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:34.161404   887 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:14:34.161690   888 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:14:34.163097   889 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:14:34.175815   861 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.129166   844 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:34.177126   781 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.177450 30553 raft_consensus.cc:2376] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7. Re-sending same reply.
I20260709 13:14:34.178887 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.159857 30558 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:34.158946 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:34.165892 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.175851 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.178298   861 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.130645   893 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:34.189152 30552 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:34.150691   734 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.184218 31383 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:34.210871 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.213032 30552 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:34.134075   855 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.216939 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.217586 31364 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:34.188591   844 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.220789   867 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.205914   594 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:34.233287 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43676.4us
I20260709 13:14:34.215026   734 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.224776   855 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:34.226825   836 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.215991   795 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:34.237731 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.234419 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.232754 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:34.182976   840 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.244704 30557 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:34.247144   527 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.256387   737 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.257757   671 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:34.276019 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 137675us
W20260709 13:14:34.261287 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:34.248903 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:34.241771   594 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:34.265664   870 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.604s	user 0.020s	sys 0.009s
I20260709 13:14:34.267725 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.236807   893 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:34.251333 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.282132 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:34.285810 30554 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.259697   786 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.280745 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.299678 30625 raft_consensus.cc:2410] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:34.303072 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.303696 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.304670 30633 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:34.307073 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.308454 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.309978 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.311923 30629 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:14:34.314265 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81619.8us
W20260709 13:14:34.314466 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63262.6us
I20260709 13:14:34.317567 30632 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:14:34.314570 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47406.9us
I20260709 13:14:34.275754   861 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:34.281116   795 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:34.321957   795 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.324967 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.234905   844 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.332422 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.332857 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.333374 30628 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:34.329360 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.294390   725 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:34.345366 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.346508 30632 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.350567 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.351914 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.350662 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.353714 30627 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:34.351542 30625 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:34.314102   724 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:34.357573 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.359107 30552 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:34.354238 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.323329   527 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:34.323521   839 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:34.287030   737 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.362399 30624 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:34.237341   867 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.266827   836 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.327010   861 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.327703   893 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:34.337347   725 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.378707 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.343793   803 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:34.361407 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.298432   870 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.316704   576 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.381450 30626 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:34.396170 31294 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 2.227s (SENT)
I20260709 13:14:34.396381 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.394994 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:34.401677 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73550.2us
I20260709 13:14:34.366195   527 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:34.404186 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79290.1us
I20260709 13:14:34.411768   893 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.413194   771 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.413873   771 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.416879 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:34.392426   725 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:34.382102   770 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:34.381460   861 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.416960 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:34.381368   839 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:34.403405   855 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:34.408098   527 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:34.443325 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.427985   770 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:34.450876 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 68564.3us
W20260709 13:14:34.452586 31294 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.762s (SENT)
I20260709 13:14:34.432957   855 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:34.483737 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 147648us
I20260709 13:14:34.455708   770 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:34.443615   771 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.489388 30550 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:34.467595   855 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:34.466856   882 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:34.498250 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:34.499399 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56169.3us
I20260709 13:14:34.500675 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.495731 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:34.509394 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.512084 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:34.512980 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74699.9us
I20260709 13:14:34.497587   590 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:34.492637   840 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.494901   891 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:34.503930   862 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:34.529775 30629 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:34.526966 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:34.514781   893 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:34.513168   786 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.536175 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:34.524827   895 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 13:14:34.545651 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.546232 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.519111   671 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:34.494874   897 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.187s
I20260709 13:14:34.577296 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.581261 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.547472 30550 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:34.585373 30630 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:34.582041 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.523247   590 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:34.581743 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:34.492308   803 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.582834   594 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.614729 30703 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:34.615463   594 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:34.539950   604 raft_consensus.cc:2764] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:14:34.615993   594 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:34.584514 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.617043 30706 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:34.525094   839 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.617069   594 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.584211 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.581395   898 raft_consensus.cc:2764] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:14:34.595468 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:34.623322 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 88151.1us
W20260709 13:14:34.559679 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45800.1us
I20260709 13:14:34.521103   899 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.211s
I20260709 13:14:34.627110 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.587991 30630 pending_rounds.cc:85] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:14:34.629396 30630 pending_rounds.cc:107] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:34.630890 30630 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.540402   875 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:34.628553   896 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 13:14:34.640890   867 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.648977   910 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.583981   909 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:34.627480   870 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:34.656847   904 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:14:34.649761 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.643644 30626 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:34.652745 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.662992 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.663981 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.591010   737 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:34.681449 30633 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:34.655248   908 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.670395 30632 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:34.547191   882 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 13:14:34.627810   771 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.640691   671 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:34.684844 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.640010   576 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:34.691941 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 165722us
I20260709 13:14:34.716315 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:34.738458 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 199114us
I20260709 13:14:34.739454 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:34.648891   875 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.592689   891 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:14:34.643296   902 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:34.746862 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.748816 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:34.680068   906 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:14:34.751353 30629 pending_rounds.cc:85] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 2
I20260709 13:14:34.751744 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:34.756820 30629 pending_rounds.cc:107] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:14:34.757220 30629 raft_consensus.cc:2889] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:34.760134 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.764175 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.767382 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.768121 30700 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:34.770251 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.754933   897 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:34.768417 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:34.776388 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:34.781497 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 465895us
I20260709 13:14:34.788740 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:34.789553 30554 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:34.790642 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.791915 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.792217 30554 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:34.689272   910 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.794684 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.756991   771 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.778146   897 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:34.716079   590 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.742252   908 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.752352   902 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.766077   898 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:14:34.806191 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:34.807592 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.808236 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.799362   839 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:34.827086 30550 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:34.827817 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.803189   897 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:34.804733   914 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:34.806778   527 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.850299 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.856529 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:34.871268 30704 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:34.857236 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 895) took 1088 ms. Trace:
I20260709 13:14:34.814249   911 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:34.822763   590 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:34.879313 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71165.1us
I20260709 13:14:34.876132 31367 rpcz_store.cc:276] 0709 13:14:33.769030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.769230 (+   200us) service_pool.cc:224] Handling call
0709 13:14:33.859838 (+ 90608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:33.995319 (+135481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.856978 (+861659us) spinlock_profiling.cc:243] Waited 74.3 ms on lock 0x7b540031d8d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:34.857155 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170994432}
I20260709 13:14:34.829538   891 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:34.858911   916 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:14:34.861575   726 raft_consensus.cc:993] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:14:34.891245 31389 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:34.902313 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.893594   913 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:34.838907   922 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:34.893615   882 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:34.910874   910 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.877533   914 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.907598   726 raft_consensus.cc:1081] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Signalling peer 6b393e79ded4461ea5da7b6b6063305f to start an election
I20260709 13:14:34.913213 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.913987 30557 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:34.914114 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:34.905516   913 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:34.915318   913 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:34.915534 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.917625 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.918282 31389 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:34.899423 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.857486   770 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.840500   863 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:34.900893   855 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:34.901680   915 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:14:34.920962   770 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:34.919694 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.927526 30624 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:14:34.919610 30627 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:34.913606   914 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.865833   771 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.906708   725 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:34.933079   863 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:34.935038 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 84838.2us
I20260709 13:14:34.934132   863 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.936729   863 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:34.936936 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.937479 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.937987   898 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:34.939395   781 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:34.875874   839 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.942591   839 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:34.944119   844 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:34.868590   907 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.203s	user 0.002s	sys 0.000s
I20260709 13:14:34.799564   671 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:34.907588 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:34.947057 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:34.947959 30551 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:34.919250 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.855221   902 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:34.953140   855 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:34.955214 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 902) took 1171 ms. Trace:
I20260709 13:14:34.955835 31361 rpcz_store.cc:276] 0709 13:14:33.783585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.865663 (+ 82078us) service_pool.cc:224] Handling call
0709 13:14:33.880359 (+ 14696us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.136418 (+256059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:34.954898 (+818480us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:34.955124 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":541269248}
I20260709 13:14:34.959887 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.847226   908 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:34.973757   725 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:34.980127 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:34.983438 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:34.985504 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:34.986980 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:34.987129 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.988843   908 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:34.989259   847 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:34.989405 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:34.989545   908 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:34.989784 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:34.990984   723 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:34.991130 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:34.991353   898 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:34.991757 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:34.991392   723 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:34.992514 30624 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:34.992349 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:34.993178   723 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.993722   770 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:34.994218 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:34.995208 31383 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:34.989956   847 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:34.993050 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.003646 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.007524   844 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:35.008147   844 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.008323 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.009130 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:35.010001 30624 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:35.011758 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:34.923225   897 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:34.994377   723 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.014853   726 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:35.016181   726 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:35.016944   726 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:35.017300   726 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:35.008553   844 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:35.019769   750 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:35.020262 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:35.022583 30699 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:35.023419   725 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:35.023973   902 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.024230   725 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:35.024601   902 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.025688   725 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.029448 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.029673 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.030511 30699 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:35.030570 30704 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:35.015007   897 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.032943 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:35.040017   902 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.046540 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.040220 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1089) took 1094 ms. Trace:
I20260709 13:14:35.035625 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.048308 30702 rpcz_store.cc:276] 0709 13:14:33.945955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.946484 (+   529us) service_pool.cc:224] Handling call
0709 13:14:35.039926 (+1093442us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b540037d2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:35.040118 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045249024}
I20260709 13:14:35.093163 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:35.094583 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.093683 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.095012 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.093078 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.097527 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.078567 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.097635 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.094430 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.032578   855 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.007067   771 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.101991 30557 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:35.074126   882 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:35.018187   726 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.042433   898 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.020308   750 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.055476   839 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:35.114377   847 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:35.075344 30700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
 from {username='slave'} at 127.0.0.1:53938
I20260709 13:14:35.115363 30381 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:14:35.118458 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.117609 30559 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:35.100847 30698 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:35.137085 30700 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 13:14:35.120035 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 127763us
I20260709 13:14:35.115089 30624 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:35.137523 30700 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.122995 30557 raft_consensus.cc:606] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:35.138412   921 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:35.138280   723 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:35.138962 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.139037 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.141180   726 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:35.074873   576 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
W20260709 13:14:35.125088   861 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.688s	user 0.014s	sys 0.000s
I20260709 13:14:35.144678 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.124107   882 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:35.151085 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:35.151430 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.156281 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:35.136008   750 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.075095   922 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:14:35.130878   771 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.139132   921 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.162278   929 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.161322 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:35.166538 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.166700 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.167428 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.171219 30624 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:35.147514   723 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:35.212987 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.214044 31364 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:35.171176   847 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.217000 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.218324   862 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.218971   862 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:35.211330   839 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:35.221120   839 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:35.224006 30701 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:35.227284 30698 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:35.152504   897 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:35.157454   882 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:35.228070 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.145251   671 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 13:14:35.165638   429 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:35.276786   929 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.277379   929 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.279600   929 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.281828 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.285288 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.285928 30630 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:35.243908   930 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.228152   723 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:35.211915   771 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.242038   576 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:35.251310 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.263262   839 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.255682 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.270215 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.270807 30555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:35.318404 30555 raft_consensus.cc:606] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:35.318689 31391 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:35.318917 30555 raft_consensus.cc:614] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:35.271541 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.319895 31383 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:35.295747 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.320369 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.321322 30558 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:35.304129 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:35.154927   861 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.317643 30624 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:35.303810   931 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:35.339349 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88143.9us
I20260709 13:14:35.340732 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.341166 31364 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:35.305317   891 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:35.385279 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:35.388581 31388 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:35.333797   576 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.360041 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.355432 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:35.315840   723 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:35.391490 30629 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:35.405365 31396 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:35.321077   870 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:14:35.332427   905 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
W20260709 13:14:35.420580 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.425961 30629 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:35.426501 30629 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:35.428005 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 136035us
I20260709 13:14:35.316941   930 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.420413 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.442190   844 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.442981   914 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:35.444017 30559 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:14:35.444234 30554 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:35.418812   891 raft_consensus.cc:2764] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:14:35.445602 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 217624us
I20260709 13:14:35.453837 30626 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:35.427361 30629 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.461205 30629 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:35.455183 31389 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:35.461621 30554 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:35.463248 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.464107 31364 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:35.455271 30700 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.456769   844 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.443450   914 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:35.356146   931 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.467132   891 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:35.471941   891 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:35.474984   914 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.476568   914 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.479249   930 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.480046   930 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:35.487933   863 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.551s	user 0.005s	sys 0.009s
I20260709 13:14:35.493355 30557 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:35.496927 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.499573 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:35.504410 30700 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.505957 31364 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:35.509829   861 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:35.512543   734 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:35.508440 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88089.6us
I20260709 13:14:35.513891   750 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.514091 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:35.524696   921 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:35.537684 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.548002 30632 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:35.527976   863 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:35.489738   862 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:35.547281 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 126763us
W20260709 13:14:35.461850   929 consensus_queue.cc:1175] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 6 status { last_received { term: 6 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:35.562162   771 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:35.563741   836 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.571951 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.545292   939 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:35.550316   863 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.591992 31394 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:35.588302 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.589377 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:35.602028 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 48366.3us
I20260709 13:14:35.603157 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:35.608790 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.610845 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.599103   863 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:35.608613   913 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.006s
I20260709 13:14:35.615245 31396 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:35.617811   930 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:35.614035 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.628826   898 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:35.625276 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.632612 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.635249 31396 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:35.636008 31383 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:35.639621   862 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:35.641688 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.643564   750 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:35.645155 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1622) took 1306 ms. Trace:
I20260709 13:14:35.645599 30552 rpcz_store.cc:276] 0709 13:14:34.338745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.406812 (+ 68067us) service_pool.cc:224] Handling call
0709 13:14:34.866117 (+459305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:34.866399 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:34.965310 (+ 98911us) log.cc:844] Serialized 14 byte log entry
0709 13:14:35.260911 (+295601us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b180045a1e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:14:35.286497 (+ 25586us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:35.286551 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:35.286598 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.286709 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:35.644937 (+358228us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54001e8cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:35.645063 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25325,"prepare.queue_time_us":215438,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":25476,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":19223,"spinlock_wait_cycles":1067210752,"thread_start_us":264937,"threads_started":2,"wal-append.queue_time_us":246149}
I20260709 13:14:35.641567   931 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:35.614235   939 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:35.641491   576 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.664968 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.665302 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.666385 31391 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:35.666425 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:35.667255 31389 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:35.669446 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.672076 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.673171 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.655098   939 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.675873 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.677023 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.695575 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.680015   875 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.683013 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.700704 30554 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:35.687592 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:35.690016   936 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:35.676817 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.695838 30557 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:35.678345 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:35.701018 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.707700 30553 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:35.708258 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:35.633873   898 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:35.710433 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:35.703133   940 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.065s	user 0.004s	sys 0.000s
I20260709 13:14:35.713821 30631 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:14:35.675351   938 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.718343 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:35.721210 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44468.2us
I20260709 13:14:35.703284   726 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:35.761109 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.762970   862 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:35.763955   944 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:14:35.765727 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.767393 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 913) took 1562 ms. Trace:
I20260709 13:14:35.767851 31388 rpcz_store.cc:276] 0709 13:14:34.205463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:34.313682 (+108219us) service_pool.cc:224] Handling call
0709 13:14:34.337707 (+ 24025us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:35.396579 (+1058872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:35.767125 (+370546us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54004b05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:35.767325 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408820608}
I20260709 13:14:35.768643 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:35.769333 30631 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.631722   913 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:35.770514 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.772635   913 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:35.723656   671 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:35.774071 31361 raft_consensus.cc:2368] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7.
I20260709 13:14:35.705477   576 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:35.727402   678 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:35.749516 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:35.780066 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57765.4us
I20260709 13:14:35.728870 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:35.781572 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.730715   527 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:35.734840 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:35.780115 30701 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:35.747736 30381 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 7, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:35.734436   839 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.735407   723 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.789000 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:35.748994   679 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:35.790239 30703 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:35.790112   679 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:35.791157 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.791759 30556 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:35.792729   679 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.792963 30381 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:35.739583 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 398921us
I20260709 13:14:35.802774 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:35.740742   590 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:35.741231   724 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:35.741488 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.741663   891 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.742331   880 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:35.745720 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:35.710637   937 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:35.751400 30698 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:35.752398   870 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:35.753997   905 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:35.746582 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:35.755710 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 413542us
I20260709 13:14:35.757460 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:35.759043 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 155) took 12483 ms (client timeout 15000 ms). Trace:
W20260709 13:14:35.760480   882 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.004s
W20260709 13:14:35.782495 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53743.6us
I20260709 13:14:35.866521 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:35.715508   861 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.806180 30381 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:14:35.762506   935 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.294s	user 0.002s	sys 0.002s
W20260709 13:14:35.813359 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71955.9us
I20260709 13:14:35.812021 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:35.821026 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:35.822062 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:35.870252 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58313us
I20260709 13:14:35.870326 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:35.825244 30551 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:35.871400 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.872357 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.870357 30550 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.806852   590 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:35.804555   770 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:35.831955 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:35.845490 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 98996.9us
I20260709 13:14:35.811002   724 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.825544 30629 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:35.868183 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:35.813573   921 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.886188 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:35.799760   671 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:14:35.820582   839 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:35.903826   921 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:35.905000 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.832651   911 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:35.907050   911 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.907557   900 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:35.908192   900 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:35.909166   900 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.909693   841 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:35.910962   864 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:35.911352   864 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:35.903898   839 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:35.902097   875 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:35.906354 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:35.747668   898 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.915454   839 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.915673   898 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:35.916788   723 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:35.919026 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51001.5us
I20260709 13:14:35.919898 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:35.921482   931 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:35.921876 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:35.922262   931 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:35.922714   931 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.923064   931 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:35.925132 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:35.926121 31364 raft_consensus.cc:2410] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:35.927419 31388 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:35.912251   864 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:35.893153   724 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:35.934778   769 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:35.845724   905 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:35.895979 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 64145.7us
I20260709 13:14:35.896250   910 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:35.836383   880 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:35.898710 30626 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:35.899461 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:35.845921 30382 rpcz_store.cc:269] 0709 13:14:23.275519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:23.275889 (+   370us) service_pool.cc:224] Handling call
0709 13:14:35.758764 (+12482875us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:28.387495 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:28.388154 (+   659us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:28.388198 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:14:28.388218 (+    20us) tablet.cc:662] Decoding operations
0709 13:14:28.389492 (+  1274us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:28.389583 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:28.389609 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:14:28.404148 (+ 14539us) tablet.cc:701] Row locks acquired
0709 13:14:28.404170 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:14:28.404353 (+   183us) write_op.cc:270] Start()
0709 13:14:28.404485 (+   132us) write_op.cc:276] Timestamp: P: 1783602868404296 usec, L: 0
0709 13:14:28.404506 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:14:28.405321 (+   815us) log.cc:844] Serialized 5154 byte log entry
0709 13:14:28.727379 (+322058us) op_driver.cc:464] REPLICATION: finished
0709 13:14:29.116253 (+388874us) write_op.cc:301] APPLY: starting
0709 13:14:29.116364 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:29.116653 (+   289us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:29.116806 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:29.118408 (+  1602us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:29.132542 (+ 14134us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:14:29.142448 (+  9906us) write_op.cc:312] APPLY: finished
0709 13:14:29.179407 (+ 36959us) log.cc:844] Serialized 94 byte log entry
0709 13:14:29.184218 (+  4811us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:29.201149 (+ 16931us) write_op.cc:454] Released schema lock
0709 13:14:29.204239 (+  3090us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:29.235735 (+ 31496us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:29.239379 (+  3644us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":44187,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":377938,"apply.run_cpu_time_us":4926,"apply.run_wall_time_us":102323,"num_ops":13,"prepare.queue_time_us":66609,"prepare.run_cpu_time_us":55385,"prepare.run_wall_time_us":117406,"raft.queue_time_us":116604,"raft.run_cpu_time_us":25533,"raft.run_wall_time_us":225013,"replication_time_us":322643,"thread_start_us":432795,"threads_started":4,"wal-append.queue_time_us":189830,"wal-append.run_cpu_time_us":14734,"wal-append.run_wall_time_us":1638212}]]}
I20260709 13:14:35.901825   930 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:35.882985   734 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:35.888267   726 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:35.882723   429 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:35.880833   882 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:35.888623   590 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:35.879760   836 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:35.890959   770 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.847271   870 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:35.954911 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:35.915091   781 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:14:35.965952 30550 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:35.964416   862 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:35.971449   947 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:35.964519   875 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 13:14:35.982779 31383 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:35.956192   551 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:35.985332 30631 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:35.984444   910 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.002089 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:35.979161   726 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:35.981602   882 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:35.985764   770 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:35.974565   880 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:36.012113 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 301759us
I20260709 13:14:36.013362 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.015410 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.021926 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:35.994459   863 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:36.062163 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:36.074530 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52705.5us
I20260709 13:14:36.006057   910 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:36.010980   726 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:36.100067 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 198636us
I20260709 13:14:36.010178   551 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:36.024448   903 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.115401 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.021224   937 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:36.124934 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:36.113408   725 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:36.125022   551 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.129565 30631 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:36.123536 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 308673us
I20260709 13:14:36.105129   891 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:14:36.135104   862 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.135715   862 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.138371   862 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:36.139968   863 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.140086 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.145099   911 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:36.086757   803 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:14:36.148241 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:36.074038   870 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.022575   841 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:36.020116   898 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:36.023463   864 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.153798   947 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:36.103434   882 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:14:36.156049 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:36.156893   725 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:36.157465 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:36.158412   930 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.158064   725 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.165293   750 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:36.165963   750 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:36.169685   724 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.170794   903 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.173941   903 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.170912   678 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:36.174675   750 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.176447   750 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:36.180516   724 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.180665   836 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:36.181332   836 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:36.182538   836 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.183985   836 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:36.009944   905 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.193248   905 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:36.193749 30555 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:36.196292   870 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.196126 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.158710   863 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:36.075937   861 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:14:36.207672   861 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:36.208113   861 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:36.209053 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.208946   861 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.209710 30556 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.212334 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.182543   590 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.216246 30705 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:36.217499   590 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.084316   671 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.270036 30556 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:36.271268 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:36.272145 31378 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.297557 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:36.020910   839 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.215s
W20260709 13:14:36.300307 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 428932us
I20260709 13:14:36.250756   861 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:36.310366 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.307812 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.319936 30629 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:36.351677 30556 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:36.277423   882 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:36.358124 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:36.363888   771 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:36.361371 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:36.365831 30632 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:36.306227   891 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 13:14:36.294400   951 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.153s	user 0.004s	sys 0.003s
I20260709 13:14:36.368326   949 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 13:14:36.383154 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.019s	sys 0.019s
W20260709 13:14:36.385632 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.019s	sys 0.020s
W20260709 13:14:36.374256 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 249727us
I20260709 13:14:36.388672 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:36.373963 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67393.3us
I20260709 13:14:36.392268 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:36.393392 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 515907us
I20260709 13:14:36.372484 30626 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:36.365582   839 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:14:36.390364   902 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:36.327337 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:36.405982 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:36.411038 30630 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.411252 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:36.406297 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.408648 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:36.370724   882 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.406675 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:36.299638   946 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.420748 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:36.433066 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:36.295150   678 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:36.433694 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:36.357390   931 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.441258 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:36.442555 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:36.450145 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 419642us
I20260709 13:14:36.407652   590 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:36.448132 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.417284 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.413573   902 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:36.462276 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48743.2us
I20260709 13:14:36.461536 30559 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:36.465492 30624 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:36.300469   671 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.470405 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.473760 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:36.476191 30705 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:36.302596   898 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.163s
I20260709 13:14:36.486287 30629 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:14:36.433449 30381 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.044s	sys 0.106s
I20260709 13:14:36.479351 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:36.480813 30633 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:36.436122   947 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.433599   847 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:36.413199   897 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 13:14:36.500016 31367 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:36.500311 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 83144.4us
W20260709 13:14:36.492764 30381 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.045s	sys 0.106s
I20260709 13:14:36.501312 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.500126 30699 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:36.511798 30556 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:36.471680   679 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:36.520222 31364 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:36.472658   902 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:36.503556   914 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:36.464736   678 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.472139   726 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.439380   946 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.484573   941 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:36.535488 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63865.6us
I20260709 13:14:36.525002   671 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.485491   862 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:36.547224 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.504937   891 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:36.551110 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 204113us
I20260709 13:14:36.532024   955 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:36.581902 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.582973 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:36.543614   841 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:36.589972 31367 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:36.504123   847 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.501123   910 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.566782   769 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 13:14:36.631771 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 130559us
W20260709 13:14:36.632974 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44062.7us
I20260709 13:14:36.500694   947 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.576192   679 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:36.657332 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68144.7us
I20260709 13:14:36.604288   898 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:36.729072   898 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:36.729764   891 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:36.730722   891 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:36.730273   898 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.732025   891 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.734192   946 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:36.737884   770 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.001s
I20260709 13:14:36.669559 31378 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:36.577980   941 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.677242 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:36.678761   897 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:36.738504   861 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:36.745587   861 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.746680   941 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:36.683933   956 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:14:36.684939   960 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:14:36.748023   861 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.693157   771 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:36.698762 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.698984   947 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:36.748780   771 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.752887   771 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:36.726480   870 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.003s
I20260709 13:14:36.753890 31394 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:36.755582 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.756332 30553 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:36.700369   781 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.654016   527 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:36.705195   931 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.661345   908 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:14:36.582633   862 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.713521   847 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.584470   955 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.775023 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.775594   955 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:36.776566   734 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:36.777819   734 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:36.778251   734 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:36.667121   968 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:36.733740   964 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.208s	user 0.004s	sys 0.000s
W20260709 13:14:36.734355   836 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.001s
W20260709 13:14:36.734133   750 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.000s
W20260709 13:14:36.734898   905 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 13:14:36.673805   863 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:14:36.747148 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:36.748617   861 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.811954   937 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:36.812506   861 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:36.814036   962 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:14:36.582391   930 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:36.762163 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.760931 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:36.758147 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:36.600836   599 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.165s
I20260709 13:14:36.744472   726 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.765961 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.757875 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:36.828751 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70981.8us
I20260709 13:14:36.830062 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.768787 30630 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:36.771238 31367 raft_consensus.cc:2368] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7.
I20260709 13:14:36.738369   724 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.725473   882 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.779120 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.732149   898 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:36.768074 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186294us
I20260709 13:14:36.812513   937 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:36.832919   974 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.839215   771 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.738518   770 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.840404 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:36.855572 30625 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:36.840904 31361 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:36.779747   781 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.775035   862 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:36.847325 30557 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:36.823011   949 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:36.850193 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.848971 30553 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:36.851564 30630 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:14:36.860435 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:36.866411 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.846719   905 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:36.871512 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.870857   900 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:36.874739   900 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.877104   900 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.877810   977 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.878477   977 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.880844   977 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:36.882756   770 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:36.884002   902 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.871428   862 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:36.885505   781 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:36.886068   910 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.863217   974 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.894388 30627 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
W20260709 13:14:36.850160 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 189719us
I20260709 13:14:36.866348 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:36.841192   931 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.846752   836 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:36.897985 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.898208 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.900916 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.901432 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.850463   724 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.906847 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:36.903613 30556 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:36.909591 30559 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:36.904044 30624 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:36.904392 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.911120 30555 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:36.905730 30625 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:36.850191   803 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:14:36.911863 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.912516 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:36.913169 30630 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:36.799732   968 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.917062 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.918174 30554 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:36.923113 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:36.923842 30699 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:36.858491   847 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:14:36.919974 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:36.930451 31378 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:36.925212 30553 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:36.906172   781 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:36.857084   771 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.885576   862 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.872957   976 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.941465 31396 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:36.872581   750 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.903644   931 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.936381 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:36.931931 30556 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:36.938452 31391 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:14:36.938364 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:36.899353   836 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:36.867151   679 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:36.955488 31389 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:36.950006 30630 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:36.907492   974 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.954499 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:36.886377   734 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.967583   734 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:36.968001   946 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:36.969065   946 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.969731   946 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:36.933808   862 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:36.935890   429 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:14:36.917021   438 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:36.972894   438 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.903705   908 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:14:36.954725   867 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:14:36.964093   931 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:36.933153   724 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:36.950846   781 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:36.932497   937 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.977172   781 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:36.927755   905 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:36.978140   726 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:36.978511   905 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.980024   957 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:36.980537   957 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:36.981091   905 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:36.981485   957 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.933566   975 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:14:36.983649   976 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:36.984283   975 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:36.989434 30553 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:36.992241 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:36.993166 30631 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:36.926980   979 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:36.999181 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79328us
I20260709 13:14:36.999389 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.006057 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:37.008287 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:36.986135   836 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:37.017567 30699 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
W20260709 13:14:37.053399 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47457.7us
I20260709 13:14:37.056900 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:37.059464 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:37.087496 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.088826 30553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:37.089811 30553 raft_consensus.cc:606] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:36.994738   975 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:37.091734 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:36.985054   750 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:37.004204   429 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:37.052812   908 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:37.051566   855 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:37.002941   875 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.110s
I20260709 13:14:37.003724   725 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.111s
W20260709 13:14:37.133632 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71744.2us
I20260709 13:14:37.024762 31367 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:37.108649   984 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:14:37.152113 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 162729us
I20260709 13:14:36.997344   931 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:36.994415   981 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:37.005623   957 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.140626 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:37.105993   975 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:37.173086 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:37.162317   984 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:37.178756 30553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:37.179539 30553 raft_consensus.cc:606] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:37.141271   855 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:37.130331   983 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:37.169956   438 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.135810   839 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:14:37.192219 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 184521us
I20260709 13:14:37.165642   968 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.198400   750 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.165251   527 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:14:37.175577   946 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.214134   977 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.180563   678 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.229091 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.182379   930 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:37.008958   979 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.190934   989 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:37.171131   931 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.239307 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:37.223536   981 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:37.224946   968 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:37.243669   976 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:37.287258 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113213us
I20260709 13:14:37.240091   902 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:37.294749 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64956.9us
I20260709 13:14:37.244238   989 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:37.301321 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:37.302127 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 129190us
I20260709 13:14:37.303061 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:14:37.293716   996 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.073s	user 0.000s	sys 0.002s
I20260709 13:14:37.236714   977 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.286309   968 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:37.306306   981 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.308223 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:37.326910 30554 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:37.331562 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.336742 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:37.338608 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:37.297839   930 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:37.302659   989 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:37.304760   946 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.307725   976 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:37.318368   978 raft_consensus.cc:993] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.352421 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.345439 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.357600 30699 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:37.357352   862 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.361167   911 raft_consensus.cc:993] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.363833 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.361663   930 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.366153   995 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.367107   995 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.368110   908 raft_consensus.cc:993] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.291771   855 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:37.372864 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.373703 30552 raft_consensus.cc:2368] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:37.348188   979 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.280401   975 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.355434   976 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:37.380383   947 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.396512   939 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.395218   985 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:14:37.378969   973 raft_consensus.cc:1081] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.351784   946 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.437995   979 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.436098   978 raft_consensus.cc:1081] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.395315   911 raft_consensus.cc:1081] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:14:37.340296   981 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:37.465181 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 457004us
I20260709 13:14:37.434731   671 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.477053   671 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.479909   671 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.480001   861 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.482384 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1076) took 1079 ms. Trace:
I20260709 13:14:37.482931 30632 rpcz_store.cc:276] 0709 13:14:36.403201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.414367 (+ 11166us) service_pool.cc:224] Handling call
0709 13:14:36.448451 (+ 34084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.670679 (+222228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.670716 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.670766 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:36.670803 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.482040 (+811237us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:37.482110 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.482305 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510455680}
I20260709 13:14:37.485263   941 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:37.485935   941 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:37.460700   975 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:37.499550   946 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.445250   939 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.466989   979 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.500309   995 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:37.501483   977 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.502168   981 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:37.474025   898 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.000s
I20260709 13:14:37.371714   968 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.515348   978 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.543921   939 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.544091   978 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:37.502195   997 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:14:37.517710   734 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.000s
I20260709 13:14:37.441076   947 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:37.548789 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1082) took 1142 ms. Trace:
I20260709 13:14:37.549201 30627 rpcz_store.cc:276] 0709 13:14:36.405884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.410978 (+  5094us) service_pool.cc:224] Handling call
0709 13:14:37.548615 (+1137637us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540007d5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:37.548723 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928368384}
I20260709 13:14:37.525683   941 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.527935   861 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.530258 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1110) took 1082 ms. Trace:
I20260709 13:14:37.533514   836 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:37.470839   931 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:37.499611   999 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:14:37.542598   955 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.004s
I20260709 13:14:37.567647   861 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.570703   955 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:37.572873   861 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.573366   955 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.554770   976 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.543347   979 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.574782 30551 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:37.562368 30701 rpcz_store.cc:276] 0709 13:14:36.447613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.447801 (+   188us) service_pool.cc:224] Handling call
0709 13:14:37.528708 (+1080907us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b540003eba0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:14:37.529993 (+  1285us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:37.530181 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1942270464}
W20260709 13:14:37.563669 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 246934us
I20260709 13:14:37.564729   978 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:37.579293   941 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:37.590776 31394 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:37.593243 31394 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.593668 31394 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.591615 30631 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:37.594564 30631 raft_consensus.cc:606] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:37.595005 30631 raft_consensus.cc:614] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:37.602689   836 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:37.591922 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.614147 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.615007 30554 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:37.615131 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.615903 30555 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:37.616925 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.617743 30552 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:14:37.606339 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.620656 30551 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:37.621133 31378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:37.639801 31378 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.641604 31378 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:37.656033 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:37.657562 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:37.657456 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:37.658706 30629 raft_consensus.cc:2368] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:37.658896 30627 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:37.641597 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:37.666860 31388 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50014
I20260709 13:14:37.667644 31388 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:14:37.668022 31388 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:37.675098  1000 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:37.677748 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:37.678864 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.679796   930 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:37.682235   930 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.676635 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:14:37.671622 31378 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.684432 30554 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:37.684700 31394 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.685854 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:37.652401   861 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:37.690866 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:37.691563 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.687672 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.702169   929 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:37.703069   929 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:37.704001 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:37.704202   929 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.706252 31394 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:37.707895   957 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.528s	user 0.013s	sys 0.000s
I20260709 13:14:37.708940 30556 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:37.712227 31378 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.682741   930 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.720045 30523 leader_election.cc:400] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:37.720616 30523 leader_election.cc:403] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 13:14:37.723472   959 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.007s	sys 0.007s
W20260709 13:14:37.724020   959 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.912s	user 0.007s	sys 0.007s
I20260709 13:14:37.746152  1006 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:37.741937   957 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.700450   861 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.748327   957 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.749963   861 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.751082   957 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.751901 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:37.756697   961 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.757576   961 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.764933   771 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.764734   989 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:37.767223 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72307.9us
I20260709 13:14:37.772774 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.742540 31388 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.768045 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:37.776181 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.777431 30699 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:37.778116 31389 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:37.782873 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.785496 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.788198 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.788986 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.791330 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.792271 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:37.794127 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.795122 30553 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:37.797466 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.799682 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.801208 30555 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:37.803282 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.803417 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.801548 30555 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:37.809803 30553 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:37.812847 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:37.804353 30707 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:37.804096 30703 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:37.815219 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.819309 31388 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:37.806981 30555 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:37.823459 30555 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.773736   975 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:37.830466 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:37.780124   957 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:37.831174 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:37.800623   891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.999s	user 0.064s	sys 0.071s
I20260709 13:14:37.777952   949 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:37.833842 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:37.837355 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.838289 30552 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:37.840194 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.841070 30704 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:37.846160   972 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:37.850036   958 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:37.855130 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:37.828186  1000 consensus_queue.cc:1175] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6b393e79ded4461ea5da7b6b6063305f" responder_term: 8 status { last_received { term: 8 index: 5 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 5 } }
I20260709 13:14:37.857211 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:37.843189   891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.064s	sys 0.071s
I20260709 13:14:37.835389   957 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:37.863616 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 700) took 1449 ms. Trace:
I20260709 13:14:37.861634 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.859891 30704 raft_consensus.cc:2368] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:37.875108 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:37.865490 30626 rpcz_store.cc:276] 0709 13:14:36.414457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:36.477616 (+ 63159us) service_pool.cc:224] Handling call
0709 13:14:36.503940 (+ 26324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:36.659550 (+155610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:36.659606 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:36.659672 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:36.659723 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:37.863356 (+1203633us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54004aa6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:37.863410 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:37.863546 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":357101952}
I20260709 13:14:37.856072  1006 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:37.865304   990 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:37.867123   957 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:37.895520 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:37.851207   949 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.860249   972 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.884827   770 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:37.900861   855 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:37.920907  1022 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:14:37.898851   990 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:37.922827 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.862257   958 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:37.936422 30558 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:14:37.959794 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:37.932458   949 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:37.916391   989 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:37.978768 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.981129 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:37.982161 30703 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:37.947163   770 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:37.970796   990 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:38.030292 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 160856us
I20260709 13:14:38.003113   961 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:37.966168   855 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:37.987334   770 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:38.139360  1027 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:14:38.175110 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:38.175590 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:38.175179 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:38.177279 30556 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:14:38.166939  1008 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.132s	user 0.002s	sys 0.000s
I20260709 13:14:38.033741   771 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 13:14:38.167657 30555 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:38.168444   855 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.168946   843 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:38.171218   984 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:38.171164 30673 leader_election.cc:400] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:38.316681 30673 leader_election.cc:403] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:14:38.317461 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 146520us
I20260709 13:14:38.171631   972 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:38.173080 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.177685 30556 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:38.167001 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 183949us
W20260709 13:14:38.310740 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:38.321574 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.321746 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:38.318493 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:38.183166   978 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.285452   903 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:38.320583 30556 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.292277   679 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:38.330931 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:38.286877  1006 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.342288 30556 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:38.364457 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:38.374539 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:38.376127 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:38.327442   902 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:38.316179   984 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:38.325973   861 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.344080   972 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:38.364447   974 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:38.350663   679 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.390600 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:38.397872 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:38.408281 30555 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:38.398716 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.362821   961 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:38.403553   875 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:38.451206   875 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.451779   875 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:38.452181   679 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:38.399505   903 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:38.381413   880 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:38.455561   903 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.399282   861 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.407038   974 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.459095   974 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:38.459486   861 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.372941  1006 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.393064   972 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.463078   972 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:38.463418  1006 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.404105   989 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:38.465574   989 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:38.377411   891 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:14:38.466774   989 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.343641   978 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:38.471829   958 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:38.472461   978 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:38.477313   930 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.002s
I20260709 13:14:38.477885   930 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:38.411020 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:38.480003 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 69139.6us
W20260709 13:14:38.508029 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.528945 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:38.565554 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.571208 30558 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:38.510325   551 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:14:38.495733   891 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.618016 30555 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:38.579473   961 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:14:38.635898 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.536s (SENT)
I20260709 13:14:38.636421   902 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:14:38.635305   551 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:38.640403   930 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.649372   551 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.614342  1035 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:38.668540 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 294082us
W20260709 13:14:38.691582 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 305404us
I20260709 13:14:38.616472   875 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.705740 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:38.707208 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.690287   921 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.697467   991 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.700081  1038 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.627537   891 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.703609   861 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.730482 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:38.734980 30633 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:38.737088 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:38.742535 30550 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:38.743639 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:38.745560 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:38.747888 30633 raft_consensus.cc:2410] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:38.746685 30626 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
W20260709 13:14:38.737959   957 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.825s	user 0.014s	sys 0.001s
I20260709 13:14:38.746543   903 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:38.766212 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:38.766741 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:38.767467 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:38.720412   921 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:38.766163   770 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.635s	user 0.013s	sys 0.002s
I20260709 13:14:38.764423   903 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:38.760866   930 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:38.783103 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.546s (SENT) [suppressed 7 similar messages]
I20260709 13:14:38.745518   861 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.699265  1035 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.733378  1038 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.778687   989 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.726003   991 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.763631   957 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:38.786885   903 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:38.801646 30550 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:14:38.812093 30550 pending_rounds.cc:85] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 4
I20260709 13:14:38.796689   930 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.807688   989 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:38.814836   930 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.815475   930 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.815625   989 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:38.816016   930 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.816377   989 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.783192   770 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.812444 30550 pending_rounds.cc:107] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:14:38.820158 30550 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:38.821381 30632 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:38.770598   974 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.817022   989 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.828672 30554 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:38.803993  1043 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.839136   957 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:38.840190  1035 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.841428   875 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.841845  1038 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.842830   891 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.845096   921 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:38.845907   947 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.845999  1046 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.847740 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:38.788444 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:38.861874 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.846722  1046 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.864730 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.866117 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.868027 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.868943 31396 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:38.870455 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.871241 31383 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:38.873010 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.873337   974 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.873945 31396 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:38.874766   989 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.875774 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.876344 31396 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:38.876598 31383 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:38.875821   861 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.878921 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.879603 31388 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:38.872853   770 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:38.882946   974 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.883069 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.883127 30550 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:38.883863   991 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:38.883994 31383 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:38.884837   989 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:38.884119   861 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:38.886540 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:38.886386   861 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:38.883888   974 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:38.888789   861 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.889315   974 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:38.889557 31383 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:38.889660 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 962) took 1299 ms. Trace:
I20260709 13:14:38.889896 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:38.890220 31364 rpcz_store.cc:276] 0709 13:14:37.590750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.591704 (+   954us) service_pool.cc:224] Handling call
0709 13:14:37.592320 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:37.594545 (+  2225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:37.594588 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:37.620944 (+ 26356us) log.cc:844] Serialized 26 byte log entry
0709 13:14:38.581418 (+960474us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b180007f3c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:14:38.827015 (+245597us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:38.827051 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:38.827111 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:38.889310 (+ 62199us) raft_consensus.cc:1685] finished
0709 13:14:38.889336 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:38.889588 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":176,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":20187,"spinlock_wait_cycles":25872128,"thread_start_us":245392,"threads_started":1,"wal-append.queue_time_us":1247216}
I20260709 13:14:38.895416  1046 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:38.866882 31394 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:38.898192 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.896963 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:38.915959 31364 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:38.868290   990 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.800s	user 0.016s	sys 0.000s
I20260709 13:14:38.823372   679 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:38.931821   930 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:38.952617 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.953751 31364 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:38.941497 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:38.943423 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.851307  1043 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:38.957324 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:38.957870 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 174802us
I20260709 13:14:38.957360 30707 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:38.916683   771 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:38.961165 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:38.962541 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.963495 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:38.968297 30704 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:38.969821 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:38.978471 31383 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:38.982298 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:38.987581 31364 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:38.958995   875 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:38.961650   859 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:38.994467 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:38.989900   930 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.998876 31364 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:38.965238   803 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:39.001737 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.966423   867 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:38.952710   990 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.004407 30700 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:39.017215 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:38.996454   980 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.026227 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.039816 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:38.994330   864 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:39.006906   958 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:39.060649 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:39.065877 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124519us
I20260709 13:14:39.028617   527 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:39.067674 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:39.069541 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.004017   859 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.023465   980 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.001869   875 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.007050   930 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.056192   921 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.012127   803 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.087873   921 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.089592   803 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.070653   527 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.090433   527 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:39.070745   958 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.024725   867 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.082158   875 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.084807   679 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.071213 31394 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:39.071794 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:39.097820 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.098838 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.231549 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46042.6us
I20260709 13:14:39.232302 31388 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:39.098963 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.235955 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.237020 30553 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:39.100231 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.102411 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.073326   859 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.073700   980 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.073063   990 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.105172   921 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.240185   867 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.241866 30559 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:39.242352 30559 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:39.230464 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.243320 30559 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.244408 30559 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:39.228665 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43908us
I20260709 13:14:39.246275 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 460) took 1459 ms. Trace:
I20260709 13:14:39.246788 30554 rpcz_store.cc:276] 0709 13:14:37.787383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.788197 (+   814us) service_pool.cc:224] Handling call
0709 13:14:37.788528 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.829350 (+1040822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.246003 (+416653us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000f0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:39.246204 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2391617280}
I20260709 13:14:39.248009  1000 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.248605  1000 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.253120 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1409) took 1432 ms. Trace:
I20260709 13:14:39.066779   864 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.152523   891 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.103415  1056 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.159590 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.150516 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:39.211354 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.211112   941 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:39.075409   862 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:39.076406   972 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.613s	user 0.009s	sys 0.004s
I20260709 13:14:39.078436   930 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.188867 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.215080 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:39.215756   527 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.216930 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:39.215777 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109383us
I20260709 13:14:39.217759   770 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.150842 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.218253 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.219579  1023 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.332s	user 0.013s	sys 0.024s
W20260709 13:14:39.220393  1054 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:14:39.204236 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:39.289119 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.308584 31391 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:39.253696 30703 rpcz_store.cc:276] 0709 13:14:37.820618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.821652 (+  1034us) service_pool.cc:224] Handling call
0709 13:14:39.252930 (+1431278us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400342454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:39.253049 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321249152}
I20260709 13:14:39.292014 31396 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:39.296497 31361 raft_consensus.cc:2410] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:39.300686 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:39.302026 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.307432 30698 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:39.309989 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.310673 31396 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:39.339532 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.312773 30700 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:39.343823 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.313827 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98816.6us
I20260709 13:14:39.345283 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.347050 31378 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:39.348685 31388 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:14:39.351423   527 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.294157   769 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.353143   769 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.354837   678 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:39.357638 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.293792   921 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.296936   958 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.363993 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.360951 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.307555   726 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.296829  1057 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:39.341382   750 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:39.367668 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:39.384876 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.386715 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.387030 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.295861   891 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.387620 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.390676 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.312440   991 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:14:39.365710   862 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.393613 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.395233 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.395927 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.340410   770 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.368113   527 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:39.401827 31394 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:39.401655   527 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.297513  1056 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.374516  1039 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.308605   972 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.382807   980 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:39.377635   726 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.408046   527 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.392179  1045 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:39.411983   864 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.413897 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.393646   987 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.365693   921 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:39.415422 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.417919 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.418488 30559 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:39.419714 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.419854 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.420224 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:39.427870 30705 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:39.427937 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:39.429638 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.405653   855 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:39.432153 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:39.435974 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.388695  1057 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.438825 30704 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:39.443508 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:39.444270 30698 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:39.449071 30705 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:39.422667  1045 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.424899  1000 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.405213   862 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.430817   980 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.487185 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.432659   968 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:39.487654  1045 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:39.497324 30551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1041) took 2168 ms (client timeout 2115 ms). Trace:
I20260709 13:14:39.442473   990 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:39.488538  1065 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:39.499751 30551 rpcz_store.cc:269] 0709 13:14:37.328922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.331225 (+  2303us) service_pool.cc:224] Handling call
0709 13:14:39.496854 (+2165629us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:39.497134 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556901632}
I20260709 13:14:39.483922   974 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.504045 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:39.511333 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 120781us
I20260709 13:14:39.511700 30705 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:39.518961 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.487175   770 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.519896 30556 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:39.533970 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.534304 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.536448 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.539624 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:39.535182 30629 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:39.537529 30633 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:39.543458 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.543949 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.499027  1053 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.549196 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.510035   990 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:39.550979 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.511734  1039 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.554168 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.555212 30628 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:39.556272 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.556602 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:39.557158 30633 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:39.558151 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.558866 30626 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:14:39.559228 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 127110us
I20260709 13:14:39.561429 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:39.562038 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.501344   980 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.564210 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:39.575033 30625 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:39.577801 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.555739   990 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:39.519747   974 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:39.564052 31292 connection.cc:256] RPC call timeout handler was delayed by -1.1736058870001216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1718s for a response
I20260709 13:14:39.586427 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.560384   861 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.587798 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.593600 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:39.588451 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.590934 30705 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:39.557545   750 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:39.591017 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.519954  1065 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.580066  1039 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.606410 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.601912 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.607431 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.563740   770 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:14:39.618834 30700 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:39.602363 30626 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:39.555100  1053 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.589854   891 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.612758  1065 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.613674   921 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:39.615175  1072 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 13:14:39.615995  1073 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:14:39.598501   861 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.636452   861 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.619536 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.610129   930 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.623962 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.619867 30698 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:39.624418 30626 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 3
I20260709 13:14:39.647796 30626 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:39.648154 30626 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:39.609751  1077 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.624384   726 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.643391   891 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.650918  1077 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.653437   937 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:39.633334  1071 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:14:39.628230  1076 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.623726 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:39.632030   972 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.653967   980 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.604419  1039 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.624979  1074 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:39.660902 30559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 468) took 1858 ms (client timeout 1682 ms). Trace:
I20260709 13:14:39.659370 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:39.662631   974 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.674808   750 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.680935  1053 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.681933 31364 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:39.682250   862 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.649928  1056 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.654042  1077 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:39.667761  1039 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.660053  1076 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.691309  1076 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.693223 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:39.696753  1001 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:39.668310 30559 rpcz_store.cc:269] 0709 13:14:37.802732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:37.814840 (+ 12108us) service_pool.cc:224] Handling call
0709 13:14:39.241709 (+1426869us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b5400498a20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:14:39.660692 (+418983us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400291154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:39.660814 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153899520}
I20260709 13:14:39.694308 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:39.700183 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76255.1us
I20260709 13:14:39.703771 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:39.709839  1077 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:39.713747 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1053) took 1097 ms. Trace:
I20260709 13:14:39.714500 30558 rpcz_store.cc:276] 0709 13:14:38.616772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.620458 (+  3686us) service_pool.cc:224] Handling call
0709 13:14:38.621024 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:38.621589 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.621634 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:39.424085 (+802451us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:39.433116 (+  9031us) log.cc:844] Serialized 26 byte log entry
0709 13:14:39.600875 (+167759us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:39.600924 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.601062 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:39.713202 (+112140us) raft_consensus.cc:1685] finished
0709 13:14:39.713240 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.713627 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5780,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":8443,"spinlock_wait_cycles":12761344,"thread_start_us":98333,"threads_started":1,"wal-append.queue_time_us":245952}
I20260709 13:14:39.697434  1001 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.718891   972 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:39.719617   972 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:39.720564   972 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:39.723187   903 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.899s	user 0.014s	sys 0.002s
I20260709 13:14:39.718922  1057 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:39.724359  1001 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.726111  1077 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:39.727895  1077 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:39.725881   984 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.731861  1043 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:39.732190   984 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:39.736515   862 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:39.737102   862 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.737483   862 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:39.737885 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1107) took 1008 ms. Trace:
I20260709 13:14:39.738296 30627 rpcz_store.cc:276] 0709 13:14:38.729556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.729681 (+   125us) service_pool.cc:224] Handling call
0709 13:14:38.730083 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.826924 (+ 96841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:38.826956 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:38.826999 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:38.827035 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.737627 (+910592us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b54004ab5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:39.737690 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.737824 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222067584}
I20260709 13:14:39.730356  1076 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:39.741568 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 358254us
I20260709 13:14:39.698836  1074 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.742416  1074 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:39.725199   903 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:39.742520 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:39.743297 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1128) took 1331 ms. Trace:
I20260709 13:14:39.757681 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.758567 31396 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:39.759461   769 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:39.760589 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.762470 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.763281 30557 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:39.770102 30703 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:39.701248   980 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:39.771090   980 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:39.771574   980 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:39.775230 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.777675 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.778630 31394 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:39.755167 30701 rpcz_store.cc:276] 0709 13:14:38.412241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.412430 (+   189us) service_pool.cc:224] Handling call
0709 13:14:38.449561 (+ 37131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.482173 (+ 32612us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:38.757795 (+275622us) log.cc:844] Serialized 14 byte log entry
0709 13:14:39.145886 (+388091us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:39.145984 (+    98us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:39.146075 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.743017 (+596942us) spinlock_profiling.cc:243] Waited 32.2 ms on lock 0x7b54005a28d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:39.743071 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:39.743206 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20229,"prepare.queue_time_us":343743,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":36505,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":6430,"spinlock_wait_cycles":74168704,"thread_start_us":544113,"threads_started":3,"wal-append.queue_time_us":242159}
I20260709 13:14:39.784053 30558 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:39.785266 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.786049 30557 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:39.749379   903 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.786634 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.752121 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.787372 30551 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:39.752856 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.788527 31367 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:14:39.752516 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.789642 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.789971 31378 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:39.786059 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:39.787600 30556 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:39.791777 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.792258 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.792490 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.792903 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.792917 30552 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:39.793730 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:39.736529 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1108) took 1006 ms. Trace:
I20260709 13:14:39.794391 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.794543 30632 rpcz_store.cc:276] 0709 13:14:38.729842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.730352 (+   510us) service_pool.cc:224] Handling call
0709 13:14:38.730755 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:38.821818 (+ 91063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:39.733864 (+912046us) spinlock_profiling.cc:243] Waited 90.3 ms on lock 0x7b54005f3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:39.736451 (+  2587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207603968}
I20260709 13:14:39.794369 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.797187 31389 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:14:39.798993 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.800625 30555 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:14:39.800832 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:39.801700 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:39.787568 31388 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:39.793752 30559 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:39.801854  1006 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.794348 30556 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:39.790341 31394 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:39.807353 30558 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:14:39.809038 30557 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:39.809543 30557 pending_rounds.cc:85] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 3
I20260709 13:14:39.809856 30557 pending_rounds.cc:107] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:39.810158 30557 raft_consensus.cc:2889] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:39.814585 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 223659us
I20260709 13:14:39.815551 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.815958 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.802529 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:39.820314 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:39.823235 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:39.823259 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.825845 31394 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:39.827683 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.825248 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:39.821367 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:39.741802  1076 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.828507 30551 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:39.829941 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.836905 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.831394 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 130307us
I20260709 13:14:39.834302 31394 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:39.805945  1006 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.902796 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.840384 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:39.916069 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 100201us
I20260709 13:14:39.919901 30551 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:39.858593 30523 leader_election.cc:400] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6.
I20260709 13:14:39.921182 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:39.921517 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63199.2us
I20260709 13:14:39.922880 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.923480 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:39.924685 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.927109 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:39.827311   984 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.903870 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.933770 31364 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:14:39.748095  1001 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:39.937283 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:39.939443 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.841377 30556 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
W20260709 13:14:39.828413 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 124707us
I20260709 13:14:39.947064 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:39.869836 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 127405us
I20260709 13:14:39.948527 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.880262   929 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:39.950121   929 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:39.950549   929 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:39.949504 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:39.840615   429 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:39.954054 31361 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:14:39.834991  1017 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:39.830756   941 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:39.837706   973 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.871845   955 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.088s
I20260709 13:14:39.955224 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1026) took 1094 ms. Trace:
I20260709 13:14:39.957544 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:39.912525  1043 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.958630  1046 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:39.829347   975 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:39.952469  1001 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:39.950593   726 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:39.952204  1057 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.944190  1053 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:39.901394  1045 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:39.958451 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:40.020144 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71724.3us
I20260709 13:14:39.972337   929 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.025647  1045 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:40.026139   929 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:39.921710  1004 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:39.978420 31361 pending_rounds.cc:85] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:14:39.983897  1017 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.032756  1017 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:39.936278   734 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:39.946098   972 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:39.832163   769 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:39.946684  1056 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:39.989833 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:39.991466 31389 rpcz_store.cc:276] 0709 13:14:38.860569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:38.860753 (+   184us) service_pool.cc:224] Handling call
0709 13:14:39.953441 (+1092688us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540044e654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:39.955112 (+  1671us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2084496128}
W20260709 13:14:40.059614 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:39.987061   973 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:40.013901 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 64481.6us
W20260709 13:14:40.019855 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62376.9us
I20260709 13:14:40.076227 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:40.021818   750 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:39.966485   980 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:40.022926 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:40.021375   986 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:39.930366  1006 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.030122 31361 pending_rounds.cc:107] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:14:40.000732  1046 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.107802  1046 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:40.109962 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 87160.2us
I20260709 13:14:40.005347   975 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.110929   975 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.005004  1080 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:40.027119   930 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:40.074285 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:40.141710 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:40.142120 30550 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.084466 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:40.102345 31361 raft_consensus.cc:2889] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:40.078594  1074 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.098965   929 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:40.093905   973 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.119575 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.162959 30705 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 5: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:40.120898 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.008519  1043 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.021775 31364 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:40.104779  1081 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:14:40.148675 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 64282.7us
I20260709 13:14:40.038797 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:40.165977 31383 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:40.166426 31396 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.076004  1056 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.155970 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:40.075618   769 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.167341 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:40.134860  1083 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:14:40.188263 30550 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:40.202466 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:40.207897 30554 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:40.157645   726 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.143321   991 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:40.151901  1057 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.157168   984 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:40.037635  1004 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:40.194185 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:14:40.215240 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48014.2us
I20260709 13:14:40.223207 31364 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:40.224264 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:40.076857   990 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.264261 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.171924   769 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.172976  1056 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:40.125401   986 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.359738 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:40.379866 30701 raft_consensus.cc:2368] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9.
I20260709 13:14:40.382411 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.385084 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:40.386013 30627 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:40.386715 30554 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:40.166754  1042 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.261830 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.265012 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.396677 30707 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:40.241294 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:40.227988 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 95905.4us
I20260709 13:14:40.397841 30705 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:40.264142 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.244488 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:40.356791 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 419783us
W20260709 13:14:40.375797 31378 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.030s	sys 0.028s
W20260709 13:14:40.406976 31378 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.030s	sys 0.029s
I20260709 13:14:40.264626 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:40.388468 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:40.393419 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 169288us
I20260709 13:14:40.410898 30703 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:40.353238   955 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:40.392544   862 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.655s	user 0.007s	sys 0.006s
I20260709 13:14:40.183105  1001 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:40.420090 30633 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
W20260709 13:14:40.419273 30673 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.020s	sys 0.040s
I20260709 13:14:40.424463 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:40.430316 30673 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.021s	sys 0.040s
W20260709 13:14:40.430867 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 603979us
I20260709 13:14:40.184195   980 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:40.214764   984 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:40.425144 30627 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:40.284535   937 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:40.435336 30698 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:40.376850   991 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:40.197947  1030 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.228s
I20260709 13:14:40.450860   972 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:40.452504  1004 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:40.454932 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:40.426771   870 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:14:40.426450   897 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.221513  1057 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:40.458725  1071 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:14:40.435118   955 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.459342 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:40.434562  1039 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
I20260709 13:14:40.460637 30632 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:40.399309   678 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.446314   937 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:40.545666 30628 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.465435 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.221071   726 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.482266 30624 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.227130  1080 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 13:14:40.413332  1076 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:40.463335 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:40.484954 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 394462us
I20260709 13:14:40.491734 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1115) took 1217 ms. Trace:
I20260709 13:14:40.512779 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1061) took 1132 ms. Trace:
W20260709 13:14:40.549374 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 86142.1us
I20260709 13:14:40.549598 31361 rpcz_store.cc:276] 0709 13:14:39.380085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.389261 (+  9176us) service_pool.cc:224] Handling call
0709 13:14:39.953610 (+564349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:40.145921 (+192311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.493034 (+347113us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540044e650. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:40.512657 (+ 19623us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285492608}
I20260709 13:14:40.549252 31383 rpcz_store.cc:276] 0709 13:14:39.274489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.277721 (+  3232us) service_pool.cc:224] Handling call
0709 13:14:40.491394 (+1213673us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540002add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:40.491611 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968865152}
W20260709 13:14:40.513594 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48720.5us
I20260709 13:14:40.444813   862 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.480206   599 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:40.445957   984 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.413880   671 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:14:40.481408   972 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.547739 30701 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:40.553503 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.551862 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:40.465292   855 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:40.531317  1082 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.322s	user 0.000s	sys 0.001s
I20260709 13:14:40.523453   870 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:40.425717  1042 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:40.546383   931 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.028s	sys 0.028s
I20260709 13:14:40.516136  1004 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.544720  1086 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.589355 30630 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:40.554821   937 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:40.539511   986 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:40.571507   972 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:40.593848 30597 leader_election.cc:400] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9.
I20260709 13:14:40.556931   903 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.523823   897 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.614897 30628 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:40.615526 30597 leader_election.cc:403] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:14:40.567780 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.569031  1076 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:40.578579   941 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:40.559113   843 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:40.558605  1039 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:40.634825 31367 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.041s	sys 0.056s
I20260709 13:14:40.590234   870 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:40.558174  1053 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
W20260709 13:14:40.594427   931 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.029s	sys 0.028s
I20260709 13:14:40.558457   678 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:40.630707 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 63039.1us
W20260709 13:14:40.646450 31367 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.041s	sys 0.056s
I20260709 13:14:40.570804   599 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.665305 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:40.586019   855 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.625573  1001 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:40.666507   843 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:40.673395 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.673925 30631 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:40.603766  1004 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.609583  1086 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:40.689957  1091 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:14:40.691174  1004 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:40.692219   599 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:40.698868   870 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.702256  1045 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.702915   990 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:40.702512   843 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.703428  1076 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.743938  1045 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.744700   862 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:40.746383   955 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:40.634472   903 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:40.719635   897 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:40.724411 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 491134us
I20260709 13:14:40.679597  1001 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:40.677243  1080 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 13:14:40.734416 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:40.682862  1074 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:40.720659  1006 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.564s	user 0.005s	sys 0.009s
I20260709 13:14:40.709362   671 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:40.625870   991 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.707875  1043 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:40.746534  1045 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:40.749799   870 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:40.758930  1001 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:40.754958 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:40.793908 30627 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:40.741899  1004 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:40.785202 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:40.795276 30559 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
W20260709 13:14:40.774660  1046 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.666s	user 0.013s	sys 0.001s
I20260709 13:14:40.788252 30554 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:14:40.789687 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:40.805078 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:40.727857 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 466143us
I20260709 13:14:40.806202 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.806447 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.807472 30627 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:40.761994  1042 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:40.811021 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:40.781054   975 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.670s	user 0.014s	sys 0.001s
I20260709 13:14:40.808499 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.820075 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:40.820928 30633 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:40.825203  1039 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:40.810966   930 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.003s
I20260709 13:14:40.803494  1004 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:40.775393  1006 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.831745 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:40.837206 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:40.837809 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.840281 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.779481   671 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.823046  1097 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:40.854075 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:40.855060 30633 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:40.850455 30624 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:40.858560 30633 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:40.816708  1046 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:40.835623  1042 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:40.834781   930 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:40.820287   975 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:40.873149 30627 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:14:40.855969 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 182689us
I20260709 13:14:40.877672 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.848234  1080 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:40.889300 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.890112 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.852177   897 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:40.901589 30632 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:40.899221 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:40.905543 30624 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:40.909188 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:40.859441   726 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:40.859719   770 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9.
I20260709 13:14:40.876116  1042 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:40.960778 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50147.2us
I20260709 13:14:40.860347  1097 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:40.963938 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 126835us
I20260709 13:14:40.964491 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 984) took 1127 ms. Trace:
W20260709 13:14:40.968221 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 114247us
I20260709 13:14:40.978582 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:40.983343 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.025s	sys 0.020s
W20260709 13:14:40.984133 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.025s	sys 0.020s
I20260709 13:14:40.973865 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:40.985297 30556 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:40.975023 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:40.987138 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:40.988140 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 106182us
I20260709 13:14:40.988648 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:40.990262 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.966720   931 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:40.989833 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:40.976209  1042 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:40.979851   836 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:40.978406  1053 raft_consensus.cc:2764] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:40.983265  1080 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:40.984015  1097 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:41.006481 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:40.962991   897 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:41.016753  1056 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.599s	user 0.006s	sys 0.006s
I20260709 13:14:41.019088  1053 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:41.000769   921 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:41.000168   942 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:41.024401 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:41.040558 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:41.030260  1113 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:41.032802   975 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.049122 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.050122 31394 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:41.050323 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.051534 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.053411 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.033933  1046 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.026775   836 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:40.999437  1086 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.002558  1110 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:41.022459  1017 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.989s	user 0.010s	sys 0.003s
I20260709 13:14:41.017993   972 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.006138   931 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:41.007673   903 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.093279 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.076618 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.096953 30633 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:41.097391 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107208us
I20260709 13:14:41.036401   921 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:41.057878   843 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:41.111498 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.068940  1105 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:41.075119  1017 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.079919   972 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:41.080775  1117 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.040930   942 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.045468  1080 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.124086  1117 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.126004  1117 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.131881 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.085064   897 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:41.038288  1056 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.133785  1004 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.134224   921 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.140583   921 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.140803  1004 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:41.085193   855 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.102787 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.103253 30633 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:41.041505 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:41.099020   671 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.076740   836 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.091077  1006 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.080847  1113 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:41.105989 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:41.107406   973 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.895s	user 0.009s	sys 0.004s
I20260709 13:14:41.161808   973 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.106922   931 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.166224  1017 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.085170  1116 raft_consensus.cc:2764] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 13:14:41.085063  1114 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:14:41.090605  1001 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.023640 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:41.153970  1110 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:41.154747 30381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 142) took 13505 ms (client timeout 15000 ms). Trace:
I20260709 13:14:41.160048   855 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.160130  1113 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:41.135206   769 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.004s
I20260709 13:14:41.164224  1042 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.164808 30552 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:41.134948   942 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:41.093775   937 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.175819 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.207948 31389 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:41.164306  1112 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:41.155706 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114313us
I20260709 13:14:41.179268  1017 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.220906  1113 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.221798  1017 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.224141  1042 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.183073  1105 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:41.225277   973 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.225972  1105 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.184434  1056 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.188403 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.189038  1030 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:41.192376  1001 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.197567 30378 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:41.154808 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52113.8us
I20260709 13:14:41.206957 30625 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
W20260709 13:14:41.169611   984 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.585s	user 0.014s	sys 0.000s
I20260709 13:14:41.213461  1095 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:41.214819   937 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.215113 31367 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:41.201123   855 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.246840 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.218465  1112 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:41.201489  1097 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.205930   769 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.253877   855 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.253997  1097 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.255224 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:41.256186 30556 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:41.245174   991 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:14:41.257715 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:41.258268  1112 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.260052 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1702) took 1244 ms. Trace:
I20260709 13:14:41.260514 30552 rpcz_store.cc:276] 0709 13:14:40.016581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:40.018140 (+  1559us) service_pool.cc:224] Handling call
0709 13:14:40.024195 (+  6055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:41.202873 (+1178678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:41.259772 (+ 56899us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400374954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:41.259977 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2622556544}
I20260709 13:14:41.261616  1017 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.263490  1057 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.263785  1100 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:41.264305  1100 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.264643  1100 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.265537   770 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:41.266700   770 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:41.267068   770 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:41.267951   770 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.268369   942 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.269367   770 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:41.269622 30628 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:14:41.278476  1100 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.281080 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.281920 31361 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:41.284235 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.284988 31394 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:41.286103  1117 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.287520 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.288265 30552 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.288915   678 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.297773 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:41.299489 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.300686   769 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:41.300969 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:41.305563 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.305897 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:41.307010 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:41.233314 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 192843us
I20260709 13:14:41.233609 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.308118 30698 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:41.234663 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:41.235018 30550 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.240515 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.302325 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.241835 30557 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:41.309914 31389 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:41.242034 30378 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 5, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:41.310047 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 69651.5us
I20260709 13:14:41.310891 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.311218 30378 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.244527  1095 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:41.247661 30629 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:41.315861 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.244577   984 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.307163 30705 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:41.309021 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.318194 30705 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:41.319191 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.320007 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:41.170140 30381 rpcz_store.cc:269] 0709 13:14:27.649282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:27.649471 (+   189us) service_pool.cc:224] Handling call
0709 13:14:41.154654 (+13505183us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:36.592369 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:36.682260 (+ 89891us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:36.682329 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:14:36.682351 (+    22us) tablet.cc:662] Decoding operations
0709 13:14:36.683541 (+  1190us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:36.683633 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:36.683659 (+    26us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:14:36.691596 (+  7937us) tablet.cc:701] Row locks acquired
0709 13:14:36.691664 (+    68us) write_op.cc:260] PREPARE: finished
0709 13:14:36.691879 (+   215us) write_op.cc:270] Start()
0709 13:14:36.692073 (+   194us) write_op.cc:276] Timestamp: P: 1783602876691789 usec, L: 0
0709 13:14:36.692099 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:14:36.811576 (+119477us) log.cc:844] Serialized 3618 byte log entry
0709 13:14:37.550951 (+739375us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b
0709 13:14:37.890890 (+339939us) op_driver.cc:464] REPLICATION: finished
0709 13:14:38.379080 (+488190us) write_op.cc:301] APPLY: starting
0709 13:14:38.402030 (+ 22950us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:38.472887 (+ 70857us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:38.472951 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:38.474491 (+  1540us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:38.475000 (+   509us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:14:38.475128 (+   128us) write_op.cc:312] APPLY: finished
0709 13:14:38.475640 (+   512us) log.cc:844] Serialized 70 byte log entry
0709 13:14:38.578845 (+103205us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:38.614240 (+ 35395us) write_op.cc:454] Released schema lock
0709 13:14:38.634234 (+ 19994us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:38.637505 (+  3271us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:38.637544 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3268414,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":423064,"apply.run_cpu_time_us":5298,"apply.run_wall_time_us":292929,"num_ops":9,"prepare.queue_time_us":609005,"prepare.run_cpu_time_us":21225,"prepare.run_wall_time_us":1178079,"raft.queue_time_us":168707,"raft.run_cpu_time_us":25090,"raft.run_wall_time_us":126190,"replication_time_us":1173302,"spinlock_wait_cycles":36663552,"thread_start_us":1652762,"threads_started":4,"wal-append.queue_time_us":867443,"wal-append.run_cpu_time_us":23532,"wal-append.run_wall_time_us":1804593}]]}
I20260709 13:14:41.323472 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.324666 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.321646 30378 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 2 to 4, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.330314 30378 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.333402 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.334059 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.326457 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:41.332581 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 327204us
I20260709 13:14:41.341665 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.301023  1117 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.343065 30625 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:41.342958 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.320254 30705 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.345618 30705 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:41.345885 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.336032 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.344519 31394 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:41.338796 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:41.349931 31388 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:41.349773 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.350955 31389 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:41.368906 30550 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:41.359601 30552 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:41.403838 31396 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:41.404011 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:41.355188 30378 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:41.405565 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70057.3us
I20260709 13:14:41.408051 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.407651 30378 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 10, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.408813 31391 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:41.410087 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.402096 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:41.382244 31383 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:41.411688 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.412039 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.406193 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:41.317052  1057 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:41.382388 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:41.416997 31361 raft_consensus.cc:2368] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:14:41.414230 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.325512  1097 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:41.412190 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:41.422224 31364 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:41.425738 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:41.426322 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.426712 31383 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:41.363981 30626 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:41.427944 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.409309 30378 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.353825   843 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:41.450446 30378 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.423835 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:40.979583 31378 rpcz_store.cc:276] 0709 13:14:39.837476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:39.838115 (+   639us) service_pool.cc:224] Handling call
0709 13:14:39.838535 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:39.838834 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:14:40.407572 (+568738us) log.cc:844] Serialized 14 byte log entry
0709 13:14:40.964038 (+556466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:40.964105 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:40.964166 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:40.964260 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:40.964415 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":691866,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":207095,"thread_start_us":1094822,"threads_started":3,"wal-append.queue_time_us":416666}
I20260709 13:14:41.447467 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.454339 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.429100  1120 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:41.353962  1042 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.401328  1117 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:41.419829  1110 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:41.417582 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:41.475154 30628 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:41.354199   769 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:41.460104 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.354100  1116 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:41.440037   941 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:41.331243  1004 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.462515 30378 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 8, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.464977 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.353941   678 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 13:14:41.455955 31391 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:41.405047  1100 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.331238   937 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 13:14:41.345538  1030 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:41.475900 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139933us
I20260709 13:14:41.532871 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:41.476951 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 143326us
I20260709 13:14:41.385828 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:41.473201 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:41.534036 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 148362us
I20260709 13:14:41.482718 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.353695   973 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:14:41.483930 31383 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:41.330946  1001 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:14:41.486779 30629 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:14:41.463179 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.468860 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.492961 30378 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.478169  1117 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.474802  1110 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:41.520107 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.552008 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:41.554239 30557 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:41.458844  1120 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:41.469970  1056 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.475169   855 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:41.513599 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.526706 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.446519  1080 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:14:41.528693 30628 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:41.489672  1004 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.536023 31367 raft_consensus.cc:2368] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9.
I20260709 13:14:41.490559   769 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:41.445897  1122 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 13:14:41.540004  1017 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:41.493429   941 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:41.538969 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.545722 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:41.540433 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57901.3us
I20260709 13:14:41.593027 31383 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:41.549201 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.588109 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.486804  1046 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:14:41.588644 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.548077 30378 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.595916 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:41.595646 30378 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.597880 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.597512 30378 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 7 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:41.485348   679 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:41.616969 30378 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.577119 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.526161  1127 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:41.571768 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.578444 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.623948 30559 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:41.625413 31388 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:41.529152  1057 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.580204 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.642453 30555 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:41.579344 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.591181 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:41.578995 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:41.580108 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.598996 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.598996 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:41.646128 31361 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:41.646723 30626 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:41.594194 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:41.647761 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53643.3us
I20260709 13:14:41.618705 30558 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:41.646142 30553 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.619515 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.651401 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:41.619856 30551 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 6: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:41.579746 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:41.556839   900 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:41.557742   897 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:41.628027 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 114553us
I20260709 13:14:41.629034 31367 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 13:14:41.633108  1132 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:14:41.633579  1017 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:41.614346   855 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:41.636245   941 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.551117   987 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:41.592319   836 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:14:41.616083  1030 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:14:41.578864  1120 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:41.641120  1128 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:14:41.548992 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.587203  1004 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.654263 30559 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:41.619639 30378 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 5, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.557209  1110 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:41.623015 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.662528 30629 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:41.721438 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:41.723162 30703 pending_rounds.cc:85] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 2
I20260709 13:14:41.723580 30703 pending_rounds.cc:107] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:41.723837 30703 raft_consensus.cc:2889] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:41.597357  1000 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:14:41.675469 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.538431   734 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.689275   429 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.691346   671 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:41.691695   984 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.637053  1134 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:41.692198  1017 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.660867   929 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.700258   855 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.702160  1117 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:41.642365  1129 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.112s	user 0.001s	sys 0.001s
I20260709 13:14:41.645279  1127 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.660334   973 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.715598  1115 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:41.716832   942 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:41.651047  1001 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.673938 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:41.789923 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 116031us
I20260709 13:14:41.696617   897 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:41.731097 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55729.7us
I20260709 13:14:41.792809 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.729692 30378 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:41.794399 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.737561 30558 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:41.745806  1097 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.747236 30555 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:41.749209 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 200287us
I20260709 13:14:41.797950 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:41.696200   937 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:41.750802 30559 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:41.797849 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.744653  1113 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:41.661314  1057 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.783324 30553 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:41.803932 30378 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 6, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:41.806836 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.778937 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:41.753690   870 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:41.760013   929 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:41.749209   734 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.823594 31389 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:41.801311   990 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.792873  1001 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.810212  1117 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:41.774253  1122 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:14:41.759604   671 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.756788   769 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.793251  1015 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:14:41.826431   679 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:41.847756 31367 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:14:41.848405 31367 pending_rounds.cc:85] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 2
I20260709 13:14:41.848825 31367 pending_rounds.cc:107] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:14:41.849159 31367 raft_consensus.cc:2889] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:41.821091  1113 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.838452  1001 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.815671   678 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:41.825676 30378 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 5, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:41.842260 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:41.789494   973 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:41.842269 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.836927   897 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:41.807248   937 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:41.848467 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 512829us
I20260709 13:14:41.831233   929 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.832495  1116 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:41.857527 30378 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 3 to 4, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 4 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:41.892043 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49871us
I20260709 13:14:41.899619 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.895840 30378 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:41.839743  1117 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:41.827998   987 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:14:41.902467 30596 leader_election.cc:400] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9.
W20260709 13:14:41.849776   921 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.709s	user 0.015s	sys 0.000s
I20260709 13:14:41.905398 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.855988   941 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:41.911506 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 292065us
I20260709 13:14:41.912451 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.905526 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:41.909102 30378 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:14:41.956182 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139782us
I20260709 13:14:41.891942  1097 raft_consensus.cc:476] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Not starting pre-election -- already a leader
I20260709 13:14:41.891223   678 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:41.915773   671 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:41.917934   937 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.976363 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:41.980324 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:41.981643 30555 raft_consensus.cc:2410] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:41.952512   941 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:41.923749   734 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.958688   921 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:41.960673   972 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:41.989230   972 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:41.990095   973 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:41.991146   678 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:41.991824   972 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:41.904925  1116 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:41.901490  1100 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:41.856397   984 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.108s
W20260709 13:14:41.916936  1135 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:14:41.908252   990 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:41.960798 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 153725us
I20260709 13:14:41.961931 31378 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:41.952839 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:41.910427  1117 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:41.971840 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:41.998126 30559 raft_consensus.cc:2368] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:14:41.969144 30553 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:42.001859   770 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.732s	user 0.016s	sys 0.000s
I20260709 13:14:41.940052 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:41.984576 31364 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:42.012063 30557 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:42.014168 30559 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:42.020259 30551 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:42.022944 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:42.025615 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72879.3us
I20260709 13:14:42.034402 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:41.968974  1136 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:42.041566 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.036253 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:42.065833 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:42.078183 30633 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:42.064497 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.072271 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:42.062745 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 122842us
I20260709 13:14:42.080732 30631 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:14:42.069111 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69542.1us
I20260709 13:14:42.081612 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:42.084702 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 111265us
I20260709 13:14:42.087276 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:42.089949 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 53791.6us
I20260709 13:14:42.092270 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:42.095852 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.096325 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.097213 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.097885 30700 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:42.098100 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.098326 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.099288 30707 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:42.099785 30704 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:42.100665 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.101256 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.102711 30673 leader_election.cc:400] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:14:42.102712 30551 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:42.103199 30673 leader_election.cc:403] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:42.041369   921 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:42.006465  1117 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.063190  1136 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:42.105476   769 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:42.112039  1140 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:14:42.112466 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.112984  1113 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:42.113590   990 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:42.113785 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.115226   942 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:42.118912  1057 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.119693   679 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:42.081985   770 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:42.079420  1127 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:42.105965  1143 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:42.145677  1120 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:42.148053 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45529.6us
I20260709 13:14:42.168514 30628 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:42.165894 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79870.1us
I20260709 13:14:42.143163  1136 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:42.188432  1110 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:42.203363  1141 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:14:42.206720  1120 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:42.208876   679 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:42.209498   679 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.203159   900 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:42.168483  1143 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:42.213862 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44926.6us
I20260709 13:14:42.218990 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:42.219851  1117 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.202322  1004 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:42.235517 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54091.3us
I20260709 13:14:42.168687  1149 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:42.202397   843 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:42.245653 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.247022 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.214834   900 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.248306 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.248723 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.210566   679 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.249457   900 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.186249   929 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.250660 30706 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:42.250917   843 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.251185   900 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:42.251787   836 consensus_queue.cc:1175] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 6 status { last_received { term: 5 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:42.252630 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.248396 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.254420 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.254830   679 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:42.215293  1143 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.254732  1122 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:42.253115 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.252789   929 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:42.227347  1015 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:42.275864  1122 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:42.276523  1122 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.279670  1110 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:42.286947  1006 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:42.287684  1006 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:42.289394   599 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:42.289028  1006 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.290095   599 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.282634  1136 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.280247  1030 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:42.214452  1150 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:14:42.227517  1147 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 13:14:42.297178  1001 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.303020  1001 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.304431  1116 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.304250   599 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.305781   599 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.256620  1155 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:14:42.298640  1030 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.302332  1136 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:42.311224  1095 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.311646  1030 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.312564  1095 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:42.312755  1030 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:42.277841  1122 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.315825  1004 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:42.316040  1122 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:42.317164   955 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:42.317723   955 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:42.318881   955 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.320395   955 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:42.316042   843 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.325455   843 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.327291  1112 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:42.328135  1112 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.329463  1112 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:42.331919   986 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:42.332621   986 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.296293  1006 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:42.333886   986 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.347144   986 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:42.354027  1112 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:42.403568  1143 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.407160  1143 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:42.466683 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 374174us
I20260709 13:14:42.474109 30554 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 13:14:42.475318 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 370306us
W20260709 13:14:42.479769 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 392871us
W20260709 13:14:42.484762 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.485821 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.490815 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 727) took 1148 ms. Trace:
I20260709 13:14:42.491510 30631 rpcz_store.cc:276] 0709 13:14:41.342114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.342323 (+   209us) service_pool.cc:224] Handling call
0709 13:14:42.490198 (+1147875us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:42.490457 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329017216}
W20260709 13:14:42.486518 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:42.493780 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.244386 30670 leader_election.cc:400] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:14:42.507090 31396 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:42.509784 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 272697us
I20260709 13:14:42.511852 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:42.512889 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.514858 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.518097   429 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:42.520133   429 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:42.521517 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.523399 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.523743 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.524967 30701 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:42.526584 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.534395 30701 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 4. Current term is 5. Ops: []
I20260709 13:14:42.527140 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.527779   671 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:42.538262 30629 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 10. Ops: [8.6-8.6]
I20260709 13:14:42.539959 30628 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:42.541663 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:42.529920 30707 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:14:42.542528 30631 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:42.531263 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54413.2us
I20260709 13:14:42.544723   769 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.546026   769 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:42.565774   678 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:42.552656 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:42.531991 30554 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:42.556957 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:42.558115 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:42.560422 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:42.568295 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:42.576463   769 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.568171   429 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:42.577168 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:42.578779   847 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:42.579473 30626 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:14:42.552369 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:42.593391 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62851.2us
I20260709 13:14:42.601799 31391 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:42.613381 30698 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:42.617652 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:42.623060 31364 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:42.407191  1149 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:14:42.627887 30551 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:42.625968 31364 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:42.640736 30628 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:42.605830   937 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:14:42.664776 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:42.628265   678 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.670847  1043 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:14:42.686738 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.674010  1143 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.676481 31394 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:42.677521   987 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:42.680007  1145 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:42.627075   847 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.697252 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:42.672248 30550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.009s	sys 0.013s
I20260709 13:14:42.706238 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:42.707463 31396 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:42.632896   991 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 13:14:42.710927 30550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.010s	sys 0.013s
W20260709 13:14:42.714651 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64535.7us
I20260709 13:14:42.710253 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:42.728420  1160 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:42.719911   429 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:14:42.731227   769 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:42.731067  1143 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.731801   769 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:42.726120  1095 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.733557   769 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.733669  1095 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:42.734650   991 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:14:42.735280   991 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:42.735750   847 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:42.703722  1145 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:42.739286  1145 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:42.737892 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 72320.4us
I20260709 13:14:42.740258  1115 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:42.741509  1156 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:42.742895  1156 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:42.743342 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:42.744531  1156 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.745240 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:42.746950  1116 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:42.747566  1116 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:42.747998  1116 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:42.755061 30628 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:42.764488   678 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:42.767076 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55502us
I20260709 13:14:42.771780 31361 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:42.772603 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51827.1us
I20260709 13:14:42.778578  1115 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:42.779687  1115 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.778450   870 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:42.778600 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67890.3us
W20260709 13:14:42.792464 30553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.006s	sys 0.025s
I20260709 13:14:42.780881   991 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.795189   937 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:14:42.809137 30553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.007s	sys 0.026s
I20260709 13:14:42.801640   429 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.824115   991 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:14:42.826134  1122 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.509s	user 0.000s	sys 0.020s
I20260709 13:14:42.826917  1115 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.835201  1115 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:42.834851  1134 consensus_queue.cc:1175] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 10 status { last_received { term: 8 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:42.831032   429 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:42.838193  1095 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:42.838274   429 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:42.840157 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 288112us
I20260709 13:14:42.842083 30703 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:42.842782 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:42.844185  1015 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:42.845590   769 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:42.842933  1145 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:42.853945  1095 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:42.854794   770 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:42.855361   770 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:42.859261 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52777.7us
W20260709 13:14:42.865118 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47020.3us
W20260709 13:14:42.872053 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 127101us
I20260709 13:14:42.880954 30552 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:42.881578 30632 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:14:42.887907 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.854926  1095 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:42.892400 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:42.894699   679 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.636s	user 0.016s	sys 0.017s
I20260709 13:14:42.899325  1155 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:42.900054  1155 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:42.910257 30626 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:42.908113 31361 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:42.914816 30700 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:42.915024 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.916218 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:42.916687 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:42.917626 30631 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:42.919427 30557 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:42.926765 30698 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:14:42.927647  1030 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.614s	user 0.014s	sys 0.018s
I20260709 13:14:42.927795   679 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:42.928507 30706 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:42.929265   679 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:42.929793   679 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:42.930362   599 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.624s	user 0.007s	sys 0.021s
W20260709 13:14:42.935595 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62000.7us
I20260709 13:14:42.930716   897 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 13:14:42.933248 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:42.940615 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.944380 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:42.952165 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:42.947295 30626 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:42.947010 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:42.910164   937 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 13:14:42.930451  1116 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:42.937917  1143 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:42.962606 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:42.963871 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:14:42.964792  1006 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.611s	user 0.016s	sys 0.004s
I20260709 13:14:42.969907 30705 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:42.971570 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:42.985261  1162 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.179s	user 0.003s	sys 0.000s
I20260709 13:14:42.985939   847 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:42.987025 30702 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:14:42.986766   847 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:42.987913  1175 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:14:42.992384 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:42.978935  1143 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.007400 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.010895 30628 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:42.967593   679 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.012710 30625 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:43.013288   679 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:43.007982 30703 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:43.011408 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:42.948536  1155 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.024260  1155 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:43.010637 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:43.026791  1176 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:14:43.026973 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.041412   931 raft_consensus.cc:2764] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:14:43.032821 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.045077   991 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:43.046068   991 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:43.034116  1116 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.035457  1145 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.047343   897 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:43.038355   855 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:14:43.048731  1145 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.031636  1095 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.049504   855 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:43.050105   855 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:43.051246   855 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.051525  1145 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:43.034232  1172 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.183s	user 0.002s	sys 0.000s
I20260709 13:14:43.052701   855 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:43.055655   986 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.012s
I20260709 13:14:43.055266  1116 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.056499  1116 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.055176  1143 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.064973  1143 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.061748 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.000681   847 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:42.968277   770 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:43.070652   770 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:43.082115 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1085) took 1665 ms. Trace:
I20260709 13:14:43.082671 31396 rpcz_store.cc:276] 0709 13:14:41.463820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:41.587795 (+123975us) service_pool.cc:224] Handling call
0709 13:14:43.081828 (+1494033us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54000113d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:43.082031 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097614208}
I20260709 13:14:43.085886 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.086813 30702 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:43.081718 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:43.070888   897 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.073660  1116 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.094641 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.099913 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:43.109894 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 62719.8us
I20260709 13:14:43.141593   991 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:43.147491 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 172767us
W20260709 13:14:43.150038 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.150709 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.153789 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1079) took 1143 ms. Trace:
I20260709 13:14:43.153757 30630 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
W20260709 13:14:43.150183  1171 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.216s	user 0.004s	sys 0.000s
I20260709 13:14:43.155769 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:43.156765 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.158813   897 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:43.163331  1160 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:43.164533  1158 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:43.165048  1158 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:43.165429  1158 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.166644   771 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.160277 30553 rpcz_store.cc:276] 0709 13:14:42.007596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:42.007732 (+   136us) service_pool.cc:224] Handling call
0709 13:14:42.008153 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:42.008427 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:42.141399 (+132972us) log.cc:844] Serialized 14 byte log entry
0709 13:14:42.984256 (+842857us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:43.007786 (+ 23530us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18004a2d80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:14:43.111446 (+103660us) log.cc:844] Serialized 14 byte log entry
0709 13:14:43.121796 (+ 10350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:43.121911 (+   115us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:43.122457 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:43.122697 (+   240us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:43.150562 (+ 27865us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35530,"prepare.queue_time_us":282570,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":3704,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":14342,"spinlock_wait_cycles":16086784,"thread_start_us":849654,"threads_started":3,"wal-append.queue_time_us":638781}
I20260709 13:14:43.167308   771 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:43.161861 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:43.172422 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.179574 30556 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:14:43.174146  1095 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:43.180962  1095 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:43.182186  1095 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:43.184182  1095 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:43.186226   847 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.186484 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.176688 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.194846 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.196964 30553 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:14:43.186975   847 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.198872   937 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.187968   986 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
W20260709 13:14:43.200541 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 288483us
I20260709 13:14:43.200829   847 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:43.200838 30555 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:14:43.201838 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75005us
I20260709 13:14:43.202888 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:43.202965   937 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:43.204933   897 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:43.204830 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.205505   897 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:43.210819   897 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:43.211180   429 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:43.212111 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59991.1us
I20260709 13:14:43.213482 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.214079 30631 raft_consensus.cc:2410] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:43.206341   771 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:43.208593   678 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:14:43.192327 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.216193 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:43.216856 30554 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:43.217540   429 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:43.218000   429 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:43.218442   429 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:43.218808  1179 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:43.219779  1179 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:43.220402 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.220786  1160 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:43.227304 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.228742   771 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.218721 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.230742   937 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.232189   429 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.232666 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:43.233390 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:43.242121 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.242156  1155 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:43.243188  1155 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:43.244539  1155 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.242547   937 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:43.248521   937 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.249091   937 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:43.250765 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:43.251605 30703 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:43.251866 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:43.252240   429 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:43.256356   678 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:43.257033  1178 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:43.257601 30550 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:43.258080  1178 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:43.258852  1178 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:43.260538 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.260317  1178 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.261296 30706 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:14:43.263486 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50267.6us
I20260709 13:14:43.265211 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:43.266729  1179 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:43.267738  1179 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:43.267771   678 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:43.268324 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.271008 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:43.271819 30707 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:43.278096   678 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:43.283643 30706 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:43.284373 30706 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 3
I20260709 13:14:43.284739 30706 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:43.285169 30706 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:43.269421  1179 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.286957 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:43.218335 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.267865 30703 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:43.293363   991 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:43.323231   991 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:43.293150   771 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:14:43.323770   991 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:43.334954 30707 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:43.338248 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:43.340039   931 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.340328 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:43.341593 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:43.341945 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:43.357714 30698 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:43.345803 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.363111 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:43.376281 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.357980 30633 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:43.277256   897 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.405303 30632 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:43.402956 30629 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:43.354504  1143 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:43.414742 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 196586us
I20260709 13:14:43.416496 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.425308 30706 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:43.408840  1006 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:14:43.402801   931 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:43.463663 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:43.474726 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.478718  1179 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.478963  1178 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:43.479723  1178 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:43.479516  1179 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.480968  1178 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.482438  1179 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.484163   897 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:43.488672 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.498441 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:43.415380  1158 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:43.507368  1158 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.509905   931 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.518591 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:43.519376 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86951us
I20260709 13:14:43.519506 30552 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:43.520990 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.522938   769 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:43.524062   937 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.556571 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:43.559966 30707 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:43.570283 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49427.2us
I20260709 13:14:43.439342  1143 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.525713   429 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:43.578181 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.557077   931 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.534299   855 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:14:43.574020  1095 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.624573  1179 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.629324 31391 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:43.617581 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.600494   986 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:43.641856 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 310378us
I20260709 13:14:43.613730  1006 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.609530  1115 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:43.652977 30707 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:43.648343 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:43.617159   991 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:43.676716 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:43.636473  1095 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:43.694741 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.683534  1116 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:43.675122  1115 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.695629  1116 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:43.696810  1116 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.697803  1115 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.635790   931 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:43.643918  1179 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.700678   931 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.701488  1179 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.687372   937 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.677552   897 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:43.695158  1095 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:43.673624   986 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.658833  1006 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.706794  1006 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:43.709211   991 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.710139   991 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:43.686782  1192 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:43.712421  1192 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:43.695148 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 87043.1us
W20260709 13:14:43.716353 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.714883 30552 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:43.739706 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.740738 30558 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:43.721009   986 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:43.752950 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.759028 31361 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:43.714915  1143 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:43.762137 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.763612 31396 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:43.759544 31361 pending_rounds.cc:85] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 5
I20260709 13:14:43.771906 31361 pending_rounds.cc:107] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:14:43.772264 31361 raft_consensus.cc:2889] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:43.766928 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:43.769846 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:43.770473 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.768203 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:43.780239 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.779942   855 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:43.784027 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.783134  1194 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.776552  1145 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:43.776170   937 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.730360   991 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:43.809686 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:43.794742   855 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:43.811785 31388 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:14:43.812389 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.813077 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:43.812947 30553 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:43.826591 30553 raft_consensus.cc:606] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:43.827126 30553 raft_consensus.cc:614] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:43.813935 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.795498  1006 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:43.813004 31367 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:43.816229 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:43.823591 31388 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 3
I20260709 13:14:43.832738 31388 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:14:43.828210 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:43.784624  1192 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:43.813958 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.831147 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:43.831362 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:43.837001 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.833048 31388 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:43.834645 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:43.856953 31394 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:43.795003  1155 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:43.811975 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:43.875952 30554 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:14:43.887615 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56385.9us
I20260709 13:14:43.889143 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:43.895874 31396 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:43.806519  1145 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:43.898911 30632 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:43.860146 31364 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:43.901433 31394 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:43.875330   847 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:43.904634   847 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.832072  1006 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:43.908958 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:43.914355 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 130033us
I20260709 13:14:43.919045 30558 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:43.919981 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:43.920512 30558 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:43.922740 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.921248   897 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:43.955896 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.967859 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:43.907135   855 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:43.917501  1006 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:43.805507  1194 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.921773  1155 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:43.917182 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.899740   931 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:43.836783   937 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:43.983322 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:43.986862 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:43.978561 30557 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:44.001096 30553 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:43.995026 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.002517 30703 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:44.002147 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:44.004658 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:43.973440   897 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:43.972469 30523 connection.cc:256] RPC call timeout handler was delayed by -1.3447663990000365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2034s for a response
W20260709 13:14:44.005743  1200 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.008s
I20260709 13:14:44.034541 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.035457 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:44.008746  1194 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.028066 31378 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:44.038347 30705 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:14:44.035650 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:44.039939 30552 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:44.024098 31391 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:44.021783   897 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.036988 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.047425 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.052261 30707 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:44.043339   847 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:44.084326 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 268242us
I20260709 13:14:44.084647 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:44.059108   679 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:44.081405  1190 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 13:14:44.096138 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.022450  1145 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.060293   937 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:44.056763  1197 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:44.100871 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:44.119203   991 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:44.127048  1199 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:14:44.127228  1095 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:44.093668   679 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:44.142560   937 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.143242   679 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.143627   937 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.148105 31361 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:44.152616 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51843.8us
I20260709 13:14:44.118944  1145 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:44.162796  1179 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:44.163487  1179 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.164283  1095 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:44.165863  1179 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:44.171576  1095 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.146585   937 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:44.176045  1198 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.337s	user 0.000s	sys 0.001s
I20260709 13:14:44.177580 30554 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 10. Ops: [8.6-8.6]
W20260709 13:14:44.179420 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 73176.8us
W20260709 13:14:44.206521 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124294us
I20260709 13:14:44.207942 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:44.197556 31292 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 1.563s (SENT) [suppressed 11 similar messages]
I20260709 13:14:44.146879  1192 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:44.216707 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:44.218240 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:44.218878 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.219602 30707 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:44.209578   769 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:44.222520 31389 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:44.222698 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.225792 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:44.229980 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.238060 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.240232 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.244021 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:44.248497 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.231235  1205 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.225874  1155 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:44.251132 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.227320   769 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.241490  1194 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:44.241360   937 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.260146 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.236315  1192 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:44.283970 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86476.3us
I20260709 13:14:44.284081 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:44.259773  1194 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.255000  1222 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.257170   769 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.268934  1004 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:44.274493  1217 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.254935  1205 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.286666  1116 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.324038 30706 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:44.273173   937 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.254786  1197 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:14:44.269586  1136 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:14:44.300814  1192 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:44.304420 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.328374 30705 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:14:44.305598 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 401985us
W20260709 13:14:44.306882 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:44.307701 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 272332us
W20260709 13:14:44.308127 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.300480 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.303577  1179 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.315645 30558 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:44.266300  1006 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.316617  1160 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.317139   897 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.347836  1160 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.349486   897 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.350293   897 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:44.351032   897 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.351498   897 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:44.360293   847 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.321688  1205 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.291818  1194 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.403291 31361 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:44.316468 30551 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:44.341003 30552 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:44.406107 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.408166 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:44.409399   987 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:44.414498  1210 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.375s	user 0.004s	sys 0.004s
I20260709 13:14:44.343858 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:44.345628  1095 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:44.345979  1143 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:44.347002  1192 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:44.363811  1219 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:14:44.434461  1136 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:44.435542  1136 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:44.435889  1136 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:44.319000  1217 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.420995   847 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.436825  1136 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.438114  1136 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:44.438592   847 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.373255  1006 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.375465 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:44.377591   937 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:44.378528 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78172.4us
I20260709 13:14:44.380030 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:44.443323 30554 raft_consensus.cc:2368] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
W20260709 13:14:44.443586 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63710.6us
I20260709 13:14:44.380815 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:44.381776 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43790.9us
I20260709 13:14:44.445195 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:44.382697   931 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.400549  1179 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:44.310752  1222 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.431919   942 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:44.460795   931 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:44.432408  1095 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.433249  1143 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:44.488535 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.433787  1192 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:44.459419 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:44.490069 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.490446 30559 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:44.459968  1179 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:44.438243  1217 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:44.463130 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:44.491931 31391 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:44.459124   770 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:44.496110 31361 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:44.464804 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:14:44.473054 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 92331us
I20260709 13:14:44.463596 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:44.484129 30380 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 3 to 6, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 6 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:44.498836 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:44.502676 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:44.499593 31394 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:44.511615 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:44.517251 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 234436us
I20260709 13:14:44.420343   679 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.518424 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:44.508045 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:44.486505  1095 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:44.494445  1158 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:44.494663   770 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.494823   942 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:44.500440  1006 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.536084 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:44.548568 30559 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:44.525162   987 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.533181  1156 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.506822  1143 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:44.562013 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58701.2us
I20260709 13:14:44.560794 30673 leader_election.cc:400] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:44.563997 30673 leader_election.cc:403] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:44.527987  1160 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:44.543355   770 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.543725  1004 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:44.566646 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:44.555564   937 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:44.599188 30380 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 8, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 8 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:14:44.601699 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67700.5us
I20260709 13:14:44.602882 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.609903 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.575572  1043 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.551393  1006 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:44.576109   855 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:44.615890 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:44.619590 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53103.4us
I20260709 13:14:44.620447 31383 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:44.600432   931 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:44.587358   937 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:44.613907  1224 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.272s	user 0.001s	sys 0.001s
I20260709 13:14:44.566651  1156 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.601130  1222 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:44.619117  1217 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.622712  1158 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.565063   987 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.697024 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:44.697805 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 198962us
I20260709 13:14:44.612943  1179 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:44.619334   986 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:44.685627 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46935.6us
I20260709 13:14:44.664130   897 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:44.613459   942 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:44.685529 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51940.4us
W20260709 13:14:44.691499  1232 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:14:44.695463 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.626349  1155 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:44.632109 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.705000 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.641368   937 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:44.641806  1205 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:44.736402 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 191941us
I20260709 13:14:44.738507 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:44.740387 31388 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:44.614197  1043 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:44.741371 30631 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:44.742524 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:44.747583 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:44.665588  1030 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:14:44.719432  1116 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:44.719542   986 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.753573 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:44.750546  1233 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:44.738162  1205 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:44.805992 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:44.751550  1227 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:14:44.772044 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:44.711406  1158 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.767364   679 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:44.830742 31378 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:44.763743   986 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:44.848572 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 76621.8us
I20260709 13:14:44.834534  1205 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.832974  1122 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:44.871778 30630 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
W20260709 13:14:44.854202 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93047.7us
I20260709 13:14:44.865476 31361 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:14:44.866451 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 367729us
I20260709 13:14:44.888155 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.891805 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.892791 30626 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:14:44.896903 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 353750us
I20260709 13:14:44.854941  1222 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:14:44.913044 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 280978us
I20260709 13:14:44.913407 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.903805 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.915939 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:44.920208 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.920993 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.924252 30633 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:44.929289 30625 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:14:44.929703 30625 pending_rounds.cc:85] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 5
I20260709 13:14:44.929939 30625 pending_rounds.cc:107] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:14:44.930281 30625 raft_consensus.cc:2889] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:44.886456  1122 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:44.920131 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.894284  1178 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:44.898910  1145 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:44.953297 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.908217  1030 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.957726 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.960668 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.961030 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:44.905536  1222 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:44.942477  1122 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:44.961656 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:44.965335 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75687.4us
I20260709 13:14:44.941756   855 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:44.968130 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:44.865115  1179 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:44.973034 30554 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:44.975123 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:44.945693  1178 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:44.974404   855 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:44.977916  1203 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:14:44.978153   855 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:44.979429 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:44.980697  1239 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:14:44.982008  1247 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:44.982630  1247 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:44.983098  1239 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:44.983515  1043 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:44.983681  1217 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:44.983744  1239 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:44.951745  1143 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:44.954454  1145 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:44.928946   931 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:14:45.000262 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:45.006726 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 62107.5us
I20260709 13:14:44.986990  1006 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:44.994339  1043 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:44.968411  1030 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.005684  1229 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:45.024482  1196 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:14:45.007807   897 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:45.012734  1143 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:45.010874  1242 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:14:45.036679  1043 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:45.003201  1145 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.041706   847 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:45.081239   847 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:45.082388   847 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:45.090047  1145 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.092237  1156 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.092835  1145 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:45.094316   769 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.726s	user 0.012s	sys 0.001s
I20260709 13:14:45.049575 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.045397  1006 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:14:45.053282  1213 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:45.055536  1231 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.291s	user 0.000s	sys 0.002s
W20260709 13:14:45.058079  1194 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
I20260709 13:14:45.059760  1255 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:45.060523  1247 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.061015   897 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:45.064335  1030 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:45.065920  1095 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.000s
I20260709 13:14:45.066504 31388 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:45.033370   987 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.098826  1192 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:45.066033   855 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.049230 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:45.103909 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.041565  1229 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:45.002939  1252 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.106s
W20260709 13:14:45.156761 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 223397us
I20260709 13:14:45.128496 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:45.071489  1143 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:45.104338  1043 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.101272 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:45.133313  1232 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:14:45.133122  1195 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.171031 30554 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:45.039752  1222 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.131575   769 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.159381  1229 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:45.136116  1194 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.169842  1143 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.199411 30706 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:45.210734 30703 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:45.050559  1178 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:45.207324  1257 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 13:14:45.207698  1234 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 13:14:45.210062  1240 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:45.232808  1240 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:45.188058  1195 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.233901  1240 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.235396  1240 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:45.235435  1195 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:45.238212  1251 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.210229   855 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.147400  1095 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.238921  1251 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.241096  1095 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.241369  1251 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.242069 31383 raft_consensus.cc:2368] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12.
I20260709 13:14:45.205585  1245 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:45.218079 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 120063us
I20260709 13:14:45.228020 30550 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:45.243221  1245 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:45.244534 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.247136  1245 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.243196  1251 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.248865  1251 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.251730  1251 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.254385 30554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:45.255159 30554 raft_consensus.cc:606] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:45.255537 30554 raft_consensus.cc:614] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:45.263183 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.263553 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.264729 30555 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:45.140123  1213 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.268626  1239 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.272994  1213 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.275082  1265 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.275827  1265 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.279019  1235 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.279732  1235 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.278447  1265 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.287145  1217 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.290059  1267 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:45.282366  1235 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.293119 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1018) took 1152 ms. Trace:
I20260709 13:14:45.297214 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:45.299412 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.302580 30558 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:45.304417 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:45.298338   770 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.713s	user 0.006s	sys 0.005s
I20260709 13:14:45.329032  1242 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:45.327870 31364 rpcz_store.cc:276] 0709 13:14:44.141068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.145731 (+  4663us) service_pool.cc:224] Handling call
0709 13:14:44.306591 (+160860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:44.307090 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:44.307121 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:44.307704 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:14:44.477715 (+170011us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:44.477783 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:44.477888 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:44.892375 (+414487us) raft_consensus.cc:1685] finished
0709 13:14:44.892395 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:45.292727 (+400332us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002f8d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:45.293009 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1355,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":369082752,"thread_start_us":140345,"threads_started":1,"wal-append.queue_time_us":256930}
I20260709 13:14:45.328452 30556 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:45.331090 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:45.294384  1136 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.856s	user 0.011s	sys 0.004s
I20260709 13:14:45.331065 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.331012 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.331509 30551 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:45.332243 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.309556 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:45.329124   770 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.370857 30552 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:45.524171 30559 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:45.524637 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 363939us
W20260709 13:14:45.529927 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 221028us
I20260709 13:14:45.531134 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:45.533399 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:45.536010 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.547260 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:45.548967 30558 raft_consensus.cc:2376] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7. Re-sending same reply.
I20260709 13:14:45.551419 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.536934 30554 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:45.567039 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:45.575687 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:45.620677  1213 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.621179  1266 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.621505  1213 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.621907  1266 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.624234  1213 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.624583  1266 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:45.620786 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 95155.5us
I20260709 13:14:45.630534 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.630820  1235 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.631471  1235 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.633813  1235 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:45.635046 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:45.635779 30703 raft_consensus.cc:2376] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8. Re-sending same reply.
I20260709 13:14:45.636956 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.637941 30626 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:45.637786 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.634385  1235 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.640224  1253 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:45.640853 30703 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:45.641131 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.641291  1253 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:45.641872 30706 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:45.639778  1235 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.643111 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:45.601193 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.534343 30553 raft_consensus.cc:2376] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9. Re-sending same reply.
I20260709 13:14:45.581725 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:45.655588 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 74004.9us
W20260709 13:14:45.655686 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.011s	sys 0.078s
W20260709 13:14:45.657752 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.011s	sys 0.078s
W20260709 13:14:45.660144 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 688748us
I20260709 13:14:45.662708 30556 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:45.580963  1195 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:45.582033  1095 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:14:45.661697  1122 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.003s
W20260709 13:14:45.667001  1160 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.028s	sys 0.072s
I20260709 13:14:45.674139 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.568938  1006 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.332752  1217 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.621196  1030 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.679759 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.673481 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:45.680222 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79362.4us
I20260709 13:14:45.588471  1239 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:45.681489 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:45.693367 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.692706 30556 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:45.570883   986 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.001s
I20260709 13:14:45.684957 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.685750 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.687649 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.671911  1266 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:45.690809 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.691989 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:45.708472 30559 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:45.686096  1160 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.029s	sys 0.072s
I20260709 13:14:45.682281 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.696978  1244 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.697297 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.716535 30552 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:45.698807  1263 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.686939  1255 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.699766  1273 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:14:45.689402  1006 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:45.700938  1235 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:45.701978  1239 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:45.691095   937 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.930s	user 0.012s	sys 0.002s
I20260709 13:14:45.679445  1095 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.679772  1195 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.706223 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:45.709539 30553 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:45.710114  1277 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.710711  1249 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:45.711340  1274 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:45.713748 30704 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:45.722095 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:45.715346   986 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.724864 30556 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:45.704006   770 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.714920  1194 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:45.740109  1195 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:45.727444 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:45.757797 30552 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:45.759233  1276 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:45.759575  1006 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:45.759974  1276 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:45.760505  1205 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.869s	user 0.009s	sys 0.004s
I20260709 13:14:45.708134  1266 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.720774  1192 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:45.733752 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.735183 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.735368 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.707151  1030 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.737546 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.739789 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.734879 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.740751 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.779666 31388 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:45.688726  1122 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.751194 30632 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:45.752507 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:45.761143  1205 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.748767 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:45.729696   769 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:45.731613   937 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.756078  1251 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:45.734193  1095 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:45.766712 30633 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:45.770048 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1116) took 1359 ms. Trace:
I20260709 13:14:45.802510 30555 rpcz_store.cc:276] 0709 13:14:44.410550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.411292 (+   742us) service_pool.cc:224] Handling call
0709 13:14:44.411648 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.282894 (+871246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.769703 (+486809us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b540052add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:45.769966 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957128960}
I20260709 13:14:45.771248  1228 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.804279  1228 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.761734  1239 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:45.753208 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47088.5us
I20260709 13:14:45.745704  1277 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.766655  1266 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:45.724367  1255 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:45.787582 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.796373 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:45.799290 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46890.2us
I20260709 13:14:45.751883  1249 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.724862  1263 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.828207 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1121) took 1198 ms. Trace:
I20260709 13:14:45.828753 30550 rpcz_store.cc:276] 0709 13:14:44.629639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.629766 (+   127us) service_pool.cc:224] Handling call
0709 13:14:44.630017 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:45.243786 (+613769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:45.827911 (+584125us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540052b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:45.828136 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372816640}
I20260709 13:14:45.723485  1244 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:45.829334 31396 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:45.831696 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:45.811685  1043 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.628s	user 0.000s	sys 0.012s
I20260709 13:14:45.807709 30707 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:45.782742 30559 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:45.819844 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:45.823433 30705 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:45.807412  1192 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
W20260709 13:14:45.826725  1143 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.004s
I20260709 13:14:45.832648 31391 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:45.779666 30628 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:45.794162  1274 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:14:45.808135  1235 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:45.839927 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.860633  1235 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:45.861027 30706 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:45.849221  1158 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:45.849561 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.863381 30700 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:45.760977  1276 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.843325 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:45.853989 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:45.871470 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.870991  1095 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:45.874589  1255 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:45.875092 31364 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:45.813781  1273 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:45.878726  1273 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:45.883345  1143 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.859766   769 raft_consensus.cc:2764] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
W20260709 13:14:45.852840  1229 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:14:45.861241  1235 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:45.715665  1242 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.809748  1275 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:45.894580 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:45.816272  1228 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:45.892825 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.899748 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1180) took 1266 ms. Trace:
I20260709 13:14:45.900290 30630 rpcz_store.cc:276] 0709 13:14:44.633005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.633176 (+   171us) service_pool.cc:224] Handling call
0709 13:14:45.899533 (+1266357us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b54009434d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:45.899675 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399962880}
I20260709 13:14:45.901753 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:45.891008  1229 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:45.904126 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1170) took 1166 ms. Trace:
I20260709 13:14:45.904582 31383 rpcz_store.cc:276] 0709 13:14:44.738296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.753193 (+ 14897us) service_pool.cc:224] Handling call
0709 13:14:45.903898 (+1150705us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:45.904061 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106327296}
I20260709 13:14:45.905894 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.907554 31378 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:45.902405 30630 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:45.913921 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.915099 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:45.888077  1043 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:45.915125 31383 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:45.932554 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:45.936475 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 93312.8us
I20260709 13:14:45.937527 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:45.940958 30629 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:45.955842  1287 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:45.956465  1287 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:45.956925  1287 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:45.961195 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1177) took 1374 ms. Trace:
I20260709 13:14:45.961781 30631 rpcz_store.cc:276] 0709 13:14:44.587285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.589171 (+  1886us) service_pool.cc:224] Handling call
0709 13:14:45.960941 (+1371770us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:45.961114 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270316800}
I20260709 13:14:45.969990 30630 raft_consensus.cc:2410] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:14:45.989240 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.004547 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.006839 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.008008 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:45.985632  1253 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:46.030315 30550 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:46.035772 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.036984 30552 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 5: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:46.038527 30555 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:14:46.035460 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98015.6us
I20260709 13:14:46.047760 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.027073  1275 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:46.019327  1259 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:46.055363 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.057621 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1156) took 1021 ms. Trace:
I20260709 13:14:46.034072  1274 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:46.028754  1251 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.059715 30559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:46.065891 30559 raft_consensus.cc:606] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:46.066380 30559 raft_consensus.cc:614] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:46.061352 30705 rpcz_store.cc:276] 0709 13:14:45.036750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.036944 (+   194us) service_pool.cc:224] Handling call
0709 13:14:46.057389 (+1020445us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b540032ad54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:46.057556 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620429696}
I20260709 13:14:46.063736 30702 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:46.032388   769 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.043242  1255 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.052336  1006 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.076996 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.078596 30699 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:46.075078  1216 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.074972  1263 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.069330  1277 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.103693  1290 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:14:46.104053 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.104770 30699 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:46.105072 30522 leader_election.cc:400] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12.
I20260709 13:14:46.105758 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:46.111331 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.096495  1267 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.065897  1259 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.098024  1275 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.098165  1288 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:46.072954   769 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.072220  1251 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.110595  1116 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:46.091904  1255 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:46.131393 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54520.2us
I20260709 13:14:46.135215 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.116806  1043 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.100955  1006 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.127972  1274 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:46.101840  1216 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.186866  1255 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.188481  1216 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.129051  1253 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.184149  1251 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.184372  1006 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:46.195693  1289 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:14:46.208981 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.141813  1290 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:46.209789  1290 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:46.210764  1290 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.212046  1290 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:46.143445  1229 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:46.185595 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50494.5us
I20260709 13:14:46.172015  1276 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.215989  1276 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.195190  1294 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:46.184558  1259 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.219475 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.127872  1288 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.127672  1275 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.198799  1244 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.249053  1235 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.199774  1249 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.123999  1267 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.196115  1253 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.196672   769 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.202893 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.201748  1266 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.241081 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.233150  1276 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.246583 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.248338  1275 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:46.198995  1286 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:14:46.262743 30628 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:46.245244  1259 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.271288 30701 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:46.289750 30699 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:46.307476 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:46.308300 30628 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.295295  1288 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.291121  1266 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.293151  1267 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.274575  1222 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:46.289014  1253 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.320894 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.328140 30628 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:46.321856 31389 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:46.321936 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.330211 30702 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:46.299050  1259 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.335580 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.321501  1235 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.343875  1122 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.344779 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.345731   769 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:46.345844 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:46.346395   769 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.346853   769 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:46.347599 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.345995   986 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.351191 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.338080  1276 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.357023 31389 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:46.358110  1244 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:46.364380  1244 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.364789  1244 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:46.358573  1194 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.368460 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.369884 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.354939  1266 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.371331  1259 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.371208  1266 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.373576  1266 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.374912 30702 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:46.362165  1276 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.376183  1276 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.380020   937 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.385156 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.386111 30550 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
I20260709 13:14:46.370468 30701 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:46.390992 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.391696  1287 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.392920 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.396068 31389 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:46.394171 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:46.394907  1195 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.009s
I20260709 13:14:46.395068 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.391791 30550 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:46.396943 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:46.398030 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.405804 30559 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:46.405951 30555 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
W20260709 13:14:46.407294  1297 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:14:46.407795 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.408998 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.409595 30704 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:46.410175 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.413319 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.402657 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:46.416242  1236 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.416712  1236 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.417053  1236 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.410517 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.418668 30704 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:46.419824 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.420444 30704 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
I20260709 13:14:46.421442 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.407331 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.406118 30556 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:46.426883 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 134) took 12499 ms (client timeout 15000 ms). Trace:
I20260709 13:14:46.400355 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.399492 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:46.428220 30553 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:46.428442  1249 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.430802 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.431805  1115 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.429116  1249 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.432340  1115 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.434047  1262 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:46.435904 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.436545 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.437218 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.447393  1194 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:46.447597  1276 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.453177 30705 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:46.449509 30553 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:46.434507  1115 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.430863  1006 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:46.434567  1262 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.461694   986 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:46.470863  1249 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:46.427430 30378 rpcz_store.cc:269] 0709 13:14:33.915021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:33.934049 (+ 19028us) service_pool.cc:224] Handling call
0709 13:14:46.403702 (+12469653us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:42.652842 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:42.751125 (+ 98283us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:42.751207 (+    82us) write_op.cc:435] Acquired schema lock
0709 13:14:42.751255 (+    48us) tablet.cc:662] Decoding operations
0709 13:14:42.753685 (+  2430us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:42.753782 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:42.753823 (+    41us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:14:42.995812 (+241989us) tablet.cc:701] Row locks acquired
0709 13:14:43.010016 (+ 14204us) write_op.cc:260] PREPARE: finished
0709 13:14:43.071181 (+ 61165us) write_op.cc:270] Start()
0709 13:14:43.071388 (+   207us) write_op.cc:276] Timestamp: P: 1783602883018538 usec, L: 0
0709 13:14:43.071413 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:14:43.124361 (+ 52948us) log.cc:844] Serialized 8623 byte log entry
0709 13:14:43.810719 (+686358us) op_driver.cc:464] REPLICATION: finished
0709 13:14:44.059176 (+248457us) write_op.cc:301] APPLY: starting
0709 13:14:44.059286 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:44.059745 (+   459us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:44.059899 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:44.130508 (+ 70609us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:44.131143 (+   635us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:14:44.131207 (+    64us) write_op.cc:312] APPLY: finished
0709 13:14:44.131858 (+   651us) log.cc:844] Serialized 151 byte log entry
0709 13:14:44.133840 (+  1982us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:44.134179 (+   339us) write_op.cc:454] Released schema lock
0709 13:14:44.134263 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:44.134897 (+   634us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:44.134926 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1178054,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":234276,"apply.run_cpu_time_us":7113,"apply.run_wall_time_us":75699,"num_ops":22,"prepare.queue_time_us":146776,"prepare.run_cpu_time_us":35145,"prepare.run_wall_time_us":778124,"raft.queue_time_us":118851,"raft.run_cpu_time_us":62522,"raft.run_wall_time_us":106489,"replication_time_us":739054,"thread_start_us":405222,"threads_started":4,"wal-append.queue_time_us":430489,"wal-append.run_cpu_time_us":9564,"wal-append.run_wall_time_us":1566089}]]}
I20260709 13:14:46.485270   986 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:46.471164 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.485764   986 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:46.485980  1030 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.486146 31389 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:46.394215  1287 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.456387  1267 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:14:46.486791   986 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.487354 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.487797  1267 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:46.488072 31389 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:46.488210  1267 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:46.489148 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:46.445597  1235 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:46.491483  1235 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.491526 31389 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:46.491916  1235 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:46.450229  1255 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:46.474386  1194 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:46.494902  1255 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.495383  1255 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:46.495102  1194 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.497426  1194 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:46.481752  1262 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:46.475246 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.479327  1252 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.499148  1252 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.500301 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.501528  1252 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.481060  1006 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:46.503178  1006 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:46.506965 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.507655 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.508379 30629 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:46.508631  1217 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.500368   599 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:46.509608 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.509742   599 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.510298   599 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:46.510468 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.513806 30632 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:46.444768  1195 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.483745 30625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54da309ba151423ab2739966b9caebd0"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:46.490191 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.515419 30625 raft_consensus.cc:606] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:46.515877 30625 raft_consensus.cc:614] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:46.516543 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.516949  1195 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.517560 31396 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:46.517818 31388 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:46.436110   769 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.518795  1004 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:46.519244  1004 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.519577 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.519627  1004 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:46.520622   769 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.521341   769 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:46.521706   769 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.522064   769 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:46.499320   986 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:46.524364  1244 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:46.527980 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1499) took 1564 ms (client timeout 1604 ms). Trace:
W20260709 13:14:46.528535 30707 rpcz_store.cc:269] 0709 13:14:44.963707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.963893 (+   186us) service_pool.cc:224] Handling call
0709 13:14:46.527723 (+1563830us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54003d27d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:46.527878 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912317824}
I20260709 13:14:46.529824 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.530508 30707 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:46.483144 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:46.489171  1267 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:46.509783  1095 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.542018 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.517623 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:46.538057 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.477425 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.535535 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:46.524358 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.540980 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.544059 30705 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:14:46.543690 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60692.3us
W20260709 13:14:46.483072  1230 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.332s	user 0.004s	sys 0.020s
I20260709 13:14:46.544275 30699 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:46.547842 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:46.547681  1249 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:46.549917  1249 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.550144  1194 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:46.547889 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.550684  1194 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.551026 30552 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:46.552315  1267 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:46.551080  1194 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:46.554117  1115 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.554574  1115 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.554725  1252 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:46.554991  1115 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.555303  1252 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.555758  1252 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.552523 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.559119  1030 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.548003  1265 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:46.559566  1030 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.560177  1030 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.539516  1259 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.561054  1259 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.563007 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:46.546339  1230 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.398s	user 0.004s	sys 0.020s
I20260709 13:14:46.530836  1004 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.472239  1236 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.563928  1259 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.550359  1249 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:46.581357  1158 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.582000  1158 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.567526 30699 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:46.532438  1205 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:46.568043  1006 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.542793  1251 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.545491 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.559641 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.562460  1194 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.577162 30553 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:46.633423 30706 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:46.548636 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:46.619988 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71677.2us
I20260709 13:14:46.620684 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.624176 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.636678 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:46.635663 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76101.3us
I20260709 13:14:46.640558 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:46.653270 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 107882us
I20260709 13:14:46.660526 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.661645 30701 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.619653  1205 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:46.654091 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.669749  1294 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:46.670702  1295 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:46.672646 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:46.628324  1265 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:46.670990  1294 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:46.670698 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 429749us
I20260709 13:14:46.679425 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.675923 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:46.643364  1122 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:46.644891  1259 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:46.652616  1245 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:46.739491 30673 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.5.190:45121, user_credentials={real_user=slave}} blocked reactor thread for 48105.3us
W20260709 13:14:46.742798 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 103691us
I20260709 13:14:46.745976 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:46.668044  1266 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.663631  1273 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.677642  1244 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:46.781906 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 114779us
W20260709 13:14:46.757804 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 157394us
I20260709 13:14:46.633103  1251 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.688532 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:46.788349 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:46.685105  1155 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:46.790999 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:46.693382  1276 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:46.794445 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 106006us
W20260709 13:14:46.798308 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 152001us
I20260709 13:14:46.677158  1043 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:14:46.668478  1303 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:46.686055  1265 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:46.766461  1306 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:46.812922 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 158904us
I20260709 13:14:46.783141  1245 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:46.779404  1266 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:46.820325 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:46.694231  1270 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 13:14:46.832645 30698 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:46.805902   991 raft_consensus.cc:2764] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:46.799273  1136 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:46.805539  1303 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:46.816725  1276 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:46.828825 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.809010  1155 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:46.837268 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 157905us
I20260709 13:14:46.822577  1266 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.824028  1158 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.865906 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:14:46.867583 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 121743us
I20260709 13:14:46.870783 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.871109 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:46.887436 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:46.812521  1122 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:14:46.838963  1281 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:46.838451  1236 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.838415  1136 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:46.835114  1255 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:46.818713  1306 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:46.904199 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75515.7us
W20260709 13:14:46.908196 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93479.9us
I20260709 13:14:46.847119  1303 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:46.898339 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.907608 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.903434  1136 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:46.867123  1235 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:46.915732  1004 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.967506  1006 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.123364 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:47.035331  1194 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:46.910347  1313 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:46.908365  1281 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:47.104399  1249 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.524s	user 0.009s	sys 0.005s
I20260709 13:14:46.872628   929 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:47.021883  1155 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:47.104633 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 195489us
I20260709 13:14:47.149634 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.150362 31389 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.150491 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:46.968169  1259 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:47.106022  1251 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:46.841338  1308 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:14:47.158967 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.106341  1306 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.160477 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.162075 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.162982 31378 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:47.155437  1259 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:47.164554  1259 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:46.892874  1179 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.109114  1255 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.109148 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:46.912312  1275 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:47.168035 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59003.8us
I20260709 13:14:46.915313  1310 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:47.168726  1310 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:47.169109  1310 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:46.901613  1192 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:46.948731  1258 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:47.175081  1258 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.175510  1258 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:46.860428  1245 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.175808 31378 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:47.104399 30707 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:47.111029 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:46.889640  1242 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:14:46.865146  1230 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.103s
W20260709 13:14:47.112723 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 207096us
I20260709 13:14:47.114425  1273 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:47.125056 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:47.109977  1262 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.004s
I20260709 13:14:47.183555 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:47.128490 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:47.133273 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:47.133901 30698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 435) took 1492 ms (client timeout 1936 ms). Trace:
I20260709 13:14:47.134744  1313 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.114235  1195 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 13:14:47.138355  1178 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:47.139019  1319 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.139578  1143 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:47.141456  1252 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.585s	user 0.018s	sys 0.002s
W20260709 13:14:47.144794  1030 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.000s
I20260709 13:14:47.145920 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:47.114176  1307 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.272s	user 0.004s	sys 0.000s
W20260709 13:14:47.146082 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 479313us
I20260709 13:14:47.153002 30699 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:14:47.152716 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.160532 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.230170 31361 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:47.157289  1194 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.168195  1251 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:14:47.107971  1115 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.008s
I20260709 13:14:47.169427 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:47.173835 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:47.178447 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67531.1us
W20260709 13:14:47.249177 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79826.9us
I20260709 13:14:47.180387 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:47.183738 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58823.1us
W20260709 13:14:47.250362 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70023.7us
I20260709 13:14:47.182857 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.190572   847 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:47.190096  1235 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.186447 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:47.185650 30698 rpcz_store.cc:269] 0709 13:14:45.641169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.642743 (+  1574us) service_pool.cc:224] Handling call
0709 13:14:47.133716 (+1490973us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540078bc54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:47.133837 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607706368}
I20260709 13:14:47.182603  1192 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.260419 30630 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:47.186199   929 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:47.142026  1281 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:47.182675  1218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.022s	sys 0.015s
I20260709 13:14:47.155812  1249 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.200518  1179 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:47.227774 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:47.321923  1320 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 13:14:47.230643 31389 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:47.323014  1136 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.323675 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.324401 31389 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:47.325351 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.229759 31383 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:47.220187  1195 consensus_queue.cc:1243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:47.219532  1287 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:14:47.232434 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:47.329326 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96976.6us
I20260709 13:14:47.329727  1304 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:47.330333  1304 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:47.330858 30706 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:47.233299  1313 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.199795  1095 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 13:14:47.199188  1262 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.331547 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.332427 30624 raft_consensus.cc:2368] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11.
I20260709 13:14:47.332427 30704 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:47.333433  1262 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.236611  1158 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:47.334344  1287 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:47.334797  1287 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:47.334918  1235 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:14:47.335731  1287 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:47.335094  1158 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:47.239344  1217 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:47.325378  1249 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.337350  1158 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.337563  1217 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:47.337862  1158 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:47.338555  1217 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.339676  1217 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:47.234093  1030 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.341010   929 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:47.342094   929 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.342720  1030 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.239634  1194 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:47.345733  1315 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.217s	user 0.000s	sys 0.000s
I20260709 13:14:47.264766  1143 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.348348 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.249279 31388 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:14:47.185696 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57330.6us
I20260709 13:14:47.255339 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:47.351524 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:47.352051 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 96821.3us
I20260709 13:14:47.355171 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.356263 30624 raft_consensus.cc:2368] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:47.250828 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:47.257954 31364 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:47.254052 31361 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:14:47.356992 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106236us
I20260709 13:14:47.357713 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:47.357861 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.257268  1308 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:47.262418 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.262552 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:47.263504 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.359529 31367 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:47.258915  1115 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.262755  1273 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:47.368151 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.267704 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.369616 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.269178 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.257625  1242 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:47.254055  1251 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:47.261261  1228 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:47.215327  1319 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.334188 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.331421 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.419615 30707 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.273495  1265 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.434020 30554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:47.435020 30554 raft_consensus.cc:606] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:47.435467 30554 raft_consensus.cc:614] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:14:47.437680 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.325423  1136 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.367434  1277 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:47.369697 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.395275 30632 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:14:47.230546  1252 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.443457 30632 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 6
I20260709 13:14:47.443922 30632 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:47.444193 30632 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:47.379590  1230 raft_consensus.cc:2764] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:14:47.402066 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.449936 30705 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:47.402277 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:47.403683 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.238201  1266 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.255555 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:47.320569  1218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.037s	sys 0.025s
I20260709 13:14:47.456215  1267 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:47.456743  1267 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:47.406966  1197 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:14:47.457711  1267 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.461143 30624 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:47.462597 30557 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:47.407961 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:47.447425   599 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.937s	user 0.020s	sys 0.000s
I20260709 13:14:47.463941  1252 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.464499   599 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.466753 30555 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12.
I20260709 13:14:47.416157 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.348599  1143 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.422996 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.471992 30703 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 2: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:47.472342 31388 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:47.238781 30525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.003s	sys 0.016s
W20260709 13:14:47.472827 30525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.004s	sys 0.016s
I20260709 13:14:47.423753 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.429284 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:47.430104 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.431011 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.474139 30698 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:47.430635 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.474880 30626 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.476006 30702 raft_consensus.cc:2368] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:47.476761 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.479882 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.441236 30707 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:14:47.243454   769 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.008s
I20260709 13:14:47.395479  1306 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:47.483201 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.395085  1255 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:47.378968  1308 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:47.409683  1251 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:47.256781   847 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.406756  1275 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.531085  1306 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:47.451812 30701 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:14:47.532490 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.532296  1306 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.533866 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:47.405241  1030 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.468593 31383 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:47.530687   769 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.469449 30551 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:47.444895  1179 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:47.534583 30701 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:47.535422 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.537842 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.337179  1287 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:47.431432 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:47.482012 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.539276 31394 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.539911 31383 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:47.540946 30558 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:14:47.540997 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.530795 30626 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:47.483937 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:47.486717 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.487346 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.488139 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:47.467324  1326 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:47.494182  1311 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:47.494920  1314 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:47.427229  1095 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 13:14:47.500725  1004 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:47.503150 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.551668 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.552485 30556 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:47.552940 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:47.553706 30553 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:47.502921  1308 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:47.508100  1194 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.512990  1319 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.520751  1267 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:47.483914 30629 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:47.447710  1228 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:47.531344 31294 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12.
I20260709 13:14:47.453212  1277 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.536356 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:47.444496   986 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.921s	user 0.006s	sys 0.009s
I20260709 13:14:47.331347  1304 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.546015 30698 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:47.548044 30554 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:47.548796 30627 raft_consensus.cc:2368] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12.
I20260709 13:14:47.601158 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1516) took 1143 ms. Trace:
I20260709 13:14:47.601603 30698 rpcz_store.cc:276] 0709 13:14:46.457585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.599843 (+1142258us) service_pool.cc:224] Handling call
0709 13:14:47.600486 (+   643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.600740 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.600777 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:47.600838 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:47.600883 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.600953 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.601086 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:47.421257  1242 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.549248 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.551625 30701 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:47.603871 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1514) took 1158 ms. Trace:
W20260709 13:14:47.604837 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.554582 30631 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:47.555372 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.539480  1274 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.604187 30701 rpcz_store.cc:276] 0709 13:14:46.445092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.551026 (+1105934us) service_pool.cc:224] Handling call
0709 13:14:47.551393 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.603624 (+ 52231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.603815 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:47.593870 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 69015.9us
W20260709 13:14:47.557297  1303 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.004s
I20260709 13:14:47.613288 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.615368 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.616153 31364 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:47.561048  1252 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:47.602105 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:47.535588   847 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:47.534906  1030 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.596241 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.616385 31388 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:47.782970 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186801us
I20260709 13:14:47.783810 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:47.787286 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.620957 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1515) took 1174 ms. Trace:
W20260709 13:14:47.649242 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47201.8us
I20260709 13:14:47.658099  1304 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:47.815765 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:47.556052  1314 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.761365 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:47.697737 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.755580  1043 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:47.734985  1228 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:47.765259 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:47.746254 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.752100 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:47.735251  1281 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:47.827086 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.618875  1303 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.752696  1030 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.593357  1326 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:47.599045  1259 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.618953 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.620960  1274 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.759841  1252 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:47.777000 31394 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:47.559118  1004 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:47.618585  1295 consensus_queue.cc:1175] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 10 status { last_received { term: 8 index: 4 } last_committed_idx: 4 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:47.736464 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.844081 31364 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:47.787966  1319 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:47.788511   929 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:47.788946  1265 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:47.788877  1277 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.778904  1006 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:47.562309 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.795639 30706 rpcz_store.cc:276] 0709 13:14:46.446676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.596912 (+1150236us) service_pool.cc:224] Handling call
0709 13:14:47.620176 (+ 23264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.620471 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.620511 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:47.620578 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:47.620640 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:47.620718 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:47.620872 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:47.830399  1245 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:47.820713 30627 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:47.851338 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:47.780397  1310 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.006s
W20260709 13:14:47.825517 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64257us
I20260709 13:14:47.865232 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:47.828095 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:47.731415 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 205955us
I20260709 13:14:47.551772  1311 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.834187 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.836087 30552 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:47.836908 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.840090 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:47.879781 30550 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:47.772303  1329 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:47.820114   897 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.820789  1143 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.668659  1179 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.847177 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.870510 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:47.850131 30625 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:47.889858  1245 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:14:47.851338 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.852661  1004 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:47.835908  1155 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:47.853183  1006 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:47.847975   929 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.856442  1259 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:47.819779 31388 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:47.843041  1194 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.868916 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:47.885001   599 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:47.907076 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.872447 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.854166  1302 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:14:47.883265 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46438.5us
W20260709 13:14:47.890969  1258 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.715s	user 0.015s	sys 0.000s
I20260709 13:14:47.909088  1258 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.909744 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:47.836577  1043 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.880256  1251 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.912499  1251 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:47.913335  1155 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:47.913787  1155 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:47.914114  1155 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:47.914765 30552 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:47.915937 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.916427 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.917404  1217 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:47.917990  1217 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:47.918788  1194 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.919230  1217 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:47.921147  1194 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.890739  1250 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:47.922806  1237 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.923005  1250 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:47.923460  1237 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.925082 30552 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:47.926115  1237 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:47.926245 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:47.885042 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:47.927420 30552 raft_consensus.cc:2410] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:14:47.927296 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:47.928328   599 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:47.928797   599 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:47.929617 30550 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:14:47.929720   599 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.930244 31383 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:47.876297   986 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:47.895726 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:47.891920 31364 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:47.947042 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.947280  1242 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:47.948082 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.948321 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.895726   769 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.846684   847 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:47.898030 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:47.933550  1321 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.586s	user 0.003s	sys 0.000s
W20260709 13:14:47.891834 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56344.1us
I20260709 13:14:47.880249  1275 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:47.841434  1281 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:47.892711   897 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.960083  1277 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:47.963842  1277 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:47.964457  1277 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:47.964522   897 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.965454   986 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:47.966813   986 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.958542 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:47.883881   937 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:14:47.831465  1255 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:47.875741  1310 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:47.931378  1158 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.003s
I20260709 13:14:47.932847  1006 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:47.887665 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:47.867833 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 122137us
I20260709 13:14:47.939206 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:47.941710 30550 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:47.942549 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:47.942694 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.893041 30706 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:47.950947 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53047.6us
I20260709 13:14:47.994755 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:47.847771  1252 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:47.902899   929 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:47.907015  1245 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:47.905078  1325 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.968248 30626 raft_consensus.cc:2368] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9.
I20260709 13:14:48.004359 31367 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:47.973116 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:47.912500  1043 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:47.978096 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:47.977620 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:47.982465  1255 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:48.014528 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:47.889744  1329 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:47.991997 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:48.003830 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:47.972054 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:48.009951 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67522us
I20260709 13:14:48.016434 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:48.019389 31378 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
W20260709 13:14:48.018992 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.022805 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.027896 31388 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:48.020498 30707 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:47.992621   897 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:47.966852  1275 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.018296  1194 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:48.033933 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.006264  1287 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:48.007642   769 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:48.040184 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:48.041870 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.003931  1158 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.047190 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.014863  1030 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:48.013959  1259 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:48.013074 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 158928us
I20260709 13:14:48.048007 31378 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:48.048918 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.051405 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.010320  1143 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:48.017455  1255 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:48.017688  1237 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:48.069947 31388 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:48.031433  1319 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:48.076391 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.087713 30624 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 11. Ops: []
I20260709 13:14:48.047425   937 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:48.089345 30706 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:48.059578  1194 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:48.100219 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.102722 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.009390  1325 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.081020  1143 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.109313 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:47.921696  1326 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.102048 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:48.124434 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112939us
I20260709 13:14:48.125475 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:48.039325   897 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.100319  1004 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:48.088770  1237 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.114949  1194 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.115608 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:48.135337 31378 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:48.136713 30706 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:47.923976  1250 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.140024 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.094084  1338 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:48.076800   769 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:48.126871 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.062162  1287 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.097810  1043 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:48.076275  1259 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.138401 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:48.178915 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 158874us
I20260709 13:14:48.158367 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.130270  1265 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:48.164917 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:48.106277   937 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.180963 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.181593 31378 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:48.182313 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:48.185235 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47718.4us
I20260709 13:14:48.181594 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 449) took 1323 ms. Trace:
I20260709 13:14:48.182384   937 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.191220 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:48.192361 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.146787  1338 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:48.187829 30701 rpcz_store.cc:276] 0709 13:14:46.857897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.621423 (+763526us) service_pool.cc:224] Handling call
0709 13:14:47.621657 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.621812 (+   155us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:14:47.904176 (+282364us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.181188 (+277012us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.181257 (+    69us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:48.181311 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.181394 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.181531 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":395708,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":8413,"thread_start_us":400242,"threads_started":3}
I20260709 13:14:48.177826  1245 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:48.134229  1004 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.186323  1270 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:48.159878  1339 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:48.208922 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:48.213536 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:48.187454  1306 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:48.186843  1217 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:14:48.215107  1334 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:14:48.210875  1345 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:48.206981 31378 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:48.193797  1314 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.250145 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.194205  1311 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:48.264147 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73080.1us
I20260709 13:14:48.189677  1265 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.257599  1158 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.245877  1345 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:48.271332  1345 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:48.271996  1004 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.272866 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:48.273635 30706 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:48.187341  1043 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.274768  1323 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:48.275266  1323 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:48.276320  1043 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.276090  1323 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.277401  1323 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:14:48.277856  1308 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.000s
I20260709 13:14:48.248154  1252 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.265234 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:48.240804  1343 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:48.242147  1310 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.270712  1275 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:48.272979 30704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1503) took 1919 ms (client timeout 2036 ms). Trace:
I20260709 13:14:48.278101  1354 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:48.290057 30704 rpcz_store.cc:269] 0709 13:14:46.353885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.422390 (+ 68505us) service_pool.cc:224] Handling call
0709 13:14:48.272729 (+1850339us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b540008be54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.272889 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056895744}
I20260709 13:14:48.279608 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1175) took 1538 ms. Trace:
I20260709 13:14:48.293008 30705 rpcz_store.cc:276] 0709 13:14:46.741355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.132907 (+391552us) service_pool.cc:224] Handling call
0709 13:14:48.279407 (+1146500us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000c9754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.279541 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607629696}
I20260709 13:14:48.280157  1265 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:48.281104  1341 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 13:14:48.264349 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82129.9us
W20260709 13:14:48.300261 30703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1166) took 1892 ms (client timeout 2145 ms). Trace:
I20260709 13:14:48.300472 30700 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:48.301653 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:48.303649 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1048) took 1036 ms. Trace:
W20260709 13:14:48.262308 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76804.2us
I20260709 13:14:48.249603  1339 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:48.288942 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.290977 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:48.280369 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 447) took 1464 ms (client timeout 1525 ms). Trace:
I20260709 13:14:48.297499 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.304028 31394 rpcz_store.cc:276] 0709 13:14:47.267518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.369329 (+101811us) service_pool.cc:224] Handling call
0709 13:14:48.303492 (+934163us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400523854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.303598 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388452096}
W20260709 13:14:48.300760 30703 rpcz_store.cc:269] 0709 13:14:46.407891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.519294 (+111403us) service_pool.cc:224] Handling call
0709 13:14:48.300035 (+1780741us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.300198 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188292864}
I20260709 13:14:48.311208 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:48.307974 30702 rpcz_store.cc:269] 0709 13:14:46.815822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.819003 (+  3181us) service_pool.cc:224] Handling call
0709 13:14:48.280153 (+1461150us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400222ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.280305 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481484544}
I20260709 13:14:48.314375 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.293694  1311 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:48.316656 30704 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [8.4-10.5]
I20260709 13:14:48.317432 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.319077  1143 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:48.319463  1340 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:14:48.305931  1354 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:48.320689 31391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1038) took 2588 ms (client timeout 1686 ms). Trace:
I20260709 13:14:48.312404  1329 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.292742  1308 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.322207  1343 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:48.322289  1287 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.304765  1306 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:48.325297 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:48.326021  1281 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:48.327920  1259 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.267123   769 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.330789 31396 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:14:48.331418 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1205) took 1516 ms. Trace:
I20260709 13:14:48.316694  1350 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:48.332885  1325 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.334574  1258 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.443380 30625 rpcz_store.cc:276] 0709 13:14:46.814686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.814919 (+   233us) service_pool.cc:224] Handling call
0709 13:14:46.827092 (+ 12173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.892323 (+1065231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.331107 (+438784us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:48.331317 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352295296}
I20260709 13:14:48.447278 30702 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:48.340312 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.339723 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.452036 31394 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:48.453210 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.453959 31394 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 1.
W20260709 13:14:48.339664 31391 rpcz_store.cc:269] 0709 13:14:45.732534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:45.871035 (+138501us) service_pool.cc:224] Handling call
0709 13:14:48.320485 (+2449450us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540057f154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:48.320594 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3585224960}
I20260709 13:14:48.296464  1237 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:48.349475 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.432287 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.324134  1274 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:48.437322 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 135768us
W20260709 13:14:48.439253 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114045us
I20260709 13:14:48.461112 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.439808 30704 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:48.461974 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:48.292457  1245 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:48.451155 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78986.9us
I20260709 13:14:48.458501  1353 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:48.457265 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.457958 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:48.456202 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.465018 31391 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:48.464841 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.465852 31394 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:48.465925 31378 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:48.468381 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.287030  1338 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.480904 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.481933 31391 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:48.483707 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.484516 31391 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:48.443548  1357 raft_consensus.cc:2764] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:14:48.488394 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:48.490049 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.491591 31394 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:48.492923 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.494211 31394 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:14:48.490998 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.465183  1344 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.309s	user 0.000s	sys 0.002s
W20260709 13:14:48.500713 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.510443 30558 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:48.512161 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 204260us
I20260709 13:14:48.512897 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1519) took 1601 ms. Trace:
I20260709 13:14:48.513391 30698 rpcz_store.cc:276] 0709 13:14:46.911269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.648688 (+737419us) service_pool.cc:224] Handling call
0709 13:14:47.742127 (+ 93439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:47.742581 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.742640 (+    59us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:47.786083 (+ 43443us) log.cc:844] Serialized 52 byte log entry
0709 13:14:48.020946 (+234863us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:48.021015 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.024046 (+  3031us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.512633 (+488587us) raft_consensus.cc:1685] finished
0709 13:14:48.512660 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.512827 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8503,"thread_start_us":173593,"threads_started":1,"wal-append.queue_time_us":251813}
I20260709 13:14:48.514183 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:48.515352 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.514076 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.518224 31394 raft_consensus.cc:2376] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10. Re-sending same reply.
I20260709 13:14:48.513787 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:48.519816 31378 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.520305 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:48.513022 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:48.496258  1267 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.934s	user 0.015s	sys 0.000s
I20260709 13:14:48.525147 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:48.529719 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.450588  1277 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.530704 31394 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:48.543431 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1055) took 1137 ms. Trace:
I20260709 13:14:48.539659 31391 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:48.492138  1343 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:48.544286 30559 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:48.543952 31394 rpcz_store.cc:276] 0709 13:14:47.405827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.530111 (+1124284us) service_pool.cc:224] Handling call
0709 13:14:48.530460 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.543017 (+ 12557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.543350 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.545928 31292 leader_election.cc:400] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12.
I20260709 13:14:48.546300 31292 leader_election.cc:403] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:14:48.546931 31394 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:48.547536 31292 leader_election.cc:400] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9.
I20260709 13:14:48.548007 31292 leader_election.cc:403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:14:48.548089 31391 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:48.548840 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1121) took 1134 ms. Trace:
I20260709 13:14:48.551525 30551 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:48.452488  1350 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.558674 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 444) took 2480 ms. Trace:
I20260709 13:14:48.560205 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1120) took 1146 ms. Trace:
I20260709 13:14:48.554818 30553 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:48.561712 30552 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:48.562111 31394 rpcz_store.cc:276] 0709 13:14:47.414234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.546110 (+1131876us) service_pool.cc:224] Handling call
0709 13:14:48.546629 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.559858 (+ 13229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.560138 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.565482 31394 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:48.549206 31391 rpcz_store.cc:276] 0709 13:14:47.414540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.547546 (+1133006us) service_pool.cc:224] Handling call
0709 13:14:48.547846 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.548615 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.548782 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.561625 30699 rpcz_store.cc:276] 0709 13:14:46.062512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.583719 (+521207us) service_pool.cc:224] Handling call
0709 13:14:46.584041 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:47.234376 (+650335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.542274 (+1307898us) spinlock_profiling.cc:243] Waited 569 ms on lock 0x7b540021fa54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:48.542527 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307858176}
I20260709 13:14:48.524084  1194 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:48.573285 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1056) took 1148 ms. Trace:
W20260709 13:14:48.573791 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60879.8us
I20260709 13:14:48.581817 30557 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:48.523051  1258 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:48.530687  1143 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:48.584623 31394 rpcz_store.cc:276] 0709 13:14:47.424673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.564761 (+1140088us) service_pool.cc:224] Handling call
0709 13:14:48.565202 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.572947 (+  7745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.573221 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.551235  1195 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12.
I20260709 13:14:48.511492  1353 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:48.503309  1325 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:14:48.596309 30558 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:48.597175  1274 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:48.598539  1308 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:48.553771  1043 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.531361  1287 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 13:14:48.620220  1043 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.621409  1205 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:48.475104  1339 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.622680  1325 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:14:48.600993  1155 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.686s	user 0.009s	sys 0.005s
I20260709 13:14:48.623273  1205 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:48.624264 31378 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:48.626011 31378 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:48.626981 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1128) took 1128 ms. Trace:
W20260709 13:14:48.613897 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 152356us
I20260709 13:14:48.602818 31367 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:48.629904 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1126) took 1147 ms. Trace:
I20260709 13:14:48.603075 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.603494  1250 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.603984 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:48.578074  1194 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:48.582161   599 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:48.543300  1006 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:48.561071  1338 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:48.605530 30557 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:48.552568  1304 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:48.585321 31391 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:48.522805  1281 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 13:14:48.661514 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:48.531618  1237 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:14:48.618113  1353 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.618944 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.622838  1043 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:48.648087  1194 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.578220  1355 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.627432 31378 rpcz_store.cc:276] 0709 13:14:47.499053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.625297 (+1126244us) service_pool.cc:224] Handling call
0709 13:14:48.625759 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.626527 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.626822 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.623613  1155 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.671979 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1135) took 1118 ms. Trace:
I20260709 13:14:48.631119 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:48.672583 31378 rpcz_store.cc:276] 0709 13:14:47.553638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.670781 (+1117143us) service_pool.cc:224] Handling call
0709 13:14:48.671223 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.671493 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.671532 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.671603 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:48.671653 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.671734 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.671897 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.674221 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:48.630406 31367 rpcz_store.cc:276] 0709 13:14:47.482464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.602051 (+1119587us) service_pool.cc:224] Handling call
0709 13:14:48.602522 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.629579 (+ 27057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.629830 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.645224 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:48.675689 30550 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:48.655751 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.575708  1306 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.660684 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1125) took 1187 ms. Trace:
I20260709 13:14:48.623307  1325 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:48.662864 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90210.6us
I20260709 13:14:48.612385  1349 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:48.631904  1205 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:48.573982   937 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:48.647351  1287 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:48.690843  1287 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:48.692180  1287 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.693609  1287 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:48.694051  1255 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.001s
I20260709 13:14:48.694906  1354 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:48.697515 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 454) took 1194 ms. Trace:
I20260709 13:14:48.697999 30703 rpcz_store.cc:276] 0709 13:14:47.503242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.309119 (+805877us) service_pool.cc:224] Handling call
0709 13:14:48.309464 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.522324 (+212860us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.522374 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.548693 (+ 26319us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.642090 (+ 93397us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:48.663769 (+ 21679us) log.cc:844] Serialized 14 byte log entry
0709 13:14:48.664156 (+   387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.664313 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.697227 (+ 32914us) raft_consensus.cc:1685] finished
0709 13:14:48.697252 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.697438 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7919,"raft.queue_time_us":127897,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":8484,"spinlock_wait_cycles":3980544,"thread_start_us":276660,"threads_started":3,"wal-append.queue_time_us":105084}
I20260709 13:14:48.699728  1325 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.701079  1325 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.702929 30706 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:48.703575  1350 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:48.703238  1205 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.704123  1155 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.704929  1205 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:48.705178  1329 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:48.705667  1329 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:48.706686  1329 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.708056  1329 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:48.711004 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 453) took 1208 ms. Trace:
I20260709 13:14:48.711508 30705 rpcz_store.cc:276] 0709 13:14:47.502919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.298668 (+795749us) service_pool.cc:224] Handling call
0709 13:14:48.299093 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.437299 (+138206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.437335 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.442737 (+  5402us) log.cc:844] Serialized 26 byte log entry
0709 13:14:48.610706 (+167969us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:48.610757 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.610845 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:48.710730 (+ 99885us) raft_consensus.cc:1685] finished
0709 13:14:48.710759 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.710933 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162624,"raft.queue_time_us":143539,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":12575,"thread_start_us":122833,"threads_started":1,"wal-append.queue_time_us":201352}
I20260709 13:14:48.679334 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:48.684374 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.656891  1304 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:48.663448   599 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:48.683089  1267 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:14:48.688413 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.655296  1362 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
W20260709 13:14:48.680665  1294 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.333s	user 0.015s	sys 0.029s
I20260709 13:14:48.697324  1311 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
W20260709 13:14:48.726711  1294 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.015s	sys 0.029s
I20260709 13:14:48.726687  1343 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:48.726018  1370 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:14:48.732103  1311 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:48.732549  1311 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:48.652340  1250 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.719105 30625 raft_consensus.cc:2368] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:48.716975 30557 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:48.736374 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:48.737520 30557 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:48.680142 31391 rpcz_store.cc:276] 0709 13:14:47.473397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.566930 (+1093533us) service_pool.cc:224] Handling call
0709 13:14:48.567321 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.657361 (+ 90040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.660585 (+  3224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.740581 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1203) took 1134 ms. Trace:
I20260709 13:14:48.728017  1343 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:48.719949 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1791) took 2124 ms. Trace:
W20260709 13:14:48.728870  1367 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:14:48.741873 30556 rpcz_store.cc:276] 0709 13:14:46.590578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.602498 (+ 11920us) service_pool.cc:224] Handling call
0709 13:14:46.604544 (+  2046us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.556190 (+951646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.715087 (+1158897us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54001e8cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:48.715317 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2179604864}
I20260709 13:14:48.741143 31391 rpcz_store.cc:276] 0709 13:14:47.606258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.739225 (+1132967us) service_pool.cc:224] Handling call
0709 13:14:48.739719 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.740006 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.740047 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.740115 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:48.740168 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.740264 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.740489 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.733533  1311 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:48.680262  1355 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:48.734838  1267 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:48.735038  1237 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:48.723572  1304 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:48.713313  1043 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 13:14:48.753700 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:48.750960 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1204) took 1144 ms. Trace:
I20260709 13:14:48.750422  1370 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:14:48.764396 31391 rpcz_store.cc:276] 0709 13:14:47.606526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.749164 (+1142638us) service_pool.cc:224] Handling call
0709 13:14:48.749694 (+   530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.749972 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.750013 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.750070 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:48.750110 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:48.750178 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:48.750806 (+   628us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:48.894975 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:48.896432 31391 raft_consensus.cc:2376] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11. Re-sending same reply.
I20260709 13:14:48.831512 30550 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:48.922052 30554 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:48.922808 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:48.924945 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:48.929450 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:48.694547  1255 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.759452  1043 consensus_queue.cc:1243] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:48.724476  1323 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.200s
I20260709 13:14:48.755208  1277 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:48.935015 30558 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:48.935465 30550 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:48.944231 30554 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:48.761637  1267 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:14:48.934368 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 180784us
W20260709 13:14:48.960043 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:48.962261 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 127728us
I20260709 13:14:48.756312  1143 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:48.966540 30557 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:48.758730  1237 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:48.967177 31292 leader_election.cc:400] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:48.930771  1281 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:48.971761 31292 leader_election.cc:403] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:14:48.972918 30550 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:48.960566 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:48.964102   929 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.944s	user 0.013s	sys 0.001s
I20260709 13:14:48.977041 30557 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:48.976177 30627 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:48.959638  1294 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:48.979327 31391 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:48.980433 30550 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:48.756212   599 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.972759  1311 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:48.975015  1273 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:48.971313  1302 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:48.756745  1343 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.978313  1323 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:14:48.987490  1371 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:49.023219 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1205) took 1416 ms. Trace:
W20260709 13:14:49.024003 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 307641us
I20260709 13:14:48.990945  1355 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:48.974061  1143 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:48.979018  1281 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:48.993772  1273 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:49.025750 31391 rpcz_store.cc:276] 0709 13:14:47.606792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.987824 (+1381032us) service_pool.cc:224] Handling call
0709 13:14:49.019672 (+ 31848us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.022386 (+  2714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.022426 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.022491 (+    65us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:14:49.022539 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.022797 (+   258us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.023134 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.017565  1115 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.038115 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1053) took 1720 ms. Trace:
I20260709 13:14:48.982916   929 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.053855 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 535) took 1683 ms. Trace:
I20260709 13:14:49.048401 31361 rpcz_store.cc:276] 0709 13:14:47.317333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.419154 (+101821us) service_pool.cc:224] Handling call
0709 13:14:47.443237 (+ 24083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:47.735886 (+292649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:47.735922 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:47.736021 (+    99us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:47.736084 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.037720 (+1301636us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400013954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.037785 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.038001 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493195648}
I20260709 13:14:49.054507 30624 rpcz_store.cc:276] 0709 13:14:47.370464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.462071 (+ 91607us) service_pool.cc:224] Handling call
0709 13:14:47.462387 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.088279 (+625892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.053504 (+965225us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b5400865bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.053765 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437573888}
I20260709 13:14:49.021621  1302 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.077397 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:49.071730  1345 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.800s	user 0.014s	sys 0.000s
I20260709 13:14:49.053048  1343 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:49.081734 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1136) took 1058 ms. Trace:
I20260709 13:14:49.022550  1237 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.082948 31361 rpcz_store.cc:276] 0709 13:14:48.023931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.073157 (+1049226us) service_pool.cc:224] Handling call
0709 13:14:49.081106 (+  7949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.081335 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.081372 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.081421 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:49.081460 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.081516 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.081671 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:49.085479 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:49.075716  1270 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.006s
I20260709 13:14:49.086131 30550 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:49.086391 30553 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:14:49.091679 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 121134us
I20260709 13:14:49.088543 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.082470  1250 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.096582 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.099143 31361 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:49.099870 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:49.101311 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.091082  1270 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:49.104415 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.105175 30624 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:49.107329 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.109153  1371 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:49.110596  1330 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.248s	user 0.004s	sys 0.016s
I20260709 13:14:49.111754   929 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.112095   897 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.113044   937 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9.
I20260709 13:14:49.095115  1237 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:49.106485 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.105250 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:49.122133  1329 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:49.086737  1345 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.125615  1329 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:49.126829 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 452694us
W20260709 13:14:49.128029 31396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1046) took 1866 ms (client timeout 2150 ms). Trace:
I20260709 13:14:49.128517  1281 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:49.128618 31396 rpcz_store.cc:269] 0709 13:14:47.261870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.262121 (+   251us) service_pool.cc:224] Handling call
0709 13:14:49.127805 (+1865684us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b540002bcd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:49.127961 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234464512}
I20260709 13:14:49.130000 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.130394  1302 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.124640 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1201) took 1519 ms. Trace:
I20260709 13:14:49.130851 30629 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:49.131107 31378 rpcz_store.cc:276] 0709 13:14:47.605697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.674187 (+1068490us) service_pool.cc:224] Handling call
0709 13:14:48.674742 (+   555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.737313 (+ 62571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.737379 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.765444 (+ 28065us) log.cc:844] Serialized 26 byte log entry
0709 13:14:49.046636 (+281192us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:49.046720 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.046825 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:49.124351 (+ 77526us) raft_consensus.cc:1685] finished
0709 13:14:49.124379 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.124558 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217225,"raft.queue_time_us":72235,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":5149,"thread_start_us":75304,"threads_started":1,"wal-append.queue_time_us":145791}
I20260709 13:14:49.095438  1266 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.133431 31378 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:49.134028 31364 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:49.134485  1143 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.137475 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:49.137926  1156 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:49.127971  1329 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.093199  1255 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.081509  1273 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.139597  1273 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:49.141116 30554 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:42734
I20260709 13:14:49.141836 30554 raft_consensus.cc:606] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:49.144160 31388 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:49.144380  1350 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:49.145076  1350 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.145475  1350 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:49.149188 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.153400 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.154829 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:49.144104 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.157609 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.160403 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.160724 30626 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:49.163553 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.164045 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.157946 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.166029 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.166524 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:49.182756 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.183519 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.184408 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.184425 30553 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:49.184537 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.185104 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.184450 30554 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:49.185803 30626 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 5.
I20260709 13:14:49.186777 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.186810 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:49.187443 30626 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:49.187830 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.189090 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.191474 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.138377  1156 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.193228 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.193651 30632 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:49.196369 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:49.199020 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:49.200798 30626 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:49.198380 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:49.203697 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.204493 30625 raft_consensus.cc:2368] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11.
W20260709 13:14:49.205446 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 521329us
I20260709 13:14:49.207975 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:49.209347 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.212885 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.213917 30630 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:49.202636 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.221580 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.220292 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.222690 30626 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:49.197156   937 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:49.224488 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:49.229785 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.234659  1158 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:49.235217  1158 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.237622 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1127) took 1751 ms. Trace:
I20260709 13:14:49.212486  1143 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:49.239132  1143 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:49.239305  1302 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:49.239756  1143 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:49.224488  1281 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:49.242647  1281 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.243232 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.230995  1294 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:14:49.244802  1294 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:49.244928 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.245378  1294 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:49.246146  1365 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.588s	user 0.000s	sys 0.002s
I20260709 13:14:49.248239 30630 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 13:14:49.249234 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.150s (SENT) [suppressed 23 similar messages]
W20260709 13:14:49.253508 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.254316 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.255888  1348 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.256670  1348 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.259043  1348 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.239846  1302 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:49.261191  1302 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:49.190274  1371 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.238157 31394 rpcz_store.cc:276] 0709 13:14:47.486326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.609924 (+1123598us) service_pool.cc:224] Handling call
0709 13:14:48.610359 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.653443 (+ 43084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.653497 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.653552 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:48.653587 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.237266 (+583679us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b540044e654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.237329 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.237533 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98483584}
I20260709 13:14:49.268277 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.271158 30703 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:14:49.272104 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.198738   929 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:49.275357 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.282191 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:49.276921 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.282819 30701 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:49.190997  1345 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:49.289793  1205 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
I20260709 13:14:49.285825   929 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.291496   929 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:49.291951  1194 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.006s
I20260709 13:14:49.292469  1194 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.294683  1266 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.295857 30554 raft_consensus.cc:2368] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11.
I20260709 13:14:49.295588  1266 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.299934  1349 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.307291 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.311986 31364 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:49.316833 31364 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:49.282122  1371 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:49.287083  1325 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.586s	user 0.014s	sys 0.000s
I20260709 13:14:49.286944  1043 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:49.288245   847 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.278575  1155 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 13:14:49.312650 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.243180  1281 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:49.313294  1304 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.004s
I20260709 13:14:49.284435 31394 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:49.314143  1354 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:49.265442  1370 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:49.317302  1294 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.319147 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.319830  1194 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.320031 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.323829  1258 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:49.224810  1339 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:49.390596 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.273579 30625 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:49.327061 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:49.381927  1287 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.688s	user 0.015s	sys 0.001s
I20260709 13:14:49.330651 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:49.332352 30523 connection.cc:256] RPC call timeout handler was delayed by -1.155090409000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:14:49.395753 30523 connection.cc:256] RPC call timeout handler was delayed by -1.155090409000195s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21450000000000002s for a response
W20260709 13:14:49.396237 30523 connection.cc:256] RPC call timeout handler was delayed by -1.1302037590000964s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1613s for a response
W20260709 13:14:49.396564 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0992218450001019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:14:49.396942 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0966835350002384s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 13:14:49.397862 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:49.332466 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60541.5us
I20260709 13:14:49.334512 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.335383 31361 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 13:14:49.403283 31361 pending_rounds.cc:85] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:49.403595 31361 pending_rounds.cc:107] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:14:49.403839 31361 raft_consensus.cc:2889] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:49.336541 30701 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:49.342293 31378 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:49.372355 30699 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:49.405994 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.340976 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.410102 31364 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:49.346699 30624 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:49.383589 30698 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:49.384383 31388 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:49.339370  1345 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:49.329403 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.393863 30704 raft_consensus.cc:2410] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 13:14:49.393926 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66968.4us
I20260709 13:14:49.416602 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.379904  1255 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:14:49.372140  1306 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:14:49.337772  1349 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.340099 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.334332  1043 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.409391 30701 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:49.436645 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.436961 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.391742  1258 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:49.420511 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.441028 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.441164 31364 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:49.385574  1304 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.449005 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:49.450268 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 201069us
I20260709 13:14:49.442358 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:49.452543 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.381208   929 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.426525 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.453748 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:49.454689 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.455255 30705 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:49.456050 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.424365 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:49.393378  1370 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.459228 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.403846 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:49.408387 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.460402 30699 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:49.435709 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.450028 30703 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:49.460959 30704 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:49.420069  1270 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:49.421449 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:49.419785  1205 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:49.467901 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46532us
I20260709 13:14:49.399572  1266 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.388464  1354 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:49.431180 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91207.1us
W20260709 13:14:49.470551 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 131437us
I20260709 13:14:49.472332 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.472591 30698 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:49.463896 30705 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:49.460845 30701 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:14:49.438539  1217 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:14:49.442023 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 216932us
I20260709 13:14:49.471025 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:49.479055 31367 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:49.425961  1194 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.399737  1311 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:14:49.439846  1345 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.441804  1287 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:49.465209 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.421808  1267 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.426414  1265 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:14:49.426389  1339 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 13:14:49.471367  1006 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:49.493662  1194 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.494711  1006 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:49.495007  1350 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.489631  1353 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 13:14:49.496141  1353 consensus_queue.cc:1243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:14:49.495888  1006 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.496507  1194 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.479606  1266 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:14:49.438095  1348 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:14:49.467324  1270 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.486195 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:49.511890 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:49.471686  1030 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:49.471410  1325 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.478144   897 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:49.491518 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.498176 30703 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:49.499369 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:49.486150  1217 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:49.528000 30673 leader_election.cc:400] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11.
W20260709 13:14:49.503798 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.505409 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1523) took 1673 ms. Trace:
I20260709 13:14:49.530815 30707 rpcz_store.cc:276] 0709 13:14:47.831539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.330187 (+498648us) service_pool.cc:224] Handling call
0709 13:14:48.330549 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.609448 (+278899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.609497 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:48.609563 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:48.609615 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.505024 (+895409us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54005f00d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.505097 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.505324 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640771840}
W20260709 13:14:49.516708 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.532225 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.532374 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.533627 31292 leader_election.cc:400] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11.
I20260709 13:14:49.534054 31292 leader_election.cc:403] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:49.518779 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:49.484597  1385 raft_consensus.cc:2764] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:14:49.497342  1350 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.537806  1030 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:49.554821  1179 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11.
I20260709 13:14:49.509784  1356 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:49.556598 31396 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:49.558662 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:49.568419 31396 pending_rounds.cc:85] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:49.568779 31396 pending_rounds.cc:107] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:49.569043 31396 raft_consensus.cc:2889] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:49.555922  1339 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:49.561683 30553 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:49.510942  1242 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:49.520079  1270 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:49.517901  1353 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:14:49.570914 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 167138us
I20260709 13:14:49.523416  1245 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:49.569656 30633 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:49.573508 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.575388 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.527396  1360 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:49.529863  1325 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.550346  1306 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:14:49.586748 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.547115   897 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:49.607354  1306 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:49.601377 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.610062 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.612411 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:49.564415  1194 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:14:49.629915 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 157698us
W20260709 13:14:49.635569 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 62219.9us
I20260709 13:14:49.636758 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.639496 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.642062 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:49.624608  1267 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.647112 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1502) took 3281 ms. Trace:
I20260709 13:14:49.596613  1339 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:49.648231 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 102126us
I20260709 13:14:49.598763  1371 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.615134  1250 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.647768 30700 rpcz_store.cc:276] 0709 13:14:46.353498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:46.585592 (+232094us) service_pool.cc:224] Handling call
0709 13:14:48.300190 (+1714598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.301194 (+  1004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.634344 (+1333150us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540042f9d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:49.634528 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3942486400}
I20260709 13:14:49.575100  1273 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:14:49.608513   897 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.622772  1306 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:49.673224 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89304.2us
W20260709 13:14:49.662906 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.627879  1245 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:49.643819  1194 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:49.556600  1370 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.693769  1392 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.694710  1392 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.697872  1392 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.692047  1311 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.699893  1311 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.648869  1295 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:49.651134   937 raft_consensus.cc:2764] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:49.648559  1274 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:14:49.706316 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 65137.1us
I20260709 13:14:49.576664  1345 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.712188 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.672448  1242 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:49.676168  1217 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.733220  1245 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.745100  1311 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:49.745836  1311 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.676937  1389 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:49.746383  1311 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:49.747648  1391 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:14:49.747242  1389 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.749101  1311 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:49.749611  1386 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:14:49.714155   769 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:49.644413  1030 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.713377  1295 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.673413  1250 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.755854   929 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.758116 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1207) took 2145 ms. Trace:
I20260709 13:14:49.707774  1325 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.744040 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.724295  1304 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.704316  1392 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
W20260709 13:14:49.760936 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 148609us
W20260709 13:14:49.790372 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 116054us
W20260709 13:14:49.768137 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 137234us
I20260709 13:14:49.778909   769 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:49.785084 31391 rpcz_store.cc:276] 0709 13:14:47.613045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.046824 (+1433779us) service_pool.cc:224] Handling call
0709 13:14:49.047325 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.050725 (+  3400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:49.050760 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:49.050807 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:49.050839 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.757771 (+706932us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b54002b0a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.757832 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:49.758005 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7129856}
I20260709 13:14:49.795442 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.729601  1242 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.796166 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.725550  1179 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:49.796116 30699 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:49.762100  1267 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.807770 30705 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:49.808218 30705 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:49.786128  1295 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:49.798314  1343 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.711s	user 0.014s	sys 0.000s
I20260709 13:14:49.809070 30705 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.823335 30705 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:49.802556   769 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:49.824903 30628 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:49.807102 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 13:14:49.849257 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.858764  1399 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:49.859601  1399 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.806668  1354 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:49.864985 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67525.1us
I20260709 13:14:49.804639  1349 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.805608  1242 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:49.872591  1143 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.008s
I20260709 13:14:49.876354 30705 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:49.873199  1143 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.874083  1371 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.875586   929 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:14:49.877410  1371 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.878074   929 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:49.878450   929 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.878762  1143 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.879633  1371 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:49.875737 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 81650.8us
I20260709 13:14:49.867515 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:49.879401   929 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.884553  1250 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.887359 30552 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:14:49.889506 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:49.889544 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:49.891160 30705 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:49.890231 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.868808  1277 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.788777  1339 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.893373 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:49.826870  1383 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.894513 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.817628  1267 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.875432  1295 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.896836 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:49.897145  1407 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:14:49.898468  1400 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
W20260709 13:14:49.905332 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:49.875890  1392 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:14:49.914935 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.912588 30556 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:49.911343  1194 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:49.911044 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.922225 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:49.920611 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:49.921641 30700 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
W20260709 13:14:49.915956  1302 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.654s	user 0.016s	sys 0.000s
I20260709 13:14:49.910336  1371 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:49.927016  1371 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.926885  1302 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.925325 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:49.908198   929 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:49.927645  1371 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:49.931303  1404 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:49.931093  1397 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:14:49.931667  1392 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:14:49.931910  1401 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:49.932500  1250 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:49.953964  1401 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.925295  1399 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.913204  1383 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.954087  1392 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.960238  1392 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.961280  1383 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:49.961627  1392 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:49.963320  1396 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:49.958702  1401 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:49.926388   897 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:49.967206 31389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1050) took 2699 ms (client timeout 1693 ms). Trace:
W20260709 13:14:49.967805 31389 rpcz_store.cc:269] 0709 13:14:47.268124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:47.325053 (+ 56929us) service_pool.cc:224] Handling call
0709 13:14:49.966936 (+2641883us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:49.967110 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5082742144}
I20260709 13:14:49.969266  1397 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:49.970005 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 541) took 1316 ms. Trace:
I20260709 13:14:49.969959  1397 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.951319  1295 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:49.971249  1205 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:49.971765  1205 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:49.970458 30629 rpcz_store.cc:276] 0709 13:14:48.653343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.656209 (+  2866us) service_pool.cc:224] Handling call
0709 13:14:48.656584 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.131307 (+474723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.969725 (+838418us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b540063ced4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.969936 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090203008}
I20260709 13:14:49.972781  1397 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:49.934402  1267 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.923112  1194 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:49.935146 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.918972  1408 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:14:49.937423 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:49.941057 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:49.942682 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:49.987226 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:49.943487 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:49.944228  1404 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:49.908072  1277 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.948915  1258 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:49.997526 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.998198  1258 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:49.998894 31364 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:50.000272 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.000662  1404 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.001551 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:49.925129  1030 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.002921  1030 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.005282  1030 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.005384  1311 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:50.005899  1311 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:50.006419  1394 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:49.919754  1409 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:14:50.006976  1311 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:49.950073 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1202) took 2344 ms. Trace:
I20260709 13:14:50.007889  1409 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.008361  1409 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:50.008080 31367 rpcz_store.cc:276] 0709 13:14:47.605995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.675545 (+1069550us) service_pool.cc:224] Handling call
0709 13:14:48.675972 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.479644 (+803672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.949823 (+470179us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b540002add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.950007 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846243072}
I20260709 13:14:49.953203 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.010262 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.008646  1311 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:49.970777  1302 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.013200  1267 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:50.009948  1409 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.013758  1267 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:49.954668 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:50.014442  1409 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.014921 31391 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:50.016181  1267 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:49.954924 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.017047 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:50.019373  1043 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.009298  1356 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:50.021000  1043 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:49.955780 31383 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:49.951797  1385 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:49.973127 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.007897 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 542) took 1349 ms. Trace:
I20260709 13:14:50.027946 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.028602 30629 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.028296 30627 rpcz_store.cc:276] 0709 13:14:48.658298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.659189 (+   891us) service_pool.cc:224] Handling call
0709 13:14:48.669042 (+  9853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.981084 (+312042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.007587 (+1026503us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54004aa1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:50.007828 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":701189504}
I20260709 13:14:49.957669  1411 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:49.978945  1194 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:50.031154  1411 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.031637  1411 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:49.980078 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.032588 30631 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:50.030467 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.033732 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.034336 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.035476 30631 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:49.980770  1158 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.037392  1158 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:49.982144 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1071) took 1679 ms. Trace:
I20260709 13:14:50.038825 31396 rpcz_store.cc:276] 0709 13:14:48.302963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.131138 (+828175us) service_pool.cc:224] Handling call
0709 13:14:49.131477 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.569556 (+438079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.981876 (+412320us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b540031d8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:49.982075 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976564096}
W20260709 13:14:50.039286 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.021s	sys 0.009s
I20260709 13:14:49.984319 30707 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:14:50.039860 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.021s	sys 0.009s
I20260709 13:14:50.041000 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.042068 30632 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
W20260709 13:14:49.989807 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:49.991489 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48890.5us
I20260709 13:14:49.994812  1295 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:49.994963  1403 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:49.995848  1306 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:50.006910  1394 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:49.933946 30626 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:14:50.048343 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 717390us
I20260709 13:14:50.043467 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.059423  1295 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.060243  1385 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:14:50.066923  1258 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.074301  1303 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.080785  1394 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.082190  1295 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.082221  1267 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:14:50.117123 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1801) took 1547 ms. Trace:
W20260709 13:14:50.123231 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75240.1us
I20260709 13:14:50.124276 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.124984 31388 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:50.125204 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:50.126384 31367 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:50.036124 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:50.131649 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62008.9us
I20260709 13:14:50.123512 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:50.136147  1255 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:14:50.137979 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.122816 30555 rpcz_store.cc:276] 0709 13:14:48.569575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.570167 (+   592us) service_pool.cc:224] Handling call
0709 13:14:48.570669 (+   502us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:48.571207 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:48.571250 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:48.571965 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 13:14:49.924030 (+1352065us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b18004c3f60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:14:49.928218 (+  4188us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:49.928259 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.928334 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:50.100780 (+172446us) raft_consensus.cc:1685] finished
0709 13:14:50.100817 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:50.117020 (+ 16203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1765,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":6730,"spinlock_wait_cycles":55842688,"thread_start_us":3965,"threads_started":1,"wal-append.queue_time_us":1494131}
I20260709 13:14:50.064414  1397 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:50.145674 30705 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 1. Current term is 2. Ops: []
I20260709 13:14:50.146176 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.047097 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.145200 31394 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
W20260709 13:14:50.147794 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 100866us
I20260709 13:14:50.129359 30698 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:50.151140 30705 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:50.151543 30705 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:50.152271 30705 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.153295 30705 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:50.153057 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.154253 31367 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:50.151912 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.163100  1277 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.164201 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.155968  1396 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:50.165000 30700 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:50.165350  1396 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.150025  1408 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.165897  1396 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:50.120450  1258 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.172940  1258 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:50.146708  1402 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:50.173954  1402 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:50.164412  1281 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.776s	user 0.015s	sys 0.007s
I20260709 13:14:50.176232  1402 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.175094  1281 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.183112  1343 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:50.189291  1250 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:14:50.192313  1270 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.577s	user 0.017s	sys 0.000s
I20260709 13:14:50.192826 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1148) took 1056 ms. Trace:
I20260709 13:14:50.193280 31361 rpcz_store.cc:276] 0709 13:14:49.136317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.136597 (+   280us) service_pool.cc:224] Handling call
0709 13:14:49.136957 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:49.404381 (+267424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:50.192485 (+788104us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b540057b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:50.192752 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455460224}
I20260709 13:14:50.195598  1401 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.183584  1281 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:50.160450 31391 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:50.197829 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 773) took 1020 ms. Trace:
I20260709 13:14:50.198765  1205 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.202719  1306 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:50.202412  1396 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:50.203665  1410 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.346s	user 0.000s	sys 0.003s
I20260709 13:14:50.204542  1306 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:14:50.205713 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.205891  1306 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:14:50.205689  1396 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.206398  1306 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:50.208323  1306 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.209728  1306 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:14:50.210466  1294 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.202481 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:50.212491  1385 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:50.212905  1385 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.213249  1385 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:50.198318 30633 rpcz_store.cc:276] 0709 13:14:49.177793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.184172 (+  6379us) service_pool.cc:224] Handling call
0709 13:14:50.197622 (+1013450us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54000555d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:50.197761 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":483415808}
I20260709 13:14:50.156335  1397 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.222407 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.228220 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.224892 30557 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:50.223521 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.238341 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:50.239173 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:50.216298 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.223037 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:50.243374 30550 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:50.244897 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:50.247851 31367 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:50.249979 31364 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:50.250136 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:50.250043 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 98214.5us
I20260709 13:14:50.251981 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.256526 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.268512 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.274226 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.274646 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.275121 30702 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:50.218252  1397 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.289907 30700 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:50.290592 30551 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:50.291832 30559 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:50.273439 30631 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:50.279656 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:50.297514 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.280201 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:50.226227   986 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 13:14:50.287549 30705 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:14:50.240800  1356 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.295261 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:50.299953 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48066.1us
I20260709 13:14:50.301354 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.420178 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.347620 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:50.363521 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60437.6us
I20260709 13:14:50.306859 30705 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 6
I20260709 13:14:50.420976 30633 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:50.421475 30705 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:14:50.355813  1343 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:50.350795  1258 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:50.302310 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.353864 30550 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:50.359645 31361 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:14:50.267652  1265 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:50.356817 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:50.228672  1389 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:50.352675 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.432458 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75741.7us
I20260709 13:14:50.434331 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.420465 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.435454 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:50.436897 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.436808 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.437434 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:50.255057  1353 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:14:50.156594  1395 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:50.343875 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.448342 30558 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:50.263386  1329 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:50.166303  1408 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.355355  1250 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:50.400357  1326 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.402796  1281 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.404069 30624 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:50.412706 30629 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:14:50.421525 30553 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:50.421881 30705 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.469404  1401 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:50.472285   929 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.007s
I20260709 13:14:50.475212 30625 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:14:50.475797 30625 pending_rounds.cc:85] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 5
I20260709 13:14:50.476104 30625 pending_rounds.cc:107] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:14:50.476408 30625 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.436697  1273 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:50.483741  1273 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.486166  1273 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:50.422096 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.427917 30559 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:50.428509 31361 pending_rounds.cc:85] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:50.488756 31361 pending_rounds.cc:107] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:14:50.489084 31361 raft_consensus.cc:2889] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:50.440294  1006 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.431394  1265 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.432438  1143 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.192857  1270 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.255471  1371 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.452569 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:50.420908  1345 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:50.431682  1255 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.355429 30552 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:50.255956  1237 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.170s
W20260709 13:14:50.490151 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69753us
I20260709 13:14:50.444361  1252 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:50.493913 30633 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 6: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:50.498142 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.464376  1303 raft_consensus.cc:2764] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:14:50.436897  1302 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:14:50.521550 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:50.522284 31389 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:50.525266 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.529203 30701 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:50.492228  1281 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.530010  1371 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.530903 31388 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:50.517501   929 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:50.531634  1281 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:50.531605   929 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.532712 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.533459 30625 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:50.533960   929 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:50.463600  1319 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:14:50.534914 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.535485 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 549) took 1378 ms. Trace:
I20260709 13:14:50.535640 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:50.535887 30628 rpcz_store.cc:276] 0709 13:14:49.157298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.157424 (+   126us) service_pool.cc:224] Handling call
0709 13:14:50.535256 (+1377832us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54000541d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:50.535424 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412819200}
I20260709 13:14:50.511586  1295 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.537520 30628 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:50.538251   769 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.670s	user 0.006s	sys 0.010s
I20260709 13:14:50.538801   769 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.539186  1295 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.540136 31383 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:50.540995  1237 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:50.541700  1237 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:50.453415  1395 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.542802  1237 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.544415  1395 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.530820 30630 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:50.512678  1143 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.519747  1255 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.508704 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:50.499140  1242 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.006s
I20260709 13:14:50.467355  1323 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:50.433449  1356 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.520463 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:50.548660  1242 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.550798  1356 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:50.551231  1343 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:50.551826  1343 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:50.553531 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.492165  1250 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:50.551267 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98812.8us
I20260709 13:14:50.503899  1006 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.555112 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:50.555956  1411 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.010s
I20260709 13:14:50.556452  1411 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.556833  1006 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:50.557660  1303 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:14:50.521283 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.553340  1345 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:50.503953  1265 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:50.561059  1345 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:50.563351  1242 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.564067  1408 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:50.564683 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.507855  1370 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:14:50.567292  1370 consensus_queue.cc:1243] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:14:50.568442  1409 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.000s
I20260709 13:14:50.508205  1396 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:50.569504  1396 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:50.523470  1273 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:50.570593  1396 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.552011  1402 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:50.571178  1273 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:50.489181  1326 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.548789  1143 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.576581 30629 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:50.572283 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.574635 30632 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:50.600821 30632 pending_rounds.cc:85] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:14:50.601255 30632 pending_rounds.cc:107] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:50.601699 30632 raft_consensus.cc:2889] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:50.598804 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.596076 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.602397 31378 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:50.603554 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:50.606139 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.606288 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.609797 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.611624 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.612823 31389 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:14:50.620574 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.622010 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.624423 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.605664 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.620764 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:50.625262 30698 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:50.624568 30700 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:50.626938 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:50.627699 31389 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:50.612560 30705 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:50.639819 30558 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:14:50.573765  1194 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.542s	user 0.016s	sys 0.002s
I20260709 13:14:50.646940  1143 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:50.647897  1143 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.572965  1250 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.554018  1343 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:50.651657 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.652432 30552 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:50.653383 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:50.653834 30702 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:50.667481 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 450) took 3165 ms. Trace:
I20260709 13:14:50.654021 30552 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:50.668109 30706 rpcz_store.cc:276] 0709 13:14:47.501733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:48.275391 (+773658us) service_pool.cc:224] Handling call
0709 13:14:48.280836 (+  5445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:48.703436 (+422600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:49.941203 (+1237767us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54000c9754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:50.667157 (+725954us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b54000c9750. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:50.667367 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981766784}
I20260709 13:14:50.630849  1411 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:50.654983 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:50.680168 30628 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:50.660418 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:50.666074 30700 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:14:50.635135 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.622740  1006 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:50.671904 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.541057   769 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:50.683676 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48673.4us
I20260709 13:14:50.628954 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.630419  1329 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:14:50.685380 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 446254us
I20260709 13:14:50.642302  1356 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:50.643947  1370 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:50.576331  1397 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.106s
I20260709 13:14:50.688575 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:50.689406 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60524us
I20260709 13:14:50.666236  1205 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:50.642028  1294 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:14:50.694587 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:50.788982 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94516.8us
I20260709 13:14:50.700277 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:50.790514 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:50.790951 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90789.4us
I20260709 13:14:50.709811 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:50.813470 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:50.815521 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:50.817828 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:50.857801 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 151118us
I20260709 13:14:50.688769  1402 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:50.863155 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.864217 30700 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:50.864722 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.865733 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.868287 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:50.872360 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:50.895622 31364 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.018s	sys 0.012s
W20260709 13:14:50.896499 31364 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.018s	sys 0.013s
I20260709 13:14:50.690512  1343 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:50.672708  1194 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:50.857358  1409 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:50.970727  1356 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:50.973541  1205 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:50.974833 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:50.975024 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:50.697849  1319 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:50.691018  1329 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:50.711117  1303 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:50.850363   897 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:51.077803 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:51.078903 30628 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:50.851056  1306 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:50.850018  1411 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:50.788079  1339 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.248s
I20260709 13:14:50.689935  1006 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:50.699407  1158 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.305s
I20260709 13:14:51.081782 30556 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:51.086668 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:51.088320 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:51.137874 30703 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:50.979784  1343 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:50.967377  1402 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:51.139380 31391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.084s	sys 0.138s
W20260709 13:14:51.139953 31391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.084s	sys 0.138s
W20260709 13:14:51.090781 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.011s	sys 0.021s
W20260709 13:14:51.141922 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.142131 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.012s	sys 0.022s
I20260709 13:14:50.698766  1265 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.977500  1353 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:50.977749  1349 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:14:51.001678 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.043s	sys 0.058s
W20260709 13:14:51.147683 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.044s	sys 0.059s
I20260709 13:14:51.034396  1409 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:51.028398  1361 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.042s	sys 0.057s
W20260709 13:14:51.154939  1361 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.043s	sys 0.058s
I20260709 13:14:51.156550 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:51.029418  1329 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:51.119048  1242 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.160338  1242 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:50.984840  1356 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.163044  1356 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:51.124796 30698 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:50.687254  1156 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.002779  1319 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:51.167984  1319 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:51.030437  1303 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:51.144770 31364 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:14:51.115949 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 298207us
I20260709 13:14:51.092644  1370 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.180862 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:51.031091  1205 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:51.175872 30557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.021s	sys 0.064s
W20260709 13:14:51.188895 30557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.022s	sys 0.065s
I20260709 13:14:51.185325 30703 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:51.133844   769 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:51.147169  1323 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:51.192448 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.150827  1343 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:51.147845  1402 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.204490 31396 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:51.182988  1303 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:51.193377  1158 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:51.177397  1353 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.210018 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:51.201332  1308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.035s	sys 0.062s
I20260709 13:14:51.214686 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:51.222770 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.171272  1349 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.227453 31364 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:51.227773 30699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.024s	sys 0.016s
I20260709 13:14:51.226459 30628 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
W20260709 13:14:51.231899 30699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.025s	sys 0.016s
I20260709 13:14:51.239100 31394 pending_rounds.cc:85] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
W20260709 13:14:51.222328  1308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.035s	sys 0.063s
I20260709 13:14:51.239539 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1233) took 1090 ms. Trace:
I20260709 13:14:51.239585 31394 pending_rounds.cc:107] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:14:51.242791 31394 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:51.243287 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:51.245255 30628 rpcz_store.cc:276] 0709 13:14:50.150788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.238312 (+1087524us) service_pool.cc:224] Handling call
0709 13:14:51.238824 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.239081 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:51.239118 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:51.239180 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:51.239228 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.239299 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.239464 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:14:51.237903 30555 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:14:51.213032  1158 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:51.186692  1370 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.248869 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:51.247890 30555 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 2.
I20260709 13:14:51.250835 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:51.253477 30554 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:51.254379 31364 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 13:14:51.228042  1258 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.017s	sys 0.021s
I20260709 13:14:51.205137  1323 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.241112  1409 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.241609  1424 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:51.265542 31364 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:51.271832 31396 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:14:51.263697 30631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.030s	sys 0.020s
I20260709 13:14:51.259596  1353 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:51.261574  1258 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.017s	sys 0.022s
W20260709 13:14:51.286526  1421 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:14:51.289278  1385 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.076s	user 0.016s	sys 0.000s
I20260709 13:14:51.289891  1385 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:51.291023  1194 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.291311 30551 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:51.291455  1006 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.292375  1385 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.266888  1425 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:14:51.264183   897 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:14:51.295897  1370 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.296213  1158 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.297566  1158 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:51.298946 30698 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:14:51.273515 30631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.030s	sys 0.020s
I20260709 13:14:51.293561  1205 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.275887 31364 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:14:51.228694  1237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.022s	sys 0.018s
I20260709 13:14:51.264009  1242 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.328670 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.330960 30698 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:51.334744 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.335963 31396 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:51.341456 31364 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:51.326807  1265 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.343966 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.349519 31364 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:51.323411  1424 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:51.356388 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:51.365985  1349 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.326823  1323 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.375355  1143 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.384529 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1176) took 1107 ms. Trace:
W20260709 13:14:51.385373 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:51.394068 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.301285  1258 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:51.266561  1308 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
W20260709 13:14:51.343899  1237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.023s	sys 0.019s
I20260709 13:14:51.415596 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.408166 30557 rpcz_store.cc:276] 0709 13:14:50.277081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.281925 (+  4844us) service_pool.cc:224] Handling call
0709 13:14:50.282310 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.282841 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.282886 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.425789 (+142903us) log.cc:844] Serialized 26 byte log entry
0709 13:14:51.189493 (+763704us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:51.189565 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.189651 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.384247 (+194596us) raft_consensus.cc:1685] finished
0709 13:14:51.384270 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.384454 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11996,"raft.run_cpu_time_us":3474,"raft.run_wall_time_us":121141,"thread_start_us":722817,"threads_started":1,"wal-append.queue_time_us":884383}
I20260709 13:14:51.429641 31396 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:51.390555  1255 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.391101  1270 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.441502 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:51.441819 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 469) took 1962 ms. Trace:
I20260709 13:14:51.442754 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:51.448341 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:51.448843 30701 rpcz_store.cc:276] 0709 13:14:49.479384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.479792 (+   408us) service_pool.cc:224] Handling call
0709 13:14:49.480147 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.529784 (+1049637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.441457 (+911673us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540078bc54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.441726 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412048640}
I20260709 13:14:51.433938  1427 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.433262   769 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:51.458453 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 258728us
W20260709 13:14:51.463213 31294 connection.cc:256] RPC call timeout handler was delayed by -1.2328041920000032s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
I20260709 13:14:51.451886  1294 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.467028 31396 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
W20260709 13:14:51.464154 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252185us
W20260709 13:14:51.468379 31294 connection.cc:256] RPC call timeout handler was delayed by -1.0618041919999541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2239s for a response
I20260709 13:14:51.429255  1343 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.446864  1428 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.453012  1237 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:51.486656 31383 raft_consensus.cc:1217] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[8.4-8.4]   Dedup: 8.4->[]
I20260709 13:14:51.487046 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:51.487370 31383 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 13:14:51.487104 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1233) took 1185 ms. Trace:
I20260709 13:14:51.488538 31391 rpcz_store.cc:276] 0709 13:14:50.301836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.302504 (+   668us) service_pool.cc:224] Handling call
0709 13:14:50.302891 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:50.397839 (+ 94948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.397883 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:50.398688 (+   805us) log.cc:844] Serialized 52 byte log entry
0709 13:14:51.140562 (+741874us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:51.475851 (+335289us) log.cc:844] Serialized 14 byte log entry
0709 13:14:51.476285 (+   434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.476379 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.486274 (+  9895us) raft_consensus.cc:1685] finished
0709 13:14:51.486296 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.487032 (+   736us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94407,"prepare.queue_time_us":119944,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":95113,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":1066799,"threads_started":2,"wal-append.queue_time_us":888715}
I20260709 13:14:51.497995 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:51.499333 30705 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:51.500329  1370 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:51.501169  1370 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.499173  1326 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.464638  1242 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.466423  1323 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.475286  1255 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:51.473357  1270 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.516050  1370 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.489167  1428 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.510924  1143 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:14:51.519861  1329 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.521143  1428 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.521692 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 562) took 1594 ms. Trace:
I20260709 13:14:51.521817  1428 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:51.522269  1428 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:51.522188 30631 rpcz_store.cc:276] 0709 13:14:49.927224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.362356 (+435132us) service_pool.cc:224] Handling call
0709 13:14:50.365798 (+  3442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.366368 (+   570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.366411 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.437030 (+ 70619us) log.cc:844] Serialized 26 byte log entry
0709 13:14:51.301687 (+864657us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:51.301751 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.301854 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:51.521426 (+219572us) raft_consensus.cc:1685] finished
0709 13:14:51.521450 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:51.521626 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17198,"raft.queue_time_us":724,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":24460,"thread_start_us":814695,"threads_started":1,"wal-append.queue_time_us":991893}
I20260709 13:14:51.513163  1242 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:51.502719  1295 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:51.527853  1323 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.529513 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1226) took 1386 ms. Trace:
I20260709 13:14:51.532649 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:51.536566 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:51.572293 30706 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:51.548652 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:51.486392  1424 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:51.485663  1411 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
W20260709 13:14:51.551959 30630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 784) took 1635 ms (client timeout 1722 ms). Trace:
I20260709 13:14:51.553901 30698 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:51.526984 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:51.579622 30702 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:51.556023 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:51.580076 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:51.581739 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:51.488348 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:51.558425 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:51.484884  1294 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:51.531867  1418 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.181s	user 0.001s	sys 0.013s
I20260709 13:14:51.571012 31388 rpcz_store.cc:276] 0709 13:14:50.142578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.143820 (+  1242us) service_pool.cc:224] Handling call
0709 13:14:50.144144 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.531391 (+387247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.529144 (+997753us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54008b2754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.529421 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888991616}
I20260709 13:14:51.493355  1385 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
W20260709 13:14:51.574913  1426 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:14:51.494582  1237 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.561128 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:51.598405 30557 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:14:51.598698  1404 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.293s	user 0.015s	sys 0.026s
W20260709 13:14:51.599359  1404 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.015s	sys 0.026s
I20260709 13:14:51.586419  1329 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.587169 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:51.513273  1255 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:51.470556  1427 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.612397 30557 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:51.613344 30559 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
W20260709 13:14:51.603559 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115306us
I20260709 13:14:51.516070  1343 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.960022  1424 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:51.570569  1439 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.961062  1439 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:51.602514 30702 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:51.606397  1294 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.963796  1306 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:51.964366  1424 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:51.966143  1306 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:51.576761  1194 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:51.966930  1194 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:51.968298  1385 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:51.969038  1440 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:51.569344  1270 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:51.970017 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 777) took 2055 ms. Trace:
I20260709 13:14:51.970420 30624 rpcz_store.cc:276] 0709 13:14:49.914163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.914303 (+   140us) service_pool.cc:224] Handling call
0709 13:14:49.914676 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.468250 (+553574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.969711 (+1501461us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400052154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.969921 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124512000}
I20260709 13:14:51.971033  1439 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.971657  1270 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.591682 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:51.615010 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54791.1us
W20260709 13:14:51.585661 30630 rpcz_store.cc:269] 0709 13:14:49.916213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:49.993457 (+ 77244us) service_pool.cc:224] Handling call
0709 13:14:51.528810 (+1535353us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b5400201054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:51.551837 (+ 23027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224671616}
I20260709 13:14:51.593478 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:51.786586 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:51.820091 30628 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:52.218406   929 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:51.836846 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1824) took 1580 ms. Trace:
W20260709 13:14:51.874375 30380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 69) took 22816 ms (client timeout 15000 ms). Trace:
I20260709 13:14:52.218477 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.219122 30554 rpcz_store.cc:276] 0709 13:14:50.256973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.259155 (+  2182us) service_pool.cc:224] Handling call
0709 13:14:50.259547 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.254284 (+994737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.836473 (+582189us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54001a9854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.836759 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2285095680}
I20260709 13:14:52.220551 30631 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:14:52.221212 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.222482 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.223858  1158 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:52.219066   929 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.224463  1158 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:51.584619 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:52.219307 30380 rpcz_store.cc:269] 0709 13:14:29.060605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:29.065743 (+  5138us) service_pool.cc:224] Handling call
0709 13:14:51.874219 (+22808476us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:45.275880 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:45.276560 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:45.276603 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:14:45.276628 (+    25us) tablet.cc:662] Decoding operations
0709 13:14:45.277143 (+   515us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:45.277209 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:45.277240 (+    31us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:14:45.277601 (+   361us) tablet.cc:701] Row locks acquired
0709 13:14:45.277618 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:14:45.277809 (+   191us) write_op.cc:270] Start()
0709 13:14:45.277962 (+   153us) write_op.cc:276] Timestamp: P: 1783602885277748 usec, L: 0
0709 13:14:45.277984 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:14:45.291659 (+ 13675us) log.cc:844] Serialized 922 byte log entry
0709 13:14:45.982499 (+690840us) op_driver.cc:464] REPLICATION: finished
0709 13:14:46.195957 (+213458us) write_op.cc:301] APPLY: starting
0709 13:14:46.196129 (+   172us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:46.196304 (+   175us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:46.196500 (+   196us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:46.197070 (+   570us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:46.197303 (+   233us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:14:46.197379 (+    76us) write_op.cc:312] APPLY: finished
0709 13:14:46.197679 (+   300us) log.cc:844] Serialized 28 byte log entry
0709 13:14:46.197944 (+   265us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:46.198092 (+   148us) write_op.cc:454] Released schema lock
0709 13:14:46.198152 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:46.243519 (+ 45367us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:46.245076 (+  1557us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":698179,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":189867,"apply.run_cpu_time_us":2752,"apply.run_wall_time_us":23775,"num_ops":2,"prepare.queue_time_us":181328,"prepare.run_cpu_time_us":65660,"prepare.run_wall_time_us":410477,"raft.queue_time_us":186543,"raft.run_cpu_time_us":42961,"raft.run_wall_time_us":161286,"replication_time_us":681435,"thread_start_us":733906,"threads_started":4,"wal-append.queue_time_us":462341,"wal-append.run_cpu_time_us":43789,"wal-append.run_wall_time_us":1501057}]]}
I20260709 13:14:52.228684 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.223873 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:51.971794 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:51.975692 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:51.975690 30523 leader_election.cc:336] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.722s (SENT)
I20260709 13:14:51.979592 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1818) took 1759 ms. Trace:
W20260709 13:14:52.233373 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 257708us
I20260709 13:14:51.985845 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.232308 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.233079 30556 rpcz_store.cc:276] 0709 13:14:50.220436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.220575 (+   139us) service_pool.cc:224] Handling call
0709 13:14:50.220975 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:51.146876 (+925901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.979281 (+832405us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b54002d0ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.979522 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1979103104}
I20260709 13:14:52.235180 30624 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:51.986655 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:52.236222 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.237291 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.237576   897 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.238155   897 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.232230 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:52.239127 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.239989  1464 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:52.240713  1464 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:51.689082 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:51.976475 31394 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:51.980131 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:52.364285 31378 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:51.978148 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.373046 31388 raft_consensus.cc:2410] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:51.981999 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.378444 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:52.379015 31396 raft_consensus.cc:2410] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:14:52.017025 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 558) took 2092 ms. Trace:
I20260709 13:14:52.380088 30632 rpcz_store.cc:276] 0709 13:14:49.924443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.083779 (+159336us) service_pool.cc:224] Handling call
0709 13:14:50.574007 (+490228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.602232 (+ 28225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.016667 (+1414435us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54000541d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.016926 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126269184}
I20260709 13:14:52.381820 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.382153  1440 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:51.826198  1155 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:52.389873  1155 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:51.898451  1156 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:51.969286  1194 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.824208  1205 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:51.635440  1237 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.665287  1242 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.638090  1255 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.423506  1440 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:51.806424  1273 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.686394  1295 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:52.426311  1250 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:52.427052  1295 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:52.429411  1295 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:51.619303  1303 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.433534  1303 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.021150  1311 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.440244  1311 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:51.966557  1306 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:52.444622  1306 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.445852  1306 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:51.625016  1319 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.449208  1319 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.665617  1323 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:52.450871 30700 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: []
I20260709 13:14:52.452360 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1540) took 1586 ms. Trace:
I20260709 13:14:52.452716 30700 rpcz_store.cc:276] 0709 13:14:50.866094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.867293 (+  1199us) service_pool.cc:224] Handling call
0709 13:14:50.971769 (+104476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.451417 (+1479648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.452046 (+   629us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540001f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.452272 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3401082240}
I20260709 13:14:51.690788  1326 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.455282  1326 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.457907  1326 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:51.823865  1349 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:52.460541 31391 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:51.652341  1356 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.464706  1356 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.820667  1350 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.474408  1350 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.969628  1361 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:52.478390  1361 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:52.479506  1361 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.480566  1361 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:51.973521  1308 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:51.512248  1258 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.585259  1402 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:52.488973  1402 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:51.860095  1403 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:52.026515 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:52.495468  1403 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:51.972800  1411 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:52.497289 30629 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:52.497598  1411 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:52.498106  1411 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:52.499106  1411 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.500031 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1235) took 2347 ms. Trace:
I20260709 13:14:51.612439  1425 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:51.648968  1429 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
W20260709 13:14:51.659610  1431 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.221s	user 0.001s	sys 0.001s
W20260709 13:14:51.656405  1433 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:14:52.508883 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1201) took 2268 ms. Trace:
I20260709 13:14:52.509344 30699 rpcz_store.cc:276] 0709 13:14:50.240142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.241559 (+  1417us) service_pool.cc:224] Handling call
0709 13:14:50.241926 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:50.394989 (+153063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:50.395040 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:50.427386 (+ 32346us) log.cc:844] Serialized 26 byte log entry
0709 13:14:51.237639 (+810253us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:51.237692 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:51.889079 (+651387us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54005a2dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:51.889152 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:52.508536 (+619384us) raft_consensus.cc:1685] finished
0709 13:14:52.508561 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.508812 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2265,"raft.queue_time_us":14152,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":14238,"spinlock_wait_cycles":144666368,"thread_start_us":738566,"threads_started":1,"wal-append.queue_time_us":938821}
I20260709 13:14:51.819846  1434 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.512632  1434 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.011483  1435 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.515508  1435 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.516693  1326 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:52.517428  1326 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:51.837707 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:51.938568  1438 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:51.671409  1427 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:51.619071  1428 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:52.113049 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.113895  1463 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:52.528055 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.527789  1463 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:52.529035 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.500536  1411 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:52.530195  1463 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:52.530471 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.127597  1466 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:52.531111  1466 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.533001  1428 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:52.533478  1466 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.132213 31361 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:52.134948 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1237) took 1886 ms. Trace:
I20260709 13:14:52.535602 30633 rpcz_store.cc:276] 0709 13:14:50.248663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.134159 (+1885496us) service_pool.cc:224] Handling call
0709 13:14:52.134396 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.134583 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.134636 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.134679 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:52.134709 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.134768 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.134885 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.536957 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.537580 30633 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:52.520952 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 683321us
W20260709 13:14:52.161381 30551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1186) took 1876 ms (client timeout 2217 ms). Trace:
I20260709 13:14:52.538717 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:52.539203 30551 rpcz_store.cc:269] 0709 13:14:50.284594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.433890 (+149296us) service_pool.cc:224] Handling call
0709 13:14:52.161038 (+1727148us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b5400187254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.161255 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968161664}
I20260709 13:14:52.539821  1349 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:52.540727 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.541989 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.162792 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 789) took 2245 ms. Trace:
W20260709 13:14:52.162935 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1177) took 1885 ms (client timeout 2237 ms). Trace:
W20260709 13:14:52.542894 30555 rpcz_store.cc:269] 0709 13:14:50.277468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.279844 (+  2376us) service_pool.cc:224] Handling call
0709 13:14:52.162733 (+1882889us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.162868 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845346176}
I20260709 13:14:52.544225 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.538558 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.167193 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.545584 30553 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:52.546166  1349 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:52.546490 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.547312 30553 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:52.547616  1319 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:52.175559 31364 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:52.548096  1319 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:52.193068  1385 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:52.549216  1385 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.549166  1319 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:51.922482 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.225623  1158 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:51.747571 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 136864us
I20260709 13:14:52.235256 30552 raft_consensus.cc:2410] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:52.554272  1462 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:52.554832 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.554941  1462 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.555693 30552 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:52.236217 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:51.620196 30706 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:14:52.562902 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1203) took 1067 ms. Trace:
I20260709 13:14:52.563381 30706 rpcz_store.cc:276] 0709 13:14:51.496082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.496342 (+   260us) service_pool.cc:224] Handling call
0709 13:14:52.562795 (+1066453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.563997  1385 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:52.229492 30550 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:52.242268  1464 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:52.249373 30631 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:52.249752  1461 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.566951  1461 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:51.588521 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 99193.2us
I20260709 13:14:52.573267 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.573649 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:52.250831   897 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:52.251703 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.575822 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.576043 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1242) took 2229 ms. Trace:
I20260709 13:14:52.576529 30559 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:52.576712 30633 rpcz_store.cc:276] 0709 13:14:50.347011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.575124 (+2228113us) service_pool.cc:224] Handling call
0709 13:14:52.575470 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.575690 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.575714 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.575760 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:52.575794 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.575854 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.575984 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.577466  1319 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:52.577617 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.252594 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.253768 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:52.579396 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 325710us
I20260709 13:14:52.579474 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.580299 30554 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 4: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:52.581132 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:52.254706 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 565704us
I20260709 13:14:52.583163 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.255712   929 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.255934  1302 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:52.583236  1475 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:52.583986  1349 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.584611  1302 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:52.282891 30630 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:52.585239  1302 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:52.585613 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:52.258833  1329 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:52.308470  1394 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:52.586825  1329 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.290051 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:52.298504 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.003s	sys 0.012s
I20260709 13:14:52.337822 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:14:52.345039 30382 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 8, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.351397 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.312057 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1168) took 1706 ms. Trace:
I20260709 13:14:51.592700   769 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.382575 30632 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:52.390336  1155 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:52.390317 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.398175 31378 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:52.399489 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.420138 30627 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:52.598030 30550 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:52.438167  1273 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:52.500535 30629 rpcz_store.cc:276] 0709 13:14:50.152212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.376728 (+1224516us) service_pool.cc:224] Handling call
0709 13:14:51.377104 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.497860 (+1120756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.499733 (+  1873us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54004aabd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.499954 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2575665920}
I20260709 13:14:52.587231 30551 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
W20260709 13:14:52.502120 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 476224us
I20260709 13:14:52.588124 31389 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:14:52.503927  1255 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:52.595691 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1238) took 2337 ms. Trace:
I20260709 13:14:52.476963  1350 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:52.506836  1311 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.518025  1435 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.523319  1438 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:52.523629  1356 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:52.524698  1434 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:52.526338 31367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1160) took 2006 ms (client timeout 1607 ms). Trace:
I20260709 13:14:52.608275  1294 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:52.164799  1143 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:52.629489  1143 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.631511 30556 raft_consensus.cc:2410] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:52.631937  1143 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:52.632758 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 562) took 1036 ms. Trace:
I20260709 13:14:52.633198 30556 rpcz_store.cc:276] 0709 13:14:51.596172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.235888 (+639716us) service_pool.cc:224] Handling call
0709 13:14:52.632544 (+396656us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.632699 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":167429504}
I20260709 13:14:52.617556   897 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:52.617789  1270 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:52.639993  1270 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:52.641191  1270 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.519799  1326 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:52.551739  1156 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.551931 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:52.644420 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 92589.3us
I20260709 13:14:52.238308 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.568750 30631 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:52.586383  1302 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.571456  1205 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:52.542783 30626 rpcz_store.cc:276] 0709 13:14:49.917090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.068167 (+151077us) service_pool.cc:224] Handling call
0709 13:14:50.068624 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:50.216703 (+148079us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:14:50.217109 (+   406us) log.cc:844] Serialized 14 byte log entry
0709 13:14:50.630683 (+413574us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:50.630747 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:50.630817 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.162417 (+1531600us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.162491 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.162711 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":754,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":459,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":24230,"spinlock_wait_cycles":339743488,"thread_start_us":413112,"threads_started":1,"wal-append.queue_time_us":969392,"wal-append.run_cpu_time_us":3814,"wal-append.run_wall_time_us":1100904}
I20260709 13:14:52.574590  1428 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:52.626713  1461 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:52.583350 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.584540  1349 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.587054 30557 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:14:52.368381 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1234) took 2217 ms. Trace:
I20260709 13:14:52.370847  1439 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:14:52.682518 30628 rpcz_store.cc:276] 0709 13:14:50.151297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.247159 (+1095862us) service_pool.cc:224] Handling call
0709 13:14:51.268710 (+ 21551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.218823 (+950113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.368002 (+149179us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.368299 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267411200}
I20260709 13:14:52.684582 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.502811  1402 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:52.605918 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:52.688022 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82167.5us
I20260709 13:14:52.590794 31383 rpcz_store.cc:276] 0709 13:14:50.605836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.656555 (+ 50719us) service_pool.cc:224] Handling call
0709 13:14:51.486483 (+829928us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:51.487837 (+  1354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.311765 (+823928us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001afed0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.311977 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907719040}
I20260709 13:14:52.606791 30632 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:14:52.720093 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.720662 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:52.588451 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.004s	sys 0.012s
W20260709 13:14:52.607318 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 217096us
W20260709 13:14:52.659945 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128029us
I20260709 13:14:52.596856 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.590152 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:52.724056 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 127341us
I20260709 13:14:52.608804 30630 rpcz_store.cc:276] 0709 13:14:50.258436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.216537 (+1958101us) service_pool.cc:224] Handling call
0709 13:14:52.216908 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.585512 (+368604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.586569 (+  1057us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b54004a88d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.595631 (+  9062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151152512}
I20260709 13:14:52.724560 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.725129 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:52.725610 30630 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:52.726260 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1248) took 2121 ms. Trace:
W20260709 13:14:52.612134 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 212742us
I20260709 13:14:52.726432 30702 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:52.726575 30630 rpcz_store.cc:276] 0709 13:14:50.604434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.724948 (+2120514us) service_pool.cc:224] Handling call
0709 13:14:52.725289 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.726026 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.726207 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.728912 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 561) took 1133 ms. Trace:
I20260709 13:14:52.593961 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:52.614202 30629 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:14:52.593720 30552 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:14:52.615221 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 560) took 1024 ms. Trace:
I20260709 13:14:52.616812 30550 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:52.620831 30559 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:52.621639 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 790) took 2482 ms. Trace:
I20260709 13:14:52.583546  1462 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:52.587319  1329 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:52.747381 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:52.625474 31367 rpcz_store.cc:269] 0709 13:14:50.520026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.520171 (+   145us) service_pool.cc:224] Handling call
0709 13:14:52.526138 (+2005967us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.526264 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4540149632}
I20260709 13:14:52.748903 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 563) took 1152 ms. Trace:
I20260709 13:14:52.749771 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.727612  1402 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:52.544970 30555 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:52.646996 30624 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:14:52.647387 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1239) took 2358 ms. Trace:
I20260709 13:14:52.754747 30699 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:52.749404 30559 rpcz_store.cc:276] 0709 13:14:51.596430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.597117 (+1000687us) service_pool.cc:224] Handling call
0709 13:14:52.620679 (+ 23562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.748544 (+127865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.748836 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.548167 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.755257 30624 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:52.755795 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:52.756055 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 794) took 1260 ms. Trace:
I20260709 13:14:52.756655 30624 rpcz_store.cc:276] 0709 13:14:51.495179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.754768 (+1259589us) service_pool.cc:224] Handling call
0709 13:14:52.755080 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.755707 (+   627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.755933 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.651734 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.578325 30633 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:14:52.650915  1435 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:14:52.645872  1294 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:52.693358 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:52.690014 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:52.767356 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:52.582348 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.726495 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.768421 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 565) took 1170 ms. Trace:
I20260709 13:14:52.768797 30554 rpcz_store.cc:276] 0709 13:14:51.604668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.582131 (+977463us) service_pool.cc:224] Handling call
0709 13:14:52.768355 (+186224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.729400 30557 rpcz_store.cc:276] 0709 13:14:51.595867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.922072 (+326205us) service_pool.cc:224] Handling call
0709 13:14:52.728723 (+806651us) spinlock_profiling.cc:243] Waited 7.51 ms on lock 0x7b54003a5f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:52.728844 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17292544}
I20260709 13:14:52.737299 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.729939   929 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.771458 30702 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:52.737362 30627 rpcz_store.cc:276] 0709 13:14:50.136398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.442911 (+306513us) service_pool.cc:224] Handling call
0709 13:14:50.466776 (+ 23865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.604333 (+2137557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.613942 (+  9609us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b5400569ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:52.618807 (+  4865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4491262720}
I20260709 13:14:52.739302 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.724608 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 7 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:14:52.775645 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 9 to 11, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.729001 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 571) took 2108 ms. Trace:
I20260709 13:14:52.777281 30630 rpcz_store.cc:276] 0709 13:14:50.620844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.727870 (+2107026us) service_pool.cc:224] Handling call
0709 13:14:52.728298 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.728556 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.728596 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:52.728664 (+    68us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:52.728712 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.728785 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:52.728923 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.778986 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.768878 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.775682 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.780330 31383 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:52.724175   897 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:52.736930 30551 rpcz_store.cc:276] 0709 13:14:51.590978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.586687 (+995709us) service_pool.cc:224] Handling call
0709 13:14:52.587033 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.614871 (+ 27838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.615152 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.737753 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1247) took 2383 ms. Trace:
I20260709 13:14:52.784795 30629 rpcz_store.cc:276] 0709 13:14:50.354643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.604899 (+2250256us) service_pool.cc:224] Handling call
0709 13:14:52.605215 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.737464 (+132249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.737695 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.745858 30550 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:14:52.611167  1265 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.753451  1461 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:14:52.748996 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 155155us
I20260709 13:14:52.753705  1338 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:52.795388 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.754601 30631 rpcz_store.cc:276] 0709 13:14:50.288777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.566218 (+2277441us) service_pool.cc:224] Handling call
0709 13:14:52.566592 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.646977 (+ 80385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.647301 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.760630 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.761974 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:52.652431  1428 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:52.720319 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1246) took 2366 ms. Trace:
W20260709 13:14:52.780656 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54234.3us
I20260709 13:14:52.645746  1156 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:52.781518 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 210590us
I20260709 13:14:52.803450 30632 rpcz_store.cc:276] 0709 13:14:50.354361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.594912 (+2240551us) service_pool.cc:224] Handling call
0709 13:14:52.595288 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:52.719951 (+124663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.720253 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.782579 30630 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:52.742239 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.729702  1311 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:52.828799 30632 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:52.829069  1311 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.784986 30382 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 7 to 10, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 10 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:52.831806  1311 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:52.750882  1158 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.643409  1434 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 13:14:52.752771  1444 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:14:52.788892 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98999.7us
I20260709 13:14:52.753047  1439 consensus_queue.cc:1243] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:14:52.844733 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59734.2us
I20260709 13:14:52.759323 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1244) took 2405 ms. Trace:
I20260709 13:14:52.850186 30633 rpcz_store.cc:276] 0709 13:14:50.353785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.577727 (+2223942us) service_pool.cc:224] Handling call
0709 13:14:52.578028 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.758946 (+180918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.759262 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:14:52.786707  1402 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:52.761686  1435 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:52.807173 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.827518 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:52.859963 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:52.858558  1435 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:52.831722 30382 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:52.747458  1143 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:52.766306  1385 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:14:52.773202  1258 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:52.781381  1252 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:52.747665  1475 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:14:52.784219   847 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.780133  1302 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:52.750670  1326 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:14:52.789904 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:52.646896  1155 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:52.795578  1343 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:52.877264  1205 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:52.854876  1444 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:52.855518  1402 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:52.874781  1252 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.879341  1402 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.834378  1408 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.882117  1252 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:52.682083  1438 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.882295  1408 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.883033  1438 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:52.884230  1156 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:52.884693  1408 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:52.879365  1294 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:52.886240  1356 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:52.794997  1265 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:52.886737  1356 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:52.887212  1356 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:52.856007  1461 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:52.857472  1158 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.860605 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.779142  1404 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:14:52.852890 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.854480 30700 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:52.787569   769 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:52.866253 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:52.865994 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:52.868269 30629 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
W20260709 13:14:52.870186 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.826763  1428 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:52.794822  1427 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:52.796638  1370 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:52.877439  1343 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.878333  1205 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:52.856688  1434 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:52.889714  1461 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.896093 30625 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:14:52.833977  1338 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:52.913072 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:52.790222   897 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.784225   929 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:52.915611   769 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:52.895861 30631 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:52.942663 31364 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:14:52.955849 31364 pending_rounds.cc:85] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 3
I20260709 13:14:52.956146 31364 pending_rounds.cc:107] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:14:52.956388 31364 raft_consensus.cc:2889] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:52.943751 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.944923 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:52.958673 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:14:52.891863 31389 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:14:52.948632 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 206450us
W20260709 13:14:52.892308  1411 consensus_queue.cc:1175] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
I20260709 13:14:52.951797 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:52.953289 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 163470us
W20260709 13:14:52.934321 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74448.5us
I20260709 13:14:52.968619 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:52.923803  1389 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:52.938635  1408 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:52.939862  1343 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:52.935989  1242 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:52.890877   847 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:52.992616 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:52.993513  1482 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:52.947144  1326 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:52.941774  1303 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.000325 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:52.971405   769 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:52.971771  1427 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:52.988327  1389 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:52.915165  1444 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:52.987108  1295 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:14:52.991688  1258 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:14:53.012751  1143 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:53.141224  1143 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.142302  1143 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:52.861861 30382 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:53.143225   897 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.143385   847 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.146157 30382 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:52.894761 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.096500 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.893478 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:52.907824 30559 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:53.004638  1242 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.115193 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 798) took 1616 ms. Trace:
I20260709 13:14:53.154379  1158 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.156157  1242 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.154808 30624 rpcz_store.cc:276] 0709 13:14:51.498392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.757695 (+1259303us) service_pool.cc:224] Handling call
0709 13:14:52.759404 (+  1709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:52.790669 (+ 31265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.790710 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:52.791255 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 13:14:52.956172 (+164917us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:52.956250 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.956324 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.114420 (+158096us) raft_consensus.cc:1685] finished
0709 13:14:53.114466 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.114884 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29000,"raft.queue_time_us":36693,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6072,"thread_start_us":164582,"threads_started":1,"wal-append.queue_time_us":213129}
W20260709 13:14:52.921873 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69092.3us
W20260709 13:14:53.117772 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.161299 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:53.117960 30654 debug-util.cc:398] Leaking SignalData structure 0x7b080019dde0 after lost signal to thread 21529
I20260709 13:14:53.034651  1389 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:53.010082 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99934.2us
W20260709 13:14:53.129199 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 160435us
W20260709 13:14:53.131345 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 130915us
W20260709 13:14:53.132123 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.129925 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 137245us
W20260709 13:14:52.900236 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.192577 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:53.158174  1205 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:53.193534 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.194008 30557 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:14:53.194242  1205 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.194795 30559 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 2.
I20260709 13:14:53.194842  1205 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:53.196372  1483 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:53.197185  1483 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:53.025180  1349 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.011925  1303 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.010824  1326 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:53.118458 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.018786  1427 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:53.037657  1484 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:14:53.166364 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 792) took 3028 ms. Trace:
I20260709 13:14:53.324115  1427 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.324857 30625 rpcz_store.cc:276] 0709 13:14:50.137172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:50.534199 (+397027us) service_pool.cc:224] Handling call
0709 13:14:50.534541 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.941483 (+2406942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.165429 (+223946us) spinlock_profiling.cc:243] Waited 2.36 s on lock 0x7b54005f37d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:53.165996 (+   567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5429959040}
W20260709 13:14:53.325608  1504 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.002s	sys 0.000s
I20260709 13:14:53.326994 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1191) took 1210 ms. Trace:
I20260709 13:14:53.327412 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:53.327592 30556 rpcz_store.cc:276] 0709 13:14:52.115979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.634109 (+518130us) service_pool.cc:224] Handling call
0709 13:14:52.634530 (+   421us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:52.634981 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:52.635024 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:14:52.635694 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 13:14:52.751222 (+115528us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:52.751267 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:52.751346 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.326386 (+575040us) raft_consensus.cc:1685] finished
0709 13:14:53.326450 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.326758 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4339,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"thread_start_us":115073,"threads_started":1,"wal-append.queue_time_us":231604}
I20260709 13:14:53.143066   929 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.181401  1500 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:53.330883  1500 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:53.332435  1500 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.181543  1258 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.211373 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:53.335103  1258 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:53.335978 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 124910us
I20260709 13:14:53.337953 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:53.215308  1370 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:53.220681 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:53.220355 30382 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:53.334468  1500 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:53.339463 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 119026us
I20260709 13:14:53.340098 30382 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:14:53.221791 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.343060  1505 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:53.235231 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 369149us
I20260709 13:14:53.344358  1505 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.345260  1507 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:53.345911  1507 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.346387  1507 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:53.228665  1408 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:53.239537 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1841) took 1005 ms. Trace:
W20260709 13:14:53.241345 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08003ecc20 after lost signal to thread 30416
I20260709 13:14:53.350786 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.350379  1408 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.352176 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 804) took 1059 ms. Trace:
I20260709 13:14:53.235255  1323 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260709 13:14:53.352809 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08001aacc0 after lost signal to thread 30583
W20260709 13:14:53.353253 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08001f3b00 after lost signal to thread 30584
I20260709 13:14:53.245579  1250 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:53.244218  1194 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:53.246414  1265 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.354884  1194 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:53.355446  1194 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:53.355816  1350 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.356407  1350 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.255620  1338 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.255815 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1837) took 1040 ms. Trace:
I20260709 13:14:53.358923 30555 rpcz_store.cc:276] 0709 13:14:52.215128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.753488 (+538360us) service_pool.cc:224] Handling call
0709 13:14:52.753838 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.754080 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:52.771348 (+ 17268us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.233973 (+462625us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b1800316000. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:14:53.254684 (+ 20711us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.254828 (+   144us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:53.254986 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.255111 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.255558 (+   447us) inbound_call.cc:177] Queueing success response
0709 13:14:53.260170 (+  4612us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
Metrics: {"mutex_wait_us":19920,"prepare.queue_time_us":42697,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":388553,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":363,"spinlock_wait_cycles":9920128,"thread_start_us":82578,"threads_started":1,"wal-append.queue_time_us":135945}
I20260709 13:14:53.195108  1489 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
W20260709 13:14:53.363142 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08000d6720 after lost signal to thread 30657
I20260709 13:14:53.261041 30557 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:14:53.363619 30654 debug-util.cc:398] Leaking SignalData structure 0x7b0800276d00 after lost signal to thread 30658
I20260709 13:14:53.197744  1483 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:14:53.261420  1486 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.322s	user 0.002s	sys 0.000s
I20260709 13:14:53.258957   769 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.263038 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.368141 31389 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:14:53.268987  1155 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:53.269536 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.369462  1326 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.370146 30631 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.271394 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.272398  1403 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.372270  1403 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.352620 30629 rpcz_store.cc:276] 0709 13:14:52.254212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.785751 (+531539us) service_pool.cc:224] Handling call
0709 13:14:52.786059 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.912700 (+126641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.282556 (+369856us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b54004aa6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:53.313239 (+ 30683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188567424}
I20260709 13:14:53.275657  1242 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.374392  1350 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.370231   769 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.275866  1490 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:53.276794  1495 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.237s	user 0.003s	sys 0.000s
I20260709 13:14:53.277947 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:53.377676 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 100336us
W20260709 13:14:53.279325  1497 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.243s	user 0.000s	sys 0.002s
I20260709 13:14:53.282164 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.379469 30633 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.279388 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:53.380064  1499 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.347s	user 0.000s	sys 0.002s
I20260709 13:14:53.305197 30551 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:53.383468  1502 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.331s	user 0.000s	sys 0.003s
I20260709 13:14:53.375052  1242 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.385779 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 800) took 1169 ms. Trace:
W20260709 13:14:53.387727  1501 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.339s	user 0.000s	sys 0.003s
I20260709 13:14:53.379482  1490 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:53.389683  1490 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.390031  1490 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:53.390331  1294 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:14:53.390661 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52903.9us
W20260709 13:14:53.306787  1503 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.139s	user 0.006s	sys 0.000s
I20260709 13:14:53.386322 30628 rpcz_store.cc:276] 0709 13:14:52.216137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.767409 (+551272us) service_pool.cc:224] Handling call
0709 13:14:52.789722 (+ 22313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.116993 (+327271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.117059 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.118551 (+  1492us) log.cc:844] Serialized 26 byte log entry
0709 13:14:53.170521 (+ 51970us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:53.170845 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.253583 (+ 82738us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b18001562e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:14:53.385255 (+131672us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54007229d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:53.385310 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.385377 (+    67us) raft_consensus.cc:1685] finished
0709 13:14:53.385393 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.385580 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169612,"raft.queue_time_us":199778,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":435191296,"thread_start_us":20900,"threads_started":1,"wal-append.queue_time_us":38199}
I20260709 13:14:53.393200 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:53.387809  1242 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.394413  1489 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:53.395118  1489 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.395591  1489 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:53.374881 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.398669  1350 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:53.399473 31361 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:14:53.402024 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.338050  1258 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.213019  1389 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:53.402426 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.343137 30382 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 5 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:53.405915 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.405762  1143 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.338394  1349 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:14:53.306849 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114611us
I20260709 13:14:53.406985 31391 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:53.377702  1428 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.407411  1349 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:53.236768 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 370376us
I20260709 13:14:53.354444  1250 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:53.215381  1434 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:53.409232  1250 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:53.343051 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.410418 31364 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:53.390813  1294 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:53.349397 30553 rpcz_store.cc:276] 0709 13:14:52.234397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.756776 (+522379us) service_pool.cc:224] Handling call
0709 13:14:52.757190 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:52.757494 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:52.896493 (+138999us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.237427 (+340934us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.238098 (+   671us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.238335 (+   237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:53.238417 (+    82us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:53.238652 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.238804 (+   152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.239270 (+   466us) inbound_call.cc:177] Queueing success response
0709 13:14:53.257607 (+ 18337us) spinlock_profiling.cc:243] Waited 46.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
Metrics: {"mutex_wait_us":145638,"prepare.queue_time_us":155548,"prepare.run_cpu_time_us":157,"prepare.run_wall_time_us":162,"raft.queue_time_us":155798,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":364,"spinlock_wait_cycles":107405568,"thread_start_us":320213,"threads_started":2,"wal-append.queue_time_us":221883}
I20260709 13:14:53.411586  1323 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:14:53.412604  1255 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.794s	user 0.020s	sys 0.008s
I20260709 13:14:53.412011  1294 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.345744 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:53.408952  1349 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.415203 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:53.416786  1356 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.013s
I20260709 13:14:53.419178  1303 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:53.419919  1329 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.010s
I20260709 13:14:53.420392  1505 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:53.427013 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08001d04a0 after lost signal to thread 30731
I20260709 13:14:53.427251 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.405928 30382 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 6 to 11, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:14:53.436414  1434 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:53.437237  1143 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:53.437906  1435 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.577s	user 0.014s	sys 0.007s
W20260709 13:14:53.438165  1302 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.547s	user 0.005s	sys 0.017s
I20260709 13:14:53.421187 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:53.421837 31383 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:53.468705 31364 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:53.474795 31391 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:53.481273  1428 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:53.484018 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 68960.2us
I20260709 13:14:53.507673  1294 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:53.486718 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.487071  1303 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:53.487706 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:53.489149 30654 debug-util.cc:398] Leaking SignalData structure 0x7b080004f720 after lost signal to thread 31426
I20260709 13:14:53.489255 31361 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:14:53.489989 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 144447us
I20260709 13:14:53.478538  1356 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:53.492367 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81324.9us
I20260709 13:14:53.493094  1155 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:53.479115  1329 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.494858  1255 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:53.478595   769 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260709 13:14:53.481274 30624 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:53.500501   897 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 13:14:53.498178  1483 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.544435 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.550813 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.511596 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.501338  1438 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:14:53.500546  1343 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.501729  1462 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:14:53.513054 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.513478  1507 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.514379 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.515614 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.517030  1194 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.525290  1270 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:53.526144 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.506896  1273 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.012s
I20260709 13:14:53.514261  1295 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.503050 30382 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 10 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.485330  1435 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.546567  1255 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:53.531170  1205 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.535229  1303 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.533133  1250 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.557974  1356 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.571713 30632 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.569816 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:53.546595  1350 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:53.575598 30628 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:53.588699 30629 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.541684 31383 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:14:53.601662 30550 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:53.585204 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.560454  1483 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.578190  1255 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:53.591166 31378 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:53.593650  1250 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.594743 30558 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:14:53.595702  1319 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.017s	user 0.017s	sys 0.008s
I20260709 13:14:53.596019  1205 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.597309 31389 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:53.582012  1303 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:53.598709  1155 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.599606 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.598300 30382 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.626585 30626 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 5: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.627456 31367 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:14:53.628227 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:53.629900 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.635118 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:14:53.655489 30558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 564) took 2058 ms (client timeout 1830 ms). Trace:
W20260709 13:14:53.656386 30558 rpcz_store.cc:269] 0709 13:14:51.596660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:51.613669 (+ 17009us) service_pool.cc:224] Handling call
0709 13:14:53.655054 (+2041385us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000bf9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:53.655311 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816607104}
I20260709 13:14:53.658083 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:53.659202 30698 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:53.659797 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.660266 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:53.661219 30627 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.661355 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 807) took 1012 ms. Trace:
I20260709 13:14:53.662016 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.661896 30626 rpcz_store.cc:276] 0709 13:14:52.648822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.651418 (+  2596us) service_pool.cc:224] Handling call
0709 13:14:53.661072 (+1009654us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54007aa654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:53.661252 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943326464}
I20260709 13:14:53.662955 30706 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:53.664527 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.665428 30703 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 4.
I20260709 13:14:53.665966  1435 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.660455 31391 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:14:53.667174 31391 pending_rounds.cc:85] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:53.666862 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.667467 31391 pending_rounds.cc:107] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:14:53.667666  1385 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.667836 31391 raft_consensus.cc:2889] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:53.668489  1385 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.668833 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.661379 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:53.669833 31367 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:53.659293 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.671249 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.671875 30700 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:53.674400 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.675271 30624 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:53.678848  1507 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.678488  1303 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.600845  1489 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.681627  1489 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.682036  1303 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.659845  1411 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.683593  1194 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.683367  1411 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.685869  1411 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.686319  1155 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.588677  1270 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.688766  1155 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:53.704770 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.706431 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:53.706867  1329 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.711418 30558 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:53.711697 30626 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.707258 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:53.713095 30632 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.708206  1343 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:53.708293 30700 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:14:53.713905 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.714386 30700 pending_rounds.cc:85] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 5
I20260709 13:14:53.606076  1295 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.714807 30550 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:53.695917  1350 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:14:53.714805 30700 pending_rounds.cc:107] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:14:53.716068 30700 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:53.709599  1483 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:14:53.708890 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.718703  1483 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:53.719202  1483 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:53.710444  1319 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:14:53.720335  1483 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.721840 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.721295 30382 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 6 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 7 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.722208 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.722661 31391 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:53.723246 30553 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:53.721834  1439 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:53.706960 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.724184  1439 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:53.723505 30382 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 6 to 8, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 8 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:14:53.724598 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.724550 30552 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:53.725334  1439 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.725221 30382 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 4 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 7 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:14:53.729246   847 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:53.730913  1303 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:53.731336  1303 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:53.732429 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.733626 30557 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:53.732334  1303 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:53.737426  1303 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:53.737843 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.737453  1483 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:53.740499 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.741549 30703 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:53.743666 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.744670 30699 raft_consensus.cc:2410] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:14:53.744889 30553 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:14:53.747407 30700 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:53.746598 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.749042  1252 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:53.709306  1270 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.750485 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:53.754411 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:53.760105 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.768409 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 329083us
I20260709 13:14:53.726470  1439 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:53.770526 30552 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
W20260709 13:14:53.780500 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.786404 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:53.782897 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 213120us
W20260709 13:14:53.771162  1482 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.074s
I20260709 13:14:53.718369  1295 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:53.798625 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.632344  1490 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.792626  1403 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:53.810705 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.811482 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.813416 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.807771 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.815838 30699 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:53.805280  1343 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:14:53.803478  1482 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.008s	sys 0.075s
I20260709 13:14:53.818725 30555 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:14:53.776592 30628 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:53.810703  1403 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.825187 30558 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:14:53.834127 30625 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 3: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:53.816555 30382 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 3 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 6 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:14:53.833796 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.840692 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:53.841495 30701 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:53.843061 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:53.843770 30698 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:14:53.844360 30707 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:53.845721 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:53.846725 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65367.2us
I20260709 13:14:53.847105 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.847681 30704 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:14:53.848516 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.848846 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.848903 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:53.849749  1343 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:53.851084  1343 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.852653 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.853070 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.853328 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:53.853773 31396 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:14:53.854327  1489 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:53.855085 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.855552  1489 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:53.856153   847 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:53.856696 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.855803 31391 raft_consensus.cc:2376] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9. Re-sending same reply.
I20260709 13:14:53.858223 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.856796   847 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:53.805706  1270 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:14:53.874603 31367 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 1.
I20260709 13:14:53.878350 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.887897 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.890734 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:53.894227  1270 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:53.894551  1462 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.894796  1270 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:53.895269  1462 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:53.895903  1270 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:53.897493  1270 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:53.899466  1462 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.899679 30702 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:14:53.904338 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:53.905722 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:53.900098 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:53.908397  1512 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:14:53.909391 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:53.871748  1500 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.532s	user 0.015s	sys 0.000s
I20260709 13:14:53.860543  1403 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:53.824126  1385 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:53.927279 30707 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:53.926725 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:14:53.929109 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:53.911350 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.934353 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.935425 30558 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:53.939088 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 810) took 1284 ms. Trace:
I20260709 13:14:53.939684 30630 rpcz_store.cc:276] 0709 13:14:52.650475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:52.828838 (+178363us) service_pool.cc:224] Handling call
0709 13:14:52.829180 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.117058 (+287878us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.117134 (+    76us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.249750 (+132616us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:53.250557 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 13:14:53.533700 (+283143us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:53.547397 (+ 13697us) log.cc:844] Serialized 14 byte log entry
0709 13:14:53.554459 (+  7062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:53.568809 (+ 14350us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:53.907677 (+338868us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:53.909994 (+  2317us) raft_consensus.cc:1685] finished
0709 13:14:53.910017 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:53.934862 (+ 24845us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2747,"prepare.queue_time_us":12738,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":25270,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":26009,"spinlock_wait_cycles":29953664,"thread_start_us":515209,"threads_started":2,"wal-append.queue_time_us":412797}
I20260709 13:14:53.874032  1356 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:53.942071  1356 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:53.943137  1356 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:53.933947 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.939162 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.974368 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:53.979349 30702 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:14:53.979390 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 89285us
I20260709 13:14:53.980667 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:53.870378  1350 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.100s
I20260709 13:14:53.969190  1385 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:14:53.987969  1385 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:53.988421  1385 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:14:53.988632 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:53.989584 31394 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 6: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:53.990788 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.991065 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:53.991252 31396 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:53.992240  1370 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:53.992537  1356 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:53.992898  1370 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:53.992700  1517 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:14:54.002046  1490 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:54.003823 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:54.004181 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:53.975697  1252 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.029963 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:54.031509 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:54.040886 30701 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:54.050064 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.067041 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:54.069597 30559 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:54.045230  1294 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.509s	user 0.012s	sys 0.002s
W20260709 13:14:54.079284 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.039105  1462 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:54.090576 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:54.096511 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.098445 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.100852 31388 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:14:54.104859 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124257us
I20260709 13:14:54.113176 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.115259 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:54.116242 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.087826  1462 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.118791 30698 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 5.
I20260709 13:14:54.141670 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.102116  1483 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:14:54.148169 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 178564us
W20260709 13:14:54.148972 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:54.113394  1520 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:14:54.068085   847 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.079798  1252 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:54.190840 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.167804 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:54.162497 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:54.178017 31383 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:14:54.160542 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:54.194917 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.142449 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.188694 30628 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:54.277385 30559 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:54.272996 31383 pending_rounds.cc:85] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:54.278932 31383 pending_rounds.cc:107] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:14:54.279232 31383 raft_consensus.cc:2889] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:54.279505 30625 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:14:54.173297  1522 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:14:54.283180  1438 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:54.283499  1252 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:54.125092  1489 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:14:54.283646  1439 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.193053 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:54.210886 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88856us
I20260709 13:14:54.288015 31389 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:54.154716 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:54.204600 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.275462 30627 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:54.297701 30554 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:54.125728  1255 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.002s
I20260709 13:14:54.294404 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:54.289419 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:54.124788  1462 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:54.159320 30555 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:14:54.162916  1194 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.295768 30551 raft_consensus.cc:2376] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10. Re-sending same reply.
I20260709 13:14:54.161921 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.284126  1438 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.307921 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:54.309350 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:54.312084 30624 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:54.288319  1483 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:54.313501 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:54.313735 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 153331us
I20260709 13:14:54.163133  1411 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.316537 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:54.318665 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 156261us
I20260709 13:14:54.319515 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:54.319861 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:54.154906  1370 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:54.210983  1303 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 13:14:54.331374  1350 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.341698 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 196554us
I20260709 13:14:54.305104  1482 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:54.337648 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:54.335474  1194 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.344334 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:54.333573 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:54.364337 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.328110  1483 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:54.327677  1329 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:54.368707 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:54.316720  1505 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:54.331367  1428 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:54.374158 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:54.346662 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.310177  1439 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:54.310098  1255 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.382802 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.431901 31378 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:54.368402  1349 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:54.384094 31394 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:54.375105  1329 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:54.442978 30632 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:14:54.446254 31389 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: [7.5-8.6]
W20260709 13:14:54.452303 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 111580us
I20260709 13:14:54.331599  1411 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.340574  1401 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.092s
W20260709 13:14:54.462796 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 134388us
I20260709 13:14:54.463646 30559 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:54.363587  1350 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:54.471315 30552 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:54.488360  1194 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:54.494812  1521 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:54.495898  1521 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:54.497263  1521 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:54.461427  1273 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:54.505826 31389 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:54.497794  1401 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:54.456524  1329 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:54.516250 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:54.517305 30631 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:14:54.524947 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.464260  1482 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:54.547420 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65805.7us
W20260709 13:14:54.536064 30700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.006s	sys 0.011s
W20260709 13:14:54.552244 30700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.007s	sys 0.011s
W20260709 13:14:54.557654 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.510468  1273 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:54.629343 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.630739 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.575656  1438 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:54.549352  1349 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:54.588546 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:54.644819 30624 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:14:54.590953 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 127078us
I20260709 13:14:54.593070 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:54.593770 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.596597 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.597268 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.600615 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.649580 30698 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:54.601930 30701 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:54.602586  1255 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:54.605005  1350 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:54.550915  1482 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.532069  1252 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:54.612203  1439 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:54.613797  1270 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.716s	user 0.017s	sys 0.000s
W20260709 13:14:54.623420  1385 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.003s
I20260709 13:14:54.625607 31361 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:54.658944  1403 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.656s	user 0.015s	sys 0.000s
I20260709 13:14:54.674753 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:54.675858 30701 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:54.432401  1505 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.686115 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:54.667727  1349 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.692101 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.693094 31364 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:14:54.694007 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 133459us
I20260709 13:14:54.694702  1194 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.663219  1252 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:54.695338  1349 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:54.695935  1270 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.696520  1270 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:54.696370  1252 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:54.685390  1385 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:54.700170  1482 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:54.700984  1252 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:54.702533 30558 raft_consensus.cc:2368] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 2. Current term is 3.
I20260709 13:14:54.701427  1385 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:54.652029 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:54.652624 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.710574 31391 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:54.652915 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:54.654258 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:54.712420 30703 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:54.695429  1194 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.655104  1411 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:54.645807 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.714355 31396 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14.
I20260709 13:14:54.651306 30554 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 3.
I20260709 13:14:54.628407 30552 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:14:54.715440 30552 pending_rounds.cc:85] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:14:54.715705 30552 pending_rounds.cc:107] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:14:54.715945 30552 raft_consensus.cc:2889] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:54.716144  1505 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:54.659941  1273 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.706020 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:54.707789 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:54.717681 30559 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:14:54.643345 30628 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:54.729931 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:54.757381 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45702.4us
I20260709 13:14:54.772135 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:54.735728 30706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.057s	sys 0.060s
W20260709 13:14:54.784793 30706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.057s	sys 0.060s
W20260709 13:14:54.794237 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:54.756839 30699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.017s	sys 0.015s
W20260709 13:14:54.819824 30699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.017s	sys 0.015s
I20260709 13:14:54.773197 31383 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:54.690627  1439 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:14:54.690230  1403 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:54.831786 30707 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:54.756397  1338 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:14:54.771044  1194 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:54.850047 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 120259us
I20260709 13:14:54.866801 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.865159 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:54.875160 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.882503 30556 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:54.879142 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:54.864771 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.865559 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.847492 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1569) took 1093 ms. Trace:
I20260709 13:14:54.885215 31391 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:54.881479 31396 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:54.865170 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:54.836407  1401 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:14:54.896782 31367 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.033s	sys 0.043s
W20260709 13:14:54.899734 31367 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.033s	sys 0.043s
I20260709 13:14:54.897933 30701 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:14:54.900797 31361 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:54.908803 30701 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:14:54.910954 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:54.868034  1270 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:54.921802 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 607881us
W20260709 13:14:54.959728 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:54.962857 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:54.860368  1329 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:54.912242  1338 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:54.913689  1294 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:54.917666  1428 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:14:54.987262 31394 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:54.997579 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.005846 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:54.983407  1401 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:55.011104 30555 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
W20260709 13:14:54.998733 30672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.014s	sys 0.033s
W20260709 13:14:55.014142 30672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.014s	sys 0.034s
I20260709 13:14:55.014387 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:55.014793 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.016140 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:55.020782 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.021034 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:55.016981 30705 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:55.012053 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:55.027318 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 162342us
I20260709 13:14:54.991400  1403 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:54.974380  1338 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.032845 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.037694 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:14:55.051708  1338 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.052552  1338 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:55.023984  1401 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.023324  1508 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:14:55.064241  1542 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:14:55.065815 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1571) took 1234 ms. Trace:
W20260709 13:14:55.067343  1462 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.702s	user 0.011s	sys 0.003s
W20260709 13:14:55.059360  1531 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.456s	user 0.002s	sys 0.002s
I20260709 13:14:55.030064   847 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:55.061548 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.061641  1329 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:55.066255 30700 rpcz_store.cc:276] 0709 13:14:53.831507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.858681 (+ 27174us) service_pool.cc:224] Handling call
0709 13:14:53.871369 (+ 12688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.942957 (+ 71588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.943002 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:53.943628 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.552879 (+609251us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:54.552944 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.553051 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.065506 (+512455us) raft_consensus.cc:1685] finished
0709 13:14:55.065535 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.065739 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47924,"raft.queue_time_us":63039,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":25404,"thread_start_us":608703,"threads_started":1,"wal-append.queue_time_us":924695}
I20260709 13:14:55.050603  1155 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:14:55.074029 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1208) took 1084 ms. Trace:
I20260709 13:14:55.027719  1356 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:55.030102  1493 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:14:55.099761  1428 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.100039  1538 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:14:55.103811  1428 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:55.105628 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44227.5us
I20260709 13:14:55.106379  1505 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:14:55.107133 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.016s	sys 0.032s
W20260709 13:14:55.115411 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.016s	sys 0.032s
W20260709 13:14:55.115932 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 699016us
W20260709 13:14:55.107623  1535 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.530s	user 0.003s	sys 0.000s
I20260709 13:14:55.087900  1401 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.096333 30557 rpcz_store.cc:276] 0709 13:14:53.989287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.009726 (+ 20439us) service_pool.cc:224] Handling call
0709 13:14:54.030473 (+ 20747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.075885 (+ 45412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.075938 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.099660 (+ 23722us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.660267 (+560607us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:54.660321 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.660401 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.073701 (+413300us) raft_consensus.cc:1685] finished
0709 13:14:55.073726 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.073921 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12184,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":12597,"thread_start_us":499050,"threads_started":1,"wal-append.queue_time_us":795456}
I20260709 13:14:55.094964  1462 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.109534  1482 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.103123 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.548s (SENT) [suppressed 20 similar messages]
I20260709 13:14:55.097545  1356 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.135598 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:55.131820 31388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.018s	sys 0.017s
W20260709 13:14:55.138648 31388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.019s	sys 0.018s
I20260709 13:14:55.135492 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:55.138131 30629 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.013s	sys 0.023s
W20260709 13:14:55.141026 30629 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.013s	sys 0.023s
I20260709 13:14:55.143118 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.151405 30558 raft_consensus.cc:2368] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:55.144218 30551 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:14:55.102418   847 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 13:14:55.159490 30630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.016s	sys 0.014s
W20260709 13:14:55.162871 30630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.016s	sys 0.014s
I20260709 13:14:55.112900  1489 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 13:14:55.138098  1356 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:55.169015 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 440820us
I20260709 13:14:55.177198 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:55.167846  1303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.022s	sys 0.082s
I20260709 13:14:55.161588  1370 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:55.138968  1482 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.186870  1462 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.187331  1539 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.188202 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:55.189224 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:55.191039 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:55.191905 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:55.220526 31396 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:14:55.193415 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1867) took 1307 ms. Trace:
I20260709 13:14:55.192830 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:55.173511  1428 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:55.192904  1489 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:55.165221  1505 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:14:54.885124 30706 rpcz_store.cc:276] 0709 13:14:53.746163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.751761 (+  5598us) service_pool.cc:224] Handling call
0709 13:14:53.813426 (+ 61665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:53.867149 (+ 53723us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:14:54.046593 (+179444us) log.cc:844] Serialized 14 byte log entry
0709 13:14:54.837790 (+791197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:54.838730 (+   940us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:54.838805 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.839303 (+   498us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:54.839488 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10803,"prepare.queue_time_us":18340,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":102574,"thread_start_us":819548,"threads_started":2}
I20260709 13:14:55.196610  1438 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.200323 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:55.202553  1540 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.204303  1546 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:55.205374  1252 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.504s	user 0.013s	sys 0.000s
I20260709 13:14:55.205549  1548 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.231539  1540 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.207424 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1103) took 1091 ms. Trace:
I20260709 13:14:55.208266  1545 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:14:55.244900  1462 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.245313  1545 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:55.245738  1462 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:55.246428  1540 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:55.246551  1545 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.247239  1438 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:55.248353  1462 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:55.169471  1205 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:55.212524 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:55.220425 31364 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:14:55.211386  1303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.022s	sys 0.082s
I20260709 13:14:55.194185 30553 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:14:55.226251 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44448us
W20260709 13:14:55.221889  1483 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.807s	user 0.016s	sys 0.000s
I20260709 13:14:55.221123 30550 rpcz_store.cc:276] 0709 13:14:53.885667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.885932 (+   265us) service_pool.cc:224] Handling call
0709 13:14:53.946923 (+ 60991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:53.982895 (+ 35972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:53.983034 (+   139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.034690 (+ 51656us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.305522 (+270832us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:54.305576 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.305651 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.106847 (+801196us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:55.193046 (+ 86199us) raft_consensus.cc:1685] finished
0709 13:14:55.193074 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.193346 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5618,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":20515,"spinlock_wait_cycles":208383232,"thread_start_us":248079,"threads_started":1,"wal-append.queue_time_us":527851}
I20260709 13:14:55.280220 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:55.231971  1548 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.232733 31367 rpcz_store.cc:276] 0709 13:14:54.115771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.115929 (+   158us) service_pool.cc:224] Handling call
0709 13:14:54.123955 (+  8026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.289986 (+166031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.290023 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.313914 (+ 23891us) log.cc:844] Serialized 26 byte log entry
0709 13:14:54.901167 (+587253us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:54.901218 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:54.901322 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.207149 (+305827us) raft_consensus.cc:1685] finished
0709 13:14:55.207172 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.207335 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77969,"raft.queue_time_us":147127,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":2789,"thread_start_us":532539,"threads_started":1,"wal-append.queue_time_us":793826}
W20260709 13:14:55.279713 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.280979 31361 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:55.224200  1489 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:55.288494 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:55.289523 31364 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:14:55.291229 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 328589us
I20260709 13:14:55.296662 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:55.253722  1205 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:55.320041  1205 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.321334  1205 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:55.324018  1482 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:55.325369  1541 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
W20260709 13:14:55.327330  1544 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:14:55.328013  1401 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.328675  1370 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:55.329210  1370 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:55.329378 30633 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:55.332162 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:55.333004 31394 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:55.335444  1294 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:55.336208  1294 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:55.297739  1252 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:55.341987  1252 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:55.342237 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1206) took 1388 ms. Trace:
I20260709 13:14:55.343248  1252 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:55.307243 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 124109us
W20260709 13:14:55.344937  1554 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:14:55.342939 30553 rpcz_store.cc:276] 0709 13:14:53.954196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:53.954405 (+   209us) service_pool.cc:224] Handling call
0709 13:14:55.193870 (+1239465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.285270 (+ 91400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.341929 (+ 56659us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54002d69d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:55.342150 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2783467264}
W20260709 13:14:55.347587  1552 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 13:14:55.349251 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.352732  1557 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:14:55.307065  1489 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.315181  1438 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:55.231475  1546 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.316715 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:55.380803 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:55.291829  1403 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:55.290117  1483 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.318975 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1104) took 1169 ms. Trace:
I20260709 13:14:55.362911 31396 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 3.
W20260709 13:14:55.363009 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 449255us
I20260709 13:14:55.369660 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:55.221514  1539 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:55.391842 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 256497us
W20260709 13:14:55.381829  1273 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.010s	sys 0.018s
W20260709 13:14:55.403263 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.330500  1401 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:55.413760 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 310634us
I20260709 13:14:55.406663 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:55.420848 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:55.403782 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.379786  1508 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:55.395366  1343 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.803s	user 0.013s	sys 0.021s
I20260709 13:14:55.391754 31388 rpcz_store.cc:276] 0709 13:14:54.149242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.149716 (+   474us) service_pool.cc:224] Handling call
0709 13:14:54.150233 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.367231 (+216998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.367286 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.420244 (+ 52958us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.139292 (+719048us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:55.139359 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.140237 (+   878us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.318699 (+178462us) raft_consensus.cc:1685] finished
0709 13:14:55.318721 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.318907 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10541,"raft.queue_time_us":23427,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":17511,"thread_start_us":640251,"threads_started":1,"wal-append.queue_time_us":838745}
I20260709 13:14:55.430536  1403 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:55.432634  1548 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.432860  1403 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.403937  1482 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 13:14:55.406167  1270 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:55.434588 30673 leader_election.cc:400] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:55.435142 30673 leader_election.cc:403] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:14:55.435665 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76108.7us
I20260709 13:14:55.431412 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:55.437701 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:55.421859 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:55.422582 30558 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:14:55.441480 30558 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:14:55.428210 30551 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:55.441756 30558 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:14:55.442698 30558 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:55.415751  1489 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:55.446275 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.405740  1438 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.441975 30551 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:55.434975  1270 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.447613 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:55.441691 31361 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:55.431545  1560 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:14:55.470674 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.443130  1343 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.015s	sys 0.021s
W20260709 13:14:55.448594  1521 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.014s	sys 0.032s
I20260709 13:14:55.443339   847 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:55.445837  1273 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:55.460701 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:55.449530  1561 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:14:55.485941 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91912us
I20260709 13:14:55.456885  1570 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:14:55.487015 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:55.456204  1270 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:55.444984  1490 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:14:55.493381 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69028.2us
I20260709 13:14:55.473855  1540 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:55.460894  1560 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:55.479496  1521 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.015s	sys 0.033s
I20260709 13:14:55.494014 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:55.485441 30551 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.473091  1438 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.498143  1482 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:14:55.494773  1155 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.498080  1329 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.490209  1548 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.498502  1303 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:14:55.508909 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 816) took 1302 ms. Trace:
I20260709 13:14:55.487398  1401 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:55.521425  1428 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:55.557793  1428 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:55.558111 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.571842  1548 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.490880  1561 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.574921  1561 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:55.540081 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 223444us
I20260709 13:14:55.579620 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.581135  1273 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:55.581630  1521 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.582340  1521 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.584933  1546 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:55.500818  1255 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:14:55.545802 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50706.8us
W20260709 13:14:55.539180 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 117421us
I20260709 13:14:55.502396  1540 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.556558  1411 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:55.531440  1560 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:55.490648  1570 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:55.553853  1329 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.574508  1548 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:55.611373  1570 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.553455 30629 rpcz_store.cc:276] 0709 13:14:54.206184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.278737 (+ 72553us) service_pool.cc:224] Handling call
0709 13:14:54.279209 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.279765 (+   556us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.279798 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.387369 (+107571us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:54.442805 (+ 55436us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.154881 (+712076us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:14:55.154992 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.155139 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.508478 (+353339us) raft_consensus.cc:1685] finished
0709 13:14:55.508508 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.508789 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":706,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":19228928,"thread_start_us":640985,"threads_started":1,"wal-append.queue_time_us":957370}
I20260709 13:14:55.561239  1294 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.550241  1155 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.539721  1549 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.566964 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 92108.8us
I20260709 13:14:55.516476  1483 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:55.515609  1343 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:55.648128  1343 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.648022  1303 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:55.648988  1343 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:55.649573  1303 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:55.650933  1546 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:55.651544  1546 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:55.652673  1546 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.574852  1462 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:55.545787 30551 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:14:55.554418 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60522.3us
I20260709 13:14:55.655383 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:55.582410  1547 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:55.604103 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:55.604393 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 121355us
I20260709 13:14:55.600946  1401 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:55.606125  1572 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:55.607198  1493 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:55.610249  1539 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.526211  1508 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.558948  1428 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:55.602674  1540 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:55.645776 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:55.647193 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.551755  1356 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.658581 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:55.658705 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43724.9us
W20260709 13:14:55.664088 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60082.8us
I20260709 13:14:55.687827 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:14:55.688412 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.615909  1570 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:14:55.667268 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 205729us
I20260709 13:14:55.688917 30559 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:55.673418 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.660748  1546 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.682392 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:55.613754  1370 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.660014  1547 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:55.735972 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53712.2us
I20260709 13:14:55.725193 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:55.738708 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.739099 30700 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:55.618376  1270 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.740883 30554 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:14:55.724435 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:55.724826 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.730803 30559 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:55.666487 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 818) took 1341 ms. Trace:
I20260709 13:14:55.667387  1483 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:55.745370 30630 rpcz_store.cc:276] 0709 13:14:54.325192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:54.325432 (+   240us) service_pool.cc:224] Handling call
0709 13:14:54.346111 (+ 20679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:54.382452 (+ 36341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:54.382490 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:54.457020 (+ 74530us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.183867 (+726847us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:55.456297 (+272430us) log.cc:844] Serialized 14 byte log entry
0709 13:14:55.666103 (+209806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.666202 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:55.666257 (+    55us) raft_consensus.cc:1685] finished
0709 13:14:55.666272 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:55.666417 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6691,"prepare.queue_time_us":47707,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":33,"raft.queue_time_us":183846,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":18451,"thread_start_us":1105319,"threads_started":3,"wal-append.queue_time_us":842272}
I20260709 13:14:55.754591 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:55.721153  1572 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.700241  1329 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:55.769107 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:55.667287  1273 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:14:55.771975 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.657506  1549 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.739954   847 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:55.774645 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.748682  1489 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:55.753103  1483 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.746691  1505 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:14:55.802407 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.681958  1579 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.098s
I20260709 13:14:55.756357  1428 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:55.806660 30703 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:14:55.671325  1560 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:55.808734 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 163070us
W20260709 13:14:55.812412 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 208169us
I20260709 13:14:55.802739  1329 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:55.775420  1563 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:55.768945  1521 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.759902  1438 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:55.858732 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86910.2us
W20260709 13:14:55.845593 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.843469 30670 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.838s (SENT)
I20260709 13:14:55.818106  1303 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:55.881765 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:55.837044  1205 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.515s	user 0.015s	sys 0.000s
I20260709 13:14:55.824620  1483 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:55.824158  1505 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:55.819145  1255 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:55.876224  1547 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.894520  1329 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:55.895567 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52191.5us
I20260709 13:14:55.896181  1273 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:14:55.896450  1549 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.897193  1549 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.897742  1549 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.898757  1580 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.899701  1155 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.824096  1489 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:55.900723  1155 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:55.900828  1255 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:55.902264  1489 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:55.837443  1401 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:55.903301 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 178631us
I20260709 13:14:55.904356  1349 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:55.907440  1490 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:55.908288 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:55.909554 30629 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:14:55.899475  1580 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.911787  1356 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:55.912526  1587 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:55.870332  1535 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:14:55.939561 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.958549 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:55.959456 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 76212.8us
I20260709 13:14:55.800719  1572 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:55.892500  1303 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:55.968931 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 143206us
I20260709 13:14:55.968034 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:55.935498  1505 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:55.964172 30630 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:55.958813  1411 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:55.992027 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76600.8us
I20260709 13:14:55.993415 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:55.996403 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:55.997325 30707 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:55.998540 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.003928 30631 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:56.004159 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:56.006071  1270 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:56.010771 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:56.012774 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:55.971923  1539 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:55.976572  1303 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:56.017153 30624 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:56.015064 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:56.013667 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 157479us
I20260709 13:14:56.040958 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:56.015064 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.016084 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:56.004555  1549 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:56.054013 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.045800 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:55.975355  1294 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:56.065996 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.067041 30698 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:14:56.068406 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:55.960464  1438 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:14:56.079180 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:56.075744 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:56.076507 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.080080 30555 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:56.074515 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:56.049506  1537 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:14:56.088999 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90580.2us
I20260709 13:14:55.975970  1490 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:56.007591  1598 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:56.123523 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:56.216735 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.083590  1252 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.064992   847 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:56.016808  1403 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
W20260709 13:14:56.319973 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 309300us
I20260709 13:14:55.974951  1587 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:56.352487 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.326751 30555 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
W20260709 13:14:56.328155 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.062139  1603 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:56.007356  1563 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:14:56.370291 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:56.392246 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 379527us
I20260709 13:14:56.321715  1370 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:56.418665 30626 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:14:56.374468  1561 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.000s
I20260709 13:14:56.420666 30554 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:56.358600  1270 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:56.368647  1591 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.374s	user 0.000s	sys 0.010s
W20260709 13:14:56.408025  1568 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.002s	sys 0.015s
I20260709 13:14:56.425436 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:56.446035 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.398593  1579 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:56.451763 30630 raft_consensus.cc:2410] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
W20260709 13:14:56.439085  1575 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:14:56.438139  1585 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.561s	user 0.000s	sys 0.010s
I20260709 13:14:56.458487 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:56.471101 31364 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:56.362424  1252 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:56.468344  1572 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:56.468766 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43545.7us
I20260709 13:14:56.469309 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:56.480813 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128489us
I20260709 13:14:56.480969 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.481932 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.512629 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:56.513839 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.517990 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.521425 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:56.526211 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.409066  1411 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:56.449796  1270 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:56.435436  1561 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:56.536770 30700 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.795s	user 0.015s	sys 0.000s
I20260709 13:14:56.552371  1598 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:56.558849 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.559881 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:56.556104 30700 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:14:56.572053 30703 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:56.552742  1370 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:56.557957  1338 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:56.575780 30556 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
W20260709 13:14:56.554157  1589 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.587s	user 0.001s	sys 0.001s
I20260709 13:14:56.634096 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:56.634963 31361 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:56.553534  1572 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:14:56.636121 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:56.632110 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119617us
I20260709 13:14:56.641355 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:56.581146  1540 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.835s	user 0.017s	sys 0.020s
W20260709 13:14:56.581468  1565 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.012s
W20260709 13:14:56.575466  1482 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.017s	user 0.006s	sys 0.009s
I20260709 13:14:56.648293 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.648814 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.554411  1580 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:14:56.653435 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 283227us
I20260709 13:14:56.587227  1252 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:56.585290  1270 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:56.634258  1620 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:56.575757  1618 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:56.647511  1603 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:56.669708 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:14:56.675027 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115251us
I20260709 13:14:56.655934 31364 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:14:56.660645 30552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.003s	sys 0.006s
W20260709 13:14:56.651294  1565 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.816s	user 0.012s	sys 0.014s
I20260709 13:14:56.661733   847 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:56.659584  1490 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:56.583032  1561 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:56.659343 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.658283  1294 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:56.655493  1482 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:56.691195 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:56.677304 31361 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:14:56.668859  1343 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.019s	user 0.015s	sys 0.000s
I20260709 13:14:56.655468  1572 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:56.697613  1572 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:56.702973  1624 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:56.703531  1624 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:56.703873  1624 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:56.704063  1561 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:14:56.704666  1561 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:56.705571  1561 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:56.581724  1579 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:56.691112  1294 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:56.677850 30552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.003s	sys 0.006s
I20260709 13:14:56.683810  1252 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.683539  1338 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:56.647233  1540 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:56.695994  1618 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:14:56.670522 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 156807us
I20260709 13:14:56.692489 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.673652  1438 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:14:56.696498  1583 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.327s	user 0.000s	sys 0.001s
W20260709 13:14:56.706003  1570 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.987s	user 0.012s	sys 0.000s
I20260709 13:14:56.706519  1579 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:56.676271 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.654554  1626 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:14:56.693285  1608 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.323s	user 0.001s	sys 0.001s
I20260709 13:14:56.681821  1563 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:56.748914 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1576) took 1025 ms. Trace:
I20260709 13:14:56.713178  1294 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:56.728349 30558 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:56.739122 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:56.727532 30626 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
W20260709 13:14:56.725127  1483 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.828s	user 0.013s	sys 0.000s
I20260709 13:14:56.712981  1343 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:56.739073 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:56.744901 30706 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:14:56.718091  1252 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:56.760298  1587 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:56.763051  1622 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 13:14:56.765133  1428 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.004s
I20260709 13:14:56.741470  1579 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:14:56.773547 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 97363.1us
I20260709 13:14:56.777024 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:56.749523 30702 rpcz_store.cc:276] 0709 13:14:55.723031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.738403 (+ 15372us) service_pool.cc:224] Handling call
0709 13:14:55.759176 (+ 20773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.759557 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.759592 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:55.760111 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:14:55.886768 (+126657us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:55.892205 (+  5437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:55.892411 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:56.748663 (+856252us) raft_consensus.cc:1685] finished
0709 13:14:56.748687 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:56.748843 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9834,"raft.queue_time_us":131582,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":15616,"thread_start_us":48943,"threads_started":1,"wal-append.queue_time_us":297190}
W20260709 13:14:56.763178  1349 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.001s
I20260709 13:14:56.779035  1349 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:56.679147  1598 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:56.759307  1483 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:56.785816  1592 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:14:56.787009 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48522.5us
I20260709 13:14:56.789151  1630 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:56.789844  1630 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:56.790884  1630 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:56.791996  1632 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 13:14:56.794029  1546 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.061s	user 0.013s	sys 0.002s
I20260709 13:14:56.788707  1343 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:56.794832 30631 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:14:56.795758  1303 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.717s	user 0.011s	sys 0.003s
I20260709 13:14:56.796532  1587 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.797108  1587 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:56.768764  1563 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:14:56.749843  1626 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:56.728451  1618 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:14:56.799597  1587 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:56.786917 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47883us
I20260709 13:14:56.801301  1626 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:56.802035 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:56.803601 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 89189.6us
I20260709 13:14:56.802874 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:56.786275  1483 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:56.767625  1572 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:56.807004 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:56.807610 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:56.796271  1303 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:56.818710  1270 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:14:56.819314 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:56.835028 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.809794  1572 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:14:56.822973 30633 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:14:56.802814  1349 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:14:56.806116  1592 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:14:56.826601  1505 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.818s	user 0.009s	sys 0.003s
I20260709 13:14:56.806375  1482 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:56.828020 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:56.830901 31367 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:56.813536  1622 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:56.839607 31388 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:14:56.830403 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:56.871469 31388 pending_rounds.cc:85] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:14:56.837988  1428 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.858925  1625 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:56.860076 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.875154 30626 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:14:56.862094 30629 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:56.837016 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:56.863034 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.863070 31367 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:56.858232  1572 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:56.833433  1547 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:56.865656 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.839620  1626 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:56.867331 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:56.867403 31383 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:14:56.887091 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50203.1us
I20260709 13:14:56.868100 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.869352  1252 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:56.870195 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.871771 31388 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:14:56.888768 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:56.835456  1597 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:56.896281 30702 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:56.896577 30554 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:14:56.874330 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:56.870855  1482 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:56.898056 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:56.827049 31294 connection.cc:256] RPC call timeout handler was delayed by -2.1513228260000687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:14:56.875849 31361 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:56.879068 30633 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:56.817451  1618 consensus_queue.cc:1243] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:14:56.880307 30627 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:56.887993 31396 raft_consensus.cc:2410] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:14:56.869187 30704 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:56.890312 31388 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:56.871692 31364 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:56.891942 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:56.893378 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:56.881428  1493 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.906898 30559 raft_consensus.cc:2376] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9. Re-sending same reply.
I20260709 13:14:56.903981 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:56.912895 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:56.881235  1630 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.913178 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:56.943531 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43486.6us
I20260709 13:14:56.944247 30559 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
W20260709 13:14:56.942273 31294 connection.cc:256] RPC call timeout handler was delayed by -1.2699203540000781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18300000000000002s for a response
W20260709 13:14:56.924788 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:56.945910 31294 connection.cc:256] RPC call timeout handler was delayed by -1.1772203539999282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19330000000000003s for a response
I20260709 13:14:56.918578 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.947362 31388 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 2. Current term is 3. Ops: []
W20260709 13:14:56.947872 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88758.8us
I20260709 13:14:56.871872  1505 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:56.943435 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64357.8us
I20260709 13:14:56.951817 30633 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:56.968806 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.970878 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.897065  1587 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.873881  1622 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:56.980141  1622 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:56.969944 30707 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:56.904615  1632 consensus_queue.cc:1243] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:14:56.900143  1634 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:56.986402 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.914461  1625 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:56.915890  1482 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.005590 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:56.899642  1597 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:56.880980  1428 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:56.981496 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.013250 31394 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:56.981768 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:56.986323 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:56.994050 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45670.2us
W20260709 13:14:57.018088 31294 leader_election.cc:336] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.830s (SENT)
I20260709 13:14:57.018136 31389 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:57.018705 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.912918  1508 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:56.994124 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:56.997993 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:56.920179  1618 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:56.897094  1626 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:56.973186 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84509.3us
I20260709 13:14:57.009757 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.025085 30633 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.011116 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.981021 31364 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:57.025178 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:56.944530  1493 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.030755 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:56.996615  1638 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:57.000607  1632 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.033120 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.035854 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:56.995410  1606 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.024227  1508 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.042378  1592 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:57.047410 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.047972  1622 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:14:57.087363  1606 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.087577 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.088498  1622 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:57.035647  1438 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:14:57.089051 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.048771  1640 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:57.051867  1635 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:14:57.090912 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.092343 31391 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 13:14:57.092681 31388 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:57.093278 31378 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:57.093453 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1237) took 1091 ms. Trace:
I20260709 13:14:57.093765 31364 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:57.093909 30552 rpcz_store.cc:276] 0709 13:14:56.002214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.002596 (+   382us) service_pool.cc:224] Handling call
0709 13:14:56.063296 (+ 60700us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.711428 (+648132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.711469 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.715491 (+  4022us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.872955 (+157464us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:14:56.872997 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.873071 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.093246 (+220175us) raft_consensus.cc:1685] finished
0709 13:14:57.093268 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.093395 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":591063,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":11412,"spinlock_wait_cycles":130304,"thread_start_us":769564,"threads_started":2,"wal-append.queue_time_us":173753}
I20260709 13:14:57.095574  1493 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.041303  1625 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.070114 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:57.096724 30705 raft_consensus.cc:2410] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:14:57.097709 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.098883 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.100114 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.100736 30705 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:14:57.101150  1584 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.126s	user 0.003s	sys 0.000s
I20260709 13:14:57.101686 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:56.994539  1587 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.104568  1587 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.105386 30551 raft_consensus.cc:2368] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:57.071466  1303 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.007396  1634 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.102509 30705 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:57.107501 30705 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:57.108233  1634 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:57.043649  1638 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:56.995530  1630 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.108946  1638 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:57.109429 30550 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:57.110553  1630 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.073282  1603 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:14:57.073501 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.073807  1632 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.111807 30630 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:14:57.113006 31383 raft_consensus.cc:2410] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:14:57.074165  1505 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.075021  1294 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.001996  1642 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:57.114367  1642 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.076802  1616 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:57.115541 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1273) took 1143 ms. Trace:
I20260709 13:14:57.115923  1616 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.116468  1616 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:57.116695  1642 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.079732  1270 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.080551 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1575) took 1358 ms. Trace:
I20260709 13:14:57.118901  1560 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.119055 30706 rpcz_store.cc:276] 0709 13:14:55.722541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.738522 (+ 15981us) service_pool.cc:224] Handling call
0709 13:14:57.080291 (+1341769us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:57.080454 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236295424}
I20260709 13:14:57.120666 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:57.020143  1597 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:57.121511 31364 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:14:57.118919  1270 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.125905  1270 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.115976 30629 rpcz_store.cc:276] 0709 13:14:55.972372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.972573 (+   201us) service_pool.cc:224] Handling call
0709 13:14:55.973221 (+   648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.884290 (+911069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.115228 (+230938us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54000712d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:57.115473 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043739008}
I20260709 13:14:57.128141 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.129676 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.129937 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.084648  1618 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.085083 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.085664 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.137221 30704 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:57.107447  1370 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:57.046945  1629 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:57.113394  1632 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.115394  1294 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.073058 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.130964  1560 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:57.138027 30629 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:57.138885 30702 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:14:57.081771  1482 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:57.139590 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172263us
I20260709 13:14:57.139698 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.146343 30633 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:14:57.145323 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.152689 30705 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:57.151224 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.154507 30629 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:57.160661  1587 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:57.161093 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.145364  1560 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.162073  1560 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:57.162671 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:14:57.166910 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.548s (SENT)
I20260709 13:14:57.169853 30633 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:57.142210  1370 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:57.172932 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.163547 30704 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:57.192420 30629 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:14:57.187486 30705 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:57.197705 31364 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:14:57.198300 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 150993us
I20260709 13:14:57.194869 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.168435  1493 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.201519 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.202570 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.204125 31364 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:14:57.183933  1294 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:57.207115 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.142429  1618 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.209097 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:57.200062  1630 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:57.220736 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.199785  1642 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:14:57.231979  1618 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.232700 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.234994 31378 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:57.260176 30707 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:57.251490 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:57.248060 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.244725 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.270253 31391 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:57.238903 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.260149 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:57.274600 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65558.1us
I20260709 13:14:57.236151 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:14:57.236654  1592 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:57.237514  1612 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:57.263788 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.292605  1612 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:57.292928 30559 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:57.270632 30553 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:14:57.293653  1612 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.208837  1600 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 13:14:57.291677  1592 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.209657  1294 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.297472  1592 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.246646  1482 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.206301  1493 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.316743 30706 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:14:57.210336  1569 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:57.295251  1612 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:57.264235  1613 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.885s	user 0.000s	sys 0.002s
W20260709 13:14:57.338406 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 137578us
I20260709 13:14:57.341784 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:57.291860  1624 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.588s	user 0.012s	sys 0.003s
I20260709 13:14:57.298328  1294 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.351325  1597 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.357013 30559 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 4
I20260709 13:14:57.357471 30559 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:14:57.357801 30559 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.276271  1489 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.358812  1641 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:57.359556  1641 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:57.359254  1489 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.360687  1641 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.361333  1581 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:14:57.362409  1581 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:14:57.308396  1603 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:14:57.362780  1581 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:57.363365  1603 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:57.363684  1548 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.364369  1548 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.365007 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1581) took 1625 ms. Trace:
I20260709 13:14:57.266924  1606 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.365489 30701 rpcz_store.cc:276] 0709 13:14:55.739833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.740003 (+   170us) service_pool.cc:224] Handling call
0709 13:14:55.759176 (+ 19173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.892197 (+133021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.892238 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:55.935130 (+ 42892us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.074087 (+138957us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:56.074148 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.074243 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.263233 (+1188990us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:57.364587 (+101354us) raft_consensus.cc:1685] finished
0709 13:14:57.364619 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.364914 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26363,"raft.queue_time_us":126248,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"spinlock_wait_cycles":30587520,"thread_start_us":170550,"threads_started":2,"wal-append.queue_time_us":109184}
W20260709 13:14:57.367316  1621 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.819s	user 0.001s	sys 0.001s
I20260709 13:14:57.345917  1638 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.367779 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.352460 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.368985 31364 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:57.369997 31389 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:57.304394  1270 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:57.343003 30553 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
W20260709 13:14:57.377996 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:57.403039  1614 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.798s	user 0.003s	sys 0.000s
I20260709 13:14:57.403229  1597 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.404933 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1579) took 1671 ms. Trace:
I20260709 13:14:57.406920  1548 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:57.405651 30699 rpcz_store.cc:276] 0709 13:14:55.733881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.734504 (+   623us) service_pool.cc:224] Handling call
0709 13:14:55.760597 (+ 26093us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.862702 (+102105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.862759 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:55.863476 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 13:14:56.095358 (+231882us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:56.095497 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:56.095719 (+   222us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.249774 (+1154055us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:14:57.404536 (+154762us) raft_consensus.cc:1685] finished
0709 13:14:57.404565 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.404857 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7747,"raft.queue_time_us":127092,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":56672640,"thread_start_us":323939,"threads_started":2,"wal-append.queue_time_us":508290}
I20260709 13:14:57.408182 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.408690  1493 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.409696 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.409514  1560 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.410437 30699 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:57.411486 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.411989  1560 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.412150 30699 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:57.413440 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:57.415076 30699 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
W20260709 13:14:57.415220  1572 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.525s	user 0.013s	sys 0.000s
I20260709 13:14:57.415777  1572 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.416857 30699 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:14:57.417335 30699 pending_rounds.cc:85] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 1
I20260709 13:14:57.417600 30699 pending_rounds.cc:107] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:14:57.417882 30699 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.418234  1572 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.363713  1581 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.363759  1603 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:57.420226  1579 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.614s	user 0.009s	sys 0.003s
I20260709 13:14:57.421067  1616 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.422202  1638 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.424854 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.404848  1610 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.408483  1489 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:14:57.417982 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 75620.2us
I20260709 13:14:57.463955 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.342572  1343 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 13:14:57.304059 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:57.434142 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.471170 30633 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:57.441119 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.317003  1630 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.431774  1579 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.351579  1482 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.451615 30629 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:14:57.208137  1645 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.227s
I20260709 13:14:57.424299  1540 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.384366 30701 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:57.430351  1294 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:14:57.472046 30630 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:57.524065 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.446146 31391 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:14:57.477223 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:57.494730 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 190783us
I20260709 13:14:57.467693  1610 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.433490  1270 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:57.472407  1581 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:57.474581  1587 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:57.389392  1624 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.509617 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.510864 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.472776  1632 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:57.526257 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49143.1us
I20260709 13:14:57.579383 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 504) took 1588 ms. Trace:
I20260709 13:14:57.579663 30630 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:57.527348 30698 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 13:14:57.529652 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.533949 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:57.496094  1607 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.019s	user 0.006s	sys 0.027s
I20260709 13:14:57.530776 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:57.574033 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:14:57.574918 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.575165 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.579931 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.579948 30703 rpcz_store.cc:276] 0709 13:14:55.990750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.991028 (+   278us) service_pool.cc:224] Handling call
0709 13:14:56.571747 (+580719us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.572672 (+   925us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.579147 (+1006475us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400435650. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:57.579316 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1280126080}
I20260709 13:14:57.575688 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.507817 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:57.583549 30699 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:57.576718 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:57.498082  1303 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:57.585417  1303 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:57.585773  1303 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.586134  1303 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:57.586635  1630 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:57.584045  1607 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.008s	sys 0.027s
I20260709 13:14:57.589798 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.590639 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.591128 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.591455 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.578473 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.583602 31388 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:57.595541 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.572559  1642 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:57.592559 31396 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:57.603946 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.473085  1598 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
W20260709 13:14:57.609987 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 291127us
W20260709 13:14:57.610996 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81435.5us
I20260709 13:14:57.613962 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.624961 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.519748  1482 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:57.630369 30629 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:14:57.576689  1618 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:14:57.577178  1343 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:57.632141 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.576094  1252 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:14:57.598789  1540 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:57.607930  1294 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:57.626559  1642 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.644127 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:57.644483  1648 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 13:14:57.644788  1647 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.443s	user 0.003s	sys 0.000s
I20260709 13:14:57.645121 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.645366 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:57.693743 30551 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:57.694729 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.695998 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.646942  1370 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.688879  1642 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:57.697782 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.648628 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.698717 30632 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:14:57.698781 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:14:57.699347 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50811.6us
I20260709 13:14:57.646301 30555 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:57.651031 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1283) took 1158 ms. Trace:
I20260709 13:14:57.649241  1600 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:57.652207 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1270) took 1740 ms. Trace:
I20260709 13:14:57.655771  1579 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.702054 30628 rpcz_store.cc:276] 0709 13:14:55.911742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.912271 (+   529us) service_pool.cc:224] Handling call
0709 13:14:55.912682 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:55.913049 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:55.913099 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.165723 (+1252624us) spinlock_profiling.cc:243] Waited 43.7 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:57.183487 (+ 17764us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.326378 (+142891us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:57.326422 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.326501 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:57.651858 (+325357us) raft_consensus.cc:1685] finished
0709 13:14:57.651879 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:57.652139 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47303,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":12827,"spinlock_wait_cycles":100509568,"thread_start_us":116817,"threads_started":1,"wal-append.queue_time_us":275680}
I20260709 13:14:57.658455  1303 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.664819  1428 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:57.668076  1270 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.654191 31367 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.662016 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:57.658653  1350 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:57.674034 30700 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:14:57.712803 30700 pending_rounds.cc:85] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 3
I20260709 13:14:57.713147 30700 pending_rounds.cc:107] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:14:57.621739  1548 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:57.689215  1294 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.620823  1634 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:14:57.641480 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.687530 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:57.689589 31396 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:57.692090 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:57.732703 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.692746  1545 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.696111 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:57.651803 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.700234 30624 rpcz_store.cc:276] 0709 13:14:56.492548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.493123 (+   575us) service_pool.cc:224] Handling call
0709 13:14:57.650735 (+1157612us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54001fbb54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:57.650922 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507902848}
I20260709 13:14:57.700251 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.710105 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.713477 30700 raft_consensus.cc:2889] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:57.714993 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.715134 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:57.718819 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.676483  1356 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:14:57.723809  1370 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.724555 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.743505 30629 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:14:57.719182  1548 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.726472 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:57.729842  1303 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:57.733793 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.705231  1428 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:57.735535 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48113us
I20260709 13:14:57.735687 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:57.576864  1603 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.703080  1644 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:57.740024 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:57.745200 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 93467.1us
I20260709 13:14:57.773270 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:57.742374 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.740067 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:57.746022 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45870.5us
I20260709 13:14:57.775238 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:57.690047  1540 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.751158 31367 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:57.712357  1350 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:14:57.736272  1652 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:14:57.733616  1294 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:57.767434 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:14:57.761502 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 230799us
I20260709 13:14:57.748986  1545 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.774688 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:57.790871  1563 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:57.740712  1356 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:57.776932 30706 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:57.835331 30557 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:57.740108 30552 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:57.750762  1582 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:57.763794  1428 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:57.759076  1600 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:14:57.753844  1547 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:14:57.787309 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45053.9us
I20260709 13:14:57.848394 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.803649 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.849785 30551 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:14:57.845227 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.760021  1656 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:57.772792  1255 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:57.763518 30625 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:57.829932 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.833050 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.775972 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:57.788843  1616 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:57.853462 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77600us
I20260709 13:14:57.793009  1343 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:14:57.854854 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:57.758792  1548 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:57.807405  1570 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:57.777271  1155 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
W20260709 13:14:57.842047 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74758.6us
I20260709 13:14:57.833418  1662 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:57.853060 31396 raft_consensus.cc:2368] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:57.863761 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:57.836898  1563 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:57.840869  1630 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:14:57.864437 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:57.865706 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:57.857571 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.845412  1624 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.846347  1610 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:57.846742  1600 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.854305  1656 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.853977  1255 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.861600  1570 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:57.922631 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:57.924721 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.877704  1343 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:57.954752 31394 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:57.888355  1600 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:57.930442 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:57.885116  1616 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:57.884759  1547 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:57.882052  1411 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:57.864508  1582 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:57.891768 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:57.885861  1370 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:57.889020  1630 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:57.979820 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:57.883574  1401 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:57.875764  1350 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:57.928275  1255 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:57.927218  1570 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:58.013929 30551 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:57.903095  1610 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:57.924510  1592 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 13:14:57.981734 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:57.890730  1656 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:14:58.038381 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 353260us
I20260709 13:14:58.017946 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.019829 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:58.049584  1540 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:58.050972  1545 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.097172 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.098887 30552 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:14:58.051602  1428 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.054738  1343 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:58.054903 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1874) took 2378 ms. Trace:
I20260709 13:14:58.101465 30556 rpcz_store.cc:276] 0709 13:14:55.638432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.639348 (+   916us) service_pool.cc:224] Handling call
0709 13:14:55.639725 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.589080 (+949355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.016304 (+1427224us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.016599 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2152017152}
I20260709 13:14:58.056294 30698 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:14:58.057252 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 595) took 1161 ms. Trace:
I20260709 13:14:58.059177  1508 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.063066  1630 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.068334  1582 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.876338  1662 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:57.972043  1598 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:58.083343 31396 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:14:58.084156 30699 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.500s	user 0.014s	sys 0.000s
W20260709 13:14:58.085316  1660 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.282s	user 0.000s	sys 0.002s
W20260709 13:14:58.088826 30631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1286) took 1501 ms (client timeout 1837 ms). Trace:
I20260709 13:14:58.092201 31394 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:58.093652 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.097625 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 77968.9us
W20260709 13:14:58.097625 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 243098us
W20260709 13:14:58.056358  1658 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:14:58.013520  1370 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.105856 31367 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:14:58.120505 31367 pending_rounds.cc:85] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:14:58.118106 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:58.120857 31367 pending_rounds.cc:107] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:14:58.121279 31367 raft_consensus.cc:2889] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.121691 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:14:58.116559 30631 rpcz_store.cc:269] 0709 13:14:56.587166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.632569 (+ 45403us) service_pool.cc:224] Handling call
0709 13:14:58.088577 (+1456008us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54004aa1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.088741 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325372160}
I20260709 13:14:57.972477  1616 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:14:58.108466 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 242844us
W20260709 13:14:58.109736 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 244097us
I20260709 13:14:58.112442 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.113799 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.125147 30551 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 3.
I20260709 13:14:58.125620 30552 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:14:58.113992 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1244) took 1013 ms. Trace:
I20260709 13:14:57.971505  1634 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 13:14:58.115262 30704 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:14:58.127058 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1200) took 2133 ms. Trace:
I20260709 13:14:58.115494 30699 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:14:58.103324 30554 rpcz_store.cc:276] 0709 13:14:56.895745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.917738 (+ 21993us) service_pool.cc:224] Handling call
0709 13:14:56.918025 (+   287us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:56.944572 (+ 26547us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:56.944609 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:56.964699 (+ 20090us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.646496 (+681797us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b1800048160. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:14:57.875776 (+229280us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:14:57.875828 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:57.875931 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.056846 (+180915us) raft_consensus.cc:1685] finished
0709 13:14:58.056872 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.057160 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26073,"raft.queue_time_us":44489,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":17069,"spinlock_wait_cycles":10459008,"thread_start_us":229072,"threads_started":1,"wal-append.queue_time_us":1068965}
I20260709 13:14:58.131763 31367 rpcz_store.cc:276] 0709 13:14:55.989247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.989411 (+   164us) service_pool.cc:224] Handling call
0709 13:14:56.011351 (+ 21940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.830775 (+819424us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400167c20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:14:57.653653 (+822878us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b5400167c20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5b06f 00007f1806ed3f60 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62
0709 13:14:58.121952 (+468299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.122263 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876871296}
I20260709 13:14:58.133692 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.133665 30702 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:58.123782 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 205172us
I20260709 13:14:58.100592  1303 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:14:58.135213 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.135213 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.136559 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.137148 31396 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:58.123989  1370 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:58.133665 30699 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:14:58.139001 30699 pending_rounds.cc:85] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 1
I20260709 13:14:58.139281 30699 pending_rounds.cc:107] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:14:58.139524 30699 raft_consensus.cc:2889] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.130084  1508 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:58.101274  1343 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.140748  1343 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:58.142346 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 509) took 1243 ms. Trace:
I20260709 13:14:58.142853 30704 rpcz_store.cc:276] 0709 13:14:56.899113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.272391 (+373278us) service_pool.cc:224] Handling call
0709 13:14:57.338293 (+ 65902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.130965 (+792672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.142014 (+ 11049us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400343354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.142278 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773552896}
I20260709 13:14:58.144529  1662 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:58.093341  1610 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.150579 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.152209 31367 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:58.126832  1616 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.154963 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.128127 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.156005 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.157562  1645 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:14:58.158111 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:58.158380  1645 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:14:58.130124  1329 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:14:58.159458  1428 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:58.159859  1428 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:14:58.160178 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.160698  1428 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.161151 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:58.162441  1428 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.162954  1610 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.163166  1638 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:58.163731  1638 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:58.164973 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1875) took 2526 ms. Trace:
I20260709 13:14:58.164629  1638 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.166731  1350 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.166752  1638 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.165572 30558 rpcz_store.cc:276] 0709 13:14:55.638847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:55.639933 (+  1086us) service_pool.cc:224] Handling call
0709 13:14:55.640215 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:56.750110 (+1109895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.164656 (+1414546us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54009ca4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.164901 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2502038784}
W20260709 13:14:58.172699 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.175369  1674 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:58.167976 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1211) took 1023 ms. Trace:
I20260709 13:14:58.163093  1370 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.131387 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:58.184454 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.176754 31389 rpcz_store.cc:276] 0709 13:14:57.145035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.145176 (+   141us) service_pool.cc:224] Handling call
0709 13:14:57.171901 (+ 26725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.371521 (+199620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.167596 (+796075us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400013954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.167881 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455101312}
I20260709 13:14:58.188139 30551 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:14:58.191159 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.192615 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:58.193328 30699 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.198129 31389 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 5. Current term is 6. Ops: []
I20260709 13:14:58.198760 31394 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:58.200116 31396 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:58.131094 30701 rpcz_store.cc:276] 0709 13:14:57.100919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.573321 (+472402us) service_pool.cc:224] Handling call
0709 13:14:57.573826 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:57.595344 (+ 21518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.596802 (+  1458us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:57.597341 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:14:57.855977 (+258636us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:57.873862 (+ 17885us) log.cc:844] Serialized 14 byte log entry
0709 13:14:58.113470 (+239608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.113593 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.113663 (+    70us) raft_consensus.cc:1685] finished
0709 13:14:58.113686 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.113901 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38626,"prepare.queue_time_us":17646,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":8614,"raft.queue_time_us":93578,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":19435,"spinlock_wait_cycles":25333120,"thread_start_us":441092,"threads_started":2,"wal-append.queue_time_us":397596}
I20260709 13:14:58.203085  1670 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:58.204165 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:58.199337 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:58.204644 30699 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:14:58.197090 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:58.128845  1581 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.001s
I20260709 13:14:58.200179 31394 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:14:58.207680 31396 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:14:58.208727 30698 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:14:57.981781  1563 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.218211 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:58.176864  1606 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:58.230451 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:58.234226 30551 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:58.129630  1270 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:58.214468 31396 pending_rounds.cc:85] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 4
I20260709 13:14:58.240195 31396 pending_rounds.cc:107] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:14:58.240620 31396 raft_consensus.cc:2889] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:58.208689  1370 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.124433  1598 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.217942  1670 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:14:58.259948 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.261626 31294 leader_election.cc:400] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
I20260709 13:14:58.259513 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.262441 31294 leader_election.cc:403] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:14:58.263103 30624 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:14:58.260756 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:14:58.255721 30670 leader_election.cc:400] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:14:58.266129 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.267367 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:14:58.267974 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.268615 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:58.269871 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.271914 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:58.272260 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:14:58.245898  1612 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.895s	user 0.003s	sys 0.017s
I20260709 13:14:58.239759  1270 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.239423  1668 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:58.274109  1676 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:58.278162 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.280028  1294 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.283388  1600 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.264551  1670 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.284920  1673 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:58.222210  1579 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:14:58.287133  1548 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.304967 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.294763  1593 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:58.276758  1493 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:14:58.291239 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:14:58.297889 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98295us
I20260709 13:14:58.295883 30553 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:14:58.299799 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 174744us
W20260709 13:14:58.287302  1663 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:14:58.299935  1329 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:14:58.293864 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.296474  1270 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:58.312358 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:58.246490  1675 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:14:58.299028  1668 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.261051  1620 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 13:14:58.311590  1493 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.318171 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:58.314038 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:58.308615 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.315052 30706 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:14:58.316826 30699 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:58.255731  1677 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:14:58.298683  1349 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:14:58.299809  1670 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:58.295028  1610 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 7. Current term is 8.
W20260709 13:14:58.316143  1632 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.732s	user 0.018s	sys 0.006s
I20260709 13:14:58.317534  1673 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:58.314510  1428 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:58.307044  1593 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.337069  1626 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.338565  1681 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:14:58.341596 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.323395  1668 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.353034 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.332341  1493 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:58.341969  1579 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.359273 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:58.333961  1632 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:58.334075  1638 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:58.335232  1483 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:14:58.361093 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47142.8us
W20260709 13:14:58.367511 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55358.5us
I20260709 13:14:58.367664 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.369042  1679 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:14:58.370850 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.372262 31367 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:14:58.373713 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:58.351085  1677 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:58.320950  1676 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:14:58.375958 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.376425 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.377173 31361 raft_consensus.cc:2368] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8.
I20260709 13:14:58.352778  1673 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:58.358074  1626 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:14:58.398483 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.347728  1508 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:14:58.360903  1610 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:14:58.414477 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.375429  1428 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.380561 31389 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:58.406549 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.412165 31396 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:58.435974 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.398864  1329 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:14:58.374984  1677 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:58.440014 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.407419  1632 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:58.476831 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:58.501480 30630 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:58.472862 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96979.1us
I20260709 13:14:58.503273 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:58.365813  1681 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:14:58.397651  1687 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:14:58.442042  1329 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:58.369997  1579 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.522364 31396 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
W20260709 13:14:58.529196 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1229) took 1683 ms (client timeout 1865 ms). Trace:
I20260709 13:14:58.503496  1610 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:14:58.529616 30707 rpcz_store.cc:269] 0709 13:14:56.846124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.993779 (+147655us) service_pool.cc:224] Handling call
0709 13:14:58.529012 (+1535233us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54005a28d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.529136 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504196352}
I20260709 13:14:58.517252  1329 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.511399  1598 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:14:58.553577  1671 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:14:58.537999  1572 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:58.544132 30552 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:14:58.546762 30557 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:14:58.547372 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.525332  1600 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.496204  1685 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:14:58.571823 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.526955  1638 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:58.578135  1428 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:58.579429 30551 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:14:58.578326 30557 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:58.528527  1548 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:58.586455 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:58.586324 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.588771 31389 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.569231  1572 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:58.560035  1508 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:14:58.594921  1593 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.584007  1679 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:58.601680 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1587) took 1488 ms. Trace:
I20260709 13:14:58.600872  1693 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:14:58.610176  1482 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.570535  1610 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:58.611347 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.611270 31367 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:58.612725 30702 rpcz_store.cc:276] 0709 13:14:57.113234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.777601 (+664367us) service_pool.cc:224] Handling call
0709 13:14:58.133478 (+355877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.134495 (+  1017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.601439 (+466944us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b540008b450. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.601606 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":731030784}
I20260709 13:14:58.615547 30632 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:58.560549  1673 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.618928  1680 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:58.619561  1616 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:14:58.621578  1570 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.003s
I20260709 13:14:58.535028  1677 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.591097  1252 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:14:58.654502 31389 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
W20260709 13:14:58.627101  1411 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.003s
I20260709 13:14:58.687443 31396 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:14:58.631103  1626 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.692840 30552 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:58.680296  1680 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:14:58.690764  1255 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
W20260709 13:14:58.698825  1343 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.001s
I20260709 13:14:58.702730  1255 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.703279  1343 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.705135  1255 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.705322  1343 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.690994  1411 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:14:58.706161 30627 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:58.697340  1642 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.999s	user 0.015s	sys 0.000s
W20260709 13:14:58.712424 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1246) took 1463 ms (client timeout 1651 ms). Trace:
I20260709 13:14:58.712291  1642 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.713222 31378 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:14:58.713645 31364 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:58.713771 30556 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:14:58.714386 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1133) took 1137 ms. Trace:
W20260709 13:14:58.700798  1401 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 13:14:58.702131  1641 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:58.715070  1401 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:58.714815 31378 rpcz_store.cc:276] 0709 13:14:57.576883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.577013 (+   130us) service_pool.cc:224] Handling call
0709 13:14:58.714189 (+1137176us) spinlock_profiling.cc:243] Waited 808 ms on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.714299 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1858260864}
I20260709 13:14:58.715068  1641 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:58.716734 31378 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
I20260709 13:14:58.716852  1641 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.718689  1428 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.719480 30632 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:58.719228  1428 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:58.720549  1656 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 13:14:58.720638  1670 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.721064  1656 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.721680  1428 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.722239 30698 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:14:58.723022  1670 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.723915  1616 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.724145 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:58.718557  1641 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:14:58.728842  1329 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.730671  1642 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.731529 30552 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:14:58.734249  1656 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.731967  1270 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.734055  1679 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.740140  1610 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.740326  1270 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.689034  1618 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.744315 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:14:58.712910 30555 rpcz_store.cc:269] 0709 13:14:57.250658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.251178 (+   520us) service_pool.cc:224] Handling call
0709 13:14:58.712219 (+1461041us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.712358 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752770944}
W20260709 13:14:58.759279 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:58.697649  1677 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:58.685784  1570 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.766079 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.742504  1581 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.754724  1618 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.756153  1343 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.696233  1626 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.816157 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1592) took 1667 ms. Trace:
I20260709 13:14:58.848886 30706 rpcz_store.cc:276] 0709 13:14:57.149247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.854884 (+705637us) service_pool.cc:224] Handling call
0709 13:14:57.871744 (+ 16860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.334295 (+462551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.815842 (+481547us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b540001f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.816091 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1015634816}
I20260709 13:14:58.799021 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1885) took 1064 ms. Trace:
I20260709 13:14:58.850229 30558 rpcz_store.cc:276] 0709 13:14:57.734837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.171625 (+436788us) service_pool.cc:224] Handling call
0709 13:14:58.547671 (+376046us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:58.547921 (+   250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:58.548225 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:58.548261 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:58.548908 (+   647us) log.cc:844] Serialized 26 byte log entry
0709 13:14:58.680459 (+131551us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:14:58.680519 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.680635 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:14:58.798465 (+117830us) raft_consensus.cc:1685] finished
0709 13:14:58.798505 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.798890 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35176832,"thread_start_us":130905,"threads_started":1,"wal-append.queue_time_us":237481}
I20260709 13:14:58.837682 30630 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:14:58.817268 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1890) took 1052 ms. Trace:
I20260709 13:14:58.853757 30559 rpcz_store.cc:276] 0709 13:14:57.765117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.179854 (+414737us) service_pool.cc:224] Handling call
0709 13:14:58.816497 (+636643us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:58.816734 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:58.816942 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:58.816967 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:58.817012 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:58.817044 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.817102 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.817208 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12158720}
I20260709 13:14:58.792305  1603 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.788218  1579 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.807361  1252 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.789703  1329 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.812040  1505 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:58.869709 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:58.852520  1694 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.815431  1615 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:58.815115  1493 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.853197  1581 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.857199  1343 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.874954  1505 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:58.789206  1680 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.892903  1610 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.892976  1505 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.893708  1610 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:58.894835  1505 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.895267  1680 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.897640 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.898491 30552 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:58.877529  1694 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:58.899106  1610 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.903748  1618 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:58.904906  1616 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:58.879303  1615 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:14:58.907430 30553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 597) took 1381 ms (client timeout 1784 ms). Trace:
W20260709 13:14:58.907927 30553 rpcz_store.cc:269] 0709 13:14:57.526043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.526727 (+   684us) service_pool.cc:224] Handling call
0709 13:14:58.907232 (+1380505us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b5400242554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:58.907354 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559715584}
I20260709 13:14:58.908624  1615 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:58.877853  1252 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.911111  1252 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.913187  1493 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.917835 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.918923 31364 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:58.918947 31388 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:14:58.921519 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.922724 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.927659 31378 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:58.929953 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.918723  1618 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:58.916071  1685 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.938735 31378 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:14:58.931430  1679 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:58.950305  1581 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:58.951792 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.952765  1572 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.953935  1572 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:58.955819 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1586) took 1842 ms. Trace:
I20260709 13:14:58.956323 30700 rpcz_store.cc:276] 0709 13:14:57.112979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.761394 (+648415us) service_pool.cc:224] Handling call
0709 13:14:57.761903 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:57.862103 (+100200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:57.862139 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:57.862189 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:14:57.862221 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:58.955533 (+1093312us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b54009cca54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:14:58.955589 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:58.955748 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229697408}
I20260709 13:14:58.957374  1489 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:14:58.957983  1489 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:58.958249  1626 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.937736  1685 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:58.959112  1489 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.960393  1489 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:14:58.960927  1685 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.935989  1618 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.961954 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.963613 31394 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:14:58.964834  1343 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.967024 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:58.965963  1679 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:14:58.969372 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:58.963562  1668 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.000s
I20260709 13:14:58.972522 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:58.972883 30552 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
W20260709 13:14:58.973392 30626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 828) took 2093 ms (client timeout 1607 ms). Trace:
W20260709 13:14:58.973690 30626 rpcz_store.cc:269] 0709 13:14:56.880179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.892868 (+ 12689us) service_pool.cc:224] Handling call
0709 13:14:58.972409 (+2079541us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8f8c 00007f1803db8ef6 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:14:58.973326 (+   917us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34704640}
I20260709 13:14:58.976754  1679 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:58.977658 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:58.978535 31394 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:58.978782  1356 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:58.980880  1356 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:14:58.975816 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:58.981981  1618 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.982128  1673 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:14:58.982746  1673 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:58.983103  1673 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:58.989542  1563 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:58.991209 30596 leader_election.cc:400] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8.
I20260709 13:14:58.991629 30596 leader_election.cc:403] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:14:58.993433  1370 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:58.997504  1694 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:58.998647  1694 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:58.999660 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:59.000591  1493 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:14:59.000985  1493 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.001364  1493 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:59.002032 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.002782  1570 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:59.006847 31389 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:59.007177 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:59.007936 31391 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.008303 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.009302  1668 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:59.009737 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:59.010592 31383 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:59.011667  1606 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:14:59.012964  1641 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:59.013319  1641 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:59.014340  1641 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.015066 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.016356  1669 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.016932  1669 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.010428 31364 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:14:59.018533 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.025527 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:58.982043 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:59.029953 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48001us
I20260709 13:14:59.030687 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.031658 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.032900 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.030696 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:59.035533 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.032446 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.037374 31378 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:14:59.026980 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:14:59.036989 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.036158 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:14:59.039628 31394 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:59.042884 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.044288 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.046120 30701 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:14:59.043285 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:58.986191  1594 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:14:59.041525 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.072005 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:59.045068 30705 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:14:59.051335 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.051494  1693 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:14:59.080387  1693 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:14:59.080822  1693 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:59.081727  1693 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.052110 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.083314 30702 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.084684  1693 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:59.085587 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1883) took 1377 ms. Trace:
I20260709 13:14:59.052273 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.043866  1674 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:14:59.034953  1679 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:14:59.034474  1634 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:59.053222  1610 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:14:59.053151  1607 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:14:59.009792  1482 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:14:59.032261  1618 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:14:59.118741  1669 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:59.121628 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.054522  1687 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:14:59.123258 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.123467 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.124260 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.125422 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.125953 30626 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:59.127137 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.128234 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.129151 30632 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:14:59.129989 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.130630 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.048065  1155 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:59.055193  1615 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:59.056021  1668 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:59.055806  1620 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:14:59.131986  1155 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:59.133077  1155 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.032902  1685 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:59.086299  1675 consensus_queue.cc:1175] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6b393e79ded4461ea5da7b6b6063305f" responder_term: 10 status { last_received { term: 8 index: 6 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:14:59.135357  1685 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:59.099784  1610 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:59.136425  1610 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:14:59.136353  1685 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:14:59.137748  1704 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:14:59.099506  1634 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:59.139158  1634 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:59.142570  1618 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:14:59.143066  1618 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:59.143931  1618 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.144503 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.145694 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.145956 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.146582 30557 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.147127 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.145220  1618 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:14:59.149650 30552 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:14:59.149514 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.095490  1294 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:59.150779 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.044077  1570 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:59.057516  1624 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:59.058969  1692 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:14:59.034870  1626 raft_consensus.cc:2764] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:14:59.061811  1677 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:14:59.048863  1270 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:14:59.064989  1370 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8.
I20260709 13:14:59.047122 31394 raft_consensus.cc:2376] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10. Re-sending same reply.
I20260709 13:14:59.021554 31391 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:59.050457 31364 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:14:59.086001 30554 rpcz_store.cc:276] 0709 13:14:57.708495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.137335 (+428840us) service_pool.cc:224] Handling call
0709 13:14:59.084803 (+947468us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:59.085028 (+   225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.085214 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.085238 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:59.085318 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:59.085351 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.085407 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:14:59.085524 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109670656}
I20260709 13:14:59.192027 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.176155 30702 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:59.094883 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.193624 31389 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:14:59.097697 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.194965 30704 raft_consensus.cc:2368] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:14:59.098085 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.196218 30706 raft_consensus.cc:2368] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:14:59.098522 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.197376 30705 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:14:59.109651  1545 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:14:59.198403  1545 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:59.199638  1545 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.200965  1545 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:14:59.112545  1706 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.069s	user 0.000s	sys 0.003s
I20260709 13:14:59.152753 30632 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:59.122275 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.056288 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.146538 30555 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:59.204005 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 81845.5us
I20260709 13:14:59.204309 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.204375 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.206063 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.123631  1687 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:14:59.168833  1630 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.169319 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.189193 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:59.101066  1592 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:14:59.177927 30630 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:14:59.178771 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.181186 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.164644 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:59.217430 31383 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:14:59.217255 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.205858 30632 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:14:59.099987  1616 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.207973 30555 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:59.220566 31389 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:14:59.224277 30550 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:14:59.225317 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1888) took 1460 ms. Trace:
I20260709 13:14:59.225730 30550 rpcz_store.cc:276] 0709 13:14:57.764456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.176298 (+411842us) service_pool.cc:224] Handling call
0709 13:14:59.218459 (+1042161us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:14:59.223754 (+  5295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.224929 (+  1175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.225230 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3284224}
I20260709 13:14:59.235817 30556 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:14:59.237651 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56565us
I20260709 13:14:59.242040 31396 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:14:59.219659 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.206779  1687 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:59.218783 30632 raft_consensus.cc:606] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:14:59.247421 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.176155  1692 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:14:59.247020 30631 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:59.247881 30554 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:14:59.260883 30627 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:14:59.261010 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.261024 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:14:59.263535 31391 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:14:59.262701 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:14:59.274530 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.185633  1570 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:59.276358 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.277354 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.279210 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.235562  1693 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:59.282639 30700 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:59.283202 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.179715  1656 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:14:59.288440 30630 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:14:59.287506 31367 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:14:59.292176 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.310871 30703 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:14:59.311720 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:14:59.312278 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.313036 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.315284 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.255038  1670 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:14:59.260272  1692 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:59.214900  1669 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:14:59.176873  1673 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.324151  1563 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:14:59.324544 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48315.9us
I20260709 13:14:59.326844 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:14:59.343600 31378 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:14:59.344158 31378 pending_rounds.cc:85] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:14:59.344440 31378 pending_rounds.cc:107] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:14:59.219779 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:14:59.344722 31378 raft_consensus.cc:2889] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:59.344612 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.334671 30626 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:14:59.340615 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:14:59.350006 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.255892  1616 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.356719 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.246914  1630 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.292229  1624 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:14:59.316514  1294 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:14:59.374516 30701 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:14:59.320982  1561 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:59.374150 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.328722  1693 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:14:59.385067 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.292071  1493 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.387961 30558 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:59.368301  1489 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:59.395275 30553 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:14:59.367076  1662 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:14:59.389004 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:59.401153 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 344929us
I20260709 13:14:59.357561  1670 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:59.376269  1624 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:59.380466  1505 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:14:59.409906 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:14:59.402115  1603 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.004s
I20260709 13:14:59.416535 31391 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:14:59.419834 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:14:59.425204 30628 raft_consensus.cc:1217] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Deduplicated request from leader. Original: 7.5->[9.6-11.7]   Dedup: 9.6->[11.7-11.7]
W20260709 13:14:59.424798 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 267251us
I20260709 13:14:59.431612 30628 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:14:59.392860  1401 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:59.400123  1681 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:59.377303  1252 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 13:14:59.404426  1592 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:14:59.488649 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 299572us
I20260709 13:14:59.510771 30557 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:14:59.421308  1674 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:59.417299  1489 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:59.426733  1270 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:14:59.432641  1626 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:14:59.454229  1294 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:14:59.578457 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.583051 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 178105us
I20260709 13:14:59.584896 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.585654 30633 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:14:59.588691 30558 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:14:59.400213  1570 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.380893  1687 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.597451  1493 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:59.549422  1626 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:14:59.607517 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.609035 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 177167us
I20260709 13:14:59.575384  1624 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:59.623576  1616 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:14:59.590535  1294 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:14:59.625028  1630 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:59.636102  1673 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:14:59.638873  1561 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:14:59.547524  1693 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.611310  1270 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.647127 30629 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:14:59.653867 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43751.5us
W20260709 13:14:59.690743 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.693138 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.698069 30553 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:14:59.698390 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.641641  1626 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:14:59.708896 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 124065us
I20260709 13:14:59.710811 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.711918  1329 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:59.712555  1356 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.731s	user 0.015s	sys 0.000s
I20260709 13:14:59.712779  1620 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:14:59.713356  1620 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:59.714258  1620 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:59.715943  1294 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:14:59.717439 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.718753  1624 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:14:59.719221 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:14:59.719698  1493 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:14:59.721056  1255 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:14:59.721556  1255 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:14:59.721951  1255 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:59.723024  1616 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:14:59.706959 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:14:59.646549  1606 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:14:59.661422  1568 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.661037  1669 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.725638 30550 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:14:59.734009 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.731747  1630 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:14:59.734928 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:14:59.735927 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.748339 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.751178 30703 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:14:59.753825 31383 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.003s
I20260709 13:14:59.754441 31383 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:14:59.754582 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.755584 30699 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:14:59.756561 30553 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:14:59.756948 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.757958 30702 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:59.732483  1715 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.767980 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.772797 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.774272 30700 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:14:59.774399 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.772689 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.750034  1719 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:14:59.778182 30523 leader_election.cc:400] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:14:59.778499 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.712574  1329 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.787190 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:14:59.786275 30523 leader_election.cc:403] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:59.782975 30702 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:14:59.655694  1670 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.792587 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:14:59.793455 30699 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:14:59.737962  1615 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:14:59.769461  1505 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:14:59.796002 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62084.7us
I20260709 13:14:59.738812  1606 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.749521  1721 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 13:14:59.752660  1669 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.750692  1568 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.724124  1489 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.790905  1462 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:14:59.782025  1719 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:59.810497 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:14:59.805588  1714 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:14:59.817129 30523 leader_election.cc:400] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
W20260709 13:14:59.818688 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.820917 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.824612 30633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1292) took 2130 ms (client timeout 1973 ms). Trace:
W20260709 13:14:59.825171 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57305.8us
I20260709 13:14:59.825815  1724 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:14:59.827368 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:14:59.828071  1570 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:14:59.840619 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:14:59.841318 30523 leader_election.cc:403] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:14:59.864060 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:14:59.836440 30633 rpcz_store.cc:269] 0709 13:14:57.694605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.695704 (+  1099us) service_pool.cc:224] Handling call
0709 13:14:59.824369 (+2128665us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:14:59.824494 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4013421952}
W20260709 13:14:59.846534 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:14:59.897681 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59619.9us
I20260709 13:14:59.899739 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:14:59.806813  1716 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:14:59.904839 31394 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:14:59.905470 31394 pending_rounds.cc:85] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 5
I20260709 13:14:59.905722 31394 pending_rounds.cc:107] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:14:59.776079  1715 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:14:59.906076 31394 raft_consensus.cc:2889] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:14:59.832233  1618 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.000s
I20260709 13:14:59.937906 31383 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:14:59.938673 31383 pending_rounds.cc:85] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:14:59.939013 31383 pending_rounds.cc:107] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:14:59.939320 31383 raft_consensus.cc:2889] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:14:59.904544  1730 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:14:59.950747  1594 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:14:59.951713  1594 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:14:59.957297  1568 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:14:59.959861  1732 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:14:59.962183  1728 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:14:59.968354  1606 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:14:59.971093 30698 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:14:59.974815  1616 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:14:59.981210  1462 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:14:59.981828  1715 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:14:59.982553  1462 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.813659  1726 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:14:59.993924  1563 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:14:59.803085  1505 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.019697 31396 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:15:00.020349 31396 pending_rounds.cc:85] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:00.020663 31396 pending_rounds.cc:107] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:15:00.020963 31396 raft_consensus.cc:2889] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:00.029989 30698 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:14:59.897680  1356 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:15:00.036983 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101234us
I20260709 13:15:00.041805  1719 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.042758  1677 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:00.043298  1719 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:00.043507  1677 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:14:59.966120  1721 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:15:00.050292 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.050685 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.053114  1568 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:15:00.055073  1641 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:00.055450  1677 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:00.061371 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 161705us
I20260709 13:15:00.054214 31389 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:00.064746 31367 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:15:00.045960  1610 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.909s	user 0.015s	sys 0.000s
I20260709 13:15:00.065774 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:00.032030  1545 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.007s
I20260709 13:15:00.097118 31383 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:00.032656  1570 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:00.046309  1618 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.105497 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.057518  1716 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:00.121513 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56359.7us
I20260709 13:15:00.130009 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:00.131258 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:00.137392 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.142916  1656 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:00.143677  1656 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:00.144202  1656 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:00.034288  1563 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:00.169240 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 352272us
I20260709 13:15:00.173367 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:00.127961  1644 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.122157  1615 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.181694 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:00.181963 30632 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:00.051383  1626 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.112121  1735 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:00.154240  1716 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:00.188023 30671 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.973s (SENT) [suppressed 21 similar messages]
I20260709 13:15:00.134464  1594 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.151364  1356 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:00.105474  1610 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.121901  1618 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.225327 30627 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:00.234241 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:00.236476 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.192566  1735 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:00.251170 31378 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:00.196687  1356 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:00.188181  1606 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:15:00.265872 30550 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.529s	user 0.012s	sys 0.003s
I20260709 13:15:00.278568 30550 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:15:00.279666 30552 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:15:00.236692  1294 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.197172  1329 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.194890  1615 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.237272  1607 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:00.192201  1462 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:00.188200  1644 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.278849  1356 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:00.275233  1738 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.330341 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.280542  1669 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:00.301723  1401 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.842s	user 0.015s	sys 0.000s
I20260709 13:15:00.295601  1462 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:00.330260  1572 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.376s	user 0.018s	sys 0.001s
I20260709 13:15:00.222751  1570 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.274924  1735 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.295426  1607 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:00.333575  1634 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.194s	user 0.001s	sys 0.015s
I20260709 13:15:00.346750  1329 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.330531  1742 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.364985  1155 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:00.366202 31394 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:00.292274  1294 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.386550 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:00.349797  1401 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:00.358304  1505 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:00.355671  1462 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:00.335150  1738 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:00.399194 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 211198us
I20260709 13:15:00.400403 31394 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:00.404175 30631 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:00.389138  1411 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.397819  1737 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:15:00.411785 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:00.413905 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:00.415274  1721 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:00.416528  1626 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:00.417007  1745 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:15:00.393291  1634 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.397046  1155 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.423857  1634 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:00.425283  1155 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.407277  1270 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.426134  1741 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:00.426365  1270 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.426832 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.427637 31394 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:00.429446  1270 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.430811 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.431165  1561 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:00.431592  1561 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:00.432818  1561 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.431833 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:00.395180  1742 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.440277  1610 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:00.440960 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.441466  1618 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:00.442731  1563 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.443205  1644 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.393795  1329 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.444634 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.445377 31394 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:00.445689  1329 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:00.447857 30629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 615) took 1328 ms (client timeout 1526 ms). Trace:
W20260709 13:15:00.448392 30629 rpcz_store.cc:269] 0709 13:14:59.119848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.122943 (+  3095us) service_pool.cc:224] Handling call
0709 13:15:00.447654 (+1324711us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54004a83d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.447786 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264405760}
I20260709 13:15:00.452684  1735 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.454842 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.456192 31394 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:00.405311  1677 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:15:00.460384 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.412143  1411 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.477332  1615 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.477916  1411 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.465444  1607 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.479107  1607 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:00.480188 31361 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:00.483595 30550 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:00.472414  1716 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.486366  1294 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:00.464910  1692 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.113s	user 0.004s	sys 0.012s
I20260709 13:15:00.438494  1742 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:00.416491  1738 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.460239  1561 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:15:00.486115  1356 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.517998 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.489817  1270 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:00.490686  1674 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.926s	user 0.015s	sys 0.000s
W20260709 13:15:00.522207 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 91015.2us
I20260709 13:15:00.525352 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.530704 30702 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:00.528659 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.532931  1738 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:00.534057  1738 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.529778  1674 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:00.537321  1741 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:00.515584  1673 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.545712 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1602) took 1288 ms. Trace:
I20260709 13:15:00.528365  1742 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:15:00.547490  1255 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.825s	user 0.015s	sys 0.000s
I20260709 13:15:00.510598  1692 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.559055 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.566138 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.549078  1738 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:00.539865  1270 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:00.559568  1724 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.629s	user 0.010s	sys 0.004s
I20260709 13:15:00.574936 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:00.576828 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.582159 30632 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:00.590777  1356 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.592074 31391 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:00.596117 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.597510 30703 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:00.600118  1718 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:00.600704  1718 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:00.601656  1718 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.605710 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:00.550704  1673 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.612334  1270 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:00.606966  1724 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:00.621713 30704 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:15:00.633498  1641 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.006s
W20260709 13:15:00.635831  1719 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.010s
I20260709 13:15:00.617292  1692 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:00.620771  1656 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.562203  1255 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.612581  1673 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.609947  1607 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:00.665645 30554 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:00.642140  1615 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:00.664007 30707 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:00.676649  1255 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.685623  1656 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.666188  1462 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:00.689015 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.690966  1462 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.691696  1370 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.692317  1370 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.694416  1370 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.695154  1370 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:00.695745  1370 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.698155  1370 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.699882  1731 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:00.700414  1731 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:00.701561  1731 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:00.711438  1731 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:00.723477 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 610) took 3012 ms. Trace:
I20260709 13:15:00.724146 30628 rpcz_store.cc:276] 0709 13:14:57.710708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:57.759625 (+ 48917us) service_pool.cc:224] Handling call
0709 13:14:57.764908 (+  5283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:14:59.432101 (+1667193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.723009 (+1290908us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54005f41d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:00.723364 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3817850240}
I20260709 13:15:00.550092 30705 rpcz_store.cc:276] 0709 13:14:59.257071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.257584 (+   513us) service_pool.cc:224] Handling call
0709 13:14:59.286407 (+ 28823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.286673 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:59.413994 (+127321us) log.cc:844] Serialized 14 byte log entry
0709 13:15:00.462781 (+1048787us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b18002bc120. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:00.545243 (+ 82462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:00.545305 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:00.545363 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.545466 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:00.545632 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127140,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":14500,"raft.queue_time_us":93,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":2898,"spinlock_wait_cycles":12781440,"thread_start_us":152258,"threads_started":2}
W20260709 13:15:00.752185 30625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 618) took 1631 ms (client timeout 1580 ms). Trace:
W20260709 13:15:00.752943 30625 rpcz_store.cc:269] 0709 13:14:59.120710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.126754 (+  6044us) service_pool.cc:224] Handling call
0709 13:15:00.751071 (+1624317us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b540063ced4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.752082 (+  1011us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076310784}
I20260709 13:15:00.758713 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1314) took 1795 ms. Trace:
I20260709 13:15:00.759395 31388 rpcz_store.cc:276] 0709 13:14:58.963315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:58.964779 (+  1464us) service_pool.cc:224] Handling call
0709 13:14:59.217682 (+252903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.217912 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.217941 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:14:59.217992 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:14:59.218027 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.218082 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:00.758346 (+1540264us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54004f8650. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.758580 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580758912}
W20260709 13:15:00.761240  1733 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.775s	user 0.000s	sys 0.004s
I20260709 13:15:00.768898 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.771422 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.772349 30703 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:00.772814  1738 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.773393 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.773559  1738 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.775039 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.775972 30698 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:00.776019 30705 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:15:00.785939 30699 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:00.778661  1739 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:00.780826  1738 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:00.781754  1718 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:00.783802 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1166) took 1663 ms. Trace:
I20260709 13:15:00.775341 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:00.791412 31396 rpcz_store.cc:276] 0709 13:14:59.120708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.243535 (+122827us) service_pool.cc:224] Handling call
0709 13:14:59.243952 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.021591 (+777639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.783515 (+761924us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400113d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:00.783735 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1782781696}
I20260709 13:15:00.793303 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:00.796663  1674 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.796895 30703 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.797303  1674 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:00.798806 30381 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:00.801725 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1306) took 1565 ms. Trace:
I20260709 13:15:00.802963  1674 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:00.807049 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1272) took 1661 ms. Trace:
I20260709 13:15:00.807315 30555 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:00.807539 30558 rpcz_store.cc:276] 0709 13:14:59.145591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.149259 (+  3668us) service_pool.cc:224] Handling call
0709 13:15:00.806833 (+1657574us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54001225d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.806981 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546125184}
I20260709 13:15:00.810050 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1902) took 1573 ms. Trace:
I20260709 13:15:00.810110  1620 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:00.810840  1561 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:00.811617  1561 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.814272  1561 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:00.815251  1687 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:00.810454 30555 rpcz_store.cc:276] 0709 13:14:59.236258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.245626 (+  9368us) service_pool.cc:224] Handling call
0709 13:15:00.807023 (+1561397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.808079 (+  1056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.809802 (+  1723us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54001225d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.809990 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3589882112}
I20260709 13:15:00.821902 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.822690 30551 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:00.802189 30624 rpcz_store.cc:276] 0709 13:14:59.236828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.278799 (+ 41971us) service_pool.cc:224] Handling call
0709 13:15:00.801492 (+1522693us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54004ab0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.801656 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079788416}
I20260709 13:15:00.832146 30629 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:00.836318 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.837757 30631 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:00.836426 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.765724 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.840366 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.833145 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.787810 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:00.795605 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.786873 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.881664 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:00.882938 30698 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:00.830106 30703 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:00.840445 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.881910 30700 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:15:00.888336 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.889163 30629 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:00.888078 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.891400 30633 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:00.887060 30626 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:00.840495 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.840967 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.982383 30628 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:00.841394 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.904162 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.906020 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.910684 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.983847 30626 raft_consensus.cc:2410] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:00.911405 30551 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:00.911305 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.984572 30627 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:00.985229 30555 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:00.902318  1736 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.919s	user 0.003s	sys 0.000s
I20260709 13:15:00.888059  1693 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.912387  1155 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:00.914258 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.982134 30625 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:15:00.842568  1592 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:00.915729 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.993055 30629 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:15:00.993325 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:00.915864 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 156) took 16924 ms (client timeout 15000 ms). Trace:
W20260709 13:15:00.918254 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:00.917236 30381 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 8 to 10, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:00.920887 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.920163 30631 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:00.921583 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:00.996090 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74621.2us
W20260709 13:15:00.923996  1734 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.937s	user 0.003s	sys 0.000s
I20260709 13:15:00.925055  1738 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.927803 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.928107  1741 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:00.928447  1294 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:00.932415  1401 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:00.933002 30705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:01.019001 30705 raft_consensus.cc:606] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:01.019392 30705 raft_consensus.cc:614] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:00.933799  1692 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:00.934733  1674 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:01.042040 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.937940 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1332) took 1076 ms. Trace:
I20260709 13:15:01.042783 31367 rpcz_store.cc:276] 0709 13:14:59.861508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.861668 (+   160us) service_pool.cc:224] Handling call
0709 13:14:59.949472 (+ 87804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:00.065218 (+115746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:00.937698 (+872480us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002b0a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:00.937875 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264155904}
W20260709 13:15:00.938685 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1251) took 1896 ms (client timeout 2236 ms). Trace:
I20260709 13:15:00.941332 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:00.944950 30559 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
W20260709 13:15:01.043910 30707 rpcz_store.cc:269] 0709 13:14:59.042201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.042604 (+   403us) service_pool.cc:224] Handling call
0709 13:15:00.938491 (+1895887us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54002233d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:00.938599 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3670242688}
I20260709 13:15:01.044525 30559 pending_rounds.cc:85] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:15:01.044905 30559 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:15:01.045207 30559 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:01.045245 30626 raft_consensus.cc:2376] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13. Re-sending same reply.
I20260709 13:15:00.948719 31388 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:01.046418 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.950762 31378 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:15:00.836549  1675 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:00.967327 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.969012 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.970850 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:00.974234 30558 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:00.974354 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:00.975524 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:00.976840 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:00.977465 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:00.978828 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:00.791145  1739 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:00.979388 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:00.979612  1670 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:01.050578 31391 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:00.991714 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:00.997694 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:01.011643 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:00.997033 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:01.058988 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61363.2us
W20260709 13:15:01.059490 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62541.4us
I20260709 13:15:01.014354 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:01.016127 30381 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:01.061460 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:01.061596 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:01.062268 30550 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:01.015570  1718 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:01.023288 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:01.051272 31364 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:01.063941 30627 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:01.023932 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:01.030264 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.021250 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:01.046499 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:01.032553 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:01.065999 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:01.066414 30631 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:15:00.980983 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 193300us
I20260709 13:15:01.049237 30552 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:15:01.067713 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:00.962971 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:01.064661 30628 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:01.052685 30704 raft_consensus.cc:2376] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8. Re-sending same reply.
I20260709 13:15:01.050845 31383 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:01.060173 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:01.080614 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:01.083668 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:01.038806  1370 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.057134 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 77843us
I20260709 13:15:01.095976 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:01.091203 30624 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:01.091652 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:01.006820  1693 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.068976  1718 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:01.100724 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:01.069233  1715 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:01.018080  1738 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:01.130919 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:01.152565 30704 raft_consensus.cc:606] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:01.087575  1561 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:01.131917 30627 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:01.025633  1294 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.040573  1401 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.078689  1606 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:01.143061 31367 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:01.086961  1482 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:01.048254  1692 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.063598  1675 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.087659  1753 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:15:01.127372  1356 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:01.158504 30702 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:01.159340 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:01.194489 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:01.196338 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:01.134835  1736 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:15:01.101210  1730 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
W20260709 13:15:01.170575 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44613.2us
W20260709 13:15:00.994520 30382 rpcz_store.cc:269] 0709 13:14:44.011793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.354212 (+342419us) service_pool.cc:224] Handling call
0709 13:15:00.915728 (+16561516us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:14:55.187136 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:14:55.187894 (+   758us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:14:55.187953 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:14:55.187989 (+    36us) tablet.cc:662] Decoding operations
0709 13:14:55.189610 (+  1621us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:14:55.189723 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:14:55.189752 (+    29us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:14:55.308630 (+118878us) tablet.cc:701] Row locks acquired
0709 13:14:55.308683 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:14:55.308903 (+   220us) write_op.cc:270] Start()
0709 13:14:55.309083 (+   180us) write_op.cc:276] Timestamp: P: 1783602895308819 usec, L: 0
0709 13:14:55.309115 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:14:55.309997 (+   882us) log.cc:844] Serialized 7079 byte log entry
0709 13:14:55.892357 (+582360us) op_driver.cc:464] REPLICATION: finished
0709 13:14:56.588134 (+695777us) write_op.cc:301] APPLY: starting
0709 13:14:56.643608 (+ 55474us) tablet.cc:1367] starting BulkCheckPresence
0709 13:14:56.684330 (+ 40722us) tablet.cc:1370] finished BulkCheckPresence
0709 13:14:56.684486 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:14:56.688127 (+  3641us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:14:56.797568 (+109441us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=18
0709 13:14:56.797643 (+    75us) write_op.cc:312] APPLY: finished
0709 13:14:56.798164 (+   521us) log.cc:844] Serialized 124 byte log entry
0709 13:14:56.798391 (+   227us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:56.803913 (+  5522us) write_op.cc:454] Released schema lock
0709 13:14:56.809111 (+  5198us) write_op.cc:341] FINISH: Updating metrics
0709 13:14:56.830407 (+ 21296us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:14:56.830443 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":262032,"reported_tablets":35,"child_traces":[["op",{"apply.queue_time_us":576004,"apply.run_cpu_time_us":6247,"apply.run_wall_time_us":277105,"num_ops":18,"prepare.queue_time_us":602403,"prepare.run_cpu_time_us":17412,"prepare.run_wall_time_us":303614,"raft.queue_time_us":195920,"raft.run_cpu_time_us":18452,"raft.run_wall_time_us":597280,"replication_time_us":527673,"thread_start_us":1023279,"threads_started":4,"wal-append.queue_time_us":249428,"wal-append.run_cpu_time_us":12642,"wal-append.run_wall_time_us":2242971}]]}
I20260709 13:15:01.134207 30624 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:15:01.143215  1771 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:15:01.052703  1674 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.129268  1718 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:01.166944 30632 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:01.173399 31394 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:01.177292 30629 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:01.165373  1606 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.097818  1570 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:15:01.197789 30381 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:01.128937  1670 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:15:01.100996  1489 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:15:01.100126  1370 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.133360  1656 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:15:01.200912 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:01.174249  1692 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:01.188508 30627 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:01.205626  1462 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:01.173560  1739 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:01.169353  1719 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:15:01.174353  1738 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:01.166079  1482 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:01.184463  1715 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:01.239579 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:01.284096 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44668.7us
W20260709 13:15:01.272910 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 233222us
I20260709 13:15:01.238718 30381 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 6, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:01.203938  1411 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:01.209774  1606 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:01.220799  1726 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:15:01.272555 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.240554 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1603) took 1983 ms. Trace:
I20260709 13:15:01.265429  1766 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:15:01.228298  1616 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:15:01.315187 30706 rpcz_store.cc:276] 0709 13:14:59.257414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.257665 (+   251us) service_pool.cc:224] Handling call
0709 13:14:59.258113 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:14:59.258599 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:14:59.258666 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:14:59.639640 (+380974us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b2c00285df8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:14:59.640151 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 13:14:59.793064 (+152913us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:14:59.815704 (+ 22640us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.998068 (+182364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:14:59.998213 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:00.901611 (+903398us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:01.240312 (+338701us) raft_consensus.cc:1685] finished
0709 13:15:01.240336 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:01.240484 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":243356,"prepare.queue_time_us":22632,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":472048,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":32227,"spinlock_wait_cycles":241283584,"thread_start_us":152457,"threads_started":1,"wal-append.queue_time_us":342635}
I20260709 13:15:01.211669  1356 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:01.322669  1772 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:15:01.325062 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:01.326443  1401 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:01.309962  1411 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.316591  1493 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:01.335577  1766 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:01.279683  1776 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:15:01.340049  1692 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:01.344422  1674 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:01.274933  1675 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:01.347052  1294 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:01.281392  1739 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:01.407240  1693 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:01.407380  1739 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:01.408252  1739 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:01.360965 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:01.313436  1482 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:01.314893  1726 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.411315  1776 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.411574  1726 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.411980  1776 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.378893  1411 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:01.416749  1692 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:15:01.417965  1692 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:01.419054  1674 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:15:01.419457  1674 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:01.419754  1766 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.420326  1674 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.421593  1674 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:15:01.425660 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 64823.2us
I20260709 13:15:01.443632 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:01.402666  1370 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.444962 30704 raft_consensus.cc:606] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:01.414449  1776 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.400509  1568 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.452951 30704 raft_consensus.cc:614] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:01.447777 30625 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:01.443406  1489 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:01.457183  1294 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.475924 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 408274us
I20260709 13:15:01.454155  1687 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.473657 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 240791us
I20260709 13:15:01.512833 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:01.483544 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:01.495003 30706 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:01.481514 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 624) took 2215 ms. Trace:
I20260709 13:15:01.486502  1294 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.455660  1568 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.485378  1679 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:01.469800  1719 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:01.575187 30630 rpcz_store.cc:276] 0709 13:14:59.266014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.354350 (+ 88336us) service_pool.cc:224] Handling call
0709 13:14:59.355209 (+   859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:14:59.355494 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:14:59.378299 (+ 22805us) log.cc:844] Serialized 14 byte log entry
0709 13:14:59.827037 (+448738us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b18001161a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:00.777574 (+950537us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b18001161a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:00.777952 (+   378us) log.cc:844] Serialized 14 byte log entry
0709 13:15:01.476901 (+698949us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:01.476992 (+    91us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:01.477173 (+   181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.479021 (+  1848us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:01.481401 (+  2380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1367456,"prepare.queue_time_us":1426556,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":99,"raft.queue_time_us":241073,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":3912,"spinlock_wait_cycles":17220864,"thread_start_us":698459,"threads_started":2,"wal-append.queue_time_us":242514}
W20260709 13:15:01.588095 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 104617us
I20260709 13:15:01.478188  1730 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:15:01.485670  1489 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:01.512478  1738 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.475672  1570 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 13:15:01.592373  1741 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:01.594064  1746 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.075s	user 0.007s	sys 0.000s
I20260709 13:15:01.494791  1687 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.463186  1715 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:01.659200 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63241.2us
I20260709 13:15:01.612483  1561 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.517251  1718 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:01.667804 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.034s	sys 0.073s
W20260709 13:15:01.724197 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.034s	sys 0.073s
W20260709 13:15:01.671402 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 384720us
W20260709 13:15:01.725268 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 704133us
I20260709 13:15:01.728266 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:01.704133 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.010s	sys 0.012s
W20260709 13:15:01.729297 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.011s	sys 0.013s
I20260709 13:15:01.604926  1719 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.692251  1738 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.726763 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:01.731357 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 733258us
I20260709 13:15:01.741279 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:01.664552  1778 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:15:01.723505  1561 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:01.725088  1741 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:01.605350  1679 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:01.774418 30525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.015s	sys 0.048s
I20260709 13:15:01.768500 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:01.778405 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:01.731422  1356 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.741039  1693 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:01.738507  1462 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:01.787022 30525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.015s	sys 0.048s
I20260709 13:15:01.781840  1620 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.775723  1675 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:15:01.809934  1776 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:01.813397  1693 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:01.802011  1462 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.819234  1788 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:01.864439 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 351744us
I20260709 13:15:01.837801  1693 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.814100  1620 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.823931  1370 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:15:01.813753  1606 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:01.869688 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:01.801194  1489 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:01.837834  1777 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:01.835947  1568 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.896603  1606 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.898116 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:01.911319  1777 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:01.846838  1294 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:01.874348  1780 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:01.874852  1693 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:01.824234  1785 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:15:01.869063  1792 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:01.924288  1294 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:01.927672  1294 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:01.928071  1294 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:01.929078  1411 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.514s	user 0.018s	sys 0.002s
I20260709 13:15:01.929607  1411 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:01.835574  1741 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:01.932103  1716 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:01.932379  1803 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:01.932771  1716 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:01.933437  1803 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:01.935011  1790 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:01.935945  1790 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:01.943816 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1313) took 1048 ms. Trace:
I20260709 13:15:01.944324 30628 rpcz_store.cc:276] 0709 13:15:00.895483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.086787 (+191304us) service_pool.cc:224] Handling call
0709 13:15:01.087212 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.087465 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:01.471323 (+383858us) log.cc:844] Serialized 14 byte log entry
0709 13:15:01.943263 (+471940us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:01.943434 (+   171us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:01.943526 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.943605 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:01.943743 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46769,"prepare.queue_time_us":84980,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":26865,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":16476,"thread_start_us":776260,"threads_started":3,"wal-append.queue_time_us":456348}
W20260709 13:15:01.953794 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 167397us
I20260709 13:15:01.922663  1719 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:01.925611  1792 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:01.915086 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:01.966636 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:01.934058  1803 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:01.875340  1788 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:01.973516 30698 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:01.954046  1718 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:01.954473  1774 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:01.971060  1739 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.562s	user 0.016s	sys 0.000s
I20260709 13:15:01.979632 30705 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:01.976969 30704 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:02.002529 30699 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:15:01.992372 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 270702us
W20260709 13:15:02.007488 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 92527.7us
W20260709 13:15:01.917671 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 176498us
I20260709 13:15:01.963677  1592 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.023921 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.008587 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:02.015923 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:02.022951 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.022951 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.023572 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.031315 30555 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:02.023880 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.030660 30558 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:02.037081 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:01.926313  1780 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.039494 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.043735 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.039821 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.046227 30703 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:01.972213  1792 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:02.047371 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.048080 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.051555 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.050769 30704 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:01.980656  1738 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.018059  1719 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.020336  1716 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.035907  1692 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:02.074978 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66508.9us
W20260709 13:15:02.075829 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60010.8us
I20260709 13:15:02.085786 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.087452 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.045648  1779 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.084470 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:02.088467 30699 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:02.093046 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.096144  1716 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:02.096994  1155 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:02.097247  1716 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:02.097513  1155 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:02.097805  1716 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:02.098819  1155 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.099154  1716 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:02.100636  1716 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:02.101540  1482 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.690s	user 0.014s	sys 0.000s
I20260709 13:15:02.102346  1482 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.037149  1739 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.080916  1785 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:15:02.038380  1805 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:02.108719 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51028.3us
I20260709 13:15:02.093693 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.117358 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.117470 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1348) took 1145 ms. Trace:
I20260709 13:15:02.077812  1568 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.095201  1738 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:02.118301 31383 rpcz_store.cc:276] 0709 13:15:00.972535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.072815 (+100280us) service_pool.cc:224] Handling call
0709 13:15:01.102199 (+ 29384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.102455 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:01.576011 (+473556us) log.cc:844] Serialized 14 byte log entry
0709 13:15:02.102349 (+526338us) spinlock_profiling.cc:243] Waited 5.48 ms on lock 0x7b18000e6440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287
0709 13:15:02.116890 (+ 14541us) log.cc:844] Serialized 14 byte log entry
0709 13:15:02.117098 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:02.117152 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:02.117202 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.117290 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.117410 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3643,"prepare.queue_time_us":636310,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":65,"raft.queue_time_us":250052,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":9797,"spinlock_wait_cycles":12612096,"thread_start_us":687473,"threads_started":2,"wal-append.queue_time_us":10652}
I20260709 13:15:02.132707 30706 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:02.094897  1807 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:02.125350  1489 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.041725  1592 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.095559  1692 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:02.124454 30552 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:15:02.143525 30552 pending_rounds.cc:85] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 1
I20260709 13:15:02.091095  1719 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.143991 30552 pending_rounds.cc:107] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:02.144660 30552 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:02.123052  1741 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:02.135597  1738 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.113516  1675 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.126348  1820 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:02.110404  1799 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:02.121965  1798 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:15:02.128523 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59294.2us
I20260709 13:15:02.148835 30552 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:02.134207 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.136876  1805 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:15:02.154791 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 70390.7us
I20260709 13:15:02.156502 30559 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:02.161463 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.141065  1620 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.126742  1791 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:15:02.164261 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:02.173740 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.178248  1776 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:02.178731  1776 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:02.179139  1776 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:02.179821 30552 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:15:02.180267 30554 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:02.181489  1811 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:02.181979  1811 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.182336  1811 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:02.182471  1739 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.187536 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.172442 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.189374 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.155504  1738 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:02.139577  1778 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:15:02.130422  1568 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.155635  1803 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.112100  1779 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.221453 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.223493  1826 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:02.163642  1675 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:02.176538  1812 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 13:15:02.284448 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63041.3us
I20260709 13:15:02.288077 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.288074  1799 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:02.157929  1820 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:02.221033 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 101991us
I20260709 13:15:02.289932  1820 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:02.290602 30556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1903) took 3030 ms (client timeout 2041 ms). Trace:
I20260709 13:15:02.290891 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:02.291353  1775 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.686s	user 0.002s	sys 0.000s
W20260709 13:15:02.291167 30556 rpcz_store.cc:269] 0709 13:14:59.259741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:59.262327 (+  2586us) service_pool.cc:224] Handling call
0709 13:15:02.290373 (+3028046us) spinlock_profiling.cc:243] Waited 2.52 s on lock 0x7b5400a27bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:02.290493 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5787568896}
I20260709 13:15:02.292119  1813 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:02.292598  1813 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:02.293058  1813 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:02.293442  1616 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:02.294224  1788 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.186414  1785 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:02.161684  1804 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:02.269805  1814 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:15:02.270046  1719 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.270493  1568 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.265762  1826 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.272948  1675 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:02.273219  1781 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.492s	user 0.003s	sys 0.000s
I20260709 13:15:02.275525  1803 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:02.289944  1401 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:02.279115  1592 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.283584  1482 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:02.284900  1809 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:15:02.285202  1780 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.225086  1825 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:02.369920  1825 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.370491  1825 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:02.194603  1805 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:02.355000  1716 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.375694  1805 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.376570  1716 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.143913  1489 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.379946  1489 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.160794  1718 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 13:15:02.330442  1826 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:02.300385 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:02.304179  1726 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.004s
W20260709 13:15:02.284371  1815 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:02.309894 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1336) took 1383 ms. Trace:
I20260709 13:15:02.241904  1739 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:02.311987 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 218990us
W20260709 13:15:02.397814 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97518.7us
I20260709 13:15:02.317866 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:02.225210  1798 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:02.320894 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90865.6us
I20260709 13:15:02.410583 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:02.412726 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94965.9us
I20260709 13:15:02.413733 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.413972 30701 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:15:02.321722 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.192899  1493 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.119s
I20260709 13:15:02.276244  1779 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.315968  1785 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:02.327888  1570 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.322930  1807 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:02.315685  1789 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 13:15:02.411727 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.393900 31396 rpcz_store.cc:276] 0709 13:15:00.926462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:00.959643 (+ 33181us) service_pool.cc:224] Handling call
0709 13:15:00.960166 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:00.961141 (+   975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:00.961185 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:00.961854 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 13:15:01.259070 (+297216us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:01.259144 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.259271 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:02.290653 (+1031382us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:02.299245 (+  8592us) raft_consensus.cc:1685] finished
0709 13:15:02.299271 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.309805 (+ 10534us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":133,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":139,"spinlock_wait_cycles":4141312,"thread_start_us":296564,"threads_started":1,"wal-append.queue_time_us":551382}
I20260709 13:15:02.357765  1401 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:02.427269 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.389561  1811 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:02.416669  1674 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.995s	user 0.012s	sys 0.008s
I20260709 13:15:02.434778 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.445253 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.419883  1803 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:02.412117  1592 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:02.439313 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.421022  1780 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:15:02.430003  1561 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.646s	user 0.015s	sys 0.000s
I20260709 13:15:02.443594 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.425688  1785 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:02.451524 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.439960 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.469709 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.420809  1824 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.447767 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.451211  1779 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:02.453956  1715 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:02.473255 30707 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:02.460530  1831 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.429054  1462 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:02.454519  1294 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.526s	user 0.015s	sys 0.001s
I20260709 13:15:02.417794  1726 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:02.482167 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.484143 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55961us
I20260709 13:15:02.519853 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:02.484763 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.486424 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.521687 30699 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:02.487264 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:02.473927 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.523219 30701 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:02.489929 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:02.524284 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.525045 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.490214  1493 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:02.456003  1803 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:02.428925  1606 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:02.492996 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.540620 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.540648 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.458228  1716 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.527415 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.543704 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:02.430853  1570 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.479144  1753 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.542258 30707 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:15:02.556447 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64378.5us
I20260709 13:15:02.441466  1813 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:02.535323 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48158.1us
I20260709 13:15:02.541350 30552 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:15:02.487272  1779 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.560734 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1355) took 1046 ms. Trace:
I20260709 13:15:02.561110 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.468930  1561 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.481315  1778 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:15:02.572896 30550 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:15:02.541071  1693 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.601s	user 0.011s	sys 0.002s
I20260709 13:15:02.573347 30557 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:02.566404 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:02.540359  1803 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:02.572767 31378 rpcz_store.cc:276] 0709 13:15:01.514505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.515496 (+   991us) service_pool.cc:224] Handling call
0709 13:15:01.535699 (+ 20203us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:01.817510 (+281811us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:01.817889 (+   379us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:01.829100 (+ 11211us) log.cc:844] Serialized 52 byte log entry
0709 13:15:02.052858 (+223758us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:02.052900 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.053057 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:02.283579 (+230522us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:02.538085 (+254506us) raft_consensus.cc:1685] finished
0709 13:15:02.538114 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.560660 (+ 22546us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":249308,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":12422,"spinlock_wait_cycles":14387456,"thread_start_us":419699,"threads_started":2,"wal-append.queue_time_us":297689}
I20260709 13:15:02.546118  1715 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:02.487275  1825 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.580199  1779 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:02.577864  1798 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:02.488126  1824 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.536281  1294 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:02.613085 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.549881  1606 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.584137  1675 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:02.507084  1401 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.602386  1493 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:02.517014  1831 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:02.633682 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 114001us
I20260709 13:15:02.627840  1718 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:02.647802  1838 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:02.631474  1717 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:02.649169  1717 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:02.594019  1592 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:02.650923  1825 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.651690  1825 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.652194  1825 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.652621  1832 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:15:02.653541  1820 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.650447  1717 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.648451  1838 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:02.572767  1753 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.661413 30556 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:02.594403  1693 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.555728  1807 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.669754  1786 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:02.624861  1715 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:02.656443  1294 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.670383  1786 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:02.628253  1568 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:15:02.663067  1838 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:02.642459  1675 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.671597  1831 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.664624  1825 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:02.717391  1836 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:02.734201  1693 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.736053  1561 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.736442  1155 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:02.743453  1570 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.749914  1820 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:02.751893  1606 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.752960  1692 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.753922  1753 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:02.754803  1816 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.605s	user 0.003s	sys 0.000s
I20260709 13:15:02.755223 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.755575  1824 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:02.756387  1715 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.760111  1777 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.762447  1777 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:02.765583  1776 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.001s
I20260709 13:15:02.763581 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.766655 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:02.693572  1811 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.767673  1726 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.767762  1811 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:02.746590  1786 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:02.769582 30553 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:02.769989  1786 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:02.705742  1568 consensus_queue.cc:1243] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:15:02.771647 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.766949 30633 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:02.766312 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:02.782487 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 100334us
I20260709 13:15:02.800762 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.790899  1693 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:02.766145  1776 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:02.791517  1808 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.660s	user 0.003s	sys 0.000s
I20260709 13:15:02.823720 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.823923 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.824980 31361 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:02.797811  1606 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.784768  1568 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:02.824465 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.832469 30628 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:02.828831 31396 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:02.829752 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.833994 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:02.831625 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 151248us
I20260709 13:15:02.834688 31391 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:15:02.834213 31361 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:02.839687 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.840229 30629 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:02.840559 30624 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:02.840273 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.841462 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.832954 31388 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:02.817170  1693 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.845181 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.853751 30632 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:15:02.842128 31378 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:02.845790 30625 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:02.848173 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.854681 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.855273 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:02.830924  1738 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.003s
I20260709 13:15:02.849638 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.849387 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.855288 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.850967 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.849321 31361 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:02.857349 31391 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:15:02.856216 30629 raft_consensus.cc:2410] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:02.857823 31388 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:02.858742 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.859612 30632 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:02.860327 31389 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:02.861572 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.862573 31361 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:02.863701 31367 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:02.846520  1568 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:02.866067  1837 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:15:02.871282 30633 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:02.864351  1738 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:02.878770  1774 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:02.879603  1675 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:02.872637  1568 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.888998 30705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:02.889959 30705 raft_consensus.cc:606] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:02.891335 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.892369 31367 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:02.894116 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 875) took 1854 ms. Trace:
I20260709 13:15:02.894760 30631 rpcz_store.cc:276] 0709 13:15:01.039384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.242361 (+202977us) service_pool.cc:224] Handling call
0709 13:15:01.277652 (+ 35291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:01.669917 (+392265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:01.669973 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:01.703760 (+ 33787us) log.cc:844] Serialized 26 byte log entry
0709 13:15:01.955197 (+251437us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:01.955258 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:01.955362 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:02.790767 (+835405us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:02.884071 (+ 93304us) raft_consensus.cc:1685] finished
0709 13:15:02.884109 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:02.894001 (+  9892us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138573,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":59,"spinlock_wait_cycles":4528896,"thread_start_us":584720,"threads_started":2,"wal-append.queue_time_us":399490}
I20260709 13:15:02.898219 30706 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:02.901387 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.902153 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.903486 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.905694 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:02.879663  1774 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:02.843364  1606 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:02.906739 31391 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:02.904417 30701 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:02.885933  1726 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:02.927378 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.929667 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:02.930713 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:02.935163 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:02.931947 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:02.938441 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.939574 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.940618 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.940963 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.942178 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.942739 30629 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:02.946732 31361 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:02.950493 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.951243 30631 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:02.952685 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.907931  1738 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.961575 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.957391 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:02.925544  1675 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:02.927048  1411 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:02.971886 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.966291 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:02.974726 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.982864 30625 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:02.933125  1561 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:15:02.949285  1677 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:02.981369 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:02.981803 31367 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:02.986246 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:02.988775 30625 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:02.995394 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:02.996520 30629 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:02.997500 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:02.998046 30629 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:03.000144 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.001469 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.003907 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:02.966181 31361 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:15:03.005827 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:02.979216  1828 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.736s	user 0.005s	sys 0.000s
I20260709 13:15:02.967370  1807 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:03.021004 30625 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:03.024233 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
W20260709 13:15:03.025718 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.036460 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1274) took 1551 ms. Trace:
I20260709 13:15:03.039691 30702 rpcz_store.cc:276] 0709 13:15:01.485480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.486323 (+   843us) service_pool.cc:224] Handling call
0709 13:15:01.486581 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:01.486823 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:01.817573 (+330750us) log.cc:844] Serialized 14 byte log entry
0709 13:15:02.793320 (+975747us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18001ef0a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:15:02.995761 (+202441us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b18001ef0a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:15:03.035614 (+ 39853us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.035945 (+   331us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:03.036012 (+    67us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:03.036068 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.036152 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.036362 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14457,"prepare.queue_time_us":464271,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":99,"raft.queue_time_us":17120,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":14072,"spinlock_wait_cycles":27097728,"thread_start_us":407933,"threads_started":3,"wal-append.queue_time_us":230296}
I20260709 13:15:02.949196  1726 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:02.973316  1824 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:03.038298 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.090682 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:03.051914 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.092005 31367 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:03.092078 30629 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:03.091069 31361 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:03.093920 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.096246 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:03.097711 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:03.098384 30706 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.099956 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:03.099848 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.102051 30631 pending_rounds.cc:85] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 5
I20260709 13:15:03.102458 30631 pending_rounds.cc:107] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:15:03.102500 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.102792 30631 raft_consensus.cc:2889] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:03.100176 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.104122 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.104159 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:02.973502  1675 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.105700 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:03.011176  1687 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:03.113195 30705 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.115221 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.091557  1847 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.124395 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.126448 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.119688 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.129259 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.130959 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:03.131908 30701 raft_consensus.cc:606] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:03.048112  1561 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:03.036027  1677 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:03.035681  1738 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:03.134393 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.143056 31367 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:03.131006 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.145398 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:03.101363  1370 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.035971  1670 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:15:03.113267  1715 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:03.127672  1462 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:03.133590  1780 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:03.148867  1738 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.130569  1825 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:03.130506  1155 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:03.119139  1778 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:15:03.163200 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:03.056123  1807 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:03.103837  1824 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.172883  1798 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:03.147720  1805 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:03.183629 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1295) took 1066 ms. Trace:
I20260709 13:15:03.168785 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:03.145718  1570 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:03.133867  1831 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:03.142436  1829 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:03.192198 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:15:03.184836  1849 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.283s	user 0.000s	sys 0.002s
I20260709 13:15:03.216797 31391 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:03.179829  1715 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.190250  1844 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:03.247952  1844 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:03.169924  1693 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:03.248631  1844 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.249562 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:03.251171 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:03.184091  1826 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.801s	user 0.016s	sys 0.000s
I20260709 13:15:03.253178  1826 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:03.254839 30630 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:03.141971  1847 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.187022  1738 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.187208  1825 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:03.186632  1679 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:03.163352  1370 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:03.275971 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:03.223837 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.278075 31367 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:03.201289 30558 rpcz_store.cc:276] 0709 13:15:02.117149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.117709 (+   560us) service_pool.cc:224] Handling call
0709 13:15:02.118046 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.118293 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.118319 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.126422 (+  8103us) log.cc:844] Serialized 26 byte log entry
0709 13:15:02.302055 (+175633us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:02.302102 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.302186 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:02.954403 (+652217us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:03.162917 (+208514us) raft_consensus.cc:1685] finished
0709 13:15:03.162943 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.183535 (+ 20592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25463040,"thread_start_us":141955,"threads_started":1,"wal-append.queue_time_us":116432}
I20260709 13:15:03.224989  1715 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:03.228125  1855 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:15:03.229264  1401 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:03.230007  1850 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:15:03.228269  1790 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.236261  1807 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:03.199043  1155 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:03.238067  1824 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:03.199985  1838 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:03.185065  1851 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:15:03.239691  1675 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:03.188268  1462 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:03.238095  1606 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:03.251276 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:03.263769 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.264811 30559 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:03.202255 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57004.6us
I20260709 13:15:03.269476 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:03.234429  1779 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.000s
I20260709 13:15:03.280578 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:03.282040 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:03.205530  1570 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:03.211055  1829 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.429948 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:03.213035  1798 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:03.256398  1826 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:03.283303  1854 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:15:03.263372  1825 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:03.284124  1692 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:03.278110 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:03.377732 31361 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:03.418435 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:03.440973 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:03.369810  1853 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:15:03.421842 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:03.396948  1785 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.006s
I20260709 13:15:03.456362 30559 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:03.424850 30630 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:03.457077 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:03.457710 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:03.457967 30555 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
W20260709 13:15:03.458323 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.458402 30630 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:15:03.459556 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:03.418720 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 136819us
I20260709 13:15:03.460624 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.461665 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:03.413851 30556 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:03.391068 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:03.464146 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:03.464861 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.000s	sys 0.016s
W20260709 13:15:03.466095 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.000s	sys 0.016s
I20260709 13:15:03.465852 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:03.466467 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 579704us
I20260709 13:15:03.454898 30706 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
W20260709 13:15:03.417611 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 317512us
W20260709 13:15:03.417452 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 483808us
W20260709 13:15:03.417999  1862 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:15:03.435796 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.218394  1677 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:03.435923  1829 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:03.437403  1851 consensus_queue.cc:1243] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:03.421957 30558 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:03.443351 30554 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:03.443831 31364 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.012s	sys 0.002s
I20260709 13:15:03.442517 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:03.444651 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 166655us
I20260709 13:15:03.257457  1561 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:03.381155  1786 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.009s
I20260709 13:15:03.470652 31378 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:03.430336  1570 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:03.499049 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.435040  1673 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:03.471417 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:03.473966 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.474648 30701 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:15:03.474481 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:03.381407  1726 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:03.423482  1790 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:03.415870  1844 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:03.370662  1679 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:03.441004  1777 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.001s
I20260709 13:15:03.429359  1779 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.444121  1826 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:03.485816 31364 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.012s	sys 0.002s
I20260709 13:15:03.709412 30706 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:03.486949 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:03.585107 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.568825 30703 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:03.455674  1692 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:03.715938 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 229111us
I20260709 13:15:03.716876 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:03.483543  1865 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:03.458425  1811 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:03.456034  1831 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:03.581907 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:03.575750  1836 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:15:03.714000 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.471460  1785 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:03.590860  1786 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:03.587144  1805 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:03.737104  1861 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.536s	user 0.002s	sys 0.000s
I20260709 13:15:03.483913  1860 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:03.486333  1687 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.229s
I20260709 13:15:03.720325  1489 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:03.706194  1792 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.642s	user 0.017s	sys 0.009s
I20260709 13:15:03.708590  1679 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:03.712687  1673 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:03.766724  1860 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:03.755031  1831 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:03.793749  1882 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:03.806928  1774 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.858s	user 0.007s	sys 0.008s
I20260709 13:15:03.807705  1774 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:03.806949  1882 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:03.809827  1882 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:03.812749  1803 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.213s	user 0.013s	sys 0.002s
I20260709 13:15:03.813434  1803 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:03.813839  1873 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:15:03.785333  1883 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:03.795504  1860 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:03.776404  1865 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:03.778841  1792 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:03.820626  1866 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:15:03.828547  1813 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 6 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:03.819743  1883 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:03.831327  1883 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:03.477880 30555 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
W20260709 13:15:03.561050 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71055.9us
W20260709 13:15:03.564697 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:03.495015 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:03.892230 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 397150us
I20260709 13:15:03.895004 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 635) took 1404 ms. Trace:
W20260709 13:15:03.702929 30705 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.000s
I20260709 13:15:03.714576 30625 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:15:03.898002 30705 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:15:03.897706 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 455042us
I20260709 13:15:03.917915 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:03.921998 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:03.923466 30701 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:03.924180 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 356706us
W20260709 13:15:03.924615 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 207704us
I20260709 13:15:03.721486 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:03.917402 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:03.932265 30706 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:03.925993 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:03.930310 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 208789us
I20260709 13:15:03.947244 30632 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:03.746639 31367 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:03.957515 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.959681 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.960686 31367 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:15:03.962016 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.949393 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.964766 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:03.951105 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1190) took 1065 ms. Trace:
I20260709 13:15:03.499368  1155 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:03.966344 31389 rpcz_store.cc:276] 0709 13:15:02.887674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.887938 (+   264us) service_pool.cc:224] Handling call
0709 13:15:02.888349 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.926095 (+ 37746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.926131 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.926734 (+   603us) log.cc:844] Serialized 26 byte log entry
0709 13:15:03.417853 (+491119us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:03.417904 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.417994 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:03.950383 (+532389us) raft_consensus.cc:1685] finished
0709 13:15:03.950423 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.950793 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48944,"raft.queue_time_us":37382,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":5644,"thread_start_us":490748,"threads_started":1,"wal-append.queue_time_us":541109}
I20260709 13:15:03.970013 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:03.966002 31378 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:03.973935 31378 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:03.985345 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1247) took 1018 ms. Trace:
I20260709 13:15:03.988266 31378 rpcz_store.cc:276] 0709 13:15:02.967055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.972179 (+1005124us) service_pool.cc:224] Handling call
0709 13:15:03.973070 (+   891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.984790 (+ 11720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.985207 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:03.981407 31367 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:03.983199 31389 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:03.997596 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1253) took 1019 ms. Trace:
I20260709 13:15:03.998268 31389 rpcz_store.cc:276] 0709 13:15:02.982272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.975966 (+993694us) service_pool.cc:224] Handling call
0709 13:15:03.982736 (+  6770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.997230 (+ 14494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:03.997493 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:04.190505 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:04.192190 30553 raft_consensus.cc:2410] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:15:04.199184  1493 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:04.200840  1462 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.199234  1370 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:03.778121  1489 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.209131  1489 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.210291  1489 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:15:04.214001  1489 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:04.214993  1489 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.215912  1489 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.217604  1892 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.222478  1892 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.214039  1462 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:04.286010 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 342261us
I20260709 13:15:04.292977 30701 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:04.355683  1851 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:04.356390  1851 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.356755  1803 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.356802  1887 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.357625  1799 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.358069  1887 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.358430  1799 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.359000  1716 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.360128  1716 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.369426  1679 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:04.370885  1719 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.370740  1718 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:15:04.372031  1719 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.374166  1832 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.375131  1832 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.372681  1716 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.380689  1880 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:15:04.376134  1875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.005s	sys 0.000s
I20260709 13:15:04.356901  1851 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:04.250900  1462 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.348363  1489 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.487648  1864 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:03.723315  1798 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.488576  1886 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.489658  1838 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.489615  1886 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:03.814716  1805 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.435647  1832 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:04.512902 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 596364us
I20260709 13:15:04.512936  1803 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.514045  1803 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:04.513844 31388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.021s	sys 0.038s
I20260709 13:15:04.517889  1807 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:04.518555 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1246) took 1551 ms. Trace:
I20260709 13:15:04.521006 31361 rpcz_store.cc:276] 0709 13:15:02.966712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.093668 (+126956us) service_pool.cc:224] Handling call
0709 13:15:04.518004 (+1424336us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54000032d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:04.518269 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651427584}
I20260709 13:15:04.517856  1803 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.526547  1673 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:04.524130 31388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.021s	sys 0.041s
I20260709 13:15:04.528676 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1339) took 1432 ms. Trace:
I20260709 13:15:04.529423 30625 rpcz_store.cc:276] 0709 13:15:03.095776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.095943 (+   167us) service_pool.cc:224] Handling call
0709 13:15:04.528079 (+1432136us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b540007d5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:04.528377 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1420225920}
I20260709 13:15:04.531632  1779 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:04.491293  1871 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.655s	user 0.001s	sys 0.001s
I20260709 13:15:04.518492  1719 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:04.524861  1883 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.691s	user 0.004s	sys 0.007s
I20260709 13:15:04.572110 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:04.576143  1739 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 2.159s	user 0.018s	sys 0.001s
W20260709 13:15:04.582821  1401 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.198s	user 0.018s	sys 0.011s
I20260709 13:15:04.515147  1887 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.493289  1847 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.597847 30626 raft_consensus.cc:2410] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:15:04.604600  1785 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.537048  1864 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:04.587428  1739 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:04.595357  1803 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:04.612221  1799 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.610483  1883 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.706648 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1189) took 1706 ms. Trace:
W20260709 13:15:04.725735 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154265us
I20260709 13:15:04.727326 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:04.728761  1882 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.918s	user 0.012s	sys 0.011s
I20260709 13:15:04.729563  1882 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.731043  1893 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.731914  1893 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.729991  1692 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:04.738368  1831 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:04.745502  1896 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 13:15:04.752285  1570 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.162s	user 0.016s	sys 0.003s
I20260709 13:15:04.753192 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1191) took 1853 ms. Trace:
I20260709 13:15:04.753024  1570 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.754884  1462 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:04.755602  1829 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.193s	user 0.002s	sys 0.023s
I20260709 13:15:04.756538  1893 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:04.759014  1792 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:04.761405  1883 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.753743 31388 rpcz_store.cc:276] 0709 13:15:02.899560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.899713 (+   153us) service_pool.cc:224] Handling call
0709 13:15:02.901640 (+  1927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.973859 (+ 72219us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:03.240510 (+266651us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18000c9640. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:03.715224 (+474714us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.531365 (+816141us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:04.597476 (+ 66111us) log.cc:844] Serialized 26 byte log entry
0709 13:15:04.597760 (+   284us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:04.597832 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.597974 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:04.752738 (+154764us) raft_consensus.cc:1685] finished
0709 13:15:04.752865 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:04.753104 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71715,"prepare.queue_time_us":271059,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":78771,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":2725,"spinlock_wait_cycles":25363328,"thread_start_us":1287716,"threads_started":2,"wal-append.queue_time_us":855617}
W20260709 13:15:04.759430  1871 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:15:04.724156  1847 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:04.742672  1739 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.743036  1887 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:04.783165  1893 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.790143  1570 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:04.790131  1893 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.789312  1692 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:04.775525  1829 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.793673  1893 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:04.797250  1693 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:04.797735  1847 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.798887  1887 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.800765  1847 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.801824  1887 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.802853  1739 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:04.806365 30556 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.343s	user 0.010s	sys 0.021s
I20260709 13:15:04.806946 30556 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:04.807543  1886 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.807868  1753 raft_consensus.cc:2764] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:15:04.764282  1803 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.807066  1677 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.811911  1803 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:04.813429  1738 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.524s	user 0.019s	sys 0.002s
I20260709 13:15:04.807560 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1930) took 1547 ms. Trace:
I20260709 13:15:04.802999  1864 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.712571 31364 rpcz_store.cc:276] 0709 13:15:02.851209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.868990 (+ 17781us) service_pool.cc:224] Handling call
0709 13:15:02.880796 (+ 11806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:02.881053 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:03.709615 (+828562us) log.cc:844] Serialized 14 byte log entry
0709 13:15:04.426456 (+716841us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:04.426658 (+   202us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:04.471984 (+ 45326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.508176 (+ 36192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:04.557912 (+ 49736us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":322690,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":416302,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":284,"thread_start_us":1538381,"threads_started":3,"wal-append.queue_time_us":120571}
I20260709 13:15:04.816625 30556 rpcz_store.cc:276] 0709 13:15:03.260031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.269068 (+  9037us) service_pool.cc:224] Handling call
0709 13:15:04.807438 (+1538370us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:04.825964  1715 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.447s	user 0.022s	sys 0.001s
I20260709 13:15:04.830209  1739 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:04.833278  1825 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.368s	user 0.011s	sys 0.005s
I20260709 13:15:04.837484 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:04.851655 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 124580us
I20260709 13:15:04.853437 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:04.815423  1892 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.818521  1829 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:04.870487  1805 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.871454  1805 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.874089  1805 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.858359  1786 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:15:04.874845 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:04.833988  1825 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:04.814013  1738 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.811964  1677 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:04.877048  1739 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:04.878849  1786 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:04.884536  1693 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:04.904132  1882 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.827880  1715 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:04.905666 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52361.9us
I20260709 13:15:04.921865  1462 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:04.923013  1462 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:04.926014  1462 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:04.925642  1786 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:04.921861  1738 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:04.942214 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 104916us
I20260709 13:15:04.966136 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:04.966343 30628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 879) took 2532 ms (client timeout 2231 ms). Trace:
W20260709 13:15:04.968600  1780 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.781s	user 0.019s	sys 0.002s
W20260709 13:15:04.986683 30628 rpcz_store.cc:269] 0709 13:15:02.434183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.434367 (+   184us) service_pool.cc:224] Handling call
0709 13:15:04.965780 (+2531413us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b54006effd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:04.966053 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888646016}
W20260709 13:15:04.973783  1790 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.261s	user 0.015s	sys 0.003s
I20260709 13:15:04.978086 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:04.940371  1875 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:04.976150  1825 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:04.993769 31396 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:04.989859  1786 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:05.006104 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1240) took 2038 ms. Trace:
I20260709 13:15:04.996482  1370 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.008922 30550 raft_consensus.cc:2410] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:05.014284 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 638) took 1767 ms. Trace:
I20260709 13:15:05.008889 31383 rpcz_store.cc:276] 0709 13:15:02.965057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.965194 (+   137us) service_pool.cc:224] Handling call
0709 13:15:02.984382 (+ 19188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.215336 (+230954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:03.215391 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:03.215462 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:03.215516 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:04.986998 (+1771482us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540002e754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:04.989616 (+  2618us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.003053 (+ 13437us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530405632}
I20260709 13:15:05.015933 30550 rpcz_store.cc:276] 0709 13:15:03.247115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.250731 (+  3616us) service_pool.cc:224] Handling call
0709 13:15:04.977751 (+1727020us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8f8c 00007f1803db8ef6 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:05.014182 (+ 36431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15723008}
I20260709 13:15:05.011449  1693 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:05.020726  1677 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:05.021713  1693 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:05.022305  1693 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:05.022882  1905 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.253s	user 0.004s	sys 0.000s
I20260709 13:15:05.011920  1780 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:05.031117 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:05.025674  1715 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:05.035135  1860 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.187s	user 0.006s	sys 0.018s
I20260709 13:15:05.035629 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.035898  1860 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.036569 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:03.895877 30552 rpcz_store.cc:276] 0709 13:15:02.490317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.563706 (+ 73389us) service_pool.cc:224] Handling call
0709 13:15:02.590178 (+ 26472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:02.606844 (+ 16666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.606880 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:02.619793 (+ 12913us) log.cc:844] Serialized 26 byte log entry
0709 13:15:02.898733 (+278940us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:02.898787 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:02.898868 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:03.894116 (+995248us) raft_consensus.cc:1685] finished
0709 13:15:03.894170 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:03.894659 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16117,"raft.queue_time_us":21513,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":14150,"thread_start_us":226109,"threads_started":1,"wal-append.queue_time_us":452659,"wal-append.run_cpu_time_us":14861,"wal-append.run_wall_time_us":1641537}
I20260709 13:15:05.018821  1875 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.043784  1882 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.047070  1882 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:05.039742  1780 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:05.039526  1825 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:15:05.051559  1851 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.004s
I20260709 13:15:05.052345  1851 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.015897  1370 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.053388 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:05.056510 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.013s	sys 0.011s
W20260709 13:15:05.057893 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.014s	sys 0.012s
W20260709 13:15:05.058209 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 1.11623e+06us
I20260709 13:15:05.042734 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:05.050325  1882 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.038493  1860 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.062408 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.066156  1493 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:05.068554 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.068910 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.069973 30556 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:05.074168 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.076272 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.079010 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.080565  1790 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:05.077289 30553 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 6.
I20260709 13:15:03.672906 30554 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:05.087595  1864 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:05.088528 30632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1330) took 2148 ms (client timeout 2387 ms). Trace:
I20260709 13:15:05.090170 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1245) took 2123 ms. Trace:
I20260709 13:15:05.090992 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:05.091048 30632 rpcz_store.cc:269] 0709 13:15:02.940586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.940726 (+   140us) service_pool.cc:224] Handling call
0709 13:15:05.088215 (+2147489us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.088428 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426381568}
I20260709 13:15:05.091112 31391 rpcz_store.cc:276] 0709 13:15:02.966400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.122594 (+156194us) service_pool.cc:224] Handling call
0709 13:15:03.151345 (+ 28751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:03.220182 (+ 68837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.089568 (+1869386us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b5400145ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:05.089965 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149998464}
I20260709 13:15:05.099109 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:05.092576  1864 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:05.101817  1864 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:05.102768  1829 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:05.089260  1693 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.105170  1885 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:05.104578 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.110003  1885 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:05.111335  1876 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:05.111896  1876 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:05.112169  1885 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.112895  1693 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:05.113842 31378 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:05.113880  1885 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:05.112936 30703 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 6: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:05.119743 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.121417 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.123042  1294 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:05.123427 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:05.112228 31396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1183) took 2264 ms (client timeout 2369 ms). Trace:
I20260709 13:15:05.124864 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:05.124709 31396 rpcz_store.cc:269] 0709 13:15:02.847825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.847945 (+   120us) service_pool.cc:224] Handling call
0709 13:15:05.111970 (+2264025us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54002f8d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.112142 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4210680192}
I20260709 13:15:05.125849 30553 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:05.127100 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:05.122380 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1188) took 2271 ms. Trace:
I20260709 13:15:05.127760 30553 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:05.129616 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.127986 31394 rpcz_store.cc:276] 0709 13:15:02.850902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.851420 (+   518us) service_pool.cc:224] Handling call
0709 13:15:02.851772 (+   352us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:02.852255 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:02.960695 (+108440us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.446174 (+485479us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:03.471697 (+ 25523us) log.cc:844] Serialized 78 byte log entry
0709 13:15:03.471888 (+   191us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:03.472160 (+   272us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.823470 (+351310us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b1800224c80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:05.121431 (+1297961us) spinlock_profiling.cc:243] Waited 4.28 ms on lock 0x7b1800224c80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287
0709 13:15:05.121686 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.121855 (+   169us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.121921 (+    66us) raft_consensus.cc:1685] finished
0709 13:15:05.121934 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.122156 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26952,"prepare.queue_time_us":314968,"prepare.run_cpu_time_us":8205,"prepare.run_wall_time_us":85677,"raft.queue_time_us":500947,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":30290,"spinlock_wait_cycles":18235648,"thread_start_us":778316,"threads_started":3,"wal-append.queue_time_us":727973}
I20260709 13:15:05.129998  1693 raft_consensus.cc:2764] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:15:05.130896  1693 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:05.131786  1790 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:05.131865 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.132277  1790 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:05.132789 30703 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:05.133810 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.133415  1790 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.135214  1790 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:05.135367 31396 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:05.141054 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.142678 30632 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:15:05.124615  1875 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.157137 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:05.158141 30550 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:05.159622 30628 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 11. Ops: []
I20260709 13:15:05.164007 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:05.165736 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.165961 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:05.167695 30707 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:05.170435  1693 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:05.174795 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:05.175065  1693 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:05.175529 31391 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.176394  1693 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.179385 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:05.180754 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:05.181080 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:05.181507 31364 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:05.182005 31389 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:05.183274  1718 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:05.183704  1718 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:05.184775  1718 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:05.187220 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:05.190678  1825 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:05.190726  1885 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.191612  1885 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.198051 31364 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:05.171293  1875 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:05.200454 30707 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:05.205461  1726 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.005s	sys 0.018s
I20260709 13:15:05.205554  1790 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.207309 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.207415  1790 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.205852 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.208863 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.209635 30699 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:05.209995  1790 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:05.210779  1726 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.006s	sys 0.019s
I20260709 13:15:05.206629  1885 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:05.212575 30554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 642) took 1963 ms (client timeout 2105 ms). Trace:
I20260709 13:15:05.212824 30556 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:15:05.213053 30554 rpcz_store.cc:269] 0709 13:15:03.248581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.251020 (+  2439us) service_pool.cc:224] Handling call
0709 13:15:05.212218 (+1961198us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5400374954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.212373 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70963072}
I20260709 13:15:05.207195  1718 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:05.223024 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.226830 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:15:05.232369 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66574.3us
I20260709 13:15:05.233635 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:05.247169  1820 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:05.249437  1820 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:05.251410 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:05.252009  1820 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:05.254163  1825 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:05.253275 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:05.254745  1825 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.262254  1825 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.256678  1807 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:05.259935  1776 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.255518  1786 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:05.286451  1786 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.287544  1786 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:05.262917 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:05.264660  1807 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:05.290599  1807 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:05.266959 30523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 2.008s (SENT) [suppressed 7 similar messages]
I20260709 13:15:05.296128  1851 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:05.295964  1739 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:05.298270 30701 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:05.301928  1776 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:05.304028  1776 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:05.305452  1776 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.306412 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:05.309019  1907 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.280s	user 0.003s	sys 0.001s
W20260709 13:15:05.315668  1920 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:15:05.318953  1875 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:05.321759  1790 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.324785 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.326520 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.328948 30706 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:05.329485 30525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.436s	user 0.004s	sys 0.010s
W20260709 13:15:05.350217 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:05.350600 30525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.457s	user 0.006s	sys 0.011s
I20260709 13:15:05.330821 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.332710 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 888) took 2578 ms. Trace:
W20260709 13:15:05.351004 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.332875 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.334604 30627 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:15:05.335119 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.335928 30632 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:05.336928 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:05.337131 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.338702  1851 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.345512 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.352775 30550 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:05.345856 30704 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:05.356393 30628 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:05.356838 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.357688 30704 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:05.346637 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.359320 30624 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:05.358835  1851 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.360436 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.347447  1881 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:05.361974  1851 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.363351 30559 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:05.362977  1790 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.364789 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.365442  1790 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.366204  1875 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.367602 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.368348 30699 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:05.368826  1875 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.369450 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.351926 30627 pending_rounds.cc:85] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 3
I20260709 13:15:05.370841 30627 pending_rounds.cc:107] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 13:15:05.370798 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.371336 30627 raft_consensus.cc:2889] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:05.373430 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.371788 30701 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:05.351287 30626 rpcz_store.cc:276] 0709 13:15:02.754746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.754893 (+   147us) service_pool.cc:224] Handling call
0709 13:15:05.332324 (+2577431us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b5400201054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.332620 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4223286016}
I20260709 13:15:05.355614 30633 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:05.383630 30624 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:05.385201 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.386255 30631 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:05.387142  1726 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:05.387872  1726 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.388248 30629 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:15:05.388705 30629 pending_rounds.cc:85] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 1
I20260709 13:15:05.388985 30629 pending_rounds.cc:107] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:15:05.389247 30629 raft_consensus.cc:2889] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:05.390848  1892 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:05.391685  1892 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.392243 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.393234  1824 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:05.393703 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.394164  1892 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:05.393095 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.392565 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.396440 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.401644 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.402853 30699 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.400323 30704 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:05.367669 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.417163 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.415354 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.418994 30629 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:05.421406 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.416814 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.424590 30630 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:05.424507 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:05.393752  1824 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:05.399709  1860 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:05.355170  1864 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.326416  1825 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.447467 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.446688 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:05.454248 30626 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:05.458986 30632 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:05.440335  1489 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:05.459817 30633 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:05.467375 30631 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:05.438515  1726 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.431711  1860 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:05.475009 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.471706 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.480047 30633 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:05.480556 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:05.465561  1489 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:05.483364 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 224347us
I20260709 13:15:05.472406  1718 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:05.498413 30559 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:05.498787 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.500588 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.503538 30631 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:05.491204 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:05.504475 30554 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:05.508710 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 212452us
W20260709 13:15:05.511556  1919 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:15:05.492321  1489 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:05.516578 30633 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:05.520188  1825 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:05.524513  1851 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:05.527153  1921 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:15:05.528060  1820 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:05.528807  1820 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.529726  1807 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:05.531690  1820 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.535876 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:05.536986  1851 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:05.537403  1851 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:05.537746  1851 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.540977  1786 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.537714  1807 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.543565  1824 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.527004 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.555265 30630 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:05.626335 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.630445 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.635865 30633 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:05.643399 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.655277 30631 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:05.636853  1718 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:05.615152  1790 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:05.669725 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.610181  1825 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:05.662884  1864 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.664187  1786 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:05.666944  1918 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.174s	user 0.000s	sys 0.002s
I20260709 13:15:05.666973 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.676468 30633 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:15:05.668471 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:05.561367  1807 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 13:15:05.683959 30630 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:05.667402  1493 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:05.676326  1790 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:05.687870  1825 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:05.719074 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 243179us
I20260709 13:15:06.005105  1944 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:05.720110 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.724434 30633 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:06.006881 30704 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:06.007769 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:05.725827 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:05.739567 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.013229 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.005321  1934 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:15:06.014580 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.949672 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1299) took 3822 ms. Trace:
I20260709 13:15:05.789731 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:05.835719  1824 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.841822  1606 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:05.946353 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.036026  1934 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:06.036505  1934 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:06.019907  1825 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:05.859159 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:06.036947  1934 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:06.038003 30630 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:06.025898 30551 rpcz_store.cc:276] 0709 13:15:02.126870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.127160 (+   290us) service_pool.cc:224] Handling call
0709 13:15:02.131061 (+  3901us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:02.170329 (+ 39268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:02.170407 (+    78us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:02.198458 (+ 28051us) log.cc:844] Serialized 52 byte log entry
0709 13:15:02.492963 (+294505us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:03.265289 (+772326us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b1800375ee0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:15:03.265521 (+   232us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.286486 (+ 20965us) log.cc:844] Serialized 14 byte log entry
0709 13:15:03.286883 (+   397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.118383 (+1831500us) spinlock_profiling.cc:243] Waited 82.5 ms on lock 0x7b1800375ee0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:05.948591 (+830208us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:05.948669 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.948725 (+    56us) raft_consensus.cc:1685] finished
0709 13:15:05.948736 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.949401 (+   665us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2936,"prepare.queue_time_us":151414,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":135,"raft.queue_time_us":8883,"raft.run_cpu_time_us":650,"raft.run_wall_time_us":126762,"spinlock_wait_cycles":345214080,"thread_start_us":299447,"threads_started":2,"wal-append.queue_time_us":326248,"wal-append.run_cpu_time_us":15025,"wal-append.run_wall_time_us":2247464}
I20260709 13:15:05.861864 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:06.039798 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 177997us
I20260709 13:15:06.041916 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:06.043006  1741 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:06.043519  1741 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.043949  1741 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:05.874101 30629 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:15:06.040169 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.047749 30625 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:05.883369 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.048071 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.900707 30554 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:05.902904 30626 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:15:05.926270 30699 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.502s	user 0.015s	sys 0.000s
I20260709 13:15:06.049633 30699 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:05.936424 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:05.938835 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 217060us
I20260709 13:15:05.945791 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.946086  1774 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:05.949333 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:04.591619 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:05.953229 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1254) took 2968 ms. Trace:
I20260709 13:15:06.060272 30551 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:06.060207 31378 rpcz_store.cc:276] 0709 13:15:02.984733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.991601 (+1006868us) service_pool.cc:224] Handling call
0709 13:15:05.113396 (+1121795us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:05.114448 (+  1052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.952895 (+838447us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002f8d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.953126 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2578025856}
W20260709 13:15:05.967010 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 64161.8us
I20260709 13:15:05.946149  1933 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.081s
I20260709 13:15:05.732720  1370 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:05.976802 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.050926 30628 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 4.
I20260709 13:15:05.689905 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.808152 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:06.005116 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.029392  1715 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:05.760702 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1333) took 2804 ms. Trace:
W20260709 13:15:05.766999 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.028811  1489 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.741292 30552 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:05.787129 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:05.954800 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:05.720043 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.073665 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:05.738531  1928 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:06.034471  1606 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:05.845800  1780 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.078035 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.050981 30629 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:06.051803 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:06.059341 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113141us
I20260709 13:15:05.989734 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1631) took 2893 ms. Trace:
I20260709 13:15:06.077726  1726 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:06.081221  1606 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:06.082023 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1288) took 1017 ms. Trace:
I20260709 13:15:06.071308 30631 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:06.074203 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.074491 30627 rpcz_store.cc:276] 0709 13:15:02.956294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:02.956430 (+   136us) service_pool.cc:224] Handling call
0709 13:15:05.333782 (+2377352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:05.372332 (+ 38550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.760272 (+387940us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400201050. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:05.760571 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5466573184}
I20260709 13:15:06.075666 30553 raft_consensus.cc:2368] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10.
I20260709 13:15:06.071111 30624 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:15:06.084316 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.076216 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.077004 30626 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:06.082829  1606 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:06.085883 30626 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
W20260709 13:15:06.071508 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 143348us
I20260709 13:15:06.050408  1875 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.069483 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64466.5us
I20260709 13:15:06.079505 30702 rpcz_store.cc:276] 0709 13:15:03.096632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:03.096781 (+   149us) service_pool.cc:224] Handling call
0709 13:15:03.097149 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:03.103451 (+  6302us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:03.144896 (+ 41445us) log.cc:844] Serialized 14 byte log entry
0709 13:15:05.757358 (+2612462us) spinlock_profiling.cc:243] Waited 892 us on lock 0x7b18001378c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:05.959175 (+201817us) log.cc:844] Serialized 14 byte log entry
0709 13:15:05.959419 (+   244us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:05.960187 (+   768us) log.cc:844] Serialized 26 byte log entry
0709 13:15:05.960359 (+   172us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:05.960393 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.960513 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:05.989257 (+ 28744us) raft_consensus.cc:1685] finished
0709 13:15:05.989284 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:05.989504 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7564,"prepare.queue_time_us":41639,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":107,"raft.queue_time_us":8120,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":15488,"spinlock_wait_cycles":2051456,"thread_start_us":190684,"threads_started":1,"wal-append.queue_time_us":2819593}
I20260709 13:15:06.086967 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.091149 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.070751  1753 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:06.093446 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.092146 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.095024 30629 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:06.096295 30628 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:06.082579 30705 rpcz_store.cc:276] 0709 13:15:05.064199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.304177 (+239978us) service_pool.cc:224] Handling call
0709 13:15:05.304720 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:05.307821 (+  3101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:05.307858 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:05.308497 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 13:15:05.508945 (+200448us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:05.509146 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:05.509495 (+   349us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:06.081755 (+572260us) raft_consensus.cc:1685] finished
0709 13:15:06.081782 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:06.081929 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1806,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":198103,"threads_started":1,"wal-append.queue_time_us":333765}
I20260709 13:15:06.098074 30624 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:06.096635 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.103322  1935 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:06.104223 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.105224 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:06.105638 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.106170  1879 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:06.106468 30551 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:06.107717 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.103507 30629 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:06.091068  1780 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.109344 30551 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:06.110093 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.093325  1944 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:06.104219  1935 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:06.111603  1780 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:05.811681  1786 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.083360  1715 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:06.111058 30553 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:15:06.075524  1936 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:06.114017  1936 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:06.091082  1875 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.818243  1864 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.114467  1936 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:06.112900  1786 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.093910  1753 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.114850  1864 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:06.115299  1753 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:06.121201 30558 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:06.112303  1935 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.124215 30558 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:06.082010  1726 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.110163 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1932) took 2220 ms. Trace:
I20260709 13:15:06.132612 30551 rpcz_store.cc:276] 0709 13:15:05.748835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.108752 (+359917us) service_pool.cc:224] Handling call
0709 13:15:06.109117 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.109905 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.110102 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:06.137966 30630 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
W20260709 13:15:06.139317 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47273.8us
W20260709 13:15:06.147926 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.148764 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51165.4us
I20260709 13:15:06.150064  1948 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:06.151139 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:06.132238  1726 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:06.121301  1879 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:06.131634  1864 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:06.175819 30631 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:06.180279 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.198048  1944 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:06.198752 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94689.7us
I20260709 13:15:06.203318  1489 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:06.204495  1568 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:06.184800  1879 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:06.219467 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 127326us
I20260709 13:15:06.221712 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:06.181305  1934 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.216946  1953 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:06.247692 30556 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:15:06.251689 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.040s	sys 0.052s
W20260709 13:15:06.252310 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.040s	sys 0.052s
I20260709 13:15:06.251871 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:06.252743 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 627117us
I20260709 13:15:06.251725  1944 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:06.254825  1944 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:05.715123  1860 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.255977  1860 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:06.256737  1860 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:06.257246  1860 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.257629  1860 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:05.741067 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:06.261577 31394 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:06.264961 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1268) took 1094 ms. Trace:
I20260709 13:15:06.265250 31394 rpcz_store.cc:276] 0709 13:15:05.170733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.172293 (+  1560us) service_pool.cc:224] Handling call
0709 13:15:05.740930 (+568637us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:06.264900 (+523970us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43883904}
I20260709 13:15:06.270757  1936 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:06.251914  1953 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.274921  1953 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:06.278748  1844 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.279464  1844 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.281131  1741 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.281778  1844 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:06.282401 30555 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:06.282850  1568 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:06.283859  1568 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:06.284197  1568 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:06.285176  1568 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.286635  1568 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:05.823937  1876 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.295320  1876 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:06.296844 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.000s	sys 0.012s
W20260709 13:15:06.297346 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.000s	sys 0.012s
W20260709 13:15:06.297640 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 654570us
I20260709 13:15:05.801546  1294 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.488s
W20260709 13:15:06.299088 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60142us
I20260709 13:15:06.300221  1294 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.300491 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:06.300920  1294 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.301695  1881 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:06.302100  1881 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:06.290994 30633 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:06.302502  1881 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:06.300297 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.303901  1882 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.304453  1882 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.290791  1958 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:06.307071  1882 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:06.275379  1936 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:06.308254 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:06.309444 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:06.310225  1864 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.310096  1882 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:06.310906  1864 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.311362  1844 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:06.311908  1844 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.312899  1844 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.313429  1864 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:06.314172 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47619.4us
I20260709 13:15:06.315701 30599 leader_election.cc:400] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10.
I20260709 13:15:06.316100 30599 leader_election.cc:403] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:15:06.316620 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76365.8us
I20260709 13:15:06.317981 30630 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:06.317348 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.330065 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:06.331415  1753 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.332022  1489 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:06.332983  1489 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:06.334009  1489 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.335260 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:05.803789  1570 raft_consensus.cc:993] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:15:06.309234  1726 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.334436 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:06.342784 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:06.343962 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 192955us
I20260709 13:15:06.343470 31391 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:06.344074 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.336627  1570 raft_consensus.cc:1081] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Signalling peer 5a03a8214bba4eb1b0ff00895c565564 to start an election
I20260709 13:15:06.351828 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.336068  1489 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:06.359352 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139319us
I20260709 13:15:06.327092  1882 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.365314 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.365914  1934 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.366554 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.366942 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:06.367336 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.317339  1718 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.369076 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:06.369331 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:06.369814 31383 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:06.370204 31388 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:06.366942 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:06.372474 31364 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:06.373185 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:06.374949 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54763.2us
I20260709 13:15:06.339690  1864 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10.
I20260709 13:15:06.348667  1774 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:06.383545 30630 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:15:06.383530 31391 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:06.399435 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:06.403873 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.371191  1807 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:06.408723 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:06.412386 31389 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:06.402707  1753 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.414384 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:06.414799 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.422427 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.403219  1774 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:06.410320  1370 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:06.411296  1739 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:06.425107 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:06.421685 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:06.414091  1780 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:06.427351 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:06.429584 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 65426.5us
I20260709 13:15:06.439280 30704 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:06.440495 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:06.396384  1881 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:06.450594 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82007.4us
I20260709 13:15:06.426412  1774 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:06.426191  1370 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:06.460111 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 238537us
I20260709 13:15:06.431646  1570 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.440853  1753 raft_consensus.cc:2764] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:15:06.423702  1864 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:06.460420 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.438829  1741 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.453302  1807 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:06.506541 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.501391 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.539709 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.450963  1824 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:06.472693 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.459899  1780 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.456862  1294 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:06.459887  1606 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.501696  1825 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:06.550336 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:06.549731 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.571053 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.574162 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:06.576284  1780 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.578797  1606 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.579478 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.580931 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.581122  1606 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.581539 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:06.428968  1882 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.571938 30559 raft_consensus.cc:2376] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13. Re-sending same reply.
I20260709 13:15:06.582391 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.585868 30554 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:06.546447  1807 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:06.594460 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 177106us
I20260709 13:15:06.547254  1964 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.594276 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.462486  1370 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:06.516216  1570 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:06.600591 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 144333us
I20260709 13:15:06.567941  1753 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:06.566161  1960 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:06.612583  1865 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:15:06.612977 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:06.613850  1865 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:06.614226  1865 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.615289  1865 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.616647  1865 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:06.612645 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.619468 30704 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 6.
I20260709 13:15:06.621184 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:06.622473  1294 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:06.622886  1739 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:06.623473  1294 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:06.623894  1294 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.624891  1294 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.632846 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.610677 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:06.633430 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.633646 30705 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:06.634052 30551 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
W20260709 13:15:06.634851  1967 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:15:06.635068 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.634073 30700 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:06.635980 30701 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:06.636795  1959 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:06.637192  1294 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:06.638090  1962 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:15:06.632000 30704 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:06.623486  1739 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.644057 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:06.645098 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:05.896098  1885 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.551158  1824 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.594094  1825 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:06.662075 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.597592  1807 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:06.659379 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.679044 30699 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 13:15:06.605954  1726 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:06.690096 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.690876 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.691519 30703 raft_consensus.cc:2376] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12. Re-sending same reply.
W20260709 13:15:06.693683 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 152266us
I20260709 13:15:06.694172 31378 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:15:06.694428 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:06.694670 31378 pending_rounds.cc:85] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:06.695205 31378 pending_rounds.cc:107] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:15:06.695232 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.695681 31378 raft_consensus.cc:2889] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:06.701239 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:06.707876 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.660344  1738 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:06.713143 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.689587  1851 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.151s	user 0.017s	sys 0.001s
I20260709 13:15:06.703441 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:06.717383 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.687255  1741 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:06.726959 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.609599  1964 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.736797 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.737289 30704 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:06.738310 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:06.711345 30554 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:49258
I20260709 13:15:06.841289 30554 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:06.841728 30554 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:06.740644  1978 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:15:06.741618  1824 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:06.741259  1739 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.759938  1948 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:06.745850  1961 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 13:15:06.693289  1879 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.750658 30701 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:06.751724  1953 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.752950  1977 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.720839  1934 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:06.708689  1876 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:06.882721  1825 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.883419  1876 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.885514 30703 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:06.726061  1738 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.887048  1882 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:06.887742  1882 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:06.888631  1738 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:06.889385  1738 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:06.799041 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:06.674578  1885 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:06.893169  1885 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:06.758805  1875 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:06.737735 30700 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
W20260709 13:15:06.837556 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.837280 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 149708us
W20260709 13:15:06.838101 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.838820 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 135453us
W20260709 13:15:06.846133 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 128880us
I20260709 13:15:06.742264  1844 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:06.726347  1851 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.875547  1570 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:06.910530 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:06.878854 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 191093us
W20260709 13:15:06.882771 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.913252 30701 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:06.902240 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:06.902217 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:06.860275  1942 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:15:06.886269  1876 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:06.909888 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:06.917402 30702 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:06.890022  1738 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:06.860652  1948 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:06.937952 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.741240  1957 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.179s
W20260709 13:15:06.942694 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:06.956948 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.866858  1881 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:06.908155  1860 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.002s
I20260709 13:15:06.890354  1882 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.125783 30704 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:07.161226 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:07.234927 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 508452us
W20260709 13:15:07.162216 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 139122us
W20260709 13:15:07.186985 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:07.207423 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:06.937597 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:06.916572  1885 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:15:07.228421 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 326294us
I20260709 13:15:06.917188  1741 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:15:07.272387 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.014s	sys 0.051s
I20260709 13:15:07.249300 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:06.906800  1844 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:07.250911 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 528070us
W20260709 13:15:07.254027 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 316548us
I20260709 13:15:06.918576  1968 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:07.256165 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 355153us
I20260709 13:15:07.125891  1876 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:07.125725  1570 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.263684  1953 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:06.925658 30728 debug-util.cc:398] Leaking SignalData structure 0x7b080017dce0 after lost signal to thread 21529
W20260709 13:15:07.299572 30728 debug-util.cc:398] Leaking SignalData structure 0x7b080035de60 after lost signal to thread 30416
W20260709 13:15:07.301471 30728 debug-util.cc:398] Leaking SignalData structure 0x7b080020fa80 after lost signal to thread 30583
W20260709 13:15:07.302776 30728 debug-util.cc:398] Leaking SignalData structure 0x7b08003df7e0 after lost signal to thread 30657
W20260709 13:15:07.272909 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.014s	sys 0.051s
I20260709 13:15:07.290212 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:07.267683  1774 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.000s
I20260709 13:15:06.963735  1948 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:07.292233 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 904) took 1932 ms. Trace:
I20260709 13:15:07.272601 30554 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:07.295429 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:07.305369 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 661336us
W20260709 13:15:07.305438 30728 debug-util.cc:398] Leaking SignalData structure 0x7b080011cd20 after lost signal to thread 30731
I20260709 13:15:07.308293 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:07.309676 30630 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:07.310153 30728 debug-util.cc:398] Leaking SignalData structure 0x7b08003b1d60 after lost signal to thread 30732
I20260709 13:15:07.306787 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.231333  1875 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:07.312592 30554 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.304697 30633 rpcz_store.cc:276] 0709 13:15:05.360008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.010048 (+650040us) service_pool.cc:224] Handling call
0709 13:15:06.290775 (+280727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.303009 (+ 12234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.291916 (+988907us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400569ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:07.292125 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644710784}
I20260709 13:15:07.315838 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:07.318151 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69032us
I20260709 13:15:06.907083  1977 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.321679 30556 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:07.325400 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.326362 30630 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:07.279222  1844 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:07.329892 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.332378 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.332959 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:07.333392 30629 raft_consensus.cc:2410] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:15:07.333853 30626 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:07.329180 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:07.334889 30704 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:07.337338 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:07.343757 30631 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:07.335496 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.289804  1882 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:07.239388  1860 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:07.340370  1489 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.978s	user 0.006s	sys 0.007s
I20260709 13:15:07.299403  1968 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:07.315717  1774 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:07.369518 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79406.2us
I20260709 13:15:07.387194 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:07.388674 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:07.370934 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75628.7us
I20260709 13:15:07.292153  1570 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:07.326797  1825 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:15:07.380548 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57648.8us
I20260709 13:15:07.353396 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.391394 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:07.392879 30698 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:15:07.344244  1953 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:07.393932 30633 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:07.374955  1974 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:07.324116  1741 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:15:07.399034 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:07.399062 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:07.403852 30632 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.002s	sys 0.014s
W20260709 13:15:07.404418 30632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.002s	sys 0.014s
W20260709 13:15:07.404280 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51008.9us
I20260709 13:15:07.377199  1824 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:07.409050 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:07.409930 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:07.411087  1980 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:15:07.403931  1957 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:07.412081  1774 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:07.413359  1879 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:07.415390 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:07.419483  1865 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.802s	user 0.014s	sys 0.001s
I20260709 13:15:07.418480 30704 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [6.6-8.7]
W20260709 13:15:07.418757 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88981.5us
I20260709 13:15:07.453672 30626 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:07.396237  1825 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.388955  1882 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:07.377954  1875 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 13:15:07.425698  1953 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:05.770047  1790 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:07.478410 30728 debug-util.cc:398] Leaking SignalData structure 0x7b0800410500 after lost signal to thread 31426
W20260709 13:15:07.479171 30728 debug-util.cc:398] Leaking SignalData structure 0x7b08000fde80 after lost signal to thread 31427
W20260709 13:15:07.417614  1294 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.768s	user 0.013s	sys 0.000s
I20260709 13:15:07.401242  1983 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:07.416909 30701 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 13:15:07.432662  1975 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:15:07.501106  1882 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:07.499637  1987 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:07.502175  1987 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.502666  1987 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:07.499711  1738 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:07.434662  1968 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.434943  1568 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:07.435441  1964 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:07.438092  1974 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:07.439733  1864 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:07.443846  1860 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.444255  1960 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.445844  1774 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:15:07.448196  1370 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.822s	user 0.012s	sys 0.000s
I20260709 13:15:07.450471 30550 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
W20260709 13:15:07.451457 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 655) took 2331 ms (client timeout 1999 ms). Trace:
I20260709 13:15:07.404326  1981 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:07.439734 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:07.455178 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:07.562887 30554 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:07.457223 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:07.421790 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:07.564682 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 143004us
I20260709 13:15:07.564208 30558 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:15:07.422864  1984 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:07.422904  1780 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.466813 30556 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:07.423789  1957 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:07.468781 30704 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:15:07.469161 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.463301  1825 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:07.471850 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:07.584848 30703 raft_consensus.cc:2368] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
W20260709 13:15:07.564885  1978 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.008s	sys 0.022s
I20260709 13:15:07.588232 30703 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:07.472877 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 897) took 2115 ms. Trace:
I20260709 13:15:07.590358 30625 rpcz_store.cc:276] 0709 13:15:05.357407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.714042 (+356635us) service_pool.cc:224] Handling call
0709 13:15:06.047519 (+333477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.048227 (+   708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.472632 (+1424405us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400410fd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:07.472800 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765917440}
I20260709 13:15:07.473255  1977 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:07.491679 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:07.491899 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.493547 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1638) took 1170 ms. Trace:
I20260709 13:15:07.494199  1570 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:07.495208  1979 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.170s	user 0.004s	sys 0.000s
I20260709 13:15:07.593017  1860 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:15:07.491914  1790 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:07.593537  1860 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:07.483086  1753 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:07.593981  1860 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:07.594547  1968 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:07.594935 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 672) took 1017 ms. Trace:
I20260709 13:15:07.595111  1968 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:07.595486 30559 rpcz_store.cc:276] 0709 13:15:06.577453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.594822 (+ 17369us) service_pool.cc:224] Handling call
0709 13:15:06.595223 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:06.638936 (+ 43713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:06.638973 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:06.639558 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:15:06.841048 (+201490us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:06.841095 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:06.841165 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:07.438593 (+597428us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:07.594561 (+155968us) raft_consensus.cc:1685] finished
0709 13:15:07.594585 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.594863 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10660,"raft.queue_time_us":45511,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":245126016,"thread_start_us":201043,"threads_started":1,"wal-append.queue_time_us":282636}
I20260709 13:15:07.506455 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:07.597854 30551 raft_consensus.cc:2368] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:15:07.506247 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1284) took 1140 ms. Trace:
I20260709 13:15:07.598664 31389 rpcz_store.cc:276] 0709 13:15:06.365808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.371781 (+  5973us) service_pool.cc:224] Handling call
0709 13:15:06.406161 (+ 34380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.412967 (+  6806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.478916 (+1065949us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b5400013954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:07.506161 (+ 27245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4440448}
W20260709 13:15:07.510937 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.523016 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1641) took 1164 ms. Trace:
I20260709 13:15:07.600338 30699 rpcz_store.cc:276] 0709 13:15:06.351125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.351469 (+   344us) service_pool.cc:224] Handling call
0709 13:15:07.477398 (+1125929us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400341f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:07.515613 (+ 38215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708281600}
I20260709 13:15:07.602056 30699 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:15:07.503014  1986 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.138s	user 0.003s	sys 0.000s
I20260709 13:15:07.592509 30702 rpcz_store.cc:276] 0709 13:15:06.323092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.323238 (+   146us) service_pool.cc:224] Handling call
0709 13:15:06.331775 (+  8537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.932449 (+600674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.493257 (+560808us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54005f05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:07.493452 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336197376}
I20260709 13:15:07.530783 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.536980  1960 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.604029 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1279) took 1269 ms. Trace:
I20260709 13:15:07.604387 31361 rpcz_store.cc:276] 0709 13:15:06.334334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.515478 (+1181144us) service_pool.cc:224] Handling call
0709 13:15:07.603937 (+ 88459us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:07.537292  1989 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 13:15:07.566329 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:07.524502  1568 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:07.606639 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 669) took 1030 ms. Trace:
W20260709 13:15:07.556241 30555 rpcz_store.cc:269] 0709 13:15:05.120067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:05.944646 (+824579us) service_pool.cc:224] Handling call
0709 13:15:07.451246 (+1506600us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54000208d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:07.451373 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530490624}
W20260709 13:15:07.580933 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.607753  1568 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.610889  1294 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:07.576471 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 136896us
W20260709 13:15:07.592777 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119926us
I20260709 13:15:07.612522 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.613379 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.606660 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.616460  1953 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:07.617646  1294 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:07.619324  1753 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:07.619922  1753 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:07.620430  1753 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:07.620620 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.621665 30631 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:07.592511 30700 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:07.624377 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.624914 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:07.607307 30552 rpcz_store.cc:276] 0709 13:15:06.576446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.581703 (+  5257us) service_pool.cc:224] Handling call
0709 13:15:06.582036 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:06.711619 (+129583us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:06.711655 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:06.758877 (+ 47222us) log.cc:844] Serialized 26 byte log entry
0709 13:15:06.910473 (+151596us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:06.910531 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.470000 (+559469us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540019b754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:07.470065 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:07.502235 (+ 32170us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:07.606319 (+104084us) raft_consensus.cc:1685] finished
0709 13:15:07.606354 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.606544 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14432,"raft.queue_time_us":22040,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":4293,"spinlock_wait_cycles":428956288,"thread_start_us":105185,"threads_started":1,"wal-append.queue_time_us":521071}
I20260709 13:15:07.618554  1294 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:07.629228 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 160189us
I20260709 13:15:07.631438 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.631404 30630 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:07.557557  1974 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:07.638770  1974 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:05.694871  1493 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:07.641196  1493 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:07.642446  1860 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:07.643817 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:07.644733 30553 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:07.645246 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 666) took 1096 ms. Trace:
I20260709 13:15:07.583498  1957 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:07.645579 30553 rpcz_store.cc:276] 0709 13:15:06.548516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.548649 (+   133us) service_pool.cc:224] Handling call
0709 13:15:07.643672 (+1095023us) spinlock_profiling.cc:243] Waited 978 us on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:07.645139 (+  1467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275456}
I20260709 13:15:07.647735  1864 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:07.648244  1864 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:07.648650  1864 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:07.649914 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1653) took 1017 ms. Trace:
I20260709 13:15:07.650326 30706 rpcz_store.cc:276] 0709 13:15:06.632672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.634685 (+  2013us) service_pool.cc:224] Handling call
0709 13:15:06.635024 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.679699 (+ 44675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:06.687219 (+  7520us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:06.687300 (+    81us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:15:06.687356 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.649654 (+962298us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b5400630454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:07.649713 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.649850 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3584,"raft.queue_time_us":95,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":8500,"spinlock_wait_cycles":55969920}
W20260709 13:15:07.585880  1978 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.009s	sys 0.022s
I20260709 13:15:07.653568  1973 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:07.654557  1824 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:07.655733  1991 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.274s	user 0.003s	sys 0.000s
I20260709 13:15:07.657531  1726 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:07.657407  1948 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:07.657999  1726 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:07.569442  1738 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:07.585078  1780 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.644747  1860 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.660117  1860 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:07.660926  1780 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.661196  1860 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:15:07.569382  1774 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:07.662551  1851 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:07.583750  1875 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:07.664119 30631 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:07.673558 31391 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: [14.7-14.7]
W20260709 13:15:07.676251 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 224537us
W20260709 13:15:07.676033 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63643.2us
I20260709 13:15:07.677416 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.677628 31364 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:07.676100 30703 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:15:07.693812 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62471.3us
I20260709 13:15:07.694854 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.557140  1370 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:07.702394 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:07.702688 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.702438 30625 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:07.703357 31396 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:07.700544 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:07.702332 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.702005 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:07.706334 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:07.706284 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 775654us
I20260709 13:15:07.704285 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1355) took 1387 ms. Trace:
I20260709 13:15:07.702306 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:07.706074 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.713588 31389 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13.
I20260709 13:15:07.701231 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:07.714938 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:07.721541 31378 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:07.663439  1774 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.724486 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:07.726675 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.723955 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.730084 31391 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:07.708424 30628 rpcz_store.cc:276] 0709 13:15:06.316903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.317964 (+  1061us) service_pool.cc:224] Handling call
0709 13:15:06.339882 (+ 21918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:06.340172 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:06.450402 (+110230us) log.cc:844] Serialized 14 byte log entry
0709 13:15:06.666250 (+215848us) log.cc:844] Serialized 14 byte log entry
0709 13:15:07.480714 (+814464us) spinlock_profiling.cc:243] Waited 9.4 ms on lock 0x7b1800350440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:07.703926 (+223212us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:07.703978 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:07.704043 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.704123 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.704227 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10372,"prepare.queue_time_us":238694,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":86,"raft.queue_time_us":424442,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":100372,"spinlock_wait_cycles":21629824,"thread_start_us":475168,"threads_started":4,"wal-append.queue_time_us":177190}
I20260709 13:15:07.699766  1964 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:07.735581 31361 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:07.732409 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:07.737473 30624 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
W20260709 13:15:07.738049 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:07.740479 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124617us
I20260709 13:15:07.764194  1984 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:07.764875  1984 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.765362  1984 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:07.779372  1949 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.780381  1949 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.783524  1949 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:07.786540  1984 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:07.715790  1978 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:07.790756 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:07.713600  1790 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:07.715595  1983 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:07.714664  1851 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:15:07.795640 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.795698 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:07.795857  1982 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.407s	user 0.002s	sys 0.000s
I20260709 13:15:07.728150  1934 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:07.810976 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:07.811713 31391 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:07.726200  1780 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:07.666862  1844 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.823854 31378 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:07.824985  1844 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:07.734930  1824 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:15:07.804477 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 77920.4us
I20260709 13:15:07.826148  1670 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 13:15:07.826788 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:07.790458  1964 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.827798  1964 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:07.797925  1953 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:07.828271  1953 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:07.829205  1953 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:07.829790 31394 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:07.796296  1988 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.119s
I20260709 13:15:07.799996  1983 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:07.835444  1983 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:07.836573 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:07.837380 31383 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:07.806579  1294 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:07.839486  1807 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.128s	user 0.012s	sys 0.002s
I20260709 13:15:07.805467 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.804723  1999 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:07.713331  1977 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:15:07.840626 31389 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:07.819727  1984 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:07.855423  1998 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:15:07.880232 31391 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:07.841593  1934 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:07.826197  1824 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:07.853135  1670 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.857807  1953 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:07.936302  2008 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:07.854984  1844 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:07.864903  1790 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:07.929009 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.929800 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.806773  1981 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 13:15:07.942164 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.931893  1786 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:07.854521  1807 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:07.934183  1996 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:07.925567  1934 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:07.949282 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 185200us
I20260709 13:15:07.952701 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:07.954200 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1360) took 1251 ms. Trace:
I20260709 13:15:07.941547 31383 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:07.942164 31389 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:07.948347 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:07.935377 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:07.961288 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 234214us
I20260709 13:15:07.915131  1999 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:07.950857 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 124205us
I20260709 13:15:07.975395 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.976318 31383 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:07.976405 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:07.978674 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:07.981848 31396 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:07.955083 30632 rpcz_store.cc:276] 0709 13:15:06.702985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.703147 (+   162us) service_pool.cc:224] Handling call
0709 13:15:06.723414 (+ 20267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:07.405088 (+681674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:07.405125 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:07.405708 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 13:15:07.514917 (+109209us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:07.733458 (+218541us) log.cc:844] Serialized 14 byte log entry
0709 13:15:07.953635 (+220177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:07.953773 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:07.953862 (+    89us) raft_consensus.cc:1685] finished
0709 13:15:07.953885 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:07.954098 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":145367,"prepare.queue_time_us":400077,"prepare.run_wall_time_us":42,"raft.queue_time_us":741227,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":16376,"spinlock_wait_cycles":24003328,"thread_start_us":1036295,"threads_started":4,"wal-append.queue_time_us":241262}
W20260709 13:15:07.994165 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:07.918467  2005 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 13:15:07.981673  1965 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:07.965502  1790 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:08.011238  2015 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:08.011787  2009 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:08.012269  2004 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:08.012943 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.013564  2001 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:08.039834 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74664us
I20260709 13:15:08.039678  2001 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.041167 30670 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13.
I20260709 13:15:07.975291  1996 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:08.015074  1999 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:08.016768  1987 consensus_meta.cc:287] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.000s
I20260709 13:15:07.980923  1493 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:08.023425  2000 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:15:08.025535  1953 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:07.962489  1875 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:07.976387  1988 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:07.940383  1670 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:08.063560 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 110940us
I20260709 13:15:08.035765 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:07.995604  1948 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:07.996157  1944 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:07.944409  1844 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.042129 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:07.996249  1774 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:07.995095  1738 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:07.966818  1786 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:15:08.049175 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1314) took 1396 ms. Trace:
I20260709 13:15:08.076437 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.042119  2001 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.088404  1933 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:08.088951  1933 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:08.031490  1974 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.059054  1370 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.072152  1568 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:08.092229  1974 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.092864  1568 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.093299  1568 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:08.095199  1960 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:08.057566 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.096112  1786 consensus_queue.cc:1243] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:08.029006  1965 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:08.056150  2009 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:08.097672  1965 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.098390  1984 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:08.098191  2009 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.099041  1984 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:08.099311  1965 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:08.100333  1984 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.067239 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:08.103101 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.044477  2004 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:08.069371 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:08.073444 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71083.9us
I20260709 13:15:08.080976  1670 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.081561  1726 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.082971  1996 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.067193  1988 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:08.137979 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68682.4us
I20260709 13:15:08.044926  2015 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.102046  1984 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:08.084657  2013 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:08.085037  1875 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:08.085348  1790 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.085891  1964 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.185168  1964 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.053726  2019 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.186695  2019 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.189299  2019 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.087194  1774 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:08.095571  1807 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.076740 30705 rpcz_store.cc:276] 0709 13:15:06.660045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.494740 (+834695us) service_pool.cc:224] Handling call
0709 13:15:07.495099 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:07.495349 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:07.695326 (+199977us) log.cc:844] Serialized 14 byte log entry
0709 13:15:08.048691 (+353365us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:08.048772 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:08.048829 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.048925 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:08.049093 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22246,"prepare.queue_time_us":303073,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":3505,"raft.queue_time_us":23444,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":16308,"thread_start_us":530050,"threads_started":3,"wal-append.queue_time_us":299645}
I20260709 13:15:08.048621  1958 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:15:08.084193  2014 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.140230 31364 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:08.203445 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.070117  2016 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:08.169642 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.171304 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.170318 31383 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:08.169863 30523 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:08.138998 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:08.175529 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.067104  1753 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.174911 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.059588  1987 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.134640  1953 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:08.217075 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.178946  1726 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:08.179697  1994 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:08.089995  1933 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.055339  1824 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.138703  1370 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:08.141382  1988 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:08.141053  1882 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:08.200212 30673 leader_election.cc:400] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
I20260709 13:15:08.235715 30673 leader_election.cc:403] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:15:08.201136 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1334) took 1559 ms. Trace:
I20260709 13:15:08.204505 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:08.224987 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:08.178025  1974 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:08.221702 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.200563  1964 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.215687  2016 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.202545  2019 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:08.225339 30551 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:08.232020 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.233501 30553 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:08.207600 30523 leader_election.cc:403] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:15:08.249279 31367 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:08.250874 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.261405 31383 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 7: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:08.263939 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 94239.6us
I20260709 13:15:08.264761 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:08.235037  1864 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.577s	user 0.013s	sys 0.001s
I20260709 13:15:08.207343  1949 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:15:08.220026  1994 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:08.270371 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.263103  1933 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:08.275269  2029 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.275980  2029 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.240314  1960 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:08.259163  2019 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:08.279011  2020 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:15:08.279340  1953 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.279757  2019 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.281837  2019 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:08.283526 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:08.251530  2017 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:08.210933  1774 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.296332 30553 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:08.293744 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:08.236901 30557 rpcz_store.cc:276] 0709 13:15:06.641493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.648219 (+  6726us) service_pool.cc:224] Handling call
0709 13:15:07.498104 (+849885us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:15:07.498354 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:07.506236 (+  7882us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:07.716351 (+210115us) log.cc:844] Serialized 14 byte log entry
0709 13:15:08.200557 (+484206us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:08.200644 (+    87us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:08.200720 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.200841 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:08.201041 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59354,"prepare.queue_time_us":352021,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":65,"raft.queue_time_us":208920,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":39815,"spinlock_wait_cycles":23622016,"thread_start_us":612232,"threads_started":3,"wal-append.queue_time_us":216888}
I20260709 13:15:08.304930 30523 leader_election.cc:400] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:15:08.306133 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:08.306747 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.309216 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.307675 31367 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:08.268708  1974 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:08.271849  1570 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:08.316860 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76472.8us
I20260709 13:15:08.240583  1370 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.286043  2008 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:08.311839  1892 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:08.353209 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53829.3us
I20260709 13:15:08.270992  2013 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
W20260709 13:15:08.366472 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61767.6us
I20260709 13:15:08.273524  1864 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.339521 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.384840 31367 raft_consensus.cc:2368] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:08.346597 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.242583  1882 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.318763  1974 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.214370  2014 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.263266  1964 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.302930  1715 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:08.319293 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 180395us
I20260709 13:15:08.385679  1960 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:08.403964 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:08.335776  1493 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.339267  1807 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:08.369462 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:08.405933 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.218660  2004 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.384074  1958 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:08.343892  2023 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:08.311790  1983 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.388482 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.411623 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.412801 31367 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:08.386143  1844 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.417232 31396 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:08.391357  1974 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:08.421713 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.412029  1738 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:08.412358  1958 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:08.392953  2025 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:15:08.417235  1493 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.349339  1994 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:08.427951  1957 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.000s
I20260709 13:15:08.414240  1960 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.444577  1864 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.444855  1964 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.436685  2004 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:08.445538  2035 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:08.446231  2035 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.442937  2023 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:08.443022  1987 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.444358  2021 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:15:08.424633  2029 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.429387  1738 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:08.465173 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61314.6us
I20260709 13:15:08.460225 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:08.456504 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.453488 30701 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:08.504561 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 100393us
W20260709 13:15:08.457768 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:08.507464 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47353.4us
I20260709 13:15:08.510326 30522 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:08.511602 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:08.506160 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:08.512727 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 125707us
W20260709 13:15:08.447504  1825 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.004s
W20260709 13:15:08.513767 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59085.1us
W20260709 13:15:08.449294  2030 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
W20260709 13:15:08.449515  2018 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.235s	user 0.002s	sys 0.000s
I20260709 13:15:08.461707  2021 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:15:08.448762  2027 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:15:08.447966  2014 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:08.525218 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.448609  2035 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.506963  1738 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:08.454386  1715 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:08.463387  2016 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:08.516146 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:08.455559  1936 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 13:15:08.547338 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.559878 30553 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:08.554311 30556 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
W20260709 13:15:08.554385 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48374.6us
I20260709 13:15:08.550100 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:08.557363 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.557783 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.517547  2029 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:08.563365 30522 leader_election.cc:400] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:08.541146  1994 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:08.464711  1493 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:08.557132  2031 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
I20260709 13:15:08.567102 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.521803  2004 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:08.450759  1957 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.455091  2034 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:15:08.543023  2021 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:08.599622  1934 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.620s	user 0.013s	sys 0.000s
I20260709 13:15:08.599743  1294 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:08.600234  1934 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.600448  1294 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.582230  1753 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.573165 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.603274 30705 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:08.604326 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.523101  1825 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.605129 30705 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:08.605623 31394 raft_consensus.cc:2368] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
I20260709 13:15:08.562987  1949 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:08.577905 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.547197  2008 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:08.586025 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:08.587235  2042 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:15:08.452876  2039 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:15:08.554035  1715 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:08.590121  2041 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:15:08.591598  1882 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.564910  2029 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:08.599146  2021 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.544272  1774 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.606226 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.650477 30522 leader_election.cc:400] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11.
W20260709 13:15:08.578631  1969 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.728s	user 0.000s	sys 0.008s
I20260709 13:15:08.614212 30707 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:15:08.598305  1568 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.004s
I20260709 13:15:08.563511  2024 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:08.549049  1958 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.571497  1370 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.634262  1493 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:08.635720  1944 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.003s
I20260709 13:15:08.635299  2008 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.635917  1825 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:08.644666 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.669513 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:08.670009 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.641816 30552 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:08.636526  1965 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.005s
I20260709 13:15:08.451994  2023 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.626953  1934 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.636162  1957 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.641705  1715 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:08.652599 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1320) took 1951 ms (client timeout 1937 ms). Trace:
I20260709 13:15:08.652179 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:08.648479  2029 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:08.647675  1294 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.663105 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:08.673898 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.792649 31388 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:08.792908 30700 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:08.667541  1958 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:08.656493  1568 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.677279  1807 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:08.656109  2024 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:08.680433 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.668921  1753 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:08.667073  2021 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:08.833585  1753 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.836906  1753 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.769555 30550 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:08.665241  1983 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:08.768119 30707 rpcz_store.cc:269] 0709 13:15:06.701281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.894697 (+193416us) service_pool.cc:224] Handling call
0709 13:15:08.456325 (+1561628us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:08.652521 (+196196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35298560}
I20260709 13:15:08.669019  1944 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:08.843782 30702 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:08.843951 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.845348 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:08.846217 30700 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:08.677335  1984 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.502s	user 0.013s	sys 0.000s
I20260709 13:15:08.807325 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:08.660173  2038 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
W20260709 13:15:08.855973  1988 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.002s
I20260709 13:15:08.856592  1988 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:08.863466  2048 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:08.867813 30705 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:08.868747  1568 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:08.829563 31367 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:08.828711  2024 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:08.892832  1570 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.003s
W20260709 13:15:08.948202 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.894294  1741 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:08.980305  1741 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.967478  2025 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.968530 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.983911 30559 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:08.983690  2025 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.985173 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:08.970803  2046 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.986054 30559 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:08.986129  2046 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:08.988981 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.990389  1741 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:08.990947 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:08.976819  2041 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.991824  1984 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.992460  1984 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:08.992439  2041 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:08.896759  1753 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:08.995352 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:08.996021  1984 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:08.996901 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:08.995409  1753 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:08.997561 30559 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:08.998314 31367 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:08.971894  1988 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:08.999518 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 571) took 2274 ms. Trace:
I20260709 13:15:09.002084 30699 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:08.989344  1570 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9.
W20260709 13:15:09.032269 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:08.998522 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:08.902442 30550 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:09.040359 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:09.041193 30550 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 7: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:09.042148 30700 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:09.037213 30703 rpcz_store.cc:276] 0709 13:15:06.725488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.589409 (+863921us) service_pool.cc:224] Handling call
0709 13:15:07.589864 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:07.681780 (+ 91916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:08.999140 (+1317360us) spinlock_profiling.cc:243] Waited 84.8 ms on lock 0x7b540008b454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:08.999439 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194993920}
I20260709 13:15:09.043836 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:09.044512 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.051793  1753 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.056082 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1372) took 1204 ms. Trace:
I20260709 13:15:09.058704 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:09.059638 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.056823 31367 rpcz_store.cc:276] 0709 13:15:07.835965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.997622 (+1161657us) service_pool.cc:224] Handling call
0709 13:15:08.998050 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.040515 (+ 42465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.040755 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.060565 30700 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:09.057574 31388 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:09.063874 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.068073 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:09.068951 30559 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:09.062343  1753 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:09.072278  1753 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.076757 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.089839 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:09.091207 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.099766 30630 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:09.100482 30630 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:09.103969 30698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1644) took 2671 ms (client timeout 1717 ms). Trace:
I20260709 13:15:09.132815 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:09.232766 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 1.10438e+06us
I20260709 13:15:09.103046 30707 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:09.312295 31394 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:09.365638 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1374) took 1529 ms. Trace:
I20260709 13:15:09.366173 31394 rpcz_store.cc:276] 0709 13:15:07.836472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.132964 (+1296492us) service_pool.cc:224] Handling call
0709 13:15:09.312013 (+179049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.365344 (+ 53331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.365577 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.150911 30630 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:09.314339 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:09.315006 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:09.360185 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46013.9us
I20260709 13:15:09.368615 30553 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:15:09.318840 30698 rpcz_store.cc:269] 0709 13:15:06.416455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:06.416668 (+   213us) service_pool.cc:224] Handling call
0709 13:15:09.087737 (+2671069us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54003d27d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:09.087966 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225477504}
I20260709 13:15:09.324030 30559 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:09.370883 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.371651 30559 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:09.309980  1974 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.011s
I20260709 13:15:09.364116 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.373387 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:09.375098 30632 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:09.374382 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.311196  2004 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.379356 30700 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
W20260709 13:15:09.359021  1738 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.755s	user 0.018s	sys 0.006s
I20260709 13:15:09.381088 30632 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:09.384982 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:09.313162  1978 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:09.381985 30559 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:15:09.393446 30559 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:09.386260 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:09.366814  1715 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.011s
I20260709 13:15:09.384187 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:09.389104 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:09.396126 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:09.395291 30698 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:09.395354 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.388979  1738 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.402554  1715 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.379562  1807 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:09.374001  1984 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:09.378535  1974 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:09.417817  1974 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:09.411522 30630 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:09.420563  1738 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:09.421934 31364 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 13. Ops: []
I20260709 13:15:09.396375  1776 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.399503  1753 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.423103  1776 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.423749  1807 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.425668  1776 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.426013  1807 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:09.414913  1984 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:09.427665 30700 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:09.428328  1984 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:09.429742 30700 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:09.435460 30629 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:09.436426  1715 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:09.437359 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:09.437198 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:09.437530 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:09.438146 30632 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:09.439221 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:09.439872 30625 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:09.437142 30628 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:09.389752  2004 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:09.454721 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.365622  1957 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:15:09.395799  1978 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:09.415463  2047 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:09.480453 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.446202  2004 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.481336 30624 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:09.408577  2041 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:09.483551 30631 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:09.486028 30630 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:09.549165 31389 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1288) took 1552 ms (client timeout 1584 ms). Trace:
I20260709 13:15:09.574441 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:09.574780 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:09.522192 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.488564 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 679) took 1816 ms. Trace:
W20260709 13:15:09.624825 31389 rpcz_store.cc:269] 0709 13:15:07.996683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.052085 (+ 55402us) service_pool.cc:224] Handling call
0709 13:15:09.548868 (+1496783us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b5400372dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:09.549055 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1730590592}
I20260709 13:15:09.628070 30630 rpcz_store.cc:276] 0709 13:15:07.672093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.693025 (+ 20932us) service_pool.cc:224] Handling call
0709 13:15:07.693376 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.099598 (+1406222us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b540084d620. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:15:09.488179 (+388581us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.488487 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52002560}
I20260709 13:15:09.496680 30625 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:09.656502 31389 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:09.657441 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1377) took 1820 ms. Trace:
W20260709 13:15:09.658021 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.657910 31389 rpcz_store.cc:276] 0709 13:15:07.837360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.655054 (+1817694us) service_pool.cc:224] Handling call
0709 13:15:09.655567 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.657073 (+  1506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.657359 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:09.498643  1994 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.930s	user 0.016s	sys 0.005s
W20260709 13:15:09.496531  2021 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.663s	user 0.020s	sys 0.002s
W20260709 13:15:09.565258  1958 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.012s
I20260709 13:15:09.487207  1957 consensus_queue.cc:1243] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:09.670909 30558 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 4. Current term is 5. Ops: []
I20260709 13:15:09.564728  2043 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:15:09.539374  1881 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:09.667954 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 307372us
I20260709 13:15:09.673666 30559 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:09.674111 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1381) took 1835 ms. Trace:
I20260709 13:15:09.488040  2047 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.681327 31389 rpcz_store.cc:276] 0709 13:15:07.838589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.660182 (+1821593us) service_pool.cc:224] Handling call
0709 13:15:09.673494 (+ 13312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.673701 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:09.673733 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:09.673784 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:09.673828 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.673891 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.674032 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.676862 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1379) took 1838 ms. Trace:
I20260709 13:15:09.595633  2041 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
W20260709 13:15:09.671324  2029 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.855s	user 0.019s	sys 0.006s
I20260709 13:15:09.683132 31391 rpcz_store.cc:276] 0709 13:15:07.837964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.657222 (+1819258us) service_pool.cc:224] Handling call
0709 13:15:09.676077 (+ 18855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.676364 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:09.676405 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:09.676471 (+    66us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:09.676519 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.676605 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.676771 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.539456  1715 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:09.673813  1974 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.690363 31389 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:09.691138 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1383) took 1761 ms. Trace:
I20260709 13:15:09.691454 31389 rpcz_store.cc:276] 0709 13:15:07.929853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.682754 (+1752901us) service_pool.cc:224] Handling call
0709 13:15:09.690177 (+  7423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.690861 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.691076 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:09.683619  2024 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.703s	user 0.012s	sys 0.008s
I20260709 13:15:09.696645 31391 pending_rounds.cc:85] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:09.696480 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.697167 31391 pending_rounds.cc:107] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:15:09.697557 31391 raft_consensus.cc:2889] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:09.698477  1741 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:09.699713 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:09.700296 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.699877  1741 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:09.680887  2043 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:09.706516 31389 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:09.707525 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1390) took 1171 ms. Trace:
I20260709 13:15:09.708143 31389 rpcz_store.cc:276] 0709 13:15:08.536035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.692221 (+1156186us) service_pool.cc:224] Handling call
0709 13:15:09.706312 (+ 14091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.707217 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.707459 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.711980 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1222) took 1879 ms. Trace:
I20260709 13:15:09.714454 30701 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:15:09.684367  1881 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.731889 31389 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:09.703176  1753 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.689175  2004 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:15:09.701258  2024 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.693150  1715 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.752902 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1393) took 1161 ms. Trace:
I20260709 13:15:09.753557 31389 rpcz_store.cc:276] 0709 13:15:08.591942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.709271 (+1117329us) service_pool.cc:224] Handling call
0709 13:15:09.731462 (+ 22191us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:09.740943 (+  9481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.752795 (+ 11852us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:09.692988  2029 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.696908  1974 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:09.717434  1984 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.703505  1741 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:09.751587 30704 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:09.733767 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
W20260709 13:15:09.754570 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 218578us
I20260709 13:15:09.760843 31389 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:09.741703  1957 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:09.750275  1885 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 13:15:09.755936 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 289365us
I20260709 13:15:09.747021  1753 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:09.762848  1879 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:09.763597  2041 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:09.765383  2029 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:09.765906 30550 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:09.768045  1715 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:09.835371  1786 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:09.770035  1984 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.772892  2024 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:09.773654 30555 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:09.788033 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1357) took 1267 ms. Trace:
I20260709 13:15:09.844797 30552 rpcz_store.cc:276] 0709 13:15:08.520504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.520980 (+   476us) service_pool.cc:224] Handling call
0709 13:15:08.521397 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:08.670601 (+149204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.787580 (+1116979us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540052b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:09.787916 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265105280}
I20260709 13:15:09.799191 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 585) took 1200 ms. Trace:
I20260709 13:15:09.814985  1997 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:09.801980 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:09.849694 31396 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:09.716183 31383 rpcz_store.cc:276] 0709 13:15:07.832886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.441255 (+608369us) service_pool.cc:224] Handling call
0709 13:15:08.499395 (+ 58140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:08.499674 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:08.545693 (+ 46019us) log.cc:844] Serialized 14 byte log entry
0709 13:15:09.492433 (+946740us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b1800124120. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:09.711578 (+219145us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:09.711642 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:09.711705 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.711782 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:09.711918 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44444,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":8988,"spinlock_wait_cycles":3905408,"thread_start_us":210919,"threads_started":1}
I20260709 13:15:09.831720 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:09.833657  1953 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:09.768589 30704 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:09.850347 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:09.851984 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:09.830535  1851 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.846640 30705 rpcz_store.cc:276] 0709 13:15:08.599117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.774333 (+175216us) service_pool.cc:224] Handling call
0709 13:15:09.798783 (+1024450us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b540008be54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:09.799076 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28499840}
I20260709 13:15:09.838711  1977 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:09.855387 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:09.868932  1974 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:09.870846  1489 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:09.871124  2047 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.871340  1489 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:09.746999  2043 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.871806  1489 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:09.872092  2043 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:09.873817  2041 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:09.874751  2041 consensus_queue.cc:1243] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:09.874802  2044 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:09.875380  2041 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:09.875814  2041 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:09.876672  2041 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:09.877322  2015 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:09.877976  2041 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:09.870187 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:09.879801 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1214) took 2153 ms (client timeout 2241 ms). Trace:
I20260709 13:15:09.880527  2038 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:09.881124  2038 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:09.882179  2038 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:09.880280 31378 rpcz_store.cc:269] 0709 13:15:07.726604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:07.726875 (+   271us) service_pool.cc:224] Handling call
0709 13:15:09.879580 (+2152705us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:09.879735 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69221760}
I20260709 13:15:09.836452  2005 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.889104 30704 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:09.833228  1957 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.849222  1997 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:09.865500  1981 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:09.910657 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:09.911852 30705 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:09.886781  1953 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:09.845454  1786 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.856591  1885 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:15:09.891415  1851 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:09.908141  1780 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:09.951956 31396 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:09.909327  1981 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:09.945955 31294 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 2.105s (SENT)
I20260709 13:15:09.894745  2005 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:09.887746  1977 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.919175  2046 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:09.963657 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.968076 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.969717 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:09.971961  1957 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:09.974148 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:09.975369  1824 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:09.975337  1953 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:09.978665  1953 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:09.979038  1824 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:09.979424 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.980285 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.981135 30558 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:09.980934 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:09.981750 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.982679 30550 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:09.979050 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:09.935770  1844 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:09.852993  1968 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:09.989602 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.991245 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.993692 30558 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:09.995152 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:09.995628 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:09.996230 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:09.903085  1741 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:09.962978  1981 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:09.992627 30599 connection.cc:256] RPC call timeout handler was delayed by -1.1194560869998895s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
I20260709 13:15:10.003665 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:10.005860 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.007539 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.010294 30551 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.007525 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.011348 31396 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:10.013123 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.010687 30550 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:10.024433 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.026753 30558 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
W20260709 13:15:10.028334 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.030546 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:09.960719  1780 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:10.030642 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 84747.3us
W20260709 13:15:10.041461 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65987.4us
I20260709 13:15:10.034052 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.035782 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:10.042017 31294 connection.cc:256] RPC call timeout handler was delayed by -1.2291880519999268s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19920000000000002s for a response
I20260709 13:15:10.042310 30631 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:10.042568 31294 connection.cc:256] RPC call timeout handler was delayed by -1.2275188699998125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2663s for a response
W20260709 13:15:10.043061 31294 connection.cc:256] RPC call timeout handler was delayed by -1.1159188699998595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2787s for a response
I20260709 13:15:10.037180 30632 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:10.040720 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.040819 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.052088 31389 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:10.056250 30626 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:15:10.060128 30627 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:10.063083 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.065114 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.066537 30625 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:10.074566 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.072239 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:10.076764 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.076875 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.077293 30629 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:10.077821 30626 raft_consensus.cc:2368] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:10.079149 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.071920 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.082306 30624 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:10.081746 30698 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:15:10.073125 30705 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:10.026470  1489 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.054843  1968 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.085547 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.086670 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.098410 30625 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:10.098730 30627 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:10.099974 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.026470  1885 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:15:10.100944  2046 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.101400  1885 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:10.101826  1885 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:10.101601  2046 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.102982  1885 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.104249  1885 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:10.105317  2046 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:10.106735 30631 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:10.088656 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1373) took 2252 ms. Trace:
W20260709 13:15:10.089704 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 81790.6us
I20260709 13:15:10.090845 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:10.094571 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89082.9us
I20260709 13:15:10.110191 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.110972 30629 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:10.107959 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.112893 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.113601 30628 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:10.110146 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.115530 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.071267  1958 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:15:10.113269 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:10.093217  1978 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.005s
I20260709 13:15:10.108549 31367 rpcz_store.cc:276] 0709 13:15:07.836212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.138994 (+1302782us) service_pool.cc:224] Handling call
0709 13:15:09.374522 (+235528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.381660 (+  7138us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:09.706018 (+324358us) log.cc:844] Serialized 14 byte log entry
0709 13:15:10.085175 (+379157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:10.085234 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:10.088350 (+  3116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.088443 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:10.088588 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":113559,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":93979,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":53,"thread_start_us":649109,"threads_started":3}
I20260709 13:15:10.086767  1753 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.122094 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.119223  1780 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.126117 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.126895 31396 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:15:10.127219 30629 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:10.128971 30626 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:10.134577 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1369) took 2299 ms. Trace:
I20260709 13:15:10.151258 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1292) took 2046 ms. Trace:
I20260709 13:15:10.151891 31391 rpcz_store.cc:276] 0709 13:15:08.105149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.692735 (+1587586us) service_pool.cc:224] Handling call
0709 13:15:09.696377 (+  3642us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:09.712774 (+ 16397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:09.712807 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:09.713444 (+   637us) log.cc:844] Serialized 52 byte log entry
0709 13:15:09.810143 (+ 96699us) spinlock_profiling.cc:243] Waited 2.76 ms on lock 0x7b18003db2e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f180adb158f 00007f1803df0571 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd
0709 13:15:09.925181 (+115038us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:09.925250 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:09.925373 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:10.150770 (+225397us) raft_consensus.cc:1685] finished
0709 13:15:10.150797 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:10.151170 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8751,"raft.queue_time_us":9599,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":15731,"spinlock_wait_cycles":7825536,"thread_start_us":186063,"threads_started":1,"wal-append.queue_time_us":298163}
I20260709 13:15:10.154793  1978 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.157182 30633 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:10.159996 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46852.3us
I20260709 13:15:10.162081  2043 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:10.162772 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:10.163264 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.163743 30550 raft_consensus.cc:2410] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:10.165143 30550 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:10.162966 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:10.166862 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1662) took 2046 ms (client timeout 2000 ms). Trace:
I20260709 13:15:10.094048  1984 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:10.167315 30702 rpcz_store.cc:269] 0709 13:15:08.120200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.246004 (+125804us) service_pool.cc:224] Handling call
0709 13:15:10.166571 (+1920567us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b5400343854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:10.166743 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310090112}
I20260709 13:15:10.175351 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.168684 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1375) took 2331 ms. Trace:
I20260709 13:15:10.171183 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.113215  2046 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:10.171535 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.146041  1741 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:10.187819 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.135298 31364 rpcz_store.cc:276] 0709 13:15:07.835053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.459339 (+624286us) service_pool.cc:224] Handling call
0709 13:15:08.459718 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.422564 (+962846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.134182 (+711618us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b540057d854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:10.134467 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212150144}
I20260709 13:15:10.194885 31394 rpcz_store.cc:276] 0709 13:15:07.836759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.385365 (+1548606us) service_pool.cc:224] Handling call
0709 13:15:09.385680 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.399090 (+ 13410us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:09.685820 (+286730us) log.cc:844] Serialized 14 byte log entry
0709 13:15:10.007749 (+321929us) log.cc:844] Serialized 14 byte log entry
0709 13:15:10.161562 (+153813us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:10.168344 (+  6782us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:10.168402 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:10.168450 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.168525 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:10.168626 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1213,"prepare.queue_time_us":98022,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":188368,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":21385,"spinlock_wait_cycles":23735040,"thread_start_us":685224,"threads_started":3,"wal-append.queue_time_us":344848}
I20260709 13:15:10.184758 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.219477 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:10.220650 30631 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:10.221743 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50339us
I20260709 13:15:10.227243 30629 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:10.233641 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:10.199821  2072 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.156s	user 0.000s	sys 0.005s
I20260709 13:15:10.176263  1950 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:10.176403  2068 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:10.176465  2070 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.176589  2080 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.176697  2082 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.246047 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:10.249944 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 159362us
I20260709 13:15:10.259658 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.264477 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.265296 30631 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:10.264428 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.195494  2069 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:15:10.266521 31364 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:10.182868  1570 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:15:10.232770  2015 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:10.270859 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.241891  1984 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.176898  2077 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 13:15:10.273612 31367 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:10.275552 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.244834  2068 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.299805 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:10.287814 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 126389us
I20260709 13:15:10.272459 31391 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 6. Current term is 7. Ops: []
I20260709 13:15:10.260875  1786 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.259582  1950 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.272975 31383 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:10.293455 31394 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:10.301478 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.200347  2046 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.327183 30553 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:10.268458  2034 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:10.311326 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.318246 31378 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:10.318528 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.335090 31394 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:10.319010 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.563s (SENT) [suppressed 18 similar messages]
I20260709 13:15:10.322306 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.297497  1851 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.324316  1786 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.271898  1995 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.301960  2005 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:10.347289 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.326118  2025 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.349845 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:10.351246 30558 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:10.259981  2070 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.328289 30559 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
W20260709 13:15:10.329006 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1324) took 1732 ms (client timeout 2052 ms). Trace:
W20260709 13:15:10.359043 30707 rpcz_store.cc:269] 0709 13:15:08.596388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.843483 (+247095us) service_pool.cc:224] Handling call
0709 13:15:10.328841 (+1485358us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400aa87d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:10.328946 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479226240}
I20260709 13:15:10.360737 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.361145  2081 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:10.361737  2081 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:10.361822 30707 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:10.328991 30551 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:15:10.329650 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.362774  2081 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.363268 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.364112 30631 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:10.364292  2081 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.364675 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.365253 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.365907 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.293016  2015 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.366871  1957 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.367391  2015 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.367482  1957 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.292519  1715 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:15:10.336736 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86920.3us
I20260709 13:15:10.337064 31391 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:10.370860  1715 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.372030 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.349344  1786 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.372961 31391 raft_consensus.cc:2368] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13.
I20260709 13:15:10.337524 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.344369 30629 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:10.345369 31383 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:10.376081 30629 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:10.379421 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1656) took 2263 ms. Trace:
I20260709 13:15:10.380008 30699 rpcz_store.cc:276] 0709 13:15:08.115746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.115900 (+   154us) service_pool.cc:224] Handling call
0709 13:15:09.001481 (+885581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.038152 (+ 36671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.379174 (+1341022us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b540008b450. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:10.379341 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985879296}
I20260709 13:15:10.381675  2046 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.383247  1950 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.389948 30706 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:10.390501 30706 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.303184  1984 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.391397 30706 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.392720 30706 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:10.393301  1984 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.267745  2082 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.347407  2084 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:10.401719 31367 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:10.354344 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.355154 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.356112 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.373965  1844 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:10.405260  1844 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.306273  1973 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:15:10.265205  2080 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.407922 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:10.327801 30701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1321) took 1776 ms (client timeout 1674 ms). Trace:
W20260709 13:15:10.408998 30701 rpcz_store.cc:269] 0709 13:15:08.551567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.551748 (+   181us) service_pool.cc:224] Handling call
0709 13:15:10.327587 (+1775839us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540055afd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:10.327719 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2563764608}
I20260709 13:15:10.409780  1489 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.396500 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.411569 30625 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:10.413466 30625 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:10.334223  2034 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.395697 31364 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:10.415747 31396 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:10.415506  2034 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.417059  2034 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:10.419027  1881 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.423545  2043 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:10.424166  2043 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.425727 31389 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:10.354264  1568 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.430739  1568 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.298056  1824 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:15:10.432039  1997 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:10.433233 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.433878  2041 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:15:10.434504 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 208705us
W20260709 13:15:10.434854 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:10.347867 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.371912  1715 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.454723 30556 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:10.400918 30633 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:10.396231  2077 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:10.459614 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.438967 30552 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:10.302347  2068 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:10.452898 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.461580 31396 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:10.458688 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.459082 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.398022  1977 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.462910 30556 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
W20260709 13:15:10.452857 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 97393.2us
I20260709 13:15:10.467585 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.468430 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1364) took 1601 ms. Trace:
I20260709 13:15:10.468986 30555 rpcz_store.cc:276] 0709 13:15:08.866630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.866793 (+   163us) service_pool.cc:224] Handling call
0709 13:15:08.867219 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:09.854540 (+987321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.468099 (+613559us) spinlock_profiling.cc:243] Waited 906 ms on lock 0x7b540028a0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:10.468349 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2083988864}
W20260709 13:15:10.470542 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:10.473210 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.349597  1988 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:10.433282  1997 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:10.479225 30558 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:15:10.483283 30558 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:10.483603 30558 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:15:10.483866 30558 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.484455 30704 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:10.481515 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.486135 30701 raft_consensus.cc:2410] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:10.482285 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.486951 30707 raft_consensus.cc:2376] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13. Re-sending same reply.
I20260709 13:15:10.488816 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.489737 31396 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:10.434355  2041 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:10.460312  1786 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:10.459658  2090 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.510439 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.518143 30704 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:10.519403 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.520253 31396 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:10.475198  1715 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:10.503379 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.355038  1995 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.533891 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.538352 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.545009 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.545186 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.546514 30698 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:15:10.439337  2043 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:10.553308 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50047.4us
I20260709 13:15:10.559923 30699 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:15:10.560448 30699 pending_rounds.cc:85] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 6
I20260709 13:15:10.560693 30699 pending_rounds.cc:107] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:15:10.560932 30699 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.562816 30702 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:15:10.543485 31394 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:10.480875  1881 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:15:10.559122 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.537637  2080 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.572163 30625 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:15:10.571735 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49521.3us
I20260709 13:15:10.489671  1988 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:10.576442 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.577126 30628 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:10.578943 30701 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:10.580981 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.575299 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.580699 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.525082  1958 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:15:10.583145 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:10.583838 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.567600  1780 raft_consensus.cc:2764] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:15:10.590286 30624 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.004s
I20260709 13:15:10.591058 30624 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:15:10.594388 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:10.596923 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.562734  1606 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:15:10.584777  2086 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:10.600311  2086 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:10.600723  2086 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.584954  2082 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.602886  2024 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:10.603483  2024 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:10.604183  1995 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:10.604581 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.607622 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.611043 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.527630  2090 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.611707 31367 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.612746 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:10.613780 30707 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:10.615934 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.617328 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.618108 31364 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:10.619045  2069 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:10.619760  1981 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.000s
I20260709 13:15:10.619587  2069 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:10.620234  1981 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.622040  2069 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:10.622308  1981 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.700107 30706 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:15:10.701258 30706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1322) took 2147 ms (client timeout 1592 ms). Trace:
I20260709 13:15:10.582594  2070 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:10.627442  1885 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.523s	user 0.015s	sys 0.002s
I20260709 13:15:10.585325  2091 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:10.582971  2041 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:10.645383  1968 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
W20260709 13:15:10.687947 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91063.6us
I20260709 13:15:10.587138  1786 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:10.631119  2021 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:10.630687  1568 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:10.586848  1776 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:10.634868 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.727523  1776 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.727799 31388 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:10.728076  1776 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:10.729804  2075 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:10.730291  2075 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:10.638504 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.731657 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.732283 30705 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:10.733145 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.633755  1988 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:10.733811 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.622464 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.735620 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.736889 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:10.737491 31396 raft_consensus.cc:2376] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9. Re-sending same reply.
I20260709 13:15:10.626760  1977 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:10.636560 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.739877 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.741082 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.628221 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.742808 30698 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:10.744364 30698 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 6. (term mismatch)
I20260709 13:15:10.744897 30698 pending_rounds.cc:85] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 5
I20260709 13:15:10.745127 30698 pending_rounds.cc:107] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:15:10.745352 30698 raft_consensus.cc:2889] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:10.747541  2024 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.748405 30625 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:10.629288 31378 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:10.754758 31396 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:10.755434 31364 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:15:10.756247 31378 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:10.756584 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:10.701777 30706 rpcz_store.cc:269] 0709 13:15:08.553494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.553655 (+   161us) service_pool.cc:224] Handling call
0709 13:15:10.389780 (+1836125us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b540003afa0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:15:10.701124 (+311344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211983104}
I20260709 13:15:10.757831 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:10.703215 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90635.1us
I20260709 13:15:10.704022  1995 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:10.704216 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:10.704627  1949 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:15:10.654402 30704 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:10.705431 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.707077  1953 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.707201 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.707830 30629 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:10.708022 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.804577 30701 raft_consensus.cc:2376] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7. Re-sending same reply.
I20260709 13:15:10.708514  1780 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:10.592134  2080 raft_consensus.cc:2764] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:15:10.709852 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 96710us
I20260709 13:15:10.710240 31394 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:10.711629 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:10.714943 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.720904 30554 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:10.723402 30700 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 13:15:10.749892  1741 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.009s
I20260709 13:15:10.751948 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:10.754364 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.733830 30705 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:10.702575 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:10.810110 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58339.3us
I20260709 13:15:10.811230  1995 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:10.808126  1780 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:10.812729 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 97890us
I20260709 13:15:10.807863  1953 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:10.814942  2035 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:10.815311  2097 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.205s	user 0.005s	sys 0.000s
I20260709 13:15:10.816327  1953 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:10.811230 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.818372  1973 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:10.818452 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.818926  1973 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.819592  1973 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:10.819504  2094 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:10.820158  1997 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:10.820717  1997 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:10.821663 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.822592  2094 raft_consensus.cc:2764] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:15:10.825728 30556 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:10.826689 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.826999 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:10.827307 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.759071 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:10.641593  1957 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:10.795861 30698 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:10.796537 30699 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:10.784247 31388 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:10.821123  1997 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:10.782120 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:10.800836 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:10.808087 30629 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:10.809091 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:10.840075 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.813107  1780 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:10.827658 30705 raft_consensus.cc:2410] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:10.827984 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:10.758989  1885 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.718228 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.761812 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.834422 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.843297 30559 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:10.843499 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:10.844240 30701 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:10.836045 30556 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:10.832309 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 125219us
W20260709 13:15:10.845053 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.710372  2015 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:10.840660 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58659.7us
W20260709 13:15:10.836907 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77949us
I20260709 13:15:10.846153 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:10.846297 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84655.9us
I20260709 13:15:10.846671 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.847151 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.848089 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:10.848186 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:10.837373 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.842159 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.849486 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:10.851943 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:10.850339 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:10.851212 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:10.808917  1978 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:10.853935 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 135856us
I20260709 13:15:10.853892 30629 raft_consensus.cc:2376] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13. Re-sending same reply.
I20260709 13:15:10.857057 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:10.857867 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.769047  1786 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.078s
I20260709 13:15:10.881884 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:10.824019  2008 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.853346 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:10.850431  2082 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:10.842938  1953 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:15:10.858397  1885 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.840418  1741 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:10.878289  2025 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:10.845285  2070 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:10.863585  2084 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:10.906129 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 685) took 1856 ms. Trace:
I20260709 13:15:10.906546 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.906942 30554 rpcz_store.cc:276] 0709 13:15:09.049996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:09.427040 (+377044us) service_pool.cc:224] Handling call
0709 13:15:09.427372 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:10.808883 (+1381511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:10.905789 (+ 96906us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540012c5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:10.906033 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2973856640}
I20260709 13:15:10.880425  2086 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:10.852128  2080 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:15:10.881217  2081 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.005s
I20260709 13:15:10.864904  2029 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:10.855378  1606 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:15:10.832402  2041 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.807055  1981 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:15:10.886432 31367 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:10.936046 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.882788 30523 leader_election.cc:400] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:10.938124 30523 leader_election.cc:403] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:10.851639  2094 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:10.891829 30624 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:15:10.922883  2095 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 13:15:10.859007  2077 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:10.855589  1824 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:10.855142  1950 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:10.896843 30627 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:10.895726 30701 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:10.898222  1741 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:10.913679 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.883237  2008 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:10.856123 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:10.856560  1957 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
W20260709 13:15:10.924671 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82611.7us
W20260709 13:15:10.958236 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102232us
I20260709 13:15:10.958895  1726 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:10.959102 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:10.964535  1741 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:10.862658  2024 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:15:10.934430 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.899071  1978 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:15:10.968253  1978 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:10.968665  1978 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:10.969012  1978 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:10.855753  2005 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.972525  2043 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:10.972203  1776 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:10.975841  2092 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.378s	user 0.002s	sys 0.000s
I20260709 13:15:10.978072 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:10.909399  1844 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.503s	user 0.004s	sys 0.012s
I20260709 13:15:10.916970  1953 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 13:15:10.945245 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84326.6us
I20260709 13:15:10.980309 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:10.981158  1844 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.981745  1977 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:10.983846  2086 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:10.985317 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:10.986348  1973 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.986820  1968 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:10.987576 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:10.987725 30632 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:10.944257 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97194.8us
I20260709 13:15:10.920683  2082 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.893707  2078 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:15:10.849814  1995 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:15:10.993748 30703 raft_consensus.cc:2410] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:10.995393 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.000571  1844 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.959417  1726 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.001282  1824 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.001922  1726 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.000214  1997 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.002394  1824 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.003695 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.004597 30628 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:11.004902 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.005477 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.004711  1997 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:10.938445  2081 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:10.957062  2094 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.010272  1957 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:11.010880  1957 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.010718  1489 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:11.010789  2094 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.011585  1489 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:11.012082  1957 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.013260  1988 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:15:11.013689  1988 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:11.009557 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:10.952308  1950 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:11.015024  1950 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:11.014705  1988 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.015168  2024 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:11.015708  2024 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.016096  1988 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:11.016672  2024 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.018052  2024 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:11.018707 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.012470  1489 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:10.953071  2025 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:11.019672  1489 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:10.947733  2029 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.021901  2029 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.023433 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:11.024892 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.025750  2070 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:11.026315  2070 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:11.026760  2070 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:11.026831  2021 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:11.027326  2021 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:11.027719  2021 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:11.029480  2025 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:10.937740 30633 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:10.936817  2080 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.030556  1953 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:11.031489 30556 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:15:11.008814 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:11.000339 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:11.032851 30557 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:11.009492  1741 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:11.030085  2025 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:11.033382  1741 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:11.031229  2080 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:11.034014 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.033783  2035 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:15:11.007519 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48511.2us
I20260709 13:15:11.005584  2015 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:10.953984  2077 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:11.036795  2046 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:11.037411  2077 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:11.025933 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.002748 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.032637 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:10.955739 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 103842us
W20260709 13:15:11.054632 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:11.042007 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.075433 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.013502  1957 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:15:11.076385 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.077088 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:11.037883  1780 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.080744 30632 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:11.033823  2025 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:11.096560 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:11.107797 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 250851us
W20260709 13:15:11.116148 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 120988us
I20260709 13:15:11.034049  1741 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:11.079609 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.118803 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.073174  1776 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:11.121559  2091 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:11.121153  1978 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.074026  1726 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:11.123665  2034 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.706s	user 0.015s	sys 0.000s
I20260709 13:15:11.125726  1780 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:11.110658  1885 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:11.128100 30557 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:11.102520  1844 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:11.144430  2086 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:15:11.137682 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102114us
W20260709 13:15:11.138916 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59445.6us
I20260709 13:15:11.152786 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:11.164258 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.105937  1973 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:11.115479  2046 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
W20260709 13:15:11.175784 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.210985  1776 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:11.213344  1776 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:15:11.214529 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50456.4us
W20260709 13:15:11.215392 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.178649  1844 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:15:11.217753  1977 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:11.218277  1977 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.219195  1977 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.220113  2086 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.220562  1977 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:11.221233  2086 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.222465  1885 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:15:11.222941 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49268us
I20260709 13:15:11.223228  1885 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.224540 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.214462  2102 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:11.224272  1885 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.222918 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.152395  1997 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:11.225905 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.246309 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.169664  1715 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.254194 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.144335  1953 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:11.259634 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 106964us
W20260709 13:15:11.225950  2107 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:15:11.267558 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.268988 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.265385  1844 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:15:11.272270  1844 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:11.272790  1844 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.273698  1844 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.274415  1973 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:11.258888  2081 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:15:11.277022  2081 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:11.283463 30632 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:11.283954 30633 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:11.257507  2103 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:11.286725  2103 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.287459  2110 raft_consensus.cc:2764] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:15:11.287834  2103 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.286432 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.288946  2110 raft_consensus.cc:2764] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:15:11.290580  1780 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.291021  1978 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:11.292217 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.293584 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:11.294358 30553 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:11.296612  2068 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.816s	user 0.008s	sys 0.004s
I20260709 13:15:11.296984 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:11.265914  1715 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.297314  2068 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.298444 30630 raft_consensus.cc:2368] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9.
I20260709 13:15:11.298728  2035 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:11.299309  2035 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:11.299407  1715 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:11.300081  1715 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:11.300529 30630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 912) took 1267 ms (client timeout 1592 ms). Trace:
I20260709 13:15:11.300320  2035 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:11.300962 30630 rpcz_store.cc:269] 0709 13:15:10.033211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.033778 (+   567us) service_pool.cc:224] Handling call
0709 13:15:11.300279 (+1266501us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:11.300459 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2888705408}
I20260709 13:15:11.302429 30626 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 9. Ops: []
I20260709 13:15:11.266041  2104 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:11.304774 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1380) took 1266 ms. Trace:
W20260709 13:15:11.305168 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.306281 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.305154 30626 rpcz_store.cc:276] 0709 13:15:10.038367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.129951 (+ 91584us) service_pool.cc:224] Handling call
0709 13:15:11.302242 (+1172291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.302965 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.304541 (+  1576us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54004a8dd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:11.304703 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2695250816}
I20260709 13:15:11.308339 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:11.309026 30699 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:11.260775 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:11.311299 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50615.3us
I20260709 13:15:11.311736 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:11.157554  2015 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:15:11.314254 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.318104 31367 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:11.314715 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:11.314905 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.315634 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.321108 31391 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:11.251648  1776 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.257541  2091 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:15:11.319126  1988 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.313628 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.323886 31361 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 13:15:11.323454  1776 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.323705  1988 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.327230 30632 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 13:15:11.327862 30632 pending_rounds.cc:85] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:15:11.328225 30632 pending_rounds.cc:107] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:15:11.328298  1988 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:11.326409  1776 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:11.328567 30632 raft_consensus.cc:2889] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:11.331431  1958 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:11.335245  2090 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:11.336519  1973 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:11.337472  2109 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:15:11.338719 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:11.341198 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.345222 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.345918 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:11.349472 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:11.338069  1978 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:11.367408 30698 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [6.6-9.7]
I20260709 13:15:11.358003 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:11.362643 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.369769 31394 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 7.
I20260709 13:15:11.370162 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.370710 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.371372 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.372130 31378 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:11.363709 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.365864 31388 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:11.375392 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.365185 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.365862 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.380107 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.383055 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.384701 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:11.387596 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:11.397221 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58613.6us
I20260709 13:15:11.397512 31396 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:11.397630 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:11.399718 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:11.400391 30704 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:11.401783 31394 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:15:11.402648  1949 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.403177  1949 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.403762 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.404709  1780 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:11.405507  1949 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:11.405943 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.397351 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:11.406563 31394 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:11.408073 30557 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:11.409590 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.405099 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.413134 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.414855 30699 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:11.415342 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.416064 31378 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:11.413182 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.344440  2104 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.430431 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:11.434228 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:11.428741 30704 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:11.356495  1981 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:11.435593 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.296706  2077 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.361603  1776 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:11.322109  2081 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
W20260709 13:15:11.443773 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 85876.1us
I20260709 13:15:11.338802  2102 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:15:11.410902  2091 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:11.452201 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 419625us
W20260709 13:15:11.462891 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.473699 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1665) took 1134 ms. Trace:
I20260709 13:15:11.455811 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:11.427071  2111 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.424691  2082 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.476787 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:11.430130  1995 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:11.451267 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:11.425850  2084 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:11.410735  2090 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:11.472280 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:11.474568 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.474998 30553 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:15:11.474152 30703 rpcz_store.cc:276] 0709 13:15:10.339718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.340125 (+   407us) service_pool.cc:224] Handling call
0709 13:15:11.470334 (+1130209us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b540062ff54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:11.473618 (+  3284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1355548928}
I20260709 13:15:11.435595  1780 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:11.427377  1974 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.045s
W20260709 13:15:11.500916 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.488837 31394 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:11.504694 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:11.505601 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.001s	sys 0.037s
W20260709 13:15:11.506074 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.001s	sys 0.037s
W20260709 13:15:11.506815 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 625012us
I20260709 13:15:11.508507 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.510299 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.451630  2115 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:11.471458  1776 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.479472  2005 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.516428  1824 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:11.529029 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.497267  2041 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:11.506440  2081 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:15:11.543974 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.502749  2102 consensus_queue.cc:1243] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:11.452082  1981 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:11.557319 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.515851  2082 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:11.567509 30624 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:11.524748  2035 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.533404  2084 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:11.533927  1780 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.466151  1953 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:11.547710  2043 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.001s
I20260709 13:15:11.572662  1489 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:11.488265  2111 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.543576  1824 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:11.603835 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 188565us
I20260709 13:15:11.613126 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.607780 31389 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:11.572057  2120 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:11.616605  1776 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.565306  1984 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
W20260709 13:15:11.618660  2116 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:15:11.619504  2111 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:11.610764  1489 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.588833  2046 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:11.622025 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.684523 31394 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:15:11.596868  2070 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.566s	user 0.013s	sys 0.003s
I20260709 13:15:11.569813  1973 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:15:11.567883  2041 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:11.671787  1741 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.626140  2121 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:15:11.632601  2082 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 13:15:11.639669 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65111.7us
W20260709 13:15:11.673695  2025 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
I20260709 13:15:11.682250  2091 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:15:11.622484  1981 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:11.685252 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:11.711964  1995 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:11.713244 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:11.713748  2049 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:11.714211  2049 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.714843  1978 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:11.715462  1978 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.715160  2049 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.716596  2049 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:11.716398  1978 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.684116  2046 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:11.690353  2108 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 13:15:11.713248 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:11.719347 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:11.669706  2005 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:11.679843  2094 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:11.745615 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:11.724118 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 217929us
I20260709 13:15:11.725169 31394 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:11.755962 31294 connection.cc:256] RPC call timeout handler was delayed by -1.477665714000068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:15:11.668948  2120 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:11.723443 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 218839us
I20260709 13:15:11.760023 30559 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:11.760674 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:11.691377  2114 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:15:11.747258  2021 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.006s
W20260709 13:15:11.686776  2092 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:15:11.759255 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:11.762065 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.774649 31378 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:11.774974 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:11.757375 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:11.764452 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.765157  2081 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:11.765391  2029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.015s	sys 0.005s
W20260709 13:15:11.765758  2080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.003s	sys 0.005s
I20260709 13:15:11.726536  2046 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:11.722810  2115 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.758481 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:11.789040 31396 raft_consensus.cc:2376] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16. Re-sending same reply.
W20260709 13:15:11.744871  2113 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 13:15:11.703152  2110 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:11.801957  2110 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:11.770310 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.803593 31388 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:11.690497  2070 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.704634  1824 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:11.779529 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 60302.8us
I20260709 13:15:11.780694  1949 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:11.781723 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.726768  2077 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:11.762813  2024 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.744s	user 0.013s	sys 0.002s
I20260709 13:15:11.753253  2078 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:11.705017  1950 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.000s
I20260709 13:15:11.795017  1974 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:11.790410  2090 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:15:11.797425  1977 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.576s	user 0.015s	sys 0.000s
I20260709 13:15:11.798422  2081 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.767853  2120 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:11.799943  2115 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.776106 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:11.851516 31378 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 4.
I20260709 13:15:11.807199 31388 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:11.808104  2110 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.809162  2104 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:11.778013  2021 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.812893 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:11.787442  2029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.015s	sys 0.005s
W20260709 13:15:11.787271  2080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.003s	sys 0.005s
I20260709 13:15:11.674985  2119 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.153s
I20260709 13:15:11.754798  2025 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.784431  2005 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:11.832695  1949 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:11.706722  1780 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:11.852293 30555 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:11.949982  1726 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.000s
I20260709 13:15:11.866076 30371 maintenance_manager.cc:419] P 9e2d0e882b2940d8b150869e4b2b2e79: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033366
I20260709 13:15:11.875334 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1383) took 1041 ms. Trace:
I20260709 13:15:11.883993 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:11.977984 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94140.8us
I20260709 13:15:11.885013  2136 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:15:11.886724 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:11.996080 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:11.901484 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1383) took 1320 ms. Trace:
I20260709 13:15:11.905128  2075 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:11.812774  2122 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:11.940402 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 127720us
I20260709 13:15:11.972929  1949 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:15:11.952035 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:11.953064 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 63200us
I20260709 13:15:11.853096 30628 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:15:11.974687 31383 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:11.975023 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:12.209750 30704 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:11.976265 30558 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:11.956360  2121 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:11.939196  2104 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:11.955730  1995 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:11.977653 30550 rpcz_store.cc:276] 0709 13:15:10.834275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.834412 (+   137us) service_pool.cc:224] Handling call
0709 13:15:10.853777 (+ 19365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.018908 (+165131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.018943 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.019595 (+   652us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.153920 (+134325us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:11.153987 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.847629 (+693642us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b54000a0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:11.847698 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:11.847757 (+    59us) raft_consensus.cc:1685] finished
0709 13:15:11.847773 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:11.875219 (+ 27446us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51537,"raft.queue_time_us":52980,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":259994496,"thread_start_us":133854,"threads_started":1,"wal-append.queue_time_us":238739}
I20260709 13:15:11.849337  1950 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:11.997539 30631 rpcz_store.cc:276] 0709 13:15:10.581457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.581584 (+   127us) service_pool.cc:224] Handling call
0709 13:15:10.581963 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.072187 (+490224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.072221 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.072798 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.358949 (+286151us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:11.359013 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.900813 (+541800us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:11.900884 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:11.900968 (+    84us) raft_consensus.cc:1685] finished
0709 13:15:11.900990 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:11.901375 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82617,"raft.queue_time_us":73143,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":15205,"spinlock_wait_cycles":958252928,"thread_start_us":281706,"threads_started":1,"wal-append.queue_time_us":411167}
W20260709 13:15:11.950268 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109702us
W20260709 13:15:12.158492 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 145610us
I20260709 13:15:12.055325  2021 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:12.130419 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:12.054831  1957 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.945s	user 0.014s	sys 0.001s
I20260709 13:15:11.878918 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:15:12.236488 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.239217 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:12.305604 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66505.6us
I20260709 13:15:12.306465 30523 leader_election.cc:400] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9.
I20260709 13:15:12.306788 30523 leader_election.cc:403] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:15:12.255173 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 239131us
W20260709 13:15:12.264204 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.289676 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1980) took 1281 ms. Trace:
I20260709 13:15:12.290125 31394 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:12.124356  2068 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:12.077432  2008 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:12.073810 30702 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:11.993701  2131 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:12.085233  2070 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.143594  1741 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:12.096474 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:12.175419  1885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.009s	sys 0.017s
I20260709 13:15:12.055440 30701 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:15:12.010250  2122 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:12.079993  2025 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:12.067516  2141 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:15:12.061015 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.011607  2110 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:15:12.120669  2102 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:11.979203  2034 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 13:15:12.063031  2005 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.196504  2075 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:12.312305 30558 rpcz_store.cc:276] 0709 13:15:11.008281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.008517 (+   236us) service_pool.cc:224] Handling call
0709 13:15:12.289400 (+1280883us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400289bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:12.289589 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170394880}
I20260709 13:15:12.320128 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:12.461205 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 141239us
I20260709 13:15:12.380328  2122 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:12.462546 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:12.464201 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:12.231452  2145 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
W20260709 13:15:12.361124 30670 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.018s	sys 0.033s
W20260709 13:15:12.467826 30670 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.019s	sys 0.033s
W20260709 13:15:12.468183 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 720746us
I20260709 13:15:12.393636  1741 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.374193 30553 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:12.226595  1950 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:12.256265  1786 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:12.280558  2015 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:12.472819 30556 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:12.475888 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:12.229163  2094 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:12.546108 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:12.372753 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1400) took 1028 ms. Trace:
I20260709 13:15:12.234448  2041 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:12.230796  2121 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:12.242962  1776 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:12.394718 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:12.559412 30704 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:12.355322  2021 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:12.354024  1957 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:12.541013  2148 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:15:12.369925  1885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.095s	user 0.009s	sys 0.017s
I20260709 13:15:12.355680  2131 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:12.234741  1949 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:12.357307  1958 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.449126  2137 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.368574  1606 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.457638 30706 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [6.6-9.7]
W20260709 13:15:12.561321  2146 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.462s	user 0.004s	sys 0.004s
I20260709 13:15:12.286806 30382 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 8, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 8 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:12.458531 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1398) took 1117 ms. Trace:
W20260709 13:15:12.464948 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 486086us
I20260709 13:15:12.368723 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:12.469319 31294 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.027s	sys 0.048s
I20260709 13:15:12.596880 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:12.596935 31294 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.028s	sys 0.049s
W20260709 13:15:12.598027 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 725934us
I20260709 13:15:12.596436 30633 rpcz_store.cc:276] 0709 13:15:11.340837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.343766 (+  2929us) service_pool.cc:224] Handling call
0709 13:15:11.380365 (+ 36599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.746235 (+365870us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.746284 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.767021 (+ 20737us) log.cc:844] Serialized 26 byte log entry
0709 13:15:12.150254 (+383233us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:12.150318 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.150430 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.458266 (+307836us) raft_consensus.cc:1685] finished
0709 13:15:12.458297 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.458470 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19349,"raft.queue_time_us":26995,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":8715,"thread_start_us":703152,"threads_started":2,"wal-append.queue_time_us":583345}
W20260709 13:15:12.605013 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59010.9us
I20260709 13:15:12.606293 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:12.555392  2094 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:12.470490  2025 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:15:12.468036  2038 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:12.475203  2147 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 13:15:12.487669  1741 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:12.473994  2015 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:12.645923  2015 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.470752  2070 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:12.656631  2155 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:12.474261  2091 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.473836  1786 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:12.588702 30704 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:12.583161 30670 leader_election.cc:400] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13.
I20260709 13:15:12.661005 30670 leader_election.cc:403] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:15:12.662166 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79249.6us
I20260709 13:15:12.573832  1776 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:12.586455  2138 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:15:12.574167  1957 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:12.588569  2049 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.871s	user 0.014s	sys 0.004s
I20260709 13:15:12.627005 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:12.590222  2131 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:12.563231  2154 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:15:12.587210  2128 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.465302  2008 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:12.558697  1968 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:15:12.589891  2103 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:12.609146  1606 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:12.657919 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 709) took 1309 ms. Trace:
I20260709 13:15:12.460764  2102 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:12.663309 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:12.548832 30626 rpcz_store.cc:276] 0709 13:15:11.344232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.677686 (+333454us) service_pool.cc:224] Handling call
0709 13:15:11.680762 (+  3076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.717941 (+ 37179us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:11.717974 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.722035 (+  4061us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.893248 (+171213us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180027d720. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:12.175199 (+281951us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:12.175244 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.175327 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.310765 (+135438us) raft_consensus.cc:1685] finished
0709 13:15:12.310796 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.372620 (+ 61824us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62334,"raft.queue_time_us":40567,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":25536,"spinlock_wait_cycles":24894208,"thread_start_us":281681,"threads_started":1,"wal-append.queue_time_us":498004}
I20260709 13:15:12.689805  1957 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:12.681274  1950 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:15:12.664397  2070 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:12.684830 30627 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 7: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:12.609735  1958 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.608961  2137 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.657199  2015 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:12.228111  2104 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:12.681787  2046 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.877s	user 0.013s	sys 0.002s
W20260709 13:15:12.688542  2140 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.008s	sys 0.026s
I20260709 13:15:12.664289  1786 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:12.732000  2153 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
I20260709 13:15:12.681274  2155 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:12.642078  2038 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:12.657102  1949 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:12.703380 30629 rpcz_store.cc:276] 0709 13:15:11.348166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.699956 (+351790us) service_pool.cc:224] Handling call
0709 13:15:11.700380 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.718487 (+ 18107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.718520 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.744590 (+ 26070us) log.cc:844] Serialized 26 byte log entry
0709 13:15:11.951385 (+206795us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:11.951440 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:11.951599 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.657645 (+706046us) raft_consensus.cc:1685] finished
0709 13:15:12.657680 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.657849 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37319,"raft.queue_time_us":47421,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":17547,"thread_start_us":168870,"threads_started":1,"wal-append.queue_time_us":366057}
W20260709 13:15:12.725745 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62576.9us
I20260709 13:15:12.698658  2049 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:12.736764  2008 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:12.665630  2091 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:12.747078  2140 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.009s	sys 0.027s
I20260709 13:15:12.748845 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:12.766719 30629 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 5.
I20260709 13:15:12.728085  2034 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:12.730775  2070 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:12.701083  2128 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.681955  2121 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:12.721709  2103 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:12.699720 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88801.1us
W20260709 13:15:12.762712  2120 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.811s	user 0.007s	sys 0.016s
I20260709 13:15:12.772904  2070 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:12.756753  2104 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:12.765142  2157 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 13:15:12.745265  2025 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:12.756752  2008 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:12.788851 30633 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:15:12.793859 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:12.815456 30633 raft_consensus.cc:606] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:12.815634 31391 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:12.797849 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:12.747754  2046 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:12.806986 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:12.808039 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:12.759459  2035 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:15:12.823709  2163 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:15:12.811174 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:12.812992 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 711) took 1462 ms. Trace:
I20260709 13:15:12.782265  2034 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.815899 30633 raft_consensus.cc:614] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:12.796928 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:12.768730  1949 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:15:12.790484  2115 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:12.822000 31388 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:12.826879 30632 rpcz_store.cc:276] 0709 13:15:11.350071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.888400 (+538329us) service_pool.cc:224] Handling call
0709 13:15:11.889045 (+   645us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:11.951901 (+ 62856us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:11.951957 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:11.952711 (+   754us) log.cc:844] Serialized 26 byte log entry
0709 13:15:12.186488 (+233777us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:12.186556 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.186706 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:12.812622 (+625916us) raft_consensus.cc:1685] finished
0709 13:15:12.812658 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:12.812897 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91130,"raft.queue_time_us":63398,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":8293,"thread_start_us":183174,"threads_started":1,"wal-append.queue_time_us":241515}
I20260709 13:15:12.811316  2154 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:12.846971  1977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.029s
I20260709 13:15:12.871562 31391 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:12.880147 30632 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:15:12.864274  1977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.009s	sys 0.030s
I20260709 13:15:12.804176  2120 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.889647  1741 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.897970 30632 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:12.835304  2034 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.881520  1776 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:12.894834  2164 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:15:12.890399  1606 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:12.897096  2161 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:15:12.915597 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1982) took 1781 ms. Trace:
I20260709 13:15:12.916122 30559 rpcz_store.cc:276] 0709 13:15:11.134143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.134361 (+   218us) service_pool.cc:224] Handling call
0709 13:15:11.134890 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.769377 (+634487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.915305 (+1145928us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54003b38d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:12.915520 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1437110016}
I20260709 13:15:12.917409 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1389) took 1895 ms. Trace:
I20260709 13:15:12.917827 30555 rpcz_store.cc:276] 0709 13:15:11.021646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.022008 (+   362us) service_pool.cc:224] Handling call
0709 13:15:11.022399 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.976913 (+954514us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:12.917201 (+940288us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:12.917357 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908086528}
I20260709 13:15:12.894508  2178 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:12.894382  2169 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.919296 30630 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:15:12.907879 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 155897us
I20260709 13:15:12.929883 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:12.924060 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 317891us
I20260709 13:15:12.925125 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:12.931999 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:12.934733 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:12.842368  2155 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:12.885926  2008 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:12.931839  2169 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:12.950765  1606 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:12.950959  2038 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:12.952195  2169 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:12.953846  2034 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.920876  2178 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:12.956416  2178 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.957386  1973 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:12.958214  2178 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:12.963090  2174 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 13:15:12.974891  2110 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.526s	user 0.011s	sys 0.000s
I20260709 13:15:12.927389  1958 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.980121  2128 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.964524  1786 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.981452  2046 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:12.973578  1606 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:12.960400  2120 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:12.959414  2008 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:12.971680  2179 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:15:12.921492  2102 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:12.947307  2103 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:12.964195  2175 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:15:12.994942 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60315.2us
I20260709 13:15:13.006197  2173 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.023012 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:12.963667  2094 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.023036  2173 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.025575  2094 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:12.960556  2021 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9.
I20260709 13:15:13.025974  2173 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.027124  1786 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:13.028697  2172 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:13.029247  2172 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:12.958056  1973 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.975503 31389 raft_consensus.cc:2410] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:13.008229  2122 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:12.934077  2154 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:13.036500 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.036931 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:13.037788  2021 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:13.038381  2021 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:13.038400  2122 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.038018 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:13.039333  2021 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.044664 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:13.045240 30557 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:13.045682 31388 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:13.048498 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:13.049130 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:13.008057  2034 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:12.936123  2137 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.055580 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:13.067004  2034 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:13.067493  2034 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:13.068259 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:13.069630 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:13.070358  2035 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:13.056113  2120 raft_consensus.cc:2764] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:15:13.056313  2181 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:13.056759 30704 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:13.083881 30704 raft_consensus.cc:606] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:13.084537 30704 raft_consensus.cc:614] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:15:13.057511 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 125598us
I20260709 13:15:13.034365  1973 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:13.062342  2131 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.063048  1780 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.064142 30706 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:15:13.071449 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:13.030215  2172 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.081321  2169 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:13.087302 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.089062 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.090227 30558 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:13.090440 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.107235 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.091589 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.095700 30552 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:13.108347 30551 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:13.109692 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.110200 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.070995  1606 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:13.054391  2091 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.114010 30554 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:13.115564 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.116139 30551 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:15:13.118860 30551 pending_rounds.cc:85] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:15:13.118961 30550 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:13.119227 30551 pending_rounds.cc:107] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:15:13.119625 30551 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:13.115337 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.078073  2041 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:13.082679  2132 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:13.132476 30559 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
W20260709 13:15:13.133242 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56425.8us
I20260709 13:15:13.134449 31388 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:13.139963 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:13.116516  1606 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:13.141355  1606 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:13.137253  2172 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:13.137759 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:13.104835  2181 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:13.137130 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:13.151046 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.150214  2155 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:13.162305  2192 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.163005  2155 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:13.163203  2192 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.164534  2155 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.167418  2192 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:13.175729 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1339) took 2543 ms (client timeout 2097 ms). Trace:
W20260709 13:15:13.176313 30702 rpcz_store.cc:269] 0709 13:15:10.632817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:10.649487 (+ 16670us) service_pool.cc:224] Handling call
0709 13:15:13.175479 (+2525992us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54005a28d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:13.175626 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1916402048}
I20260709 13:15:13.176648 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.181026 30551 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:13.115154  1780 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.187448 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.192473 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.195204 30551 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:13.196810 31361 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:13.196972 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.194070  2070 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.205292  2070 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:13.202432  1957 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.207494 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:13.208235 31388 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:15:13.208455  1957 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.210101 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1407) took 1897 ms. Trace:
I20260709 13:15:13.210557 31383 rpcz_store.cc:276] 0709 13:15:11.312933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.313066 (+   133us) service_pool.cc:224] Handling call
0709 13:15:11.313406 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.219101 (+905695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.209769 (+990668us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54001e0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:13.210031 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511240960}
W20260709 13:15:13.211405  2186 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
W20260709 13:15:13.196378 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59352.7us
I20260709 13:15:13.217480 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:13.218405 31364 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:15:13.233506 31367 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:13.238509 31364 raft_consensus.cc:1217] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Deduplicated request from leader. Original: 5.5->[6.6-9.7]   Dedup: 6.6->[9.7-9.7]
I20260709 13:15:13.238862 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:13.159328 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 87971.1us
I20260709 13:15:13.240772 31364 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [6.6-9.7]
I20260709 13:15:13.242805 31378 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:13.242817 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1402) took 1816 ms. Trace:
I20260709 13:15:13.243582 30625 rpcz_store.cc:276] 0709 13:15:11.426587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.123119 (+696532us) service_pool.cc:224] Handling call
0709 13:15:12.123547 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.123811 (+   264us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:12.124192 (+   381us) log.cc:844] Serialized 14 byte log entry
0709 13:15:12.729974 (+605782us) log.cc:844] Serialized 14 byte log entry
0709 13:15:13.242432 (+512458us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:13.242487 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:13.242541 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.242607 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:13.242754 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":520517,"prepare.queue_time_us":2056,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":6160,"raft.queue_time_us":640721,"thread_start_us":596397,"threads_started":2,"wal-append.queue_time_us":523550}
I20260709 13:15:13.186170  2154 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.249616 30551 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:13.251864 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.256049 31383 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:15:13.184180  2191 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:13.271479 30700 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:13.275632 30702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:13.276489 30702 raft_consensus.cc:606] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:13.278273 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.283329 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.287621 30706 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:15:13.288367 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.287693 31388 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:13.242964  2090 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:13.290800 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:13.284896  2015 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.005s
I20260709 13:15:13.303162 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.298225 30704 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:13.284794  2132 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:13.245642  2034 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.300513  1776 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:13.300349  2090 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:13.301419 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.342890 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.302368 30554 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:15:13.302870  1977 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.303148 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.305434  2191 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:13.307389 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.310133 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1404) took 1250 ms. Trace:
I20260709 13:15:13.361074 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.310885  2111 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:13.311468 30550 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:15:13.315078  2144 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.240s	user 0.005s	sys 0.001s
I20260709 13:15:13.315227  2131 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:13.319696 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.392153 30704 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:13.390301 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.395282 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:13.299597 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57505.7us
I20260709 13:15:13.321290 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.324669 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.327385 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:13.397490 30705 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:13.397650 30700 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:15:13.338780 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48134.7us
I20260709 13:15:13.341521 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:13.347379 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.353943 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.359953 30703 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:15:13.361529 30628 rpcz_store.cc:276] 0709 13:15:12.059890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.283014 (+223124us) service_pool.cc:224] Handling call
0709 13:15:12.283363 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.695322 (+411959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:12.695360 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:12.695416 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:12.695449 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.309825 (+614376us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b5400094554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:13.309887 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:13.310058 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":946702720}
I20260709 13:15:13.361764 30698 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:13.340461 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.400424 30632 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:15:13.408947 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:13.352509  1950 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.610s	user 0.014s	sys 0.000s
I20260709 13:15:13.422205  1950 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13.
I20260709 13:15:13.371116 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.338455  2194 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:15:13.425745 30556 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:13.425927 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:13.426810 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.450938 30702 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:13.394876 30558 raft_consensus.cc:2376] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13. Re-sending same reply.
I20260709 13:15:13.292069  1957 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:15:13.346561  1726 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:13.276424  2154 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:13.450208 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55055us
I20260709 13:15:13.360576  2102 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.372035  2111 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:13.468256 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.451040 30633 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:13.371429  1977 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:13.455832 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48647.9us
I20260709 13:15:13.340065  1958 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 13:15:13.474136 30628 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:13.481244 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:13.433240  2070 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.449012  2169 consensus_queue.cc:1243] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:13.439055  1780 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.454833  2194 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.346963  1776 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.459019  1885 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.495518 30559 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:13.497659 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.498036 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.275655  2181 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.433454  2121 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.468791  2131 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:13.490574 30702 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:15:13.509883 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 584881us
I20260709 13:15:13.511263 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.519330 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:13.525406 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:13.526362 31367 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:13.527143 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.529767 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 712) took 2177 ms. Trace:
I20260709 13:15:13.530400 30624 rpcz_store.cc:276] 0709 13:15:11.352649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.944740 (+592091us) service_pool.cc:224] Handling call
0709 13:15:11.945230 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:11.945588 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:11.966928 (+ 21340us) log.cc:844] Serialized 14 byte log entry
0709 13:15:12.285684 (+318756us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18002946a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:12.585686 (+300002us) log.cc:844] Serialized 14 byte log entry
0709 13:15:13.040723 (+455037us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b18002946a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:13.213646 (+172923us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18002946a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:13.529206 (+315560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:13.529298 (+    92us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:13.529372 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.529474 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:13.529679 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":546948,"prepare.queue_time_us":21184,"prepare.run_cpu_time_us":835,"prepare.run_wall_time_us":10433,"raft.queue_time_us":712427,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":27836,"spinlock_wait_cycles":22901120,"thread_start_us":594467,"threads_started":2,"wal-append.queue_time_us":731081}
I20260709 13:15:13.433063  2201 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.114s
I20260709 13:15:13.495806  2046 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.492964  1726 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:13.500111  2178 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.595836 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.601505 30628 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:15:13.500449  2194 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:13.615496  2197 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:13.639254 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.639787  2197 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:13.640137 30550 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:13.640275  2197 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:13.640605  2180 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.478235  2102 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.501387  2070 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.622118  2209 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.657222 31388 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:15:13.657716  2209 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.658866  2070 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.594977  2103 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:13.589501  2131 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:13.642436 30633 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:13.662070 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.665150 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.614629 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.614884 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:13.666332 30627 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:13.667071 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.509518  1885 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.714568 30624 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:15:13.522266  2121 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.717145 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.714159  1973 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:15:13.624954  2046 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.741009  2154 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.720403  2204 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.133s
I20260709 13:15:13.720607 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.744755 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.746042 30628 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:13.720985  2090 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:13.721413 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.748682 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:13.749645  2191 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:13.750231  2191 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:13.721593  2034 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:13.750735  2191 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:13.587694  2132 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.752206  2102 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.619448  2199 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.723081  2125 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:13.787103 30627 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:13.788913 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1246) took 2032 ms. Trace:
I20260709 13:15:13.789412 31394 rpcz_store.cc:276] 0709 13:15:11.756832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.757270 (+   438us) service_pool.cc:224] Handling call
0709 13:15:12.289790 (+532520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.313688 (+ 23898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.788627 (+1474939us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54004b05d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:13.788822 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1223818496}
I20260709 13:15:13.722651  1726 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.724159 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:13.791898 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67848.4us
I20260709 13:15:13.792114 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.726866  2168 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:15:13.615051  2170 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.622509  1780 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.619323  2200 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.709713  2110 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:15:13.728937  2167 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.648135  2005 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.662163  2068 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.154s
I20260709 13:15:13.651467  2111 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.750319 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:13.619065  2205 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:13.644392  2091 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.798148 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:13.727807  2089 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.799858 30624 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:13.641644  2140 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:13.787174  2120 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.012s	sys 0.021s
I20260709 13:15:13.827539 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:13.828290 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.828358 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:13.871548 30628 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:13.872601 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44346.1us
I20260709 13:15:13.716408  2103 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:13.660207  2209 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.729873  2187 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.649200  2178 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.653177  2180 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.798193  2137 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:13.808882  2102 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:13.839278 30550 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:13.800117  2046 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.827477  1489 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:13.837549 31367 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:15:13.865947 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:13.868062 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.871665 30553 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:13.826931  2015 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
W20260709 13:15:13.847296  2172 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.004s
I20260709 13:15:13.876506 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1403) took 1817 ms. Trace:
I20260709 13:15:13.879204 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:13.833962  2167 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:13.863217  1606 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.721s	user 0.014s	sys 0.000s
I20260709 13:15:13.838375  2168 consensus_queue.cc:1243] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:13.809580  2125 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.831077  2094 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:15:13.832119 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:13.838809  2090 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:15:13.901031  2154 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:13.831795  1780 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.910048 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:13.911654 30633 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:13.914413 30631 rpcz_store.cc:276] 0709 13:15:12.059575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.224758 (+165183us) service_pool.cc:224] Handling call
0709 13:15:12.225192 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:12.741300 (+516108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:12.741347 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:12.741407 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:12.741454 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.862019 (+1120565us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54005f3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:13.875374 (+ 13355us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:13.876413 (+  1039us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186302208}
I20260709 13:15:13.920576 30555 raft_consensus.cc:2410] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:13.919190  2046 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:13.927978 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.869418  2140 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:13.930819  1606 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:13.932101  2178 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.932142  2196 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:13.864295  2089 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.834111  2170 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.844239  2205 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:13.835549  2005 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:13.830435  2199 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.026922 30702 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:14.029073  2199 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:13.939778  2103 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.015617  2196 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.034267 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:14.035432  2196 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.035990 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.010131  2168 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:13.899798  2137 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.041229  2090 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.041790  2090 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.043712  2137 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.044034  2090 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:13.839129  2200 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.046475  2200 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.947043  2218 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.047441  2218 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.052397  2167 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:14.053632  1885 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:14.054404  2162 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.308s	user 0.002s	sys 0.000s
W20260709 13:15:14.056412  2160 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.309s	user 0.001s	sys 0.001s
I20260709 13:15:14.061025  2005 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.062080 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:14.063150 30624 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:14.064031  2140 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.064328 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:14.065100 30558 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:14.067384 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.065109 30624 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:14.072017 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:13.989114  1974 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:13.967309  2102 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:13.967837 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.968942 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:14.088115 30627 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:14.089128 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.901670 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:14.090446 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 188874us
I20260709 13:15:14.093086  2121 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:14.093354  1780 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.089857  1974 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.097589  1974 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:13.959766 30628 raft_consensus.cc:2410] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:15:13.869889  2091 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.100495  2091 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:13.990417 30701 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:15:13.993206 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 83272.2us
I20260709 13:15:13.975453 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:13.869081  2131 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.105443 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:14.107059  2005 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:14.008277 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.891525  2187 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.020346  1988 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.020736  2089 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:14.024818 30630 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1401) took 2599 ms (client timeout 2053 ms). Trace:
I20260709 13:15:13.995680  2197 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.040613  2094 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:14.029978 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.074096 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:13.944058  2191 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.079689  2115 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:14.044360 31378 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:13.955951  2128 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:15:14.085356 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:14.139859 30631 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:14.140213 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.141527 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.142931  2191 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:14.086668  2200 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.143750  2200 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.105480 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.145823 30706 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:15:14.145859 30699 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:13.969673  2180 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.050383  2218 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.025342  1606 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:14.117622 30630 rpcz_store.cc:269] 0709 13:15:11.425673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:11.433889 (+  8216us) service_pool.cc:224] Handling call
0709 13:15:14.024592 (+2590703us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540007d5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:14.024723 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3413031168}
I20260709 13:15:14.156761  2200 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.156850 30706 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:14.118245 30628 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:14.087466  2102 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:14.108377  1885 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:14.090170  2168 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:14.108002  2005 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:14.113224  2046 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:14.108330  2205 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.131805 30702 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:14.145370  2115 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.137202 31378 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:14.187717 30702 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:15:14.136214 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.143034 30705 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:14.138705  2094 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.144876  1776 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.160508 31389 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:14.164283 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:14.119890  1988 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.103955 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:14.204356 30627 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:14.190272 30626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.014s	sys 0.020s
W20260709 13:15:14.193118 30554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.002s	sys 0.012s
I20260709 13:15:14.195946 30628 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:15:14.207808 30554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.003s	sys 0.012s
W20260709 13:15:14.207615 30626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.014s	sys 0.020s
I20260709 13:15:14.202149 30624 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:14.188495 31378 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.190973  2046 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:14.228530 31378 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:14.229357  2046 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.175004  1885 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:14.231469  1885 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.232611  1885 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:14.187391  2225 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.234112  2102 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.235109  2225 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.235685  2187 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.236697  2168 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:14.204027  2194 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.540s	user 0.007s	sys 0.007s
I20260709 13:15:14.243116  2094 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:14.196422  2041 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.005s	sys 0.017s
I20260709 13:15:14.243269 30628 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:14.245129 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 80945.7us
I20260709 13:15:14.235499  2102 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:14.255832  2224 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:14.256489  2224 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:14.256865  2224 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.164994  1949 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:15:14.225647  2205 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.264402  2205 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.267400  2197 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.267686 30629 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:14.268064 30629 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:14.269078 30557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 1986) took 1960 ms (client timeout 1761 ms). Trace:
I20260709 13:15:14.268934 30629 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:14.269557 30557 rpcz_store.cc:269] 0709 13:15:12.308794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:12.309142 (+   348us) service_pool.cc:224] Handling call
0709 13:15:14.268860 (+1959718us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400110dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:14.268989 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988276608}
I20260709 13:15:14.270344  2199 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:14.270885  2199 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:14.271703 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1319) took 1054 ms. Trace:
I20260709 13:15:14.271905 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:14.243119  2194 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.272349 31391 rpcz_store.cc:276] 0709 13:15:13.217346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.217477 (+   131us) service_pool.cc:224] Handling call
0709 13:15:13.217849 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:13.347165 (+129316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:13.347290 (+   125us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:13.426434 (+ 79144us) log.cc:844] Serialized 52 byte log entry
0709 13:15:13.833053 (+406619us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:13.833098 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:13.833175 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:14.271426 (+438251us) raft_consensus.cc:1685] finished
0709 13:15:14.271451 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.271635 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99480,"raft.queue_time_us":276300,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":10632,"thread_start_us":466925,"threads_started":2,"wal-append.queue_time_us":808686}
I20260709 13:15:14.274277  2194 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.276721 30706 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: []
I20260709 13:15:14.287303 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.288779 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:14.244233  2041 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.005s	sys 0.018s
I20260709 13:15:14.290448 30629 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:14.251005 30624 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:14.306777  1949 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:14.307283  1949 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:14.307354  2170 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.307741  1949 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:14.266291  1988 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:14.309820  2225 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:14.312551 30628 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:14.312217  2122 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.314294  2122 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.271726  2199 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.284643  2205 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.316479  2122 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.324834 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.326606 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:14.327821 30671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.015s	sys 0.008s
I20260709 13:15:14.329334 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.330463 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.298995 30627 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:14.336385 30552 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:14.328351 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:14.355679 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:14.360342 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:14.362778 30671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.015s	sys 0.009s
W20260709 13:15:14.363598 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 844408us
I20260709 13:15:14.366304 30627 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:15:14.366498 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:14.368815 31378 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:15:14.369717 31378 pending_rounds.cc:85] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:14.370223 31378 pending_rounds.cc:107] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:15:14.370744 31378 raft_consensus.cc:2889] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:14.371837 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1327) took 1126 ms. Trace:
I20260709 13:15:14.380457 30629 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:14.384393 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:14.386662  2170 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:14.386972 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:14.387283  2178 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:15:14.391639  1949 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.398695 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.482493 31378 rpcz_store.cc:276] 0709 13:15:13.245790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.245915 (+   125us) service_pool.cc:224] Handling call
0709 13:15:13.246232 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.044244 (+798012us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400281520. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:15:14.371466 (+327222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.371759 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567596288}
I20260709 13:15:14.492688 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:14.402927 30625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.038s	sys 0.045s
W20260709 13:15:14.494320 30625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.038s	sys 0.046s
I20260709 13:15:14.495609 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1410) took 1151 ms. Trace:
I20260709 13:15:14.496064 30625 rpcz_store.cc:276] 0709 13:15:13.343882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.344431 (+   549us) service_pool.cc:224] Handling call
0709 13:15:13.344850 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.345123 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:13.449670 (+104547us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.495045 (+1045375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:14.495143 (+    98us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:14.495220 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.495320 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.495523 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15763,"prepare.queue_time_us":88720,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":5135,"thread_start_us":1044809,"threads_started":1}
I20260709 13:15:14.403183 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:14.406579 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:14.408593 31396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.007s	sys 0.012s
W20260709 13:15:14.499207 31396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.007s	sys 0.012s
I20260709 13:15:14.406384  2199 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:14.407428  2070 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:15:14.437163 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.163s	user 0.008s	sys 0.015s
I20260709 13:15:14.440308 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:14.510437 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.009s	sys 0.015s
W20260709 13:15:14.511356 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 1.23732e+06us
W20260709 13:15:14.493764  2136 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.411s	user 0.015s	sys 0.025s
I20260709 13:15:14.442149 30624 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:14.518862 30702 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
W20260709 13:15:14.459532 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.483145 30705 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:14.484781 30629 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:14.493556 30552 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:14.526299 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.527451 30552 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:14.498919  2216 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:15:14.520756  2070 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:14.519333  2136 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.468s	user 0.015s	sys 0.026s
I20260709 13:15:14.532146 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.533910 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:14.530397 30632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.012s	sys 0.019s
W20260709 13:15:14.540692 30632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.012s	sys 0.019s
W20260709 13:15:14.404378 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44166.3us
I20260709 13:15:14.541874 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:14.537563 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:14.541002 30556 raft_consensus.cc:2410] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:14.543323 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1414) took 1138 ms. Trace:
I20260709 13:15:14.521468  2035 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.555172 30632 rpcz_store.cc:276] 0709 13:15:13.405068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.405248 (+   180us) service_pool.cc:224] Handling call
0709 13:15:13.405702 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.406009 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:13.533708 (+127699us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.542577 (+1008869us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:14.542659 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:14.542715 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.542791 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.543253 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29144,"prepare.queue_time_us":94266,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":13853,"raft.queue_time_us":1374,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":153,"thread_start_us":1106390,"threads_started":2}
I20260709 13:15:14.524230  2197 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:14.528352  2181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.008s	sys 0.014s
I20260709 13:15:14.555318  1953 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 13:15:14.564504  1953 consensus_queue.cc:1243] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:14.564424  2220 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:14.564749  2218 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:14.566269  2218 consensus_queue.cc:1243] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:14.508265  2115 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.563736 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:14.567422  2220 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:14.568055  2220 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:14.569286 30557 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:14.569187  2220 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:14.572492  2218 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:14.573511 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.574273 31391 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:14.574900 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.575748 31378 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:14.576005 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:14.577960 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:14.578749 31391 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:14.584774 31367 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:14.586246 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.555883  1885 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:14.587358 31378 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:14.591672 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.594774  1949 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.599126  2035 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.539952  2205 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:14.610435 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.611474 30556 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
W20260709 13:15:14.604019  2001 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.014s	sys 0.012s
W20260709 13:15:14.538019  2104 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.897s	user 0.010s	sys 0.012s
W20260709 13:15:14.608093  2181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.874s	user 0.009s	sys 0.015s
W20260709 13:15:14.634805  2001 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.134s	user 0.014s	sys 0.012s
W20260709 13:15:14.639314  2104 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.010s	sys 0.012s
I20260709 13:15:14.672950 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.671566 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.672556 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:14.635913  2120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.011s	sys 0.015s
I20260709 13:15:14.670293 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.694468 31367 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:14.609100  2197 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:14.671975 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.698061 31383 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:14.614436  2205 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:14.693658  2120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.011s	sys 0.016s
I20260709 13:15:14.671351  2024 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:14.663859  2035 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:14.693869  2070 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.704738  2024 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:14.709064  2197 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.709748  2024 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:14.711653  2024 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:14.720546  2205 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.722960  2224 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:14.725308  2224 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:14.739653  2103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.038s	sys 0.044s
W20260709 13:15:14.884837  2103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.039s	sys 0.045s
W20260709 13:15:14.807163 30554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.006s	sys 0.003s
W20260709 13:15:14.886515 30554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.007s	sys 0.003s
W20260709 13:15:14.886715  2046 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.013s	sys 0.021s
W20260709 13:15:14.887430  2046 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.013s	sys 0.021s
W20260709 13:15:14.809187 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:14.811028 30550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.005s	sys 0.011s
W20260709 13:15:14.890103 30550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.005s	sys 0.011s
I20260709 13:15:14.891296  2231 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:14.893646 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:14.819602 30703 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.010s	sys 0.013s
W20260709 13:15:14.821125 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 279090us
I20260709 13:15:14.807477 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:14.824221 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:14.824991 31294 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.201s	user 0.021s	sys 0.045s
I20260709 13:15:14.826468 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.826519 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.827805 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.828136 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:14.829543 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.668587 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:14.831453 31383 raft_consensus.cc:1217] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Deduplicated request from leader. Original: 10.6->[11.7-13.8]   Dedup: 13.8->[]
I20260709 13:15:14.831939 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1322) took 1611 ms. Trace:
I20260709 13:15:14.833235 31394 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:15:14.454461  2198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.006s	sys 0.024s
W20260709 13:15:14.872489  2168 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.007s	sys 0.017s
W20260709 13:15:14.701764  1958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.180s	user 0.014s	sys 0.017s
W20260709 13:15:14.959286  2198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.311s	user 0.006s	sys 0.025s
I20260709 13:15:14.934799 31396 rpcz_store.cc:276] 0709 13:15:13.219930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.220097 (+   167us) service_pool.cc:224] Handling call
0709 13:15:13.220535 (+   438us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:13.473355 (+252820us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:13.588569 (+115214us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.499616 (+911047us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:14.509266 (+  9650us) log.cc:844] Serialized 52 byte log entry
0709 13:15:14.509501 (+   235us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:14.516553 (+  7052us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.516829 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.516920 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:14.674921 (+158001us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b18000e3b60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:14.831084 (+156163us) raft_consensus.cc:1685] finished
0709 13:15:14.831198 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:14.831612 (+   414us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120553,"prepare.queue_time_us":181332,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":118,"raft.queue_time_us":279141,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":9922,"spinlock_wait_cycles":5221760,"thread_start_us":1085092,"threads_started":2,"wal-append.queue_time_us":1082519}
W20260709 13:15:14.959865  1958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.458s	user 0.014s	sys 0.018s
W20260709 13:15:14.872679  2132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.003s	sys 0.028s
W20260709 13:15:14.877233  1489 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.016s	sys 0.010s
W20260709 13:15:14.878069  2131 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.012s	sys 0.018s
I20260709 13:15:14.879213  2024 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:14.893110  2231 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:14.746271  2195 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.025s	sys 0.004s
W20260709 13:15:14.896025 30559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 705) took 1704 ms (client timeout 1620 ms). Trace:
W20260709 13:15:14.810310 30626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.001s	sys 0.015s
W20260709 13:15:14.872714  2041 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.018s	sys 0.003s
W20260709 13:15:14.970705  2132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.077s	user 0.003s	sys 0.029s
W20260709 13:15:14.972548  2195 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.890s	user 0.026s	sys 0.004s
W20260709 13:15:14.959347  2168 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.008s	sys 0.018s
I20260709 13:15:14.976217 30556 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:14.977427  2115 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:14.977610  2231 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.977882 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:14.743057  2206 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.016s	sys 0.034s
W20260709 13:15:14.978598  2131 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.013s	sys 0.018s
W20260709 13:15:14.979416  2206 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.356s	user 0.018s	sys 0.034s
I20260709 13:15:14.979554  2125 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:14.983390 31364 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
W20260709 13:15:14.968662 30559 rpcz_store.cc:269] 0709 13:15:13.191834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.191979 (+   145us) service_pool.cc:224] Handling call
0709 13:15:14.895367 (+1703388us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540013a954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:14.895679 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588353408}
I20260709 13:15:14.985126 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:14.985584 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:14.982007  2024 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:14.989001  2024 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:14.922227 31294 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.021s	sys 0.045s
W20260709 13:15:14.879279 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 690) took 1792 ms (client timeout 1675 ms). Trace:
W20260709 13:15:14.931054 30703 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.011s	sys 0.013s
I20260709 13:15:14.934697 31383 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:15:14.938211 31388 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:14.940953 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1433) took 1416 ms. Trace:
I20260709 13:15:14.945472  2205 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:14.947597  2169 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:14.960805 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:14.942216 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 114894us
W20260709 13:15:14.969072 30626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.001s	sys 0.016s
W20260709 13:15:14.969630  1489 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.017s	sys 0.011s
I20260709 13:15:14.986303 31364 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:15:14.971014  2041 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.019s	sys 0.003s
W20260709 13:15:14.991140 30555 rpcz_store.cc:269] 0709 13:15:13.086427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.086580 (+   153us) service_pool.cc:224] Handling call
0709 13:15:14.878467 (+1791887us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54000208d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:14.878934 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1822904064}
I20260709 13:15:14.992321 31389 rpcz_store.cc:276] 0709 13:15:13.524520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.525061 (+   541us) service_pool.cc:224] Handling call
0709 13:15:13.601995 (+ 76934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.194130 (+592135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:14.940083 (+745953us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400680454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:14.940670 (+   587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1283844864}
W20260709 13:15:15.023713 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.009470  2111 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:14.946465  1977 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:15.048919 30556 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:15.030861 30559 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:15:15.057796  2244 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.310s	user 0.002s	sys 0.004s
I20260709 13:15:14.999949  2205 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:15.055163 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 1.43201e+06us
W20260709 13:15:15.071686 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56944.5us
W20260709 13:15:15.070938  2246 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
W20260709 13:15:15.090205  2254 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.063s	user 0.002s	sys 0.000s
I20260709 13:15:15.103255 31364 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:15:15.123106  2029 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.123917  2170 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.124462  2029 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:15.125033  2170 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:15.126739 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.128150  1953 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.128899 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.129094  1953 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.130375 31383 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:15.096480  2122 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:15.123461 30558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.018s	sys 0.023s
I20260709 13:15:15.133049  1953 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.090183  2232 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:15.146467  2170 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:15.142266 30558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.019s	sys 0.023s
W20260709 13:15:15.151276 30630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.038s	sys 0.030s
I20260709 13:15:15.170589 31388 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:15.173293 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:15.175679  2029 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:15.165117 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:15.138118  2232 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:15.183998 30630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.038s	sys 0.031s
I20260709 13:15:15.182350  2232 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:15.121416  2169 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.138s
W20260709 13:15:15.275278  2220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.013s	sys 0.058s
W20260709 13:15:15.271697 30599 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.012s	sys 0.046s
W20260709 13:15:15.287812 30599 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.003s	user 0.014s	sys 0.046s
W20260709 13:15:15.284969 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 169676us
W20260709 13:15:15.309424 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.312108 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1358) took 1178 ms. Trace:
I20260709 13:15:15.312762 30706 rpcz_store.cc:276] 0709 13:15:14.133978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.182956 (+ 48978us) service_pool.cc:224] Handling call
0709 13:15:14.204327 (+ 21371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.298658 (+ 94331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.311285 (+1012627us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:15.311791 (+   506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165921664}
W20260709 13:15:15.291199 30671 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.038s	sys 0.085s
I20260709 13:15:15.285095 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:15.288609  2220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.014s	sys 0.059s
W20260709 13:15:15.354934  2192 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:15:15.277999  2115 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:15.353116 30671 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.039s	sys 0.085s
I20260709 13:15:15.358886 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.388617 30555 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:15.390431 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1412) took 1144 ms. Trace:
I20260709 13:15:15.391041 30557 rpcz_store.cc:276] 0709 13:15:14.131938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.286922 (+154984us) service_pool.cc:224] Handling call
0709 13:15:15.239104 (+952182us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:15.276219 (+ 37115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620053760}
I20260709 13:15:15.392100  2122 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.392784  2122 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:15.388681 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.396548 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1411) took 1265 ms. Trace:
I20260709 13:15:15.397178 30555 rpcz_store.cc:276] 0709 13:15:14.131554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.044915 (+913361us) service_pool.cc:224] Handling call
0709 13:15:15.102211 (+ 57296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:15.389530 (+287319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.390495 (+   965us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b5400174b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:15.396399 (+  5904us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":575456000}
W20260709 13:15:15.399770  2258 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:15:15.400517  2201 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:15.401263  2201 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:15.401965  2218 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:15.403113 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.727s (SENT)
I20260709 13:15:15.403846 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:15.405570 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 148107us
I20260709 13:15:15.406206 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1400) took 1909 ms. Trace:
I20260709 13:15:15.393137 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:15.406581 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.840s (SENT) [suppressed 18 similar messages]
W20260709 13:15:15.441768 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 1.15778e+06us
I20260709 13:15:15.458846 30552 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
W20260709 13:15:15.460263  2242 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.703s	user 0.006s	sys 0.000s
I20260709 13:15:15.460840 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:15.462838 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:15.463054 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 172012us
I20260709 13:15:15.464135 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:15.466481  2240 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.710s	user 0.001s	sys 0.005s
W20260709 13:15:15.466668  2211 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.210s	user 0.000s	sys 0.002s
I20260709 13:15:15.470914 30557 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:15.473116 30557 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:15.473616  2247 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:15:15.472723 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.473989 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2007) took 1193 ms. Trace:
I20260709 13:15:15.434840  2041 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:15.474769 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1410) took 1322 ms. Trace:
I20260709 13:15:15.407107  2228 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:15.441313  2104 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.006s	sys 0.027s
I20260709 13:15:15.493577 31364 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:15:15.490526 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 1.01124e+06us
I20260709 13:15:15.481060 30558 rpcz_store.cc:276] 0709 13:15:14.129982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.275074 (+145092us) service_pool.cc:224] Handling call
0709 13:15:14.275506 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.275765 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:14.322315 (+ 46550us) log.cc:844] Serialized 14 byte log entry
0709 13:15:15.276093 (+953778us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:15.291064 (+ 14971us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:15.366121 (+ 75057us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.385750 (+ 19629us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.451917 (+ 66167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49380,"prepare.queue_time_us":55227,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":65225,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":69129,"thread_start_us":827499,"threads_started":1,"wal-append.queue_time_us":1060568}
I20260709 13:15:15.484236 30382 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:15.504436  1606 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.504722  2232 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.505476  1606 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:15.505694  2232 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:15.507732 30558 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:15:15.508581  2216 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.508646 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2012) took 1226 ms. Trace:
I20260709 13:15:15.509258 30627 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:15.509409 30558 rpcz_store.cc:276] 0709 13:15:14.282557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.504630 (+1222073us) service_pool.cc:224] Handling call
0709 13:15:15.507261 (+  2631us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.508292 (+  1031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.508558 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:15.511469 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:15.512554 30624 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:15.513911 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:15.509567  2216 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:15.515015 30628 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:15.515213 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:15.515661  2021 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:15.516148 30625 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:15.516548 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:15.517328 30629 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:15.518060 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 1998) took 1690 ms. Trace:
I20260709 13:15:15.518658  2232 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:15.519279 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2009) took 1238 ms. Trace:
I20260709 13:15:15.514094 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:15.519804 30552 rpcz_store.cc:276] 0709 13:15:14.281109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.531778 (+250669us) service_pool.cc:224] Handling call
0709 13:15:15.519017 (+987239us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:15.519203 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011794304}
I20260709 13:15:15.513978 30553 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:15.523324 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:15.524255 30631 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:15.518586 30550 rpcz_store.cc:276] 0709 13:15:13.827905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.084688 (+256783us) service_pool.cc:224] Handling call
0709 13:15:14.099927 (+ 15239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:14.100243 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:14.100281 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:14.100918 (+   637us) log.cc:844] Serialized 26 byte log entry
0709 13:15:15.002553 (+901635us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:15.011108 (+  8555us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.014359 (+  3251us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:15.517685 (+503326us) raft_consensus.cc:1685] finished
0709 13:15:15.517721 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.517977 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":789227,"threads_started":1,"wal-append.queue_time_us":989783}
I20260709 13:15:15.524307 30553 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:15.517225  2216 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:15.528180 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 707) took 2245 ms. Trace:
I20260709 13:15:15.528795 30551 rpcz_store.cc:276] 0709 13:15:13.282192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.283386 (+  1194us) service_pool.cc:224] Handling call
0709 13:15:14.882431 (+1599045us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:15.064518 (+182087us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:15.064574 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:15.068643 (+  4069us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:15.072383 (+  3740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.078416 (+  6033us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.527765 (+449349us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54000208d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:15.527934 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31910,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":215,"spinlock_wait_cycles":3676280064}
W20260709 13:15:15.530962 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 444627us
I20260709 13:15:15.532397 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:15.540177 30629 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6.
I20260709 13:15:15.539368 30382 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.552874 30382 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.555917 30382 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 11 to 14, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:15.559536 30382 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.406903 30554 rpcz_store.cc:276] 0709 13:15:13.496711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.503057 (+  6346us) service_pool.cc:224] Handling call
0709 13:15:13.504675 (+  1618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:13.659882 (+155207us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:14.208295 (+548413us) log.cc:844] Serialized 14 byte log entry
0709 13:15:14.887571 (+679276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:14.887736 (+   165us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:14.887940 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.405724 (+517784us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400a27bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:15.405795 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.406102 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":362860,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1910,"raft.run_cpu_time_us":432,"raft.run_wall_time_us":432,"spinlock_wait_cycles":356240128,"thread_start_us":1226404,"threads_started":2,"wal-append.queue_time_us":873216}
I20260709 13:15:15.543618  2154 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:15.544741  2170 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:15:15.547485 30381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 143) took 31591 ms (client timeout 15000 ms). Trace:
I20260709 13:15:15.481012  2041 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.570448 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.579774 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:15.566910  2104 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.006s	sys 0.028s
I20260709 13:15:15.474543 30557 rpcz_store.cc:276] 0709 13:15:14.280473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.472169 (+1191696us) service_pool.cc:224] Handling call
0709 13:15:15.472795 (+   626us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:15.473656 (+   861us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.473899 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:15.587777 31389 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:15:15.578081 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 98017.6us
W20260709 13:15:15.581238 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 465717us
I20260709 13:15:15.596760  2228 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:15.596279 30382 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.598264  2228 consensus_queue.cc:1243] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:15.598800  2121 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:15.599020  2232 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:15.599537  2232 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:15.599941 31364 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:15:15.601290 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69126us
I20260709 13:15:15.601306  2232 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.447484  2115 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:15.604410 30703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.046s	sys 0.062s
W20260709 13:15:15.572952  2239 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.761s	user 0.003s	sys 0.003s
W20260709 13:15:15.577474  2250 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.420s	user 0.000s	sys 0.002s
W20260709 13:15:15.608079 30703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.047s	sys 0.062s
I20260709 13:15:15.592262 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:15.578043  2041 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:15.572460  2205 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.614329 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1409) took 2270 ms. Trace:
I20260709 13:15:15.456280  2201 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:15.626514 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:15.606534  1489 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.041s	sys 0.098s
W20260709 13:15:15.572341 30381 rpcz_store.cc:269] 0709 13:14:44.010509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:44.051114 (+ 40605us) service_pool.cc:224] Handling call
0709 13:15:15.547139 (+31496025us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:01.918219 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:02.093497 (+175278us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:02.107791 (+ 14294us) write_op.cc:435] Acquired schema lock
0709 13:15:02.110697 (+  2906us) tablet.cc:662] Decoding operations
0709 13:15:02.277650 (+166953us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:02.277733 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:02.277771 (+    38us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:15:02.278410 (+   639us) tablet.cc:701] Row locks acquired
0709 13:15:02.278430 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:15:02.278593 (+   163us) write_op.cc:270] Start()
0709 13:15:02.278757 (+   164us) write_op.cc:276] Timestamp: P: 1783602902278547 usec, L: 0
0709 13:15:02.278778 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:15:02.279540 (+   762us) log.cc:844] Serialized 2078 byte log entry
0709 13:15:02.846430 (+566890us) op_driver.cc:464] REPLICATION: finished
0709 13:15:03.142735 (+296305us) write_op.cc:301] APPLY: starting
0709 13:15:03.148816 (+  6081us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:03.184533 (+ 35717us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:03.196986 (+ 12453us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:03.252002 (+ 55016us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:03.252270 (+   268us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 13:15:03.252334 (+    64us) write_op.cc:312] APPLY: finished
0709 13:15:03.252618 (+   284us) log.cc:844] Serialized 46 byte log entry
0709 13:15:03.252844 (+   226us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:03.252977 (+   133us) write_op.cc:454] Released schema lock
0709 13:15:03.253035 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:03.253548 (+   513us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:03.253582 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":389739,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":235654,"apply.run_cpu_time_us":3563,"apply.run_wall_time_us":132362,"num_ops":5,"prepare.queue_time_us":503290,"prepare.run_cpu_time_us":29203,"prepare.run_wall_time_us":588268,"raft.queue_time_us":128618,"raft.run_cpu_time_us":21130,"raft.run_wall_time_us":167385,"replication_time_us":549031,"thread_start_us":743146,"threads_started":4,"wal-append.queue_time_us":302179,"wal-append.run_cpu_time_us":11231,"wal-append.run_wall_time_us":1832808}]]}
W20260709 13:15:15.641911 30633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 942) took 1784 ms (client timeout 1858 ms). Trace:
I20260709 13:15:15.616596  2125 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.643028 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1706) took 1688 ms. Trace:
I20260709 13:15:15.643968  2232 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:15.644460  2228 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:15.645037  2228 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:15.645509  2228 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:15.618326 30626 rpcz_store.cc:276] 0709 13:15:13.343566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.344326 (+   760us) service_pool.cc:224] Handling call
0709 13:15:13.350170 (+  5844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:13.471693 (+121523us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:14.209634 (+737941us) log.cc:844] Serialized 14 byte log entry
0709 13:15:15.069840 (+860206us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.123293 (+ 53453us) log.cc:844] Serialized 26 byte log entry
0709 13:15:15.123611 (+   318us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:15.613061 (+489450us) log.cc:844] Serialized 14 byte log entry
0709 13:15:15.613813 (+   752us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.613961 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:15.614011 (+    50us) raft_consensus.cc:1685] finished
0709 13:15:15.614032 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.614240 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":190928,"prepare.queue_time_us":130724,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":127267,"raft.run_cpu_time_us":553,"raft.run_wall_time_us":17515,"thread_start_us":1988282,"threads_started":3,"wal-append.queue_time_us":728531}
I20260709 13:15:15.643261  2125 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.648936 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:15.650146  2230 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:15.642714 30633 rpcz_store.cc:269] 0709 13:15:13.672862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:13.749976 (+ 77114us) service_pool.cc:224] Handling call
0709 13:15:15.363408 (+1613432us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54006effd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:15.457009 (+ 93601us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165314816}
I20260709 13:15:15.651156 31364 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:15.650851  2230 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:15.651592 31364 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:15.652141  2125 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.643641 30703 rpcz_store.cc:276] 0709 13:15:13.954395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.068516 (+114121us) service_pool.cc:224] Handling call
0709 13:15:14.068901 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.069199 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:15.028671 (+959472us) log.cc:844] Serialized 14 byte log entry
0709 13:15:15.634962 (+606291us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:15.642429 (+  7467us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:15.642576 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.642747 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:15.642954 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":829722,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":86,"raft.queue_time_us":6092,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":17480,"thread_start_us":1452499,"threads_started":2}
I20260709 13:15:15.652691 31364 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:15.653893 31364 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:15.616217 30382 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 9, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:15.560712  2024 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 13:15:15.636722  1489 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.042s	sys 0.098s
I20260709 13:15:15.671607 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:15.668776 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 92113.9us
I20260709 13:15:15.678033 30382 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.661608  2046 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:15.681309 30382 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:15.682257  2046 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:15.682771  2046 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:15.687654 30382 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.689611 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:15.690434  1726 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:15.690789  2216 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.691731  2216 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.694964 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.690986  1726 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:15.676074 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:15.705585 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:15.705724 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48503.5us
I20260709 13:15:15.691812 30382 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.654351  2230 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:15.715018 31364 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:15.752542 31389 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:15.756531 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:15.765560  1489 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.766870 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:15.769086 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 157367us
I20260709 13:15:15.770699 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:15.754715  2024 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:15.772208  2024 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:15.772678  2024 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.773144  2024 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:15.774732 31378 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:15.760874 30382 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:15.707432  2029 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:15:15.763741 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.779345 31388 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:15.780283  2122 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:15.782106 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 121245us
I20260709 13:15:15.784243  2205 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:15.785125  2205 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:15.786895  2169 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:15.789032 31361 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 13:15:15.789805 31391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.043s	sys 0.122s
W20260709 13:15:15.790347 31391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.043s	sys 0.122s
I20260709 13:15:15.792563  2170 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:15.794342  1606 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:15.763491  2230 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:15:15.798826  2015 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.799374 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.799677  2015 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:15.807230 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.808307 31378 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:15.800212 31361 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:15.811079  1726 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:15.812744  1726 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:15.777197 30382 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 11, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:15.816274 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:15.821234 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:15.823820 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:15.818199  2122 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.779345  2216 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.819832 30382 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 9, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:15.839466  2015 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:15.774359 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:15.863374 30382 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 1 to 2, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 2 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.867697  2216 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:15.868283  2216 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:15.868728  2216 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:15.865607 30382 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 9, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 9 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:15.871960  2169 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:15.872447 30382 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:15.873925 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:15.874024  2015 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:15:15.884616 30556 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.805s	user 0.013s	sys 0.002s
I20260709 13:15:15.897388  2015 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:15.898494  2015 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:15.903826  2015 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:15.889496 31361 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:15.908095 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.904156  2115 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:15.910521 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.911592  2015 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:15.917434  2015 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.920959  2015 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:15.916196 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.935582 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.937628 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.827706  1489 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:15.885329 30556 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:15.957931 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1418) took 1139 ms. Trace:
I20260709 13:15:15.913128  2228 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:15.945343 31361 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:15.958369 30556 rpcz_store.cc:276] 0709 13:15:14.818550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.977460 (+158910us) service_pool.cc:224] Handling call
0709 13:15:15.957855 (+980395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:15.962843 31378 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:15.964473 30631 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:15.974112 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:15.975055 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:15.976624 31378 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:15.958822  2169 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:15.985105 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 111350us
I20260709 13:15:15.986852 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:15.988233  2205 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:15.989151 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:15.990505  2205 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:15:15.993613 30559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.021s	sys 0.028s
W20260709 13:15:15.994901 30559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.021s	sys 0.028s
I20260709 13:15:15.997272 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.006718 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.007743 31378 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:16.012823  2015 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:16.014037  2015 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:16.014079 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.018378  2115 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.019155  2115 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:16.019667  2199 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.503s	user 0.005s	sys 0.024s
I20260709 13:15:16.019441  2015 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:16.021845  2115 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:15.993705  2201 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:16.005103  2111 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:16.009086  2131 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:16.022459  2115 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:16.048743 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59787.3us
W20260709 13:15:16.059214 30630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.002s	sys 0.023s
W20260709 13:15:16.059983 30630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.002s	sys 0.023s
W20260709 13:15:16.060503 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 286348us
W20260709 13:15:16.042990 31396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.013s
W20260709 13:15:16.063024 31396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.012s	sys 0.013s
I20260709 13:15:16.043423  2131 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:16.081748 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:16.083583 31378 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:16.084565  2131 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:16.062002  2115 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:16.110342  2122 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:16.113464 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 521271us
W20260709 13:15:16.132742 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:16.140810 31394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.008s	sys 0.016s
W20260709 13:15:16.141443 31394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.008s	sys 0.016s
W20260709 13:15:16.141954 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 421393us
I20260709 13:15:16.102898  2046 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.109935  2024 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:16.139859  2198 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.007s	sys 0.023s
I20260709 13:15:16.150686 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 723) took 2537 ms. Trace:
I20260709 13:15:16.166054 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:16.170132 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.145153  2125 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:16.147336  2228 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.196385 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:16.198969 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 87157.5us
W20260709 13:15:16.190726 30632 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.020s	sys 0.040s
W20260709 13:15:16.202347 30632 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.020s	sys 0.040s
W20260709 13:15:16.203316 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.146876  2122 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:16.205426 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.151062  2216 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:16.177402  2199 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.168782  1489 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:16.160842  2198 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.008s	sys 0.023s
I20260709 13:15:16.226188  2015 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.226938  2015 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:16.227922  2216 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.229113  2015 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:16.230535  2024 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:16.241516 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1440) took 1235 ms. Trace:
W20260709 13:15:16.243471  2271 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:15:16.244858 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.245042 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1347) took 1418 ms. Trace:
I20260709 13:15:16.245556 31391 rpcz_store.cc:276] 0709 13:15:14.826303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.952562 (+126259us) service_pool.cc:224] Handling call
0709 13:15:14.953408 (+   846us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:14.954205 (+   797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:14.954266 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.007275 (+ 53009us) log.cc:844] Serialized 26 byte log entry
0709 13:15:15.790880 (+783605us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:15.790942 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:15.791050 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.244761 (+453711us) raft_consensus.cc:1685] finished
0709 13:15:16.244790 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.244973 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":716710,"threads_started":1,"wal-append.queue_time_us":977568}
I20260709 13:15:16.247032 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.247905 30704 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:16.248036 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.249501 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.249752 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.250782 31391 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 8.
I20260709 13:15:16.251864 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:16.252257 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.252676 31391 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:16.253036 30702 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:16.254403 31391 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:16.250494 30698 raft_consensus.cc:2410] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:16.256035 31391 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:16.246328 30701 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:16.258399 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.260087  2280 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:16.256093 30702 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:16.261740 31361 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:16.180505  2169 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:15:16.256349 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:16.242118 31396 rpcz_store.cc:276] 0709 13:15:15.006208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.018853 (+ 12645us) service_pool.cc:224] Handling call
0709 13:15:15.073542 (+ 54689us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.238093 (+164551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.238158 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.307645 (+ 69487us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.078439 (+770794us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:16.078518 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.078682 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.241225 (+162543us) raft_consensus.cc:1685] finished
0709 13:15:16.241250 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.241426 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35426,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":302,"thread_start_us":629616,"threads_started":1,"wal-append.queue_time_us":867396}
I20260709 13:15:16.263154 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2005) took 1942 ms. Trace:
I20260709 13:15:16.164048 30630 rpcz_store.cc:276] 0709 13:15:13.598167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:14.156718 (+558551us) service_pool.cc:224] Handling call
0709 13:15:14.157055 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:14.157299 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:14.157670 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 13:15:16.078043 (+1920373us) log.cc:844] Serialized 14 byte log entry
0709 13:15:16.107747 (+ 29704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:16.110784 (+  3037us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:16.113189 (+  2405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.124985 (+ 11796us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.135248 (+ 10263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3553,"prepare.queue_time_us":501470,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":102,"raft.queue_time_us":16694,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":31371,"thread_start_us":1723496,"threads_started":2,"wal-append.queue_time_us":1348542}
I20260709 13:15:16.267691 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:16.266510 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46469.2us
W20260709 13:15:16.267588 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61815.5us
I20260709 13:15:16.216239 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:16.285336 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.288372 31367 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:16.272114  2024 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:16.219128  1726 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:15:16.263825 30555 rpcz_store.cc:276] 0709 13:15:14.279816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.457966 (+1178150us) service_pool.cc:224] Handling call
0709 13:15:15.475273 (+ 17307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.690534 (+215261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.690571 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.691251 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.130921 (+439670us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:16.131008 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.221944 (+ 90936us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:16.222020 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.222081 (+    61us) raft_consensus.cc:1685] finished
0709 13:15:16.222095 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.222368 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23891,"raft.queue_time_us":25380,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"spinlock_wait_cycles":438674688,"thread_start_us":429769,"threads_started":1,"wal-append.queue_time_us":504922}
I20260709 13:15:16.301851 31378 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:16.217866 30703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:16.284044  2228 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:16.309188  2115 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:16.309762  2228 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.310330  2228 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:16.311795  2194 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.312427  2194 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.315564  2194 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.317727  2279 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:16.318832  1606 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.319802  2232 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:16.320756  2232 consensus_queue.cc:1243] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:15:16.321090 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 64910.5us
I20260709 13:15:16.321204  2281 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.274098  2280 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:16.318584  2279 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.323324  2280 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:16.325186  2279 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:16.325675 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 502044us
I20260709 13:15:16.322136  2281 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:16.327080  2290 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:16.327251 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:16.332188  2265 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:15:16.332654  2290 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:16.333251  2290 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.333655  2290 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:16.307037 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 85609.5us
I20260709 13:15:16.335616 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:16.338009 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:16.340734  2216 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:16.341207  2216 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.341640  2216 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:16.341741  2122 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:16.279088  2199 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.343853  2125 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:16.344457  2125 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:16.344497  2199 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.345077  2281 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:16.338954  2286 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:16.338853 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:16.319437  2273 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:15:16.321350  2232 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:16.347154 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.347405 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.348734 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:16.348960 30631 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:16.350054  2046 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:16.350445 30633 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:16.352098 30703 raft_consensus.cc:606] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:16.354257 30626 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:16.355687 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:16.338218  2278 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:16.417420 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2004) took 2137 ms. Trace:
I20260709 13:15:16.356758 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.354138  2111 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:16.401540 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:16.401106 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:16.359848 31367 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:16.300395 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:16.361681  2121 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:15:16.408435 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 100341us
I20260709 13:15:16.409112 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:16.453758 31383 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:16.319631  1606 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.416411 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.459820 30624 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:15:16.418818  2041 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.758s	user 0.017s	sys 0.000s
I20260709 13:15:16.412640  2286 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.453373  2281 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:16.451974 30559 rpcz_store.cc:276] 0709 13:15:14.279536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.086673 (+807137us) service_pool.cc:224] Handling call
0709 13:15:15.140035 (+ 53362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.224934 (+ 84899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.224973 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.354664 (+129691us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.000225 (+645561us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:16.000301 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.000435 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.417199 (+416764us) raft_consensus.cc:1685] finished
0709 13:15:16.417222 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.417358 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21230,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":22142,"thread_start_us":642253,"threads_started":1,"wal-append.queue_time_us":808419}
I20260709 13:15:16.485327 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.416981  2232 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:16.447075  2194 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.453393  2111 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:16.500938 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.504860 30629 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:16.510234 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 109260us
I20260709 13:15:16.511646 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:16.513036  2046 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:16.513556  2046 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.514024  2046 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:16.513238 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:16.519040  1726 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:16.519598  1726 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:16.519994  1726 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:16.523365  2232 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.525063  2232 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:16.527447 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:16.528276 31389 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16. Re-sending same reply.
I20260709 13:15:16.529691  2169 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.531085 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:16.539968  1726 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:16.456825  2205 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:15:16.503129 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.481876  2024 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:16.407387  2115 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:15:16.563465 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1258) took 1106 ms. Trace:
W20260709 13:15:16.560431 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 211797us
I20260709 13:15:16.560671 31367 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:16.566879 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.564361 30624 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:16.487325  2286 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:16.579257 30630 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:16.568006 31394 rpcz_store.cc:276] 0709 13:15:15.477841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.485194 (+  7353us) service_pool.cc:224] Handling call
0709 13:15:15.586872 (+101678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:15.601670 (+ 14798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:15.601703 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:15.602373 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.149474 (+547101us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:16.149545 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.149653 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.563204 (+413551us) raft_consensus.cc:1685] finished
0709 13:15:16.563231 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.563392 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4574,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":7401,"thread_start_us":538983,"threads_started":1,"wal-append.queue_time_us":612391}
I20260709 13:15:16.549718  1606 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:16.565474  2201 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.538s	user 0.012s	sys 0.003s
I20260709 13:15:16.595007 30706 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:16.595701 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:16.542558  1726 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:16.596982  1726 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:16.598861  1489 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:16.600664  1489 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:16.600819  2279 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:16.601253  1489 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:16.601325  2279 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:16.504559  2194 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.602571  2279 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:16.483776  2041 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:16.580142 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:16.605973 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.530468  2169 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.629134 30624 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:15:16.637677 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.604502  2279 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:16.673020 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.545159  2216 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:16.605070  2194 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.545249  2046 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:16.685825 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.690982 30624 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:15:16.687542 30705 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
W20260709 13:15:16.693454 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 134316us
I20260709 13:15:16.699156 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.705958 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.698459 30555 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:16.706928 30703 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:16.605273  2131 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:16.724112 30556 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:16.728349 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.726688 30706 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:16.729698 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.744661 30557 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:16.602607  1489 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:16.724067  2154 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:16.721875  2194 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:16.755322 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.754177 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:16.759266 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.759334 30554 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:16.763212 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 459666us
W20260709 13:15:16.756168 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.766196 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:16.766805 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.711225  2290 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:16.768409 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 246223us
I20260709 13:15:16.749292 31383 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:16.768358 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.768584 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.801354 30551 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:16.757470  2154 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:16.808751 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.809837 30556 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:16.817344 30630 raft_consensus.cc:1217] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.5->[6.6-10.8]   Dedup: 6.6->[8.7-10.8]
W20260709 13:15:16.824101  2296 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:15:16.825649 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 537540us
I20260709 13:15:16.827011 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.831100 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1452) took 1016 ms. Trace:
I20260709 13:15:16.811510 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.821293 30630 pending_rounds.cc:85] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 6
W20260709 13:15:16.835666 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 434168us
I20260709 13:15:16.837517 30627 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:16.801354  2104 raft_consensus.cc:2764] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:15:16.758754  2228 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:16.811494  2281 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:16.831605 31391 rpcz_store.cc:276] 0709 13:15:15.814928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.257159 (+442231us) service_pool.cc:224] Handling call
0709 13:15:16.257568 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.258160 (+   592us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:16.258209 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.258808 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.602527 (+343719us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:16.602586 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.602705 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.830846 (+228141us) raft_consensus.cc:1685] finished
0709 13:15:16.830870 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.831031 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1711,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"thread_start_us":343192,"threads_started":1,"wal-append.queue_time_us":447899}
I20260709 13:15:16.834765 30630 pending_rounds.cc:107] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:15:16.851840 30630 raft_consensus.cc:2889] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:16.848493 30703 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:16.852051 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.858950  2154 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:16.860448  2154 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:16.863554  2024 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:16.864890 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.866048 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:16.863603  2046 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:16.866947 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:16.822600  2125 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:16.879148 30703 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:16.843109  2015 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:16.846184  2295 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.207s
I20260709 13:15:16.883569 30554 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:16.889938 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:16.891494 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51728.3us
I20260709 13:15:16.891001 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:16.915478 30702 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:16.892944 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.839480  2198 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:15:16.899282 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.890647 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.853431  2170 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:15:16.891898 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.937523 30703 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:16.894068 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:16.946331 30701 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:16.888381  2299 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:16.947983 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:16.890554  2125 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:16.883625  2216 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:16.952479 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 261428us
I20260709 13:15:16.974992 30698 raft_consensus.cc:2410] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:16.972147 30700 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:15:16.888670  2041 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:16.981731 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:16.983156 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1453) took 1167 ms. Trace:
I20260709 13:15:16.978381 30704 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:16.925009  2304 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:16.990245 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:16.983752 31361 rpcz_store.cc:276] 0709 13:15:15.815323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.262922 (+447599us) service_pool.cc:224] Handling call
0709 13:15:16.271603 (+  8681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.280120 (+  8517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.280172 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.289643 (+  9471us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.646935 (+357292us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:16.647005 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.647119 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:16.982821 (+335702us) raft_consensus.cc:1685] finished
0709 13:15:16.982855 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.983069 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8793,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":5557,"thread_start_us":319780,"threads_started":1,"wal-append.queue_time_us":428158}
I20260709 13:15:16.977697 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1423) took 1466 ms. Trace:
I20260709 13:15:16.938184  2170 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:16.971323  2280 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:16.888556  2305 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.102s
W20260709 13:15:16.997020 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64557.3us
I20260709 13:15:16.998178 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:16.999469 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:17.000770 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1454) took 1185 ms. Trace:
I20260709 13:15:17.002367 31396 rpcz_store.cc:276] 0709 13:15:15.815679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.273026 (+457347us) service_pool.cc:224] Handling call
0709 13:15:16.285456 (+ 12430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:16.286035 (+   579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:16.286082 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:16.308262 (+ 22180us) log.cc:844] Serialized 26 byte log entry
0709 13:15:16.726809 (+418547us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:16.726863 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.726946 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:17.000416 (+273470us) raft_consensus.cc:1685] finished
0709 13:15:17.000449 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.000687 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1342,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":40932,"thread_start_us":309708,"threads_started":1,"wal-append.queue_time_us":634127}
W20260709 13:15:17.008517 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 181835us
I20260709 13:15:16.985884  2169 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:17.009128 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:17.010203 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.001446  2310 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:17.000874  2131 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:16.992274 30632 rpcz_store.cc:276] 0709 13:15:15.511019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:15.511273 (+   254us) service_pool.cc:224] Handling call
0709 13:15:15.511741 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:15.512349 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:16.214513 (+702164us) log.cc:844] Serialized 14 byte log entry
0709 13:15:16.706067 (+491554us) log.cc:844] Serialized 14 byte log entry
0709 13:15:16.945995 (+239928us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:16.946086 (+    91us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:16.977317 (+ 31231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:16.977433 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:16.977609 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":294922,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":571241,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":9461,"thread_start_us":1314277,"threads_started":4,"wal-append.queue_time_us":322743}
I20260709 13:15:17.054188 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:17.022864  2154 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:17.064903 30556 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:17.064904 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.066694 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.067418 30557 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:17.072101 30551 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
W20260709 13:15:17.079344 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 301310us
I20260709 13:15:17.059682  2125 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:17.072932  2154 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.084452  2046 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.103451 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:17.134604 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 148793us
I20260709 13:15:17.033604  2309 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:17.107240 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 109223us
I20260709 13:15:17.137537 30627 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:17.118682 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:17.055142  2232 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:17.099536  1606 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:17.115363 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:17.131222 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.132778 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:17.115914 30631 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:15:17.082136 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46808.3us
I20260709 13:15:17.023288  2199 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:15:17.151559 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.062564  1726 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.130712 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.006937  2280 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:17.080564 30523 connection.cc:256] RPC call timeout handler was delayed by -1.8374243880000449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 13:15:17.164143 30523 connection.cc:256] RPC call timeout handler was delayed by -1.1247654070000408s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15300000000000002s for a response
W20260709 13:15:17.157227 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.152621 30701 raft_consensus.cc:2410] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:15:17.134321  2111 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.580s	user 0.005s	sys 0.005s
I20260709 13:15:17.114964  2125 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:17.124334  2313 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:15:17.161430 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:17.147032  2300 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.278s	user 0.000s	sys 0.001s
I20260709 13:15:17.176661 30702 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:17.075433  2312 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:17.178561 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.179651 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.125754  2154 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:17.149031  2232 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.153826  1606 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.161156  2309 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.187173 31361 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:17.188582 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.160866  2199 consensus_queue.cc:1243] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:17.162302  2115 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:17.192531 31396 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:17.209229 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:17.210119 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:17.176012 30732 heartbeater.cc:646] Failed to heartbeat to 127.21.5.190:45121 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.5.190:45121 timed out after 15.000s (SENT)
I20260709 13:15:17.051430  2194 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:17.196691 30632 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:17.195972  2232 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:17.180640  2111 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.159850  2306 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:17.180861  2125 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:17.202754 30558 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:17.203068 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.200798 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:17.241449 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:17.213317 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:17.220850  2314 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.141s	user 0.001s	sys 0.002s
I20260709 13:15:17.221716 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.246593 30550 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:17.222362  2294 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.362s	user 0.001s	sys 0.001s
I20260709 13:15:17.223434 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.250480 30551 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:17.250731 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 244875us
I20260709 13:15:17.183364  2316 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:17.200506  1606 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:17.193816  2228 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:17.232332  2286 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.005s
I20260709 13:15:17.148784  2046 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.234668  2111 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.070976  2310 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.196653  2312 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:17.239144  2115 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:17.281782 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.102998  2170 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.196589  1489 raft_consensus.cc:2764] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:15:17.247861 30702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:17.287787 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.288237 30702 raft_consensus.cc:606] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:17.288645 30702 raft_consensus.cc:614] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:17.170905  1726 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:17.251400 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.254869 31361 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:15:17.256170 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72736.3us
I20260709 13:15:17.261826  2198 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:17.262645  2308 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:17.262830  2104 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:15:17.263793 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63901.9us
I20260709 13:15:17.264636  2309 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.264845 30701 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:17.265842 30705 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:17.269029 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:17.236276 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 178067us
I20260709 13:15:17.237726 30557 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:17.283263 30626 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:17.251876  2281 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:17.289043 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.309335  2198 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:17.316565 30627 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:17.314258  2281 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.317158  2281 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:17.317337 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.317692 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.245827  2024 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:15:17.293915  2322 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:15:17.297916 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.321120  2322 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:17.298813  2170 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:15:17.321734 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.322817 30626 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:17.322329  2322 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.323948 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.324832 30626 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:17.299888 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.326081  2319 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:17.327232  2303 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:17.327775  2303 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:17.328640 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.328771  2303 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.329442 30631 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:17.330281  2303 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:17.331087  2308 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:17.314292  2121 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.332373  2121 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.318511 30627 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:17.337520 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51281us
I20260709 13:15:17.298592 30554 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:17.298126 30555 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:17.315588 30551 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:17.341595 30628 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:15:17.308622 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61340.5us
I20260709 13:15:17.312598 30625 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 13:15:17.343158 30625 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 5
I20260709 13:15:17.343495 30625 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:15:17.343955 30625 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:17.325767 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.357374 30626 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
W20260709 13:15:17.366887  2279 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.682s	user 0.004s	sys 0.010s
I20260709 13:15:17.368253 30706 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:17.354275 30550 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:17.371979 30626 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:17.331579  2308 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:17.375219 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:17.376554 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63576.9us
I20260709 13:15:17.378916 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.377731 30705 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:17.316658  2198 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:17.281270  2286 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:17.383737 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.367950  2104 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:17.393265 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.410667 30629 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:17.395738 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.411051 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:17.388608 30631 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:17.394475 30627 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:15:17.413751 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56265.1us
W20260709 13:15:17.396209  2307 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.414s	user 0.004s	sys 0.000s
I20260709 13:15:17.421340 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:17.425282 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:17.425977 30701 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:17.426934 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:17.427677 30702 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:17.438858 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.334843  2320 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.077s
W20260709 13:15:17.440254 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68364.1us
W20260709 13:15:17.440944 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 141206us
I20260709 13:15:17.452206 30554 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:17.419355  2279 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.473793  2046 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.484915  2046 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:17.484709  2279 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.486840 30704 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 13:15:17.487213 30704 pending_rounds.cc:85] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 8
I20260709 13:15:17.487478 30704 pending_rounds.cc:107] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:15:17.487574  2279 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.409667  2115 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.489494 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:17.473940  2320 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:15:17.483862  2216 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:17.487746 30704 raft_consensus.cc:2889] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:17.398851  2169 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:15:17.447201  2321 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:15:17.494341 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:17.502211 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106595us
I20260709 13:15:17.512176 30629 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:15:17.514391 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71297.7us
I20260709 13:15:17.490427 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:17.517210 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.524586 30703 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:17.528208 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.534011 30631 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:15:17.534474 30705 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:17.536953 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.538204 30629 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:17.539913 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.540759 30626 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:17.548036 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.551682 30702 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:17.553579 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.554240 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:17.556080 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.502808  1726 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.571844 30627 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:17.556113 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:17.560026 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.535946  2220 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:17.562925  2320 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:17.567202 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.568475 30551 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:17.397980  2194 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.578722 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.579784 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:17.513640  2216 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:17.595331 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.600006 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:17.540666  2323 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:17.550482  2280 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:17.597510 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.556694  2316 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:17.600096 30705 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:17.588593 30699 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:17.605573 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.603044  2170 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:17.608943 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 145646us
I20260709 13:15:17.589462  1726 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:17.583146  2220 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:17.627535  2220 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.553438  2169 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:15:17.588253  2320 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:17.629515 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.636234  2345 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:15:17.636556 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:17.504207  2308 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:17.631217 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 116928us
I20260709 13:15:17.650406  2341 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:15:17.655781  1726 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:17.661353  1606 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.650875 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:17.653527 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.572285  2198 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.639868  2320 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:17.666371 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 129051us
I20260709 13:15:17.653585 30707 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:15:17.649972 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.655051 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.639075  2312 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:17.653661  2332 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:15:17.654479  2308 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:17.689217 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 129336us
I20260709 13:15:17.646957  2220 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:17.690359 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.695930  2346 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:17.651372  2339 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:17.711910 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:17.698581  2338 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:15:17.698274  2216 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.702323 30550 pending_rounds.cc:85] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:15:17.713829 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.649902  2347 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:17.716189 30558 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:17.720454 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.721534 30707 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:17.720407 30550 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:15:17.722991 30550 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:17.730221  2279 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:17.732349  1489 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:17.732893  1489 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:17.734150  1489 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.736497  1489 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:17.745307  2317 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:17.746851  2228 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:17.747365  2228 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:17.748633  2228 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.751029  2228 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:17.771829 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:17.652699  2170 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
W20260709 13:15:17.705308  2330 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.309s	user 0.000s	sys 0.002s
I20260709 13:15:17.713232  2341 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:17.789026  2354 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:17.712206  2339 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:17.795215 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:17.688851  2309 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
W20260709 13:15:17.787669 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 175555us
I20260709 13:15:17.717268  2312 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:17.716555  2286 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:17.712330  2280 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.794059  2354 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:17.783224  2345 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:17.801008 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:17.712630  2232 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:17.793972  2125 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.542s	user 0.014s	sys 0.000s
I20260709 13:15:17.786713  2347 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:17.806697  2354 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:17.818964  2343 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:15:17.820883  2343 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:17.786257 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 231753us
I20260709 13:15:17.825069  1606 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:17.798817  2339 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:17.827076 30703 raft_consensus.cc:2368] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:17.827821 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 736) took 1106 ms. Trace:
I20260709 13:15:17.834527  1606 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:17.834823  2232 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.817301  2125 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:17.823810  2216 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:17.843016  2345 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:17.870218 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46618.2us
I20260709 13:15:17.845732  2220 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:17.870010  1606 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:17.921219 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69566.1us
I20260709 13:15:17.927605  1606 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:17.929456 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:17.930828  2319 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:17.931252  2319 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:17.931645  2319 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:17.933547 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:17.915477 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 143798us
W20260709 13:15:17.863548  2340 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:15:17.916389 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 202685us
I20260709 13:15:17.917632  2323 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:17.853351 30624 rpcz_store.cc:276] 0709 13:15:16.721347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.727837 (+  6490us) service_pool.cc:224] Handling call
0709 13:15:16.751105 (+ 23268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:17.073409 (+322304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.073461 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:17.174247 (+100786us) log.cc:844] Serialized 26 byte log entry
0709 13:15:17.285132 (+110885us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:17.285190 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.285292 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:17.827502 (+542210us) raft_consensus.cc:1685] finished
0709 13:15:17.827540 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:17.827735 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130438,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":373665,"threads_started":2,"wal-append.queue_time_us":187620}
I20260709 13:15:17.919021  2367 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:17.859799  2354 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:17.923573  2355 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:17.924552 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:17.925096 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 617) took 1178 ms. Trace:
I20260709 13:15:17.952390 30706 rpcz_store.cc:276] 0709 13:15:16.746856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.756215 (+  9359us) service_pool.cc:224] Handling call
0709 13:15:17.924808 (+1168593us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b5400050854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:17.924969 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307090816}
W20260709 13:15:17.916512 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.944514 30705 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:15:17.954036 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.943662 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.823491  2286 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:15:17.817004  2280 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:17.954530 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.856652  2121 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:17.941653  2220 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:17.951560  2041 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.956593  2367 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:17.958192 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.965831 30553 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:17.958966 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:17.968858 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.966859 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:17.971306 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.971985 30556 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:17.976289 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:17.957850  2355 consensus_queue.cc:1243] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:17.972601  2121 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:17.982888 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 204587us
I20260709 13:15:17.985472 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:17.987159 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 337295us
I20260709 13:15:17.966477  2341 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:17.986269 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:17.986268 31391 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:17.993803 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.995937 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:17.996802 30670 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.505s (SENT)
I20260709 13:15:17.997308 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:17.850795  2347 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:17.998454  2370 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:17.998823  2343 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:17.999207  2368 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:17.999905 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.000247  2368 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:18.001248  2350 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.255s	user 0.003s	sys 0.000s
I20260709 13:15:18.001348 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.002749  2312 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:17.941980  2345 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:17.999329  2343 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:17.965862  2316 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:15:17.990875  2170 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:17.985085  2333 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:18.007761 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.010038 30553 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:18.008332 30707 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:17.973820  2041 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:17.997900 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:17.989032  2121 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:17.998301  2220 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:18.015556  2104 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.005s
W20260709 13:15:18.016436  2349 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.323s	user 0.000s	sys 0.002s
I20260709 13:15:18.018095  2367 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:17.998344  2366 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:15:18.019539  2369 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:18.003796  2122 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
W20260709 13:15:18.027856 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.613s (SENT)
I20260709 13:15:18.030157  2316 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:18.002789  2365 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:18.013736  2343 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:18.037595 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43919.6us
I20260709 13:15:18.018114  2368 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:18.044456 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.020934  2323 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:18.034662  2121 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:18.043635  2303 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.699s	user 0.007s	sys 0.008s
I20260709 13:15:18.035898  2322 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:18.061480  2316 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.066262 30556 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:18.043098  2104 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:18.069953 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.056828  2360 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:18.034366  2170 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:18.059783 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.840s (SENT)
W20260709 13:15:18.088779 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 110456us
W20260709 13:15:18.090504 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.097635 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:18.099227 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 54896.1us
I20260709 13:15:18.100047 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:18.086186  2369 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:18.089223  2316 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:18.070964  2312 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:15:18.116173 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56405.1us
W20260709 13:15:18.177593 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.875s (SENT)
W20260709 13:15:18.095736  2281 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.778s	user 0.018s	sys 0.001s
I20260709 13:15:18.093848  2115 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:18.097486 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:18.095399  2125 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:18.213831 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.183828 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.105631  2303 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:15:18.219307 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 214565us
I20260709 13:15:18.094835  2304 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 13:15:18.174891  2199 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.061177  2341 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.174142  2322 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:18.180558  2333 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:18.240103 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.212663  2115 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:18.240381 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 60356.8us
I20260709 13:15:18.181819  2354 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:18.249595 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.220327  2303 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:18.243844  2320 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.544s	user 0.008s	sys 0.011s
I20260709 13:15:18.258520 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.214782  2281 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:18.259596 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:18.247994  2345 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:18.271565 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:15:18.273342  2308 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.001s
I20260709 13:15:18.276893 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.257195  2303 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:18.262243  2373 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 13:15:18.325703 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 321328us
W20260709 13:15:18.283501  2376 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
W20260709 13:15:18.258443  2374 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:15:18.284533 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.340838 31383 raft_consensus.cc:2410] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:15:18.283620  2170 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:18.313639 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.254768  2194 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:18.323578 31378 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:18.244503  2199 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.343519 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:18.345772 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 168224us
W20260709 13:15:18.346493 30670 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.962s (SENT)
W20260709 13:15:18.349824 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.350772  2378 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:18.351389 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.352267 31394 raft_consensus.cc:2376] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15. Re-sending same reply.
I20260709 13:15:18.353569 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.354157 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.354321 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.354462 31394 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:18.355386 30624 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:18.355243 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.356045 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.356135 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.353642  2199 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:18.355113 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.362232 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.358230 30626 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:18.364719 30633 raft_consensus.cc:2368] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:15:18.364933 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.366849 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:18.362015  2228 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.610s	user 0.007s	sys 0.007s
I20260709 13:15:18.369141 30626 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:18.371332 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.373275 30627 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:18.368746  2378 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:18.376842 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.377813 31383 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:18.377743  2378 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.378978 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.380264 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.382234 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.370483  2388 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:15:18.384768 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.392571 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 632) took 1261 ms. Trace:
I20260709 13:15:18.392992  2281 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:18.395869 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.265617  2320 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.381712 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.396992 31394 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
W20260709 13:15:18.405634 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.407143 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.393142 30700 rpcz_store.cc:276] 0709 13:15:17.131679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.131830 (+   151us) service_pool.cc:224] Handling call
0709 13:15:17.132270 (+   440us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:17.209020 (+ 76750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.209055 (+    35us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:17.227360 (+ 18305us) log.cc:844] Serialized 78 byte log entry
0709 13:15:17.489282 (+261922us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b18002f19a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9
0709 13:15:17.490373 (+  1091us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18002f19a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:17.681539 (+191166us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:17.681590 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.844777 (+163187us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b18002f19a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:18.015582 (+170805us) spinlock_profiling.cc:243] Waited 14.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:18.392148 (+376566us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b540008b454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:18.392229 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.392281 (+    52us) raft_consensus.cc:1685] finished
0709 13:15:18.392294 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.392477 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10320,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":6024,"spinlock_wait_cycles":223590016,"thread_start_us":190179,"threads_started":1,"wal-append.queue_time_us":464900}
I20260709 13:15:18.409123 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:18.409687 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:18.410511 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:18.410290 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 175245us
I20260709 13:15:18.411271 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.411711 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.410260 30626 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:18.416179 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.418324 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:18.418447 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 54218.1us
I20260709 13:15:18.420962 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.422194 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.422673 30557 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:18.422318 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.424244 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.424998 31383 raft_consensus.cc:2376] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13. Re-sending same reply.
I20260709 13:15:18.425184 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.426204 30628 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:18.428865 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.430083 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.430738 31394 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:18.430837 30631 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:18.432252 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.409135  2169 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:18.438004 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 91545.5us
I20260709 13:15:18.440136 31383 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 13:15:18.440815 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.998s (SENT)
I20260709 13:15:18.441296 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.418922  2388 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:18.429701  2170 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:18.445567 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.445670 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.448886 30705 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:18.449328 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.451370 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.453238 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.436435  2347 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:18.460942 30670 leader_election.cc:336] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.216s (SENT)
I20260709 13:15:18.402890  2333 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.439904  2366 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.455530 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.482019 31388 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:18.488026 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.496057 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.501814 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.475677  2322 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:18.470376  2383 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:18.453755  2388 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:18.494414  2216 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.003s
I20260709 13:15:18.444943  2169 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.533839 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:18.536095 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.540480 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:18.541312 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.541551 30700 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:18.536888 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.528702 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.542630 30705 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:18.540055 30559 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:18.537340 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.512828 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.528122 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.534264 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:18.538386 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99404us
I20260709 13:15:18.555555 30628 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:18.560629  2169 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.561599 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.543165 30552 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:18.564018 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.564670 31394 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:18.567865 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:18.568393 30552 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:18.510154  2368 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:18.536583  2389 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:15:18.493876  2366 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.582470 30627 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:18.511687  2387 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:18.565907  2360 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:18.591452 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.593542 31394 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:18.588286 30624 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:18.571691  2320 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.595198 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.600545 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.601624 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.602257 30705 raft_consensus.cc:2376] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12. Re-sending same reply.
W20260709 13:15:18.605118  2343 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.518s	user 0.003s	sys 0.011s
I20260709 13:15:18.605692  2343 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.607303 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.608830 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.609591 30558 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:18.615520 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.609656  2366 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.534585  2121 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.600052 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:18.616233  2305 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:18.618593  2310 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.628227 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.628996 30705 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:18.638469 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.643571 30551 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:18.535445  2388 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:18.626392  2339 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.000s
I20260709 13:15:18.591493  2385 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.198s
I20260709 13:15:18.651366 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1377) took 1169 ms. Trace:
I20260709 13:15:18.625429  2305 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:18.624536  2343 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:18.655367  2319 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.723s	user 0.014s	sys 0.000s
W20260709 13:15:18.656339 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 89643.2us
I20260709 13:15:18.575871  2368 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.651878 30702 rpcz_store.cc:276] 0709 13:15:17.482337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.573225 (+ 90888us) service_pool.cc:224] Handling call
0709 13:15:18.361571 (+788346us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:15:18.361752 (+   181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.362143 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.362173 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.362958 (+   785us) log.cc:844] Serialized 26 byte log entry
0709 13:15:18.446756 (+ 83798us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:18.515043 (+ 68287us) log.cc:844] Serialized 14 byte log entry
0709 13:15:18.519762 (+  4719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.519870 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.647585 (+127715us) raft_consensus.cc:1685] finished
0709 13:15:18.651104 (+  3519us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.651288 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":139529,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":3343,"raft.queue_time_us":1245,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":6949,"spinlock_wait_cycles":5692032,"thread_start_us":113636,"threads_started":2,"wal-append.queue_time_us":181878}
I20260709 13:15:18.658337 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.663434 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.664124 30552 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:15:18.665290 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.670363 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.676306 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:18.681169  1606 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:18.682601 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1478) took 1046 ms. Trace:
I20260709 13:15:18.656095  2316 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.683133  2198 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:18.685081 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.663797  2400 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:18.686451 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.686574 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.687383 30628 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:18.687892 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.688293 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.688680 30701 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:18.689116 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:18.689595 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.689982 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.690304 30701 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:18.690526 30628 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:18.691215 30671 leader_election.cc:336] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 2.053s (SENT)
I20260709 13:15:18.691715 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.692060  2352 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:18.692512  2352 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:18.693405  2352 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.656633  2339 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:18.695013  2322 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:18.695469 30558 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:18.696744  2339 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:18.683145 31364 rpcz_store.cc:276] 0709 13:15:17.636001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.637533 (+  1532us) service_pool.cc:224] Handling call
0709 13:15:17.665033 (+ 27500us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:17.665566 (+   533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:17.665616 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:17.687071 (+ 21455us) log.cc:844] Serialized 52 byte log entry
0709 13:15:18.681984 (+994913us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b5400163b20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806e546b8 00007f1806edd92b 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c
0709 13:15:18.682060 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:18.682118 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:18.682195 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.682240 (+    45us) raft_consensus.cc:1685] finished
0709 13:15:18.682253 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:18.682515 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15605,"raft.queue_time_us":13185,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":2288,"spinlock_wait_cycles":40167168,"thread_start_us":217112,"threads_started":1,"wal-append.queue_time_us":213164}
I20260709 13:15:18.697870 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.698172  2121 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.698858  2368 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.688591 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.694485  2352 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:18.685076  2316 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.702574 30557 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:15:18.706919 30627 raft_consensus.cc:2376] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 6. Re-sending same reply.
W20260709 13:15:18.712673 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47494.3us
I20260709 13:15:18.713733 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.715014  2104 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:18.717303 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.718690 30550 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:18.719000 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:18.719522  2400 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:18.720381 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.723053 30705 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:18.724766 30628 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:18.726498 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.728142 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.731302 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.732172 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.732908 31383 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:18.733973 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:18.734382  2325 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.030s	user 0.011s	sys 0.000s
I20260709 13:15:18.734905 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:18.735009  2325 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.011s	sys 0.000s
I20260709 13:15:18.735466 31383 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:18.736575 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.737703 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.738579 30628 raft_consensus.cc:2368] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:15:18.732326 31367 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:18.729748 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.742161 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.748175 30557 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:15:18.747527 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:18.747679 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:18.753687 31364 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:18.754762 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.750344 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.755349 31364 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:18.724941  2322 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:15:18.737331  2385 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:18.730544  2316 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.758581 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.662498  2319 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.741119  2368 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:18.765122 30557 pending_rounds.cc:85] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:18.768206 30557 pending_rounds.cc:107] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:15:18.768471 30557 raft_consensus.cc:2889] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:18.738988  2194 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:15:18.770259 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.772037 31383 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:18.763681  2385 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.766024  2395 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:18.727231  2305 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:18.780270  2194 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:18.780778  2305 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:18.783574  2369 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.775889 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.792131 30628 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:18.786337 31389 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:18.809981 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:18.778484  2402 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 13:15:18.779227  2385 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:18.780306  2401 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:15:18.768352  2316 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.784207  2369 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:18.837795 31383 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:18.837845 30628 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:18.839190 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:18.840852 31364 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:18.843169 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:18.843158 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104902us
I20260709 13:15:18.846529 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.847250 30628 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:18.829818  2339 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:18.862300 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.819065  2303 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:18.865157 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.871799 30628 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
W20260709 13:15:18.867975 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 150756us
I20260709 13:15:18.874457 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:18.876518 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:18.856606  2339 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:18.878252 31361 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:18.877578 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:18.886251 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.884253  2319 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:18.897854 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.898010 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.898604 30628 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:18.900184  2412 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:15:18.904132  2316 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:18.904493  2393 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:18.910360 31394 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:18.885499  2339 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:15:18.914431 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 184747us
I20260709 13:15:18.915768 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:18.918802 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.914333 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:18.904264  2369 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:18.926285 31396 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:18.910689  2322 consensus_queue.cc:1243] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:15:18.938755 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67234.5us
I20260709 13:15:18.940615 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.918385  2345 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:18.942015  2345 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:18.937538 30628 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:18.944281 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:18.918562  2393 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:18.917871  2405 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:18.949252 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.942447  2345 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:18.955348 30628 raft_consensus.cc:2410] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:15:18.974817  2369 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:18.979924  2359 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.015s	user 0.004s	sys 0.000s
W20260709 13:15:18.982770 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:18.974861  2115 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:18.987298  2420 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:18.989037 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1476) took 1354 ms. Trace:
W20260709 13:15:19.016047 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75409.9us
I20260709 13:15:19.016816 31378 rpcz_store.cc:276] 0709 13:15:17.635186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.635340 (+   154us) service_pool.cc:224] Handling call
0709 13:15:18.988687 (+1353347us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400460ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:18.988913 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477032320}
I20260709 13:15:18.992619 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:18.993628  2322 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:18.993708  2413 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:18.994791  2421 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:18.922669  2308 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:15:18.995328  2417 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:18.995654  2415 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:15:18.996098  2335 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.438s	user 0.019s	sys 0.038s
I20260709 13:15:18.996992 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:18.997445  2419 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:18.998821  2303 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:19.001493  2343 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:18.926939  2360 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:19.004794 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:19.041348 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:19.020730 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:19.019502 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:19.022764 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 78560.7us
I20260709 13:15:18.948359  2400 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:18.955111  2393 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:19.031759 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:19.035336 30631 raft_consensus.cc:2376] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13. Re-sending same reply.
I20260709 13:15:19.021653  2369 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:19.029567  2131 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.032793  2415 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.042387 30556 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:19.023754 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:19.032624  2417 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.044376 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:19.087764 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172055us
I20260709 13:15:19.065171 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:19.064083 30624 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:19.109794 31383 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:19.065435 30553 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:15:19.108626 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88179.3us
I20260709 13:15:19.031075  2420 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.072476 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 737) took 2315 ms. Trace:
I20260709 13:15:19.119160 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:19.121951 30630 rpcz_store.cc:276] 0709 13:15:16.763700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:16.767373 (+  3673us) service_pool.cc:224] Handling call
0709 13:15:16.801691 (+ 34318us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:17.159674 (+357983us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:17.159729 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:17.185973 (+ 26244us) log.cc:844] Serialized 52 byte log entry
0709 13:15:17.375341 (+189368us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:17.375398 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:17.375490 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:18.995264 (+1619774us) spinlock_profiling.cc:243] Waited 4.27 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:19.072030 (+ 76766us) raft_consensus.cc:1685] finished
0709 13:15:19.072059 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.072379 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5683,"raft.queue_time_us":129302,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"spinlock_wait_cycles":9825536,"thread_start_us":399465,"threads_started":2,"wal-append.queue_time_us":221987}
I20260709 13:15:19.135462 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:19.129261 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:19.152768 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:19.185561 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54348.5us
W20260709 13:15:19.189572 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 247370us
I20260709 13:15:19.190781 30553 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 13:15:19.106264 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:19.193171 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:19.050720  2344 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.223s	user 0.002s	sys 0.019s
I20260709 13:15:19.189656 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:19.194190 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:19.042042  2308 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:15:19.194406 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84634.4us
W20260709 13:15:19.196921 30670 leader_election.cc:336] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.211s (SENT)
W20260709 13:15:19.199671 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 155414us
I20260709 13:15:19.199877 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.152168  2322 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:19.195448  2424 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:19.047323  2419 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:19.195219 30630 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:19.201807 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:19.071240  2316 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.238237 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:19.240023 30556 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:19.242238 30628 raft_consensus.cc:2410] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:19.237607 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:19.199296  2344 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.496s	user 0.005s	sys 0.023s
I20260709 13:15:19.056340  1606 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:19.203414 30553 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:19.246501 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:19.084637  2280 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:19.085810  2415 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:19.079020  2131 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:19.244278 31396 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:19.057840  2232 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.181s
I20260709 13:15:19.230679  2424 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.252242  2305 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:15:19.257819  2104 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.542s	user 0.014s	sys 0.000s
I20260709 13:15:19.256124 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:19.171978  2393 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:19.277769 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1372) took 1797 ms. Trace:
I20260709 13:15:19.078123  2345 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.278257 30704 rpcz_store.cc:276] 0709 13:15:17.480341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.480525 (+   184us) service_pool.cc:224] Handling call
0709 13:15:17.480896 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:17.531534 (+ 50638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.277452 (+1745918us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b54002c4cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:19.277692 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12525952}
I20260709 13:15:19.070856  2343 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:19.202116  2308 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:19.228848  2413 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:15:19.274739 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:19.252948  2198 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:19.252308  2417 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.259454  2355 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:19.259891  2322 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:19.265375  2280 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:19.312973  2360 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.078651  2369 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:19.194846  2383 raft_consensus.cc:2764] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:15:19.248813  2316 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.273515  2424 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:19.305004 31396 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:19.305217 31378 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:19.310413 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:19.278443  2305 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:19.285528  2319 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:19.287757  2104 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:19.251428  2290 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:19.260185  1606 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.358724 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:19.444774  2348 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:19.277490  2420 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.429330  2431 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:19.430908  2417 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:19.432358  2345 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:19.360716  2355 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:19.470602 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:19.436681  2343 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:19.438218  2131 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:19.312464  2421 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:19.361840 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1287) took 1033 ms. Trace:
I20260709 13:15:19.475519 31389 rpcz_store.cc:276] 0709 13:15:18.328647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.376110 (+ 47463us) service_pool.cc:224] Handling call
0709 13:15:19.361483 (+985373us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b54002f8d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:19.361711 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921436032}
W20260709 13:15:19.465898  2433 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 13:15:19.457125  2306 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:19.471757 30704 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:19.462214  2385 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.616s	user 0.007s	sys 0.008s
I20260709 13:15:19.455727  2322 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.452524  2169 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:19.449725  2360 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.459920  2368 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:15:19.451512  2198 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:19.481643 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 292122us
I20260709 13:15:19.633100 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1728) took 1352 ms. Trace:
I20260709 13:15:19.633484 31391 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:19.465113  2305 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:19.634203 30705 rpcz_store.cc:276] 0709 13:15:18.280798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.729975 (+449177us) service_pool.cc:224] Handling call
0709 13:15:18.753935 (+ 23960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.907359 (+153424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.907410 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.934844 (+ 27434us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.266120 (+331276us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:19.267207 (+  1087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.267326 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.632836 (+365510us) raft_consensus.cc:1685] finished
0709 13:15:19.632864 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.633029 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34584,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":6947,"thread_start_us":440534,"threads_started":2,"wal-append.queue_time_us":323529}
I20260709 13:15:19.467200  2348 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:19.474640  2355 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:19.467700  2431 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:19.640695 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.643481 31391 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:15:19.648466  2306 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:19.680915 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 387837us
I20260709 13:15:19.636505  2385 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.691025 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.696291 30557 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:15:19.697160 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:19.697927 30633 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:19.470300  2308 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.598732  2419 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.703872 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:19.710384 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.638949  2169 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.717360 30553 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:15:19.694283  2378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.313s	user 0.006s	sys 0.021s
I20260709 13:15:19.719643 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:19.720585 30556 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:19.658012  2232 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:15:19.693053  2131 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.599691  1606 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:19.724301  2431 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:19.726984 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:19.754477 30559 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:15:19.727226 30627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1445) took 1237 ms (client timeout 1608 ms). Trace:
I20260709 13:15:19.664105  2306 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.728365 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 716) took 1275 ms. Trace:
I20260709 13:15:19.755738 30550 rpcz_store.cc:276] 0709 13:15:18.452483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.452635 (+   152us) service_pool.cc:224] Handling call
0709 13:15:18.452971 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.719145 (+266174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.727998 (+1008853us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54000f0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:19.728284 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610078592}
W20260709 13:15:19.739645  2339 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.813s	user 0.015s	sys 0.000s
W20260709 13:15:19.714581  2395 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.925s	user 0.000s	sys 0.017s
I20260709 13:15:19.762825  2322 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:19.763453  2395 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:19.740525 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:19.764220  2385 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:19.764979 30553 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:19.765858 31364 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:19.743439 30557 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:19.760958  2131 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:19.755475 30627 rpcz_store.cc:269] 0709 13:15:18.489437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.594585 (+105148us) service_pool.cc:224] Handling call
0709 13:15:19.726961 (+1132376us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54004a9cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:19.727133 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":239635328}
I20260709 13:15:19.769166 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:19.769757 30627 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
W20260709 13:15:19.748944  2378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.006s	sys 0.021s
I20260709 13:15:19.762431  2339 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.770330  2131 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:19.771193 30627 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:19.771870 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1450) took 1111 ms. Trace:
I20260709 13:15:19.772260  2339 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:19.705232  2345 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:19.776108 30558 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:15:19.779023 30552 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:15:19.780839 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 728) took 1251 ms. Trace:
I20260709 13:15:19.781208 30552 rpcz_store.cc:276] 0709 13:15:18.529907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.673959 (+144052us) service_pool.cc:224] Handling call
0709 13:15:19.778746 (+1104787us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.779540 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.780603 (+  1063us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001a9850. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:19.780771 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539775872}
I20260709 13:15:19.784121  2232 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:19.784869  2115 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:19.772202 30627 rpcz_store.cc:276] 0709 13:15:18.660799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.770538 (+1109739us) service_pool.cc:224] Handling call
0709 13:15:19.770862 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.771592 (+   730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.771812 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:19.778198 30554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2020) took 2582 ms (client timeout 2919 ms). Trace:
W20260709 13:15:19.792325 30554 rpcz_store.cc:269] 0709 13:15:17.195320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.195650 (+   330us) service_pool.cc:224] Handling call
0709 13:15:19.777962 (+2582312us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b54001a9854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:19.778111 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120587520}
W20260709 13:15:19.794569 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.018s	sys 0.032s
W20260709 13:15:19.795176 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.018s	sys 0.032s
W20260709 13:15:19.795641 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50361.1us
I20260709 13:15:19.722595  2400 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:19.717516 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.747936  2416 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:15:19.808043 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 611123us
I20260709 13:15:19.791218  2308 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:19.800324  2400 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:19.806236 30556 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:19.716519  2438 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:15:19.830144 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:19.830122 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 112675us
W20260709 13:15:19.857668 30671 leader_election.cc:336] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.608s (SENT)
I20260709 13:15:19.858256 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.858384 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.700532  2348 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.800354  2346 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:15:19.822705  2416 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:19.830520  2316 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:19.883579 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.869045 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 986) took 1246 ms. Trace:
I20260709 13:15:19.886421 30631 rpcz_store.cc:276] 0709 13:15:18.622593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.249918 (+627325us) service_pool.cc:224] Handling call
0709 13:15:19.259480 (+  9562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.260795 (+  1315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.260848 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:19.261584 (+   736us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.486574 (+224990us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:19.782678 (+296104us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b4800550158. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:15:19.847295 (+ 64617us) log.cc:844] Serialized 14 byte log entry
0709 13:15:19.847627 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.847715 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:19.847765 (+    50us) raft_consensus.cc:1685] finished
0709 13:15:19.868482 (+ 20717us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.868902 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19567,"raft.queue_time_us":4001,"raft.run_cpu_time_us":1755,"raft.run_wall_time_us":523747,"spinlock_wait_cycles":7445760,"thread_start_us":452471,"threads_started":2,"wal-append.queue_time_us":392930}
W20260709 13:15:19.863616 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.817854  2424 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:19.896241 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:19.866712  2400 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:19.873626  2447 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.894582  2104 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:19.902490  2348 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:19.894291  2198 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:19.928133 30631 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:19.903616  2440 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:15:19.908499  2220 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:19.935446 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 757) took 1178 ms. Trace:
I20260709 13:15:19.911556  2395 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:19.911289  2385 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:19.937050 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79432.9us
I20260709 13:15:19.911800  2360 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:19.922112  2417 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:15:19.926432  2436 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
W20260709 13:15:19.922844 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.025s	sys 0.070s
W20260709 13:15:19.944890 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.026s	sys 0.070s
W20260709 13:15:19.945417 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 670852us
I20260709 13:15:19.937988 30631 rpcz_store.cc:276] 0709 13:15:18.757038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.923726 (+1166688us) service_pool.cc:224] Handling call
0709 13:15:19.927776 (+  4050us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.934867 (+  7091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.935374 (+   507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:19.944198 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:19.946641 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 644) took 1217 ms. Trace:
I20260709 13:15:19.950380 30704 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:19.904821  2419 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:19.944228  2417 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:19.967314 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:19.929886  2447 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:19.977723 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 227738us
W20260709 13:15:19.984056 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:19.984884 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:19.945629  2385 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:19.948239  2395 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.011919 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.001427  2378 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:19.957000  2104 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:15:19.970541  2169 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.015290  2378 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.015848  2369 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:20.016306  2369 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:20.016930  2448 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:15:20.017259  2369 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.018847  2369 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.019788 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:20.020382  2390 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.541s	user 0.000s	sys 0.003s
I20260709 13:15:20.020581 30554 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
W20260709 13:15:20.022234  2452 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:15:20.029178 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.001335  2441 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.031155  2441 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.033864  2441 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:20.034633  2443 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:15:20.030220 30707 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 6.
I20260709 13:15:20.015322 31396 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:20.040102 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:19.996148  2316 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:20.054531 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 756) took 1309 ms. Trace:
I20260709 13:15:20.055423 30627 rpcz_store.cc:276] 0709 13:15:18.745394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.798466 (+1053072us) service_pool.cc:224] Handling call
0709 13:15:19.799005 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.799327 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:19.948229 (+148902us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.053615 (+105386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.053878 (+   263us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:20.054037 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.054247 (+   210us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.054436 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10807,"prepare.queue_time_us":170195,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":52,"thread_start_us":233646,"threads_started":2}
I20260709 13:15:20.073736 30627 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:20.073823 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 989) took 1439 ms. Trace:
W20260709 13:15:20.071991  2449 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:15:20.050303  2316 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.050073  2378 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.090866 30627 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:15:20.104008 30523 leader_election.cc:400] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:15:20.106484 30523 leader_election.cc:403] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:15:20.107704 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:20.109611 30627 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:15:20.102728  2316 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:20.099963 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.079968 30633 rpcz_store.cc:276] 0709 13:15:18.634439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.756117 (+1121678us) service_pool.cc:224] Handling call
0709 13:15:19.756609 (+   492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.756925 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:19.930826 (+173901us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.073093 (+142267us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.073246 (+   153us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:20.073425 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.073544 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.073742 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20199,"prepare.queue_time_us":190805,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3139,"thread_start_us":265047,"threads_started":2}
I20260709 13:15:20.114243 30627 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:20.135387 31396 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:20.141707 30633 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:20.141960 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.132334  2440 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:20.114778  2450 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:15:20.143076 31396 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 5.
I20260709 13:15:20.080720  2417 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:20.140196  2415 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.873s	user 0.017s	sys 0.000s
I20260709 13:15:20.155216  2447 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.124722  2367 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:20.168524 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 129309us
W20260709 13:15:20.190215 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.155198  2385 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:20.170305  2305 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.006s
I20260709 13:15:20.145511  2440 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.211321 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.215420 30555 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:20.212025 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 758) took 1443 ms. Trace:
I20260709 13:15:20.216197 30631 rpcz_store.cc:276] 0709 13:15:18.759966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.951498 (+1191532us) service_pool.cc:224] Handling call
0709 13:15:19.951847 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.952108 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:19.964843 (+ 12735us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.202112 (+237269us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.202419 (+   307us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.202494 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:20.202565 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.202688 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.202881 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8064,"prepare.queue_time_us":80018,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":24848,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":6249,"spinlock_wait_cycles":11930496,"thread_start_us":212336,"threads_started":2,"wal-append.queue_time_us":232833}
I20260709 13:15:20.192800  2104 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:20.221428  2104 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.224382  2104 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.194536  2131 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:20.194854  2471 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:15:20.164660  2415 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.167186  2367 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.219259  2417 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:19.581336 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:20.196923  2387 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:15:20.253966 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.235306 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.219238  2115 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.255261  2378 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:20.255877  2378 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:20.255463  2115 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.256976  2378 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.258004  2115 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.258689  2115 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.259182  2115 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.261348  2115 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.261842  2115 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.262084 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.206450  2305 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.265506  2281 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:20.268242 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 160659us
I20260709 13:15:20.270901 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.269089 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.271540 30557 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:20.273574 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.276515 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.277877 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.247601  2131 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:20.262338  2115 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.283377  2115 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.274493  2306 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:20.291077  2306 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.291491  2306 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:20.292717  2415 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.299317  2305 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.264602  2471 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:20.263824  2395 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.263375  2445 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:15:20.315207 30557 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:20.320921 30705 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:20.299805  2131 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:20.325860 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.314915  2115 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:15:20.329523  2459 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:15:20.329661  2115 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:20.331319  2468 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:15:20.330835  2115 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:20.319339  2355 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.666s	user 0.014s	sys 0.001s
I20260709 13:15:20.326694  2305 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
W20260709 13:15:20.328620  2470 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:15:20.327406 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.313118  2480 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:15:20.334981 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.337677 31396 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:20.336582 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:20.339594 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61768.7us
I20260709 13:15:20.339754 30557 raft_consensus.cc:2376] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9. Re-sending same reply.
I20260709 13:15:20.340402 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.320751  2477 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:20.309635  2471 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.341671 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:20.330521  2041 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:20.343552  2469 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 13:15:20.351850 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.344293  2475 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.357173  2475 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.360270  2352 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.346570  2478 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:20.363075  2419 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:20.364881  2477 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:20.365655  2400 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.363085  2367 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.368296  2471 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.355362  2348 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:20.369364  2348 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.369732  2348 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:20.369913  2383 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:20.370368  2383 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:20.371376  2383 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.336802  2355 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.380746  2104 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:20.381340  2104 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:20.374279  2306 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.385051  2306 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.385461  2424 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.385825 30370 maintenance_manager.cc:643] P 9e2d0e882b2940d8b150869e4b2b2e79: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.337s	user 0.181s	sys 0.089s Metrics: {"bytes_written":30168,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":200980,"dirs.run_cpu_time_us":515,"dirs.run_wall_time_us":11802,"drs_written":1,"lbm_read_time_us":64624,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":1,"lbm_write_time_us":385973,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":4,"lbm_writes_gt_100_ms":2,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":192678,"threads_started":1,"update_count":19}
I20260709 13:15:20.390443 30371 maintenance_manager.cc:419] P 9e2d0e882b2940d8b150869e4b2b2e79: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7642 bytes on disk
I20260709 13:15:20.368497  2400 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:20.404413 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.404870 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:20.405884 30557 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:20.360996  2352 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.396844 30370 maintenance_manager.cc:643] P 9e2d0e882b2940d8b150869e4b2b2e79: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.005s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":387,"lbm_reads_lt_1ms":4}
W20260709 13:15:20.409439  2342 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 2.716s	user 0.008s	sys 0.008s
W20260709 13:15:20.410558 30703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1376) took 2928 ms (client timeout 1952 ms). Trace:
W20260709 13:15:20.368443  2403 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.709s	user 0.003s	sys 0.000s
I20260709 13:15:20.397542  2445 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:20.350543  2290 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:15:20.413277 30703 rpcz_store.cc:269] 0709 13:15:17.482049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.566793 (+ 84744us) service_pool.cc:224] Handling call
0709 13:15:20.410180 (+2843387us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b540062ff54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.410406 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547966848}
I20260709 13:15:20.426815 30632 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:20.429975 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.430330 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.430819 30631 raft_consensus.cc:2368] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:15:20.431070 30633 raft_consensus.cc:2376] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12. Re-sending same reply.
I20260709 13:15:20.418282 30557 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:20.432510 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.432919 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1435) took 2732 ms. Trace:
I20260709 13:15:20.433312 30551 rpcz_store.cc:276] 0709 13:15:17.701012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.701962 (+   950us) service_pool.cc:224] Handling call
0709 13:15:17.702240 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.644219 (+941979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.432681 (+1788462us) spinlock_profiling.cc:243] Waited 941 ms on lock 0x7b5400a27bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:20.432857 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2164318592}
I20260709 13:15:20.434049 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.429311 31396 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:20.435904  2440 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.437871 30671 leader_election.cc:400] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 7 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8.
I20260709 13:15:20.438687  2440 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:20.435940 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.440605  2421 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:20.441093 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.441160  2421 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:20.442421 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.443063 31396 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:20.443691 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.442544  2421 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:20.416553 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63045.3us
W20260709 13:15:20.447600  2444 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.638s	user 0.003s	sys 0.000s
I20260709 13:15:20.448957 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 985) took 1826 ms. Trace:
I20260709 13:15:20.449359 30630 rpcz_store.cc:276] 0709 13:15:18.622238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.245518 (+623280us) service_pool.cc:224] Handling call
0709 13:15:19.257577 (+ 12059us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.264841 (+  7264us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.264903 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:19.279217 (+ 14314us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.646198 (+366981us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:19.646263 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.646348 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.446920 (+800572us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:20.448609 (+  1689us) raft_consensus.cc:1685] finished
0709 13:15:20.448641 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.448881 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":270,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":7819,"spinlock_wait_cycles":11593472,"thread_start_us":327637,"threads_started":1,"wal-append.queue_time_us":415645}
W20260709 13:15:20.451800  2457 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.371s	user 0.002s	sys 0.000s
I20260709 13:15:20.452018  2290 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:20.453256  2290 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:20.454497 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 721) took 1927 ms (client timeout 2284 ms). Trace:
W20260709 13:15:20.456612  2308 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.590s	user 0.009s	sys 0.005s
W20260709 13:15:20.457263 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 109225us
I20260709 13:15:20.457589  2194 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:20.458108  2194 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:20.459311  2339 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:20.459205  2194 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.460474  2194 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:20.461431  2339 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:20.463050 31388 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:15:20.372602  2383 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:20.457151 31367 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1295) took 2098 ms (client timeout 2180 ms). Trace:
W20260709 13:15:20.456660 30555 rpcz_store.cc:269] 0709 13:15:18.527432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.527637 (+   205us) service_pool.cc:224] Handling call
0709 13:15:20.454208 (+1926571us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b54001a9d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.454423 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3838584576}
I20260709 13:15:20.466126 31389 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:20.466856 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:20.465039 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:20.476512  2352 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.463901  2445 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.478376 30558 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:20.478902  2339 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:15:20.479313  2479 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 13:15:20.482335 31367 rpcz_store.cc:269] 0709 13:15:18.359319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.394797 (+ 35478us) service_pool.cc:224] Handling call
0709 13:15:20.456942 (+2062145us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54004b05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.457086 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548643072}
I20260709 13:15:20.400391  2355 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.494390 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.495601 31367 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:20.492664 30630 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:15:20.502357 30597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 1.533s (SENT) [suppressed 44 similar messages]
I20260709 13:15:20.445501  2421 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:20.382246  2104 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.515131 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1387) took 1892 ms. Trace:
I20260709 13:15:20.514844  2279 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:20.515618 30701 rpcz_store.cc:276] 0709 13:15:18.622596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.764332 (+141736us) service_pool.cc:224] Handling call
0709 13:15:18.767581 (+  3249us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:18.911416 (+143835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:18.911465 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:18.916573 (+  5108us) log.cc:844] Serialized 26 byte log entry
0709 13:15:20.124135 (+1207562us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18001637e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:20.220246 (+ 96111us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:20.265046 (+ 44800us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.265309 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.265406 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.514719 (+249313us) raft_consensus.cc:1685] finished
0709 13:15:20.514858 (+   139us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.515057 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69617,"prepare.queue_time_us":28948,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":129214,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":8671,"spinlock_wait_cycles":42143872,"thread_start_us":238680,"threads_started":2,"wal-append.queue_time_us":1404559}
I20260709 13:15:20.517099  2348 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.519717  2348 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:20.504668 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.525259  2480 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:20.526656  2480 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:20.527921 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.530910 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.534137 31367 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:20.535802  2194 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.536569  2194 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.539768  2194 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.499926  2316 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.527715  2488 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:20.542269 31396 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:20.535233  2339 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:20.574323 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:20.580719 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:20.584348 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.585196 30700 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:15:20.585779 30700 pending_rounds.cc:85] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 4
I20260709 13:15:20.585806 30631 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:15:20.586190 30700 pending_rounds.cc:107] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:15:20.586566 30700 raft_consensus.cc:2889] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:20.587432 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:20.588013 30703 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:20.590972 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:20.592738 31367 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:20.592549 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.596064 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.548763  2488 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:20.585623  2486 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:15:20.598179 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.598904 30703 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:20.577471  2279 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:20.546113  2358 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:20.600944 30701 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:20.557973  2131 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:15:20.607889 30629 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 7. Current term is 8. Ops: []
I20260709 13:15:20.608652 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1427) took 2256 ms. Trace:
I20260709 13:15:20.604666  2279 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:20.591660  2339 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:20.598986  2488 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.618757 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:20.605022 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.619594 30627 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:20.609032 30629 rpcz_store.cc:276] 0709 13:15:18.352117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.354723 (+  2606us) service_pool.cc:224] Handling call
0709 13:15:20.607422 (+2252699us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:15:20.607682 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.608364 (+   682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.608591 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2834304}
I20260709 13:15:20.601671  2486 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.604254  2348 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.632151  2486 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:20.632596  2348 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.634903  2194 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:20.635553 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:20.636520 30629 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:20.635944  2194 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.634972 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:20.642856  2496 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:15:20.650424 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1388) took 2020 ms. Trace:
I20260709 13:15:20.651109 30706 rpcz_store.cc:276] 0709 13:15:18.630276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.999844 (+369568us) service_pool.cc:224] Handling call
0709 13:15:19.000241 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.052596 (+ 52355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.052639 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:19.053309 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.260104 (+206795us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:19.260159 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.789840 (+529681us) spinlock_profiling.cc:243] Waited 42.5 ms on lock 0x7b54005a28d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:19.789891 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.619414 (+829523us) spinlock_profiling.cc:243] Waited 9.34 ms on lock 0x7b2400329d38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2efb0 00007f1806aa5488 00007f1806aa5be8 00007f1806a7d398 00007f1806a7876a 00007f1806a77bb6
0709 13:15:20.650086 (+ 30672us) raft_consensus.cc:1685] finished
0709 13:15:20.650126 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.650338 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9318,"raft.queue_time_us":12925,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":108,"spinlock_wait_cycles":119240704,"thread_start_us":206076,"threads_started":1,"wal-append.queue_time_us":307076}
I20260709 13:15:20.658721 30626 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:20.662930  2445 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.626451  2279 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.665045 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.647531  2339 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:20.666298 31383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1367) took 1900 ms (client timeout 1939 ms). Trace:
W20260709 13:15:20.666914 31383 rpcz_store.cc:269] 0709 13:15:18.766123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.838869 (+ 72746us) service_pool.cc:224] Handling call
0709 13:15:20.666067 (+1827198us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54000032d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.666217 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485225216}
I20260709 13:15:20.670635 30558 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
W20260709 13:15:20.672920 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.607620  2358 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:20.680542  2441 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:20.682287  2041 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.682819  2194 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.683060  2041 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.685491 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.688964 30627 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:20.669991  2316 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.701900  2041 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.702327 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:20.702504  2475 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:20.703773  2409 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.970s	user 0.000s	sys 0.008s
W20260709 13:15:20.705025  2394 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 2.381s	user 0.004s	sys 0.000s
I20260709 13:15:20.717821 30557 raft_consensus.cc:1217] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Deduplicated request from leader. Original: 13.8->[14.9-14.9]   Dedup: 14.9->[]
I20260709 13:15:20.691645  2445 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.710649  2348 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.725912 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.728394 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1371) took 1881 ms. Trace:
I20260709 13:15:20.728936 31364 rpcz_store.cc:276] 0709 13:15:18.846952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.076713 (+229761us) service_pool.cc:224] Handling call
0709 13:15:19.077040 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.766325 (+689285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.728012 (+961687us) spinlock_profiling.cc:243] Waited 689 ms on lock 0x7b5400372dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:20.728307 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1583676160}
I20260709 13:15:20.718761 30557 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:15:20.705964 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 647) took 1475 ms. Trace:
I20260709 13:15:20.730538 30705 rpcz_store.cc:276] 0709 13:15:19.230235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.637366 (+407131us) service_pool.cc:224] Handling call
0709 13:15:19.655073 (+ 17707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.321411 (+666338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.705705 (+384294us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54001f22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:20.705897 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530775936}
I20260709 13:15:20.731756 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.733814 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:20.736778 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:20.739832 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:20.743345 30698 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:15:20.737902  2348 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:15:20.722440  2486 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.755980  2486 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.757661 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:20.759223 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1370) took 1931 ms. Trace:
I20260709 13:15:20.759738 31378 rpcz_store.cc:276] 0709 13:15:18.828353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.042658 (+214305us) service_pool.cc:224] Handling call
0709 13:15:19.135908 (+ 93250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.463253 (+327345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.758952 (+1295699us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b540002e754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:20.759156 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":388843776}
I20260709 13:15:20.738783  2024 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:15:20.736941  2490 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.762042  2490 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.762071  2431 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:20.766032  2358 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.736156  2339 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:20.737466  2503 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:20.770323 30701 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:20.772960 30558 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:15:20.745573  2405 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:15:20.727898  2445 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.769245  2280 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:15:20.770187  2368 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:20.775333  2368 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.776111  2445 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:20.739148  2480 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:15:20.777331 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1484) took 2512 ms. Trace:
I20260709 13:15:20.778016  2503 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.778544  2503 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:20.777777 31361 rpcz_store.cc:276] 0709 13:15:18.265110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.265305 (+   195us) service_pool.cc:224] Handling call
0709 13:15:18.267067 (+  1762us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:18.878929 (+611862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.777035 (+1898106us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b54002b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:20.777236 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1405188480}
I20260709 13:15:20.773046  2471 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:20.781426  2471 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:20.781769  2471 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:19.947171 30702 rpcz_store.cc:276] 0709 13:15:18.729319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.486844 (+757525us) service_pool.cc:224] Handling call
0709 13:15:19.487399 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.487745 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:19.524573 (+ 36828us) log.cc:844] Serialized 14 byte log entry
0709 13:15:19.730273 (+205700us) spinlock_profiling.cc:243] Waited 1.09 ms on lock 0x7b18004285c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:19.946231 (+215958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:19.946292 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:19.946336 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.946414 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:19.946555 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35270,"prepare.run_cpu_time_us":562,"prepare.run_wall_time_us":881,"raft.queue_time_us":92,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3782,"spinlock_wait_cycles":2498944,"thread_start_us":215696,"threads_started":1}
I20260709 13:15:20.787236 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.764587  2490 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:20.793761 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.769881  2388 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:20.803274 30625 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:20.794212  2405 consensus_queue.cc:1243] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:20.801797  2490 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:20.812268  2388 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:20.812361 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.798501  2041 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:20.812549  2490 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.820755 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1365) took 2076 ms. Trace:
I20260709 13:15:20.825198 31394 rpcz_store.cc:276] 0709 13:15:18.744336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:18.915244 (+170908us) service_pool.cc:224] Handling call
0709 13:15:19.362485 (+447241us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:19.452943 (+ 90458us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:19.452999 (+    56us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:19.464718 (+ 11719us) log.cc:844] Serialized 78 byte log entry
0709 13:15:19.847112 (+382394us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:19.847198 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:19.847326 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:20.198054 (+350728us) raft_consensus.cc:1685] finished
0709 13:15:20.198088 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.820428 (+622340us) spinlock_profiling.cc:243] Waited 447 ms on lock 0x7b54002f8d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.820675 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47794,"raft.queue_time_us":47947,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":7111,"spinlock_wait_cycles":1027527552,"thread_start_us":306033,"threads_started":1,"wal-append.queue_time_us":468658}
I20260709 13:15:20.819314  2368 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.832337  2199 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:20.814420  2445 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.832548  2279 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.833185  2417 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.833112  2445 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.833441  2279 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.833835  2417 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.836105  2417 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.836659  2279 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.838781  2279 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.839643  2279 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.840693 30698 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:20.843328  2490 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.843618 30705 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:20.853657 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.854964 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.856057 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.857151 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.856848  2503 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.857681 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:20.844903 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.858299 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.861753 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.862943 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.863327 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.863934 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.864230 30559 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:20.865077 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:20.860126 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.852293 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.859972 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:20.860531 30698 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:20.845973  2279 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.817601  2041 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:20.846789 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.823474  2478 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:20.848721 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:20.885336 30707 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:20.850054 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.895210 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:20.897318 30628 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 987) took 2274 ms (client timeout 2688 ms). Trace:
I20260709 13:15:20.897127 30703 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:20.897541 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.899667 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:20.900429 31383 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:15:20.831945 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 649) took 1591 ms. Trace:
W20260709 13:15:20.899236 30628 rpcz_store.cc:269] 0709 13:15:18.622995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.019177 (+396182us) service_pool.cc:224] Handling call
0709 13:15:20.887044 (+1867867us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:20.897246 (+ 10202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458080640}
I20260709 13:15:20.901760 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.858707  2490 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:15:20.913055 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.913362 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.913932 31378 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:20.914194 31364 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:15:20.914973 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.836771  2445 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:20.868899 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:20.866374 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.870774 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.878962  2441 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:20.879913 30552 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:20.881264  2198 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:20.862958 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.882331 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.883023 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.883329 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:20.884106 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.932374 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.906486 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.932453 30551 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:20.851306  2488 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.893087  2478 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:20.918869 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.934968 31394 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:20.939342 31383 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:20.941309 30698 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:20.901759 30699 rpcz_store.cc:276] 0709 13:15:19.240365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.660650 (+420285us) service_pool.cc:224] Handling call
0709 13:15:19.693832 (+ 33182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:19.694104 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:19.713953 (+ 19849us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.787455 (+1073502us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b18001ab3c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:20.812054 (+ 24599us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.827851 (+ 15797us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.831373 (+  3522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:20.831438 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:20.831493 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:20.831637 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:20.831862 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19210,"prepare.queue_time_us":18370,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":59,"raft.queue_time_us":20064,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":22554,"spinlock_wait_cycles":12155904,"thread_start_us":22612,"threads_started":1}
I20260709 13:15:20.919775 30703 pending_rounds.cc:85] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 6
I20260709 13:15:20.942785 30703 pending_rounds.cc:107] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:15:20.943192 30703 raft_consensus.cc:2889] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:20.942831  2488 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:20.925546 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.927508 30550 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:20.896865  2169 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.932729 30556 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:20.884286 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:20.946837  2169 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:20.907483 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.949647  2169 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:20.949707  2478 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.951671 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.953148 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.950464  2169 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:15:20.953532  2478 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:20.954471 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:20.973289 30707 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:20.978340 30553 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:20.882499  2502 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:15:20.926774  2405 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:20.996933  2220 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:20.997378  2405 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:20.997874  2220 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.000023  2405 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.901502  2388 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:20.982790  2041 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.009040  2041 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:21.009299 30632 raft_consensus.cc:1217] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Deduplicated request from leader. Original: 10.6->[12.7-16.8]   Dedup: 12.7->[16.8-16.8]
I20260709 13:15:21.009946 30632 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: [12.7-16.8]
I20260709 13:15:20.918653  2492 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
I20260709 13:15:20.976143  2198 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:21.015641  2198 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.016382 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1394) took 1280 ms. Trace:
I20260709 13:15:21.016965  2198 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:21.017058 30704 rpcz_store.cc:276] 0709 13:15:19.736223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.938480 (+202257us) service_pool.cc:224] Handling call
0709 13:15:21.016139 (+1077659us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:21.016300 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5929728}
I20260709 13:15:21.020474  2393 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:20.976094  2480 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:20.993805 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:20.993003 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:20.975980  2506 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:15:20.992951 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.028040 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:21.029397 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.032823 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.036087 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.041006 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.044334 30704 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:21.043320 30706 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:21.043466 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.046484 30551 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:21.046936 30628 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:21.049669 30707 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:21.045773 30631 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:21.062252 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:21.064225 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.065663 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:21.042655 31361 raft_consensus.cc:2376] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14. Re-sending same reply.
I20260709 13:15:21.070143 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.071115 30698 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:21.075719 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.076844 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.075719 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.066380 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:21.079962 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.081183 30701 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:21.059985 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:21.084450 30628 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:20.928057  2386 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:15:21.092989 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:21.098704 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.193647 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.167176 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.271128 30625 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:21.235069 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 362546us
I20260709 13:15:21.099653 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.155586 30698 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:21.273307 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.179519 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.149724 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.140000 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:21.274567 30627 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:21.224486 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1372) took 2358 ms. Trace:
W20260709 13:15:21.222997 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.189281 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.108193 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:21.103953 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:21.277411 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 173535us
W20260709 13:15:21.255537 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.185552 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.281253 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:21.288983 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.290066 30706 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:21.276533 31388 rpcz_store.cc:276] 0709 13:15:18.865841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.255560 (+389719us) service_pool.cc:224] Handling call
0709 13:15:20.462738 (+1207178us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:20.463505 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.224093 (+760588us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54004b05d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:21.224362 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2740290816}
W20260709 13:15:21.222527  2512 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:15:21.304685 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:21.166414  2506 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:15:21.126241  2386 consensus_queue.cc:1243] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:21.124965  2503 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:21.059512  2368 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.245s
I20260709 13:15:21.319486 30552 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:21.289650  2280 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:21.339959 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 984) took 2718 ms. Trace:
I20260709 13:15:21.045123  2480 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.094156  2405 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:21.124369  2316 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:21.064054  2445 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.235s
I20260709 13:15:21.088376  2393 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:21.362578  2393 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:21.347389  2280 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.363339  2393 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:21.365365  2280 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:21.366060  2280 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:21.164162  2220 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:21.368482  2220 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:21.368938  2220 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:21.369333  2220 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:21.104674  2290 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:21.378294  2316 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:21.379523  2316 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.380304  2339 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:21.380818  2339 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:21.382681  2484 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:15:21.383291  2431 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 13:15:21.386243 30698 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:21.386041  2405 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.319000  2441 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:15:21.392982  2486 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.366459  2280 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.416147 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:21.420022  2405 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:21.343521  2378 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:21.294883  2194 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.293920  2306 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:21.341677 30624 rpcz_store.cc:276] 0709 13:15:18.621907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:19.231131 (+609224us) service_pool.cc:224] Handling call
0709 13:15:19.242707 (+ 11576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:19.478140 (+235433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:19.478190 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:19.584821 (+106631us) log.cc:844] Serialized 26 byte log entry
0709 13:15:19.895756 (+310935us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:20.011576 (+115820us) log.cc:844] Serialized 14 byte log entry
0709 13:15:20.011928 (+   352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.339474 (+1327546us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54001ea854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:21.339534 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:21.339585 (+    51us) raft_consensus.cc:1685] finished
0709 13:15:21.339688 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:21.339874 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4969,"prepare.queue_time_us":135959,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":213,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":43521,"spinlock_wait_cycles":540070656,"thread_start_us":332117,"threads_started":2,"wal-append.queue_time_us":324154,"wal-append.run_cpu_time_us":3148,"wal-append.run_wall_time_us":1099594}
I20260709 13:15:21.316373  2478 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:21.352626  2368 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:21.427817  2445 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:21.428296  2445 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:21.429193  2445 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.421201  2280 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:21.353328  2131 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:21.431912  2131 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:21.432942  2131 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:21.355914 30627 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:21.434300  2131 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:21.438239  2131 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:21.357220 30553 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
W20260709 13:15:21.304867 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 373773us
I20260709 13:15:21.339529  2520 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:21.446813  2194 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.447649  2520 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.448625 30707 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:21.423836  2306 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:21.451339  2306 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:21.205138  2471 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:21.453649  2471 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:21.356621  2480 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:21.455299 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.457620 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.458007  2198 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:21.418350 30701 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:21.458251 30707 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:21.422721  2378 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:21.486969  2378 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.488437  2378 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:21.399291  2486 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.450270  2520 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:21.485224  2041 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:21.492755  2486 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:21.508455  2522 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.135s	user 0.002s	sys 0.000s
W20260709 13:15:21.511721  2513 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 13:15:21.514218 30699 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:15:21.450214  2194 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:21.536840 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79314.8us
I20260709 13:15:21.438901 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.542419 30704 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:21.554344 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:21.554455 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 207760us
I20260709 13:15:21.558300 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.559298 30704 raft_consensus.cc:2376] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10. Re-sending same reply.
I20260709 13:15:21.560606 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.559372 30631 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:21.577351  2111 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:21.591406 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:21.592401 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 484327us
I20260709 13:15:21.594732 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.595556 30628 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:15:21.602578 30628 pending_rounds.cc:85] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 4
I20260709 13:15:21.602632 30704 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:15:21.602986 30628 pending_rounds.cc:107] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:15:21.603307 30628 raft_consensus.cc:2889] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:21.595391 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:21.552384  2405 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:21.593259 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.601135 30627 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:21.601430 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.613040 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.618927 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.625855 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.626201 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:21.627148 31361 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:21.630822  2486 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.631119 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:21.631217 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.631868 31383 raft_consensus.cc:2376] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12. Re-sending same reply.
I20260709 13:15:21.633361 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.634196 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.601311  2521 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:21.635334  2431 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:21.635931  2431 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:21.636996  2431 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.639153 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.599068  2345 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:21.640090  2345 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:21.640282 31364 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:21.640631  2345 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:21.641934 31364 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:21.625855 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.643393 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:21.644143 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:21.643723 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.644963 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.643435 30704 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:21.648027 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.649410 31389 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:15:21.649809 31389 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:21.650036 31389 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:15:21.650311 31389 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:21.652644 30624 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:21.659193 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.660259 31389 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:21.661630 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:21.666054 31367 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:21.652614 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:21.671613 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.616650  2413 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:21.679422 31388 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:21.635356  2521 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:21.675988  2520 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:21.687613  2475 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:21.688597 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.651664  2400 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:21.657135  2279 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:21.692902 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.618889  2518 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:21.670734  2290 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:21.711994 31364 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:21.657500  2386 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.688835  2490 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:15:21.665359  2488 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.577890  2339 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.676000  2378 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:21.714648 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 389061us
I20260709 13:15:21.700753  2521 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:21.720559 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.708285  2199 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:15:21.722338 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:21.677809  2041 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:15:21.727842 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:21.712563  2510 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.668s	user 0.006s	sys 0.000s
I20260709 13:15:21.687641  2306 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.702085  2440 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:15:21.717433  2198 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:21.744051 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:21.744154 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:21.723465  2518 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:21.747705 30556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.009s	sys 0.022s
I20260709 13:15:21.706411  2520 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:21.726984  2490 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:21.713116  2400 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.766716 30629 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
W20260709 13:15:21.767246 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 433399us
I20260709 13:15:21.768144 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:21.773077 30704 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:21.762585 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:21.794376 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:21.795342 31361 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:21.748250 30556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.009s	sys 0.023s
I20260709 13:15:21.749321 31394 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:21.741645  2378 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.775378 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:21.745617 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 464473us
I20260709 13:15:21.723227  2386 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:21.743171  2290 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:21.787449 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:21.806846 30552 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:21.807564 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:21.831211 30703 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:21.850872 30704 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:15:21.774748  2480 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:21.850630 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.853211 30558 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:21.919394  2386 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:21.854102 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.855593 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:21.855888 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:21.921562  2421 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.856478  2529 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.868355  2306 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:21.869728  2198 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:21.771358  2280 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:21.871898  2393 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.001s
I20260709 13:15:21.743180  2413 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:15:21.881829  2520 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:21.882979  2490 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.885967 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:21.887506  2518 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:21.890499  2358 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:21.887565 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:21.895102  2510 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:15:21.897116  2290 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:21.809990  2488 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:21.904441  2104 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:21.900074 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:21.915241 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 312560us
I20260709 13:15:22.000283 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:21.910209 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:21.903964 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 87802.6us
I20260709 13:15:22.005489 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:22.006444 30624 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:22.007719 31389 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:21.903544  2220 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.001s
I20260709 13:15:22.022017 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.023255 30633 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:22.026377 30557 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:21.952978  2041 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:15:21.980268 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94436.9us
I20260709 13:15:21.952838  2378 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:22.039420 30633 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:15:22.017400  2388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.025s	sys 0.049s
W20260709 13:15:22.041504  2388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.025s	sys 0.050s
I20260709 13:15:21.986961  2111 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:21.934293  2421 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:21.987399 30559 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:22.057013 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.017537  2527 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:21.900766  2400 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.029229 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:22.043130 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:22.075348 30705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.014s	sys 0.014s
I20260709 13:15:22.075697 31383 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:22.075881 30705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.014s	sys 0.015s
I20260709 13:15:22.118104 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:22.118958 30633 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:22.122812 30629 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:21.986265  2393 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:21.985306 31361 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:22.019160  2369 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:22.067906 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.133562 30630 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:22.069083 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47214.7us
I20260709 13:15:22.136157 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:21.947386  2383 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.021s
I20260709 13:15:21.979800 30555 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:22.060025 30701 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.016s
I20260709 13:15:22.034855  2308 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:22.043490  2367 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:22.069904  2111 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.026655  2220 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:22.142040 30701 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.004s	sys 0.017s
I20260709 13:15:22.030213  2198 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:15:22.027382  2488 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:22.140367  2383 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.022s
I20260709 13:15:21.952266  2529 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.067443  2527 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:22.148217 30626 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:22.143541  2397 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:22.159107  2526 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:15:22.161010  2280 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.166760  2431 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:22.123106  2360 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:22.148432 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:22.028158  2521 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.062934  2531 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.060148  2378 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.144814  2369 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:22.188545  2345 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.000s
I20260709 13:15:22.196367  2530 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:22.042727  2041 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.212575 31367 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:15:22.213240 31367 pending_rounds.cc:85] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:22.213739 31367 pending_rounds.cc:107] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:15:22.214057 31367 raft_consensus.cc:2889] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:22.175315 30629 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:22.144507  2316 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.180346 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 1748) took 1204 ms. Trace:
I20260709 13:15:22.214857 30699 rpcz_store.cc:276] 0709 13:15:20.975943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.980119 (+  4176us) service_pool.cc:224] Handling call
0709 13:15:22.172389 (+1192270us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b54003d27d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:22.180244 (+  7855us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075574528}
I20260709 13:15:22.147248  2111 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:22.144320  2367 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:22.163043  2393 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.192073 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:22.220881 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1408) took 1182 ms. Trace:
I20260709 13:15:22.221382 30700 rpcz_store.cc:276] 0709 13:15:21.038556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.089751 (+ 51195us) service_pool.cc:224] Handling call
0709 13:15:21.090271 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.104523 (+ 14252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.104554 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.141496 (+ 36942us) log.cc:844] Serialized 26 byte log entry
0709 13:15:21.689772 (+548276us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:21.689816 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.689890 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.220627 (+530737us) raft_consensus.cc:1685] finished
0709 13:15:22.220651 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.220810 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":26805,"thread_start_us":498620,"threads_started":1,"wal-append.queue_time_us":746289}
I20260709 13:15:22.224980 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.173878 31361 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:22.234858 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:22.253051 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.211248  2530 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.255683 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:22.258635 30704 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:22.255602 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.213186  2527 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.269083  2521 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.269325  2527 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:22.205634  2345 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.270941  2480 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:22.271611  2480 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:22.272097  2345 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.272625  2480 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.274317  2521 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:22.274973  2521 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.289757 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.290793 30707 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:22.292222 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.293314 30632 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:22.295027  2220 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.297591  2279 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:15:22.300773 31396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.000s	sys 0.036s
W20260709 13:15:22.301396 31396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.000s	sys 0.036s
I20260709 13:15:22.298259  2279 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:22.306265 30627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.010s	sys 0.026s
W20260709 13:15:22.309406 30627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.011s	sys 0.026s
I20260709 13:15:22.312075 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1380) took 2159 ms. Trace:
I20260709 13:15:22.316239 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.312621 31396 rpcz_store.cc:276] 0709 13:15:20.152566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.545515 (+392949us) service_pool.cc:224] Handling call
0709 13:15:20.545847 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.546674 (+   827us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:20.551016 (+  4342us) log.cc:844] Serialized 14 byte log entry
0709 13:15:21.705963 (+1154947us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b180003fd60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:22.311660 (+605697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.311737 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:22.311799 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.311866 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.312009 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4170,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":250,"spinlock_wait_cycles":10142464,"thread_start_us":595707,"threads_started":1}
I20260709 13:15:22.322992 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:22.246861  2360 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:22.317705 30704 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
W20260709 13:15:22.349017 31391 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.020s	sys 0.049s
W20260709 13:15:22.350886 31391 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.021s	sys 0.050s
I20260709 13:15:22.201604  2531 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.342238 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.357051 30629 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:22.304256  2530 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:22.347146 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:22.331188 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.019s	sys 0.031s
W20260709 13:15:22.349409 30706 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.014s	sys 0.038s
I20260709 13:15:22.312896  2388 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.220643  2316 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.313611  2520 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:22.376816 30706 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.015s	sys 0.038s
W20260709 13:15:22.377703 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.021s	sys 0.032s
W20260709 13:15:22.378437 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 657942us
I20260709 13:15:22.345221 31396 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:22.384999 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1489) took 2257 ms. Trace:
I20260709 13:15:22.378450 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1410) took 1336 ms. Trace:
I20260709 13:15:22.387255 30706 rpcz_store.cc:276] 0709 13:15:21.042046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.317366 (+275320us) service_pool.cc:224] Handling call
0709 13:15:21.317839 (+   473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.318161 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:21.777646 (+459485us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.377762 (+600116us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.377843 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:22.377953 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.378174 (+   221us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.378367 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":357394,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":124,"thread_start_us":1029411,"threads_started":2}
I20260709 13:15:22.288483  2367 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.338055  2400 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.393733 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:22.366282  2475 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.020s	sys 0.026s
I20260709 13:15:22.403980 30624 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:22.349748  2198 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:22.406929 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.020s	sys 0.052s
I20260709 13:15:22.385574 31391 rpcz_store.cc:276] 0709 13:15:20.127101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:20.263693 (+136592us) service_pool.cc:224] Handling call
0709 13:15:20.264073 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:20.265250 (+  1177us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:20.306371 (+ 41121us) log.cc:844] Serialized 14 byte log entry
0709 13:15:21.013698 (+707327us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b180002f3e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:21.507559 (+493861us) spinlock_profiling.cc:243] Waited 29.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:22.383490 (+875931us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.384038 (+   548us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.384163 (+   125us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:22.384296 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.384688 (+   392us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.384916 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73012,"prepare.queue_time_us":40248,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":2954,"raft.queue_time_us":174297,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"spinlock_wait_cycles":71048320,"thread_start_us":1263814,"threads_started":3,"wal-append.queue_time_us":1066391}
W20260709 13:15:22.417080 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.020s	sys 0.053s
I20260709 13:15:22.418159 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.418891 30633 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:22.425913 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:22.434386 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:22.384498  2393 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:22.435240 31394 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:22.419348  2475 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.023s	sys 0.027s
I20260709 13:15:22.435885 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:22.381170  2360 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:22.435853 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.430740  2367 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:22.461850 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:22.390765  2520 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.437316  2290 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:15:22.452677 30629 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:22.440281  2528 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.274s	user 0.001s	sys 0.001s
I20260709 13:15:22.392032  2388 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.444574  2405 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.396550  2279 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.415861  2400 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:22.507125 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45458.2us
W20260709 13:15:22.440207  2368 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.000s	sys 0.016s
I20260709 13:15:22.451256  2421 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.433324  2198 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:22.502030  2538 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:15:22.480192  2345 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:22.508734  2378 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.437388  2549 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:15:22.445237  2393 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:22.547349  2368 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.000s	sys 0.017s
W20260709 13:15:22.513099  2115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.000s	sys 0.023s
I20260709 13:15:22.501741  2521 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.549753 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 653) took 1517 ms. Trace:
W20260709 13:15:22.561239 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 532110us
I20260709 13:15:22.511461  2529 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:22.559875 31391 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:22.563479 31396 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:22.576027 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.552738 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:22.512410  2369 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.546008  2447 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:22.577069 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:22.575320 30705 rpcz_store.cc:276] 0709 13:15:21.031891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.032053 (+   162us) service_pool.cc:224] Handling call
0709 13:15:21.032485 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.032800 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:21.524970 (+492170us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.511730 (+986760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.511800 (+    70us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:22.511870 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.511972 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.549510 (+ 37538us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39118,"prepare.queue_time_us":610019,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":70,"raft.queue_time_us":73891,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":1291,"spinlock_wait_cycles":18591104,"thread_start_us":1408078,"threads_started":3,"wal-append.queue_time_us":866334}
I20260709 13:15:22.583285 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.583288 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:22.591938 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2052) took 1448 ms. Trace:
I20260709 13:15:22.580168 31394 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:22.559041  2280 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.587049 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:22.592629 30556 rpcz_store.cc:276] 0709 13:15:21.143746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.144396 (+   650us) service_pool.cc:224] Handling call
0709 13:15:21.165124 (+ 20728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.167701 (+  2577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.167737 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.189672 (+ 21935us) log.cc:844] Serialized 26 byte log entry
0709 13:15:21.796681 (+607009us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:15:21.796746 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.796851 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.591578 (+794727us) raft_consensus.cc:1685] finished
0709 13:15:22.591617 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.591835 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6081,"raft.queue_time_us":7503,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":6493,"thread_start_us":572956,"threads_started":1,"wal-append.queue_time_us":861321}
I20260709 13:15:22.558497  2279 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:22.599390 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.603340  2548 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:22.603973  2548 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:22.604321  2548 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:22.606190  2413 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:22.606854  2378 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:22.607636  2228 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:22.608852 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:22.614882 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.616657 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:22.616869  2280 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:22.621248  2428 raft_consensus.cc:2764] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:15:22.621930  2542 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:15:22.615114  2477 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.623656  2194 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:22.623476  2477 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:22.623952  2445 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:22.624333  2194 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:22.625492  2194 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.625990  2477 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:22.627542  2543 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 13:15:22.628538  2280 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.623346 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:22.630359  2544 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:15:22.630650  2441 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:22.631134  2441 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:22.632038  2441 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.633423  2441 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:22.633594 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.635051 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:22.635903 30557 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:22.635149 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.638973 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1413) took 1539 ms. Trace:
W20260709 13:15:22.629137  2537 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:15:22.639613 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1412) took 1548 ms. Trace:
I20260709 13:15:22.648670 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:22.607126  2535 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:22.639449 30701 rpcz_store.cc:276] 0709 13:15:21.099696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.505230 (+405534us) service_pool.cc:224] Handling call
0709 13:15:21.509522 (+  4292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.511915 (+  2393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.511954 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.512547 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.147750 (+635203us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:22.147797 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.147882 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.638700 (+490818us) raft_consensus.cc:1685] finished
0709 13:15:22.638729 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.638894 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1523,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":112,"thread_start_us":634832,"threads_started":1,"wal-append.queue_time_us":967886}
I20260709 13:15:22.645211 31394 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:15:22.638409 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.656096 31388 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:22.647754 30698 rpcz_store.cc:276] 0709 13:15:21.090868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.453558 (+362690us) service_pool.cc:224] Handling call
0709 13:15:21.454029 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:21.479826 (+ 25797us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:15:21.479870 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:21.480494 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 13:15:21.976971 (+496477us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:21.977032 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:21.977119 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.639380 (+662261us) raft_consensus.cc:1685] finished
0709 13:15:22.639404 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.639552 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25263,"raft.queue_time_us":26932,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":496016,"threads_started":1,"wal-append.queue_time_us":771855}
W20260709 13:15:22.575246  2115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.000s	sys 0.024s
I20260709 13:15:22.666326 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1328) took 1041 ms. Trace:
I20260709 13:15:22.516399  2552 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:22.649883 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:22.653637 30555 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:22.514808  2405 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.663149 30702 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:15:22.670012 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.664047 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.648993 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:22.644649  2104 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.619s	user 0.020s	sys 0.003s
I20260709 13:15:22.672792 30557 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:22.588776  2431 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:22.610452  2488 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.607667  2378 consensus_queue.cc:1243] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:22.627619  2413 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:22.702883 31388 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:22.680222 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:22.684894 30559 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:22.658042  2534 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:22.646587  2521 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:22.691128 31378 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:22.668903 31364 rpcz_store.cc:276] 0709 13:15:21.625029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.922545 (+297516us) service_pool.cc:224] Handling call
0709 13:15:21.923067 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:21.923367 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:22.443525 (+520158us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.665672 (+222147us) log.cc:844] Serialized 14 byte log entry
0709 13:15:22.665925 (+   253us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.665973 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:22.666036 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.666106 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.666243 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":113,"thread_start_us":687038,"threads_started":2}
W20260709 13:15:22.713246 30625 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.011s	sys 0.027s
I20260709 13:15:22.695778 30698 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:22.647229  2169 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.676307 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.654359  2471 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.630651  2280 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:22.696414 30699 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:22.653703  2358 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:15:22.700109  2539 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 13:15:22.726774 30703 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:22.728170 31388 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [15.9-16.10]
W20260709 13:15:22.737993 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 390978us
I20260709 13:15:22.682873  2490 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:22.714381 30625 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.012s	sys 0.028s
W20260709 13:15:22.729435 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49369.7us
W20260709 13:15:22.739821 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.740454 30698 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:22.708688  2558 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.642077  2447 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.744550  2431 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:22.742930 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:22.714605  2386 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:22.753098 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1468) took 1205 ms. Trace:
I20260709 13:15:22.753533 30630 rpcz_store.cc:276] 0709 13:15:21.547486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.549025 (+  1539us) service_pool.cc:224] Handling call
0709 13:15:21.549380 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.140722 (+591342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.752802 (+612080us) spinlock_profiling.cc:243] Waited 584 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:22.753027 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342805248}
I20260709 13:15:22.744740 30699 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:15:22.754485 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:22.708144  2369 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:22.661404  2547 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:15:22.723117  2531 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.641616  2360 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.723908  2559 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:22.767370 30705 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:22.730700  2521 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:22.732952  2316 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.776176  2400 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.777069  2400 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:22.751840  2558 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.781450  2558 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:22.723978  2535 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:15:22.797508 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:22.768998  2551 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:15:22.729765  2169 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.733354  2471 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.824183  2447 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.764990  2386 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.825191  2471 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.806136  2400 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:22.826376  2400 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:22.827430  2400 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.766274  2431 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:22.829515  2431 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:22.835103  2562 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:15:22.769836  2369 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.843765  2369 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:22.845996  2527 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.000s
I20260709 13:15:22.847360  2535 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:22.851943  2535 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:22.851732  2521 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.754302  2490 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.853574  2535 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.819981  2198 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:22.782965  2559 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:22.821592  2563 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:22.822099  2388 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:22.823182  2540 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.382s	user 0.003s	sys 0.002s
I20260709 13:15:22.849570  2279 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:22.870471  2169 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:22.870726  2279 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.873132  2279 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:22.849869  2316 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:22.875020  2316 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:22.875455  2339 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:22.877404  2316 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:22.813380 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.879120  2535 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:22.721491  2534 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:22.880961 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.881737  2534 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:22.885653 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1330) took 1004 ms. Trace:
I20260709 13:15:22.886303 31361 rpcz_store.cc:276] 0709 13:15:21.881079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.242159 (+361080us) service_pool.cc:224] Handling call
0709 13:15:22.276630 (+ 34471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.306418 (+ 29788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.306454 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.307049 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.569817 (+262768us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:22.569888 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.570021 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.885338 (+315317us) raft_consensus.cc:1685] finished
0709 13:15:22.885373 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.885566 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29334,"raft.queue_time_us":35866,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6696,"thread_start_us":262369,"threads_started":1,"wal-append.queue_time_us":576120}
W20260709 13:15:22.819439  2111 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.002s
I20260709 13:15:22.888862 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.889345  2111 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:22.889717 31361 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:22.890515 30556 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:22.869921 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1507) took 1163 ms. Trace:
I20260709 13:15:22.869858 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1511) took 1152 ms. Trace:
I20260709 13:15:22.818043  2405 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.891855 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.891666 31367 rpcz_store.cc:276] 0709 13:15:21.717077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.079706 (+362629us) service_pool.cc:224] Handling call
0709 13:15:22.139519 (+ 59813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.219997 (+ 80478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.869519 (+649522us) spinlock_profiling.cc:243] Waited 67.6 ms on lock 0x7b54008b15d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:22.869786 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155536128}
I20260709 13:15:22.892807 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:22.912987 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:22.896363 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.897167 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:22.902282 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:22.903568 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.916457 30633 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:22.917984 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.905036 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.920436 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.908934 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.922374 30551 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:22.910521 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:22.911476 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:22.923944 30550 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:22.936340 30705 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:22.947043 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.854698  2490 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:22.950126 30699 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:15:22.954885 30552 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:22.960363 30555 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 13:15:22.951058 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.951212 30553 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:15:22.951303  2545 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 13:15:22.952665  2279 raft_consensus.cc:2764] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:15:22.891237 31389 rpcz_store.cc:276] 0709 13:15:21.706273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.025644 (+319371us) service_pool.cc:224] Handling call
0709 13:15:22.026246 (+   602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.068479 (+ 42233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.068518 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.140629 (+ 72111us) log.cc:844] Serialized 26 byte log entry
0709 13:15:22.473451 (+332822us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:22.473502 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.473591 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:22.869713 (+396122us) raft_consensus.cc:1685] finished
0709 13:15:22.869737 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:22.869865 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41730,"raft.queue_time_us":53877,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":20985,"thread_start_us":295193,"threads_started":1,"wal-append.queue_time_us":580418}
I20260709 13:15:22.970589 31389 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:22.957886 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.962517 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:22.963621 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.974853 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:22.983831 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:22.985311 30551 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 13:15:22.988266 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:22.976032 31367 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:22.929195  2471 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:23.007215  2548 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:23.012418  2530 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.002s
I20260709 13:15:23.014027  2280 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:23.015820  2111 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.017730  2559 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.019713 31361 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:22.966825  2421 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.020898  2421 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:23.026693  2367 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.004s
I20260709 13:15:23.019531  2471 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.027652 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1477) took 1383 ms. Trace:
I20260709 13:15:23.036962 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.040827 30551 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 13:15:23.063797 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 460952us
I20260709 13:15:23.069042 30550 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:23.038478 30627 rpcz_store.cc:276] 0709 13:15:21.644207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.645875 (+  1668us) service_pool.cc:224] Handling call
0709 13:15:21.677042 (+ 31167us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:21.687953 (+ 10911us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:21.688006 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:21.689070 (+  1064us) log.cc:844] Serialized 78 byte log entry
0709 13:15:22.309995 (+620925us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:22.310058 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:22.310165 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:23.027369 (+717204us) raft_consensus.cc:1685] finished
0709 13:15:23.027396 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.027584 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10309,"raft.queue_time_us":34868,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":7586,"thread_start_us":620379,"threads_started":1,"wal-append.queue_time_us":869943}
I20260709 13:15:23.070065 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.013101  2530 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.075822 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:23.080144 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.081421 30557 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:23.090058 30704 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:23.087134 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.069307  2559 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.117102 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.117491 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.072402  2111 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:23.168488 30524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.018s	sys 0.024s
W20260709 13:15:23.175508 30524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.019s	sys 0.025s
I20260709 13:15:23.133333 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:23.178185 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45008.2us
I20260709 13:15:23.182417 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:23.183710 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.123272  2428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.025s	sys 0.056s
I20260709 13:15:23.121659  2358 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:23.120604  2413 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:23.115168  2445 raft_consensus.cc:2764] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:15:23.172670  2306 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:23.173758  2488 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
W20260709 13:15:23.229616  2518 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.023s	sys 0.066s
I20260709 13:15:23.115329  2104 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.177634  2548 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.171765  2115 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 13:15:23.234505 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.174640  2480 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
W20260709 13:15:23.232947  2428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.027s	sys 0.058s
I20260709 13:15:23.245042 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.281072 30553 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:15:23.283802 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1015) took 1273 ms. Trace:
I20260709 13:15:23.228255  2547 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:15:23.246461 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 425089us
W20260709 13:15:23.294687 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.227767 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45453.8us
I20260709 13:15:23.177788  2559 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:23.243446  2518 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.023s	sys 0.067s
I20260709 13:15:23.228672  2573 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:23.323848  2530 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.327397 31364 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:23.328873  2393 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:23.296965 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.017s
W20260709 13:15:23.344506 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.013s	sys 0.017s
W20260709 13:15:23.345163 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 590739us
I20260709 13:15:23.286397 30628 rpcz_store.cc:276] 0709 13:15:22.010203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.010369 (+   166us) service_pool.cc:224] Handling call
0709 13:15:22.010915 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.011259 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:22.514878 (+503619us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.065954 (+551076us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b180020abe0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:15:23.282257 (+216303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.282328 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:23.282391 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.283590 (+  1199us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.283736 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":503282,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":4471,"raft.queue_time_us":40381,"spinlock_wait_cycles":3164544,"thread_start_us":695390,"threads_started":3}
I20260709 13:15:23.304841 30550 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [8.5-9.6]
I20260709 13:15:23.307792  2559 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.308842  2571 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.348948 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.348852  2571 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.350203 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:23.351224  2571 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.309695 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.319826  2471 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.352154  2471 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.352255  2559 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:23.352675  2471 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.287537  2358 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:23.354351  2558 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:23.320713  2383 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:23.354921 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:23.290648  2549 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:23.357241 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:23.357199  2558 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:23.358145  2549 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:23.358806  2549 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:23.358860  2393 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:23.359375  2393 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:23.359730  2393 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:23.360780  2471 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:23.332782 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.364003 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:23.288902  2534 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:23.337190  2541 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.839s	user 0.001s	sys 0.005s
I20260709 13:15:23.343765 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1331) took 1412 ms. Trace:
I20260709 13:15:23.367197  2471 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:23.367715  2431 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:23.368310  2431 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:23.366545 31396 rpcz_store.cc:276] 0709 13:15:21.931644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.639777 (+708133us) service_pool.cc:224] Handling call
0709 13:15:22.640175 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:22.640467 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.640501 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:22.654940 (+ 14439us) log.cc:844] Serialized 26 byte log entry
0709 13:15:23.092483 (+437543us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:23.092543 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.092680 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:23.343495 (+250815us) raft_consensus.cc:1685] finished
0709 13:15:23.343522 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.343685 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":437142,"threads_started":1,"wal-append.queue_time_us":686567}
I20260709 13:15:23.369385  2431 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.348102  2421 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.279852  2548 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:23.358317  2530 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:23.359841 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1513) took 1618 ms. Trace:
I20260709 13:15:23.226189  2532 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:15:23.360605 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:23.354709  2358 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:23.331278  2386 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.366160 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:23.376663 31383 rpcz_store.cc:276] 0709 13:15:21.741242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.118220 (+376978us) service_pool.cc:224] Handling call
0709 13:15:22.118537 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.169075 (+ 50538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.169127 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.169190 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:22.169240 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.359557 (+1190317us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b540002add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:23.359613 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.359753 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115531520}
I20260709 13:15:23.367728  2471 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.387902 30557 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:23.389699 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:23.392102 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 272470us
I20260709 13:15:23.393349 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.392179  2569 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:23.377244  2431 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:23.420569 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.375368  2339 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:15:23.425484  2339 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:23.426433  2490 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:23.426960  2490 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:23.427837  2490 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.429029  2490 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.429774  2393 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.403901  2521 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
W20260709 13:15:23.433364  2572 consensus_queue.cc:1175] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acf852f85f47483d99cbe4020baee627" responder_term: 11 status { last_received { term: 11 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 11 index: 6 } }
I20260709 13:15:23.440150  2388 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:23.383934  2535 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 13:15:23.444128  2393 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.444777  2569 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:23.391716  2471 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:23.446082 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.387156  2548 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:23.448954 30550 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:23.371361  2534 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.418190  2115 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:23.404994  2194 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:23.405946  2530 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:23.472633 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.474710 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.475458 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.476284 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.476445 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:23.483326 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:23.484663 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.485749 30624 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:23.489894 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.484426 30699 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:23.492558 30630 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:15:23.493284 30630 pending_rounds.cc:85] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 7
I20260709 13:15:23.493662 30630 pending_rounds.cc:107] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:15:23.494043 30630 raft_consensus.cc:2889] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:23.496959 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:23.405524  2421 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:15:23.419763  2488 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:15:23.534763 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.555744 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:23.556656 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 148863us
I20260709 13:15:23.559438 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.559576 30633 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:23.507802 30704 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:23.506877 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:23.561053 31383 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:23.532949 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:23.535684 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.557818 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:23.563364 30625 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.560356 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.555976 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:23.535238 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:23.567265 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.567898 30707 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:23.534746 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.532361 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.499861 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.576575 31367 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:23.570323 30632 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:23.574985 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 774) took 1332 ms. Trace:
W20260709 13:15:23.577163 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77470us
I20260709 13:15:23.572618 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.584059 30626 rpcz_store.cc:276] 0709 13:15:22.242431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.242996 (+   565us) service_pool.cc:224] Handling call
0709 13:15:22.243595 (+   599us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.243905 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:23.373569 (+1129664us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b1800288ca0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:15:23.373812 (+   243us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.553252 (+179440us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.553666 (+   414us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.559452 (+  5786us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.559518 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:23.559569 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.559644 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.574909 (+ 15265us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":321457,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":2376,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":17282,"spinlock_wait_cycles":110573568,"thread_start_us":521914,"threads_started":2}
I20260709 13:15:23.586134 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.587990 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.454834  2584 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:23.464718  2115 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:23.595942 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:23.600957 30624 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:23.603471 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.604470 30627 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:23.556434  2560 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.608547 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.567651  2475 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:23.611392 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.607074  2115 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:23.581347  2386 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:23.614921 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.632232 30627 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 7.
I20260709 13:15:23.599454  2518 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.641685 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.642443 30633 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:23.598737  2529 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:23.644095 30625 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
W20260709 13:15:23.617373  2591 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:23.618850  2521 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.615456  2486 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.633476  2584 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.656989  2486 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.620003  2488 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:23.660975  2488 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.636379  2559 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:23.594667  2597 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:15:23.625890  2534 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:23.622911  2530 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.613332  2560 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.662441  2486 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:23.679560  2589 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:15:23.659785  2521 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.646003  2529 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.622947  2569 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.612797  2561 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:15:23.685637  2521 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.689105  2431 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:23.689757  2431 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:23.657172  2584 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.605636  2548 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.690857  2431 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.684635  2529 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:23.695851  2584 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.696655  2471 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.698009  2573 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:23.694550  2549 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.701143  2115 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.704444  2115 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.708433  2600 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.666855 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1336) took 1111 ms. Trace:
I20260709 13:15:23.710754 30624 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:23.712201 30707 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:23.709780 31361 rpcz_store.cc:276] 0709 13:15:22.555733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.021086 (+465353us) service_pool.cc:224] Handling call
0709 13:15:23.021577 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:23.026041 (+  4464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:23.026090 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:23.026734 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:15:23.364786 (+338052us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:23.364837 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.364916 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:23.666538 (+301622us) raft_consensus.cc:1685] finished
0709 13:15:23.666565 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.666772 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3745,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":153,"thread_start_us":337581,"threads_started":1,"wal-append.queue_time_us":545763}
I20260709 13:15:23.720790 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:23.725898 31361 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:15:23.668799  2546 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.017s	user 0.000s	sys 0.003s
I20260709 13:15:23.731642 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:23.663286  2488 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.738801  2358 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:23.676802  2560 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.664078  2480 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 13:15:23.741482  2471 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.727862  2549 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.743008  2529 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.749809 30632 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:23.714282  2531 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:23.701143  2518 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:23.751350 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.714099  2477 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:23.767293 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 167217us
I20260709 13:15:23.769927 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.775996 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1014) took 1771 ms. Trace:
I20260709 13:15:23.795001 30631 rpcz_store.cc:276] 0709 13:15:22.004510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.027837 (+ 23327us) service_pool.cc:224] Handling call
0709 13:15:22.044190 (+ 16353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.206359 (+162169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:22.206397 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:22.206449 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:22.206487 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.775545 (+1569058us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:23.775634 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:23.775894 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372338432}
I20260709 13:15:23.795948 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.733003  2584 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:23.778191  2358 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:23.782593 31361 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:23.790441 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 135866us
I20260709 13:15:23.797546 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.780339  2115 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.802927 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:23.793545  2531 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:23.782193  2552 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.055s	user 0.003s	sys 0.007s
I20260709 13:15:23.714809  2600 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:23.811776 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1321) took 2190 ms (client timeout 2217 ms). Trace:
I20260709 13:15:23.805058 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:23.812354 31378 rpcz_store.cc:269] 0709 13:15:21.618171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:21.618323 (+   152us) service_pool.cc:224] Handling call
0709 13:15:23.808582 (+2190259us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400717ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:23.808807 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447521152}
I20260709 13:15:23.813056 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.802469  2569 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:23.804049  2393 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:23.801340  2358 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:23.807271  2552 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.084s	user 0.003s	sys 0.007s
I20260709 13:15:23.807955  2518 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:23.801581  2477 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.834802  2535 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:23.835578  2535 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.837481 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.838099  2535 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.838405  2520 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.818434  2386 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:23.818656 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.827240 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.823385  2552 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:23.828903  2290 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:23.820045  2393 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:23.831765  2529 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.818316  2569 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:23.832587 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.834059  2531 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:23.827436  2518 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:23.805439  2573 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:15:23.818212  2471 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:23.845038 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:23.871091 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.871593 31378 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:23.829236 31388 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:23.806110  2115 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.871932 30629 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:23.873499 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.874560  2115 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:23.875887  2502 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:23.876426  2502 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:23.867154  2518 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:23.877504  2502 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.879129 30633 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:23.879542 30633 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:23.804857  2584 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.880333 30633 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.881336 30633 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:23.881644 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.825498  2607 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.881699  2584 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:23.845809  2290 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:23.851346 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.893656 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.854596 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:23.862373 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.895247 30625 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:23.861140  2569 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:23.865173 30699 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:15:23.886762  2368 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.867796 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:23.847493  2386 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:23.849649  2393 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:23.868291 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:23.919708 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51521.6us
I20260709 13:15:23.882788 30632 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:23.883364  2552 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.883533  2607 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.883980 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:23.885399  2471 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:23.863147  2441 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:23.886354 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:23.827191  2530 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:23.939214  2530 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:23.939401  2471 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:23.940392  2530 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:23.941864 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1415) took 1293 ms. Trace:
I20260709 13:15:23.846005  2490 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:23.942382  2441 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:23.945700 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:23.946223 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.848716  2358 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:23.946580 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.918215  2368 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:23.949482 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.949970  2547 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:23.864365  2194 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:23.913262 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45561.8us
I20260709 13:15:23.950570  2547 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.951195 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.951408  2368 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:23.952103  2194 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:23.953433  2598 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:23.954077  2598 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:23.954342 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.955684  2475 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:23.956229  2475 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:23.868403 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55436.3us
I20260709 13:15:23.924670 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:23.958045 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.958124 30625 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:23.902189 31378 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:23.888787  2502 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:23.936600 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:23.961565 30707 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:23.937060 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:23.942313 30705 rpcz_store.cc:276] 0709 13:15:22.648240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.648381 (+   141us) service_pool.cc:224] Handling call
0709 13:15:23.941639 (+1293258us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540032a354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:23.941793 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271169792}
I20260709 13:15:23.959872 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:23.960131 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:23.931152 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.963909 31378 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:23.965299 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:23.968250 31378 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:15:23.967186 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:23.961062  2559 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:23.954366  2194 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:23.971410  2607 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:23.964061 30629 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:23.973667 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:23.930637  2549 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:23.978686 30698 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:15:23.976514 31364 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:23.990033 30633 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:23.996105 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.001153 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.008425 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.010254 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.014362 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.019361 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:24.015499 30698 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:23.980000  2534 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:23.980219  2549 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.033450 30625 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:24.024536 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.039299 31378 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:24.046357 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:23.998739  2559 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:23.963903  2560 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:15:24.056041 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.059325 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:23.957118  2475 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.066210 30698 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:15:24.038077 30625 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:24.061460  2559 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.051036  2549 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:24.120293 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 153170us
I20260709 13:15:24.127152 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.138605  2471 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:23.978832  2290 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.143440 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.145236 30705 raft_consensus.cc:2410] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:15:24.110989  2535 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:24.139611  2471 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:24.129930  2618 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:15:24.154016 30705 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:24.116340  2552 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.157881 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.124485  2559 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:24.163137 30698 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:15:24.166640 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.160841 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.168536 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.175884 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.176694 30705 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:24.051606  2386 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:24.159685  2471 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.191536 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.202744 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.209205 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.210108 30707 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:15:24.210372 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.211160 30706 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:15:24.211308 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.174696  2306 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:24.219070 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.225013  2529 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:24.226029  2529 consensus_queue.cc:1243] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:15:24.226649  2529 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:24.227066  2529 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:24.228173  2529 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.229832  2529 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:24.157570  2368 consensus_queue.cc:1175] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "015ca80eba674552b34856a4f20271f1" responder_term: 17 status { last_received { term: 14 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [15.9-16.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:24.237848 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.224009 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.241422  2104 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:15:24.241799 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:24.243221 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.241899  2104 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.243683  2104 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:24.249778 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.250885  2534 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.250403  2530 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:24.251736 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.255328 31361 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:24.253515 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1396) took 1906 ms. Trace:
I20260709 13:15:24.258915 31361 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:15:24.257258 31391 rpcz_store.cc:276] 0709 13:15:22.346742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.982843 (+636101us) service_pool.cc:224] Handling call
0709 13:15:22.991617 (+  8774us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.991986 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:23.022085 (+ 30099us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.249030 (+226945us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.253956 (+  4926us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.279363 (+ 25407us) log.cc:844] Serialized 14 byte log entry
0709 13:15:23.290239 (+ 10876us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:23.290300 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:23.290357 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:23.290434 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.253272 (+962838us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b54002f8d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:24.253441 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26120,"prepare.queue_time_us":56443,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":94,"raft.queue_time_us":18198,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":57574,"spinlock_wait_cycles":78759424,"thread_start_us":226203,"threads_started":1,"wal-append.queue_time_us":414410}
I20260709 13:15:24.252313  2534 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.261888 31367 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:24.251001  2421 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:24.263288  2421 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.264256 31367 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:24.265594  2421 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.266431  2534 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.259677 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1394) took 1913 ms. Trace:
I20260709 13:15:24.267367 31388 rpcz_store.cc:276] 0709 13:15:22.346119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.738651 (+392532us) service_pool.cc:224] Handling call
0709 13:15:22.759536 (+ 20885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.872554 (+1113018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.259381 (+386827us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002b0a54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:24.259607 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459116928}
I20260709 13:15:24.268265 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.269130 30558 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:24.271077 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.272619 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.274201 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.274251 30706 pending_rounds.cc:85] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 7
I20260709 13:15:24.274602  2530 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:24.293154 30706 pending_rounds.cc:107] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:15:24.294507 30702 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:15:24.170604  2552 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:24.301462 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:24.307696 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67352.6us
I20260709 13:15:24.307981 31391 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:15:24.312819 30552 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:24.305902 30706 raft_consensus.cc:2889] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:24.314606 30705 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:24.304589 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.320179  2104 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.322054 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.326084 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.300124  2480 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.339615 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:24.343844 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.309317  2530 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:24.379789 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53964.1us
I20260709 13:15:24.305645  2634 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.222359  2306 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:24.385704 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.335625 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.399189 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.334779  2607 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:24.422598 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:24.407846 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 68346.6us
I20260709 13:15:24.424525 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:24.405746  2518 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.005s
I20260709 13:15:24.393263  2530 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:24.392753  2619 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:24.422847  2584 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:24.423107  2607 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:24.429397 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 467194us
I20260709 13:15:24.384899  2480 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:24.491775 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.418877  2621 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:15:24.432449 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96903.5us
W20260709 13:15:24.438670 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.405416  2634 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.502727 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.489949  2607 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:24.512936 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.437852  2421 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:24.442868  2548 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:15:24.516840  2627 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:15:24.522965 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:24.532334 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.533339 30629 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:24.496037  2584 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:24.526782  2636 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:15:24.434414  2619 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:24.491290  2552 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.514672 31388 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:24.536731 30629 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:24.435670  2518 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:24.532370  2631 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:15:24.542377  2104 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:24.605481  2569 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.004s
I20260709 13:15:24.613561  2535 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:24.574543  2421 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:24.574731  2632 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 13:15:24.540388  2619 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:24.553001  2228 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:15:24.639169 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.640580 30555 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:15:24.629822  2393 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.694s	user 0.013s	sys 0.001s
I20260709 13:15:24.626935  2634 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:24.640614  2631 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:15:24.639029  2535 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:24.680045 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.665897  2306 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:24.680869  2534 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.685762  2584 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:24.696940  2640 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:15:24.699692 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:24.644946  2421 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.639894  2569 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:24.709592 30625 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 13:15:24.711272 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.690385  2634 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:24.680284  2393 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.722999 31378 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:24.717489  2638 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.227s	user 0.002s	sys 0.000s
I20260709 13:15:24.698757  2534 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.716655  2480 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.716009  2634 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.732371  2518 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:24.740851  2634 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:24.764153  2607 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.768131  2607 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:24.770783  2637 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.337s	user 0.000s	sys 0.005s
I20260709 13:15:24.771373  2041 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.752619  2386 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:24.776952  2386 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.779367  2386 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.782312  2441 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:24.788745 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 787) took 1224 ms. Trace:
I20260709 13:15:24.789299 30630 rpcz_store.cc:276] 0709 13:15:23.564889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.566935 (+  2046us) service_pool.cc:224] Handling call
0709 13:15:24.788488 (+1221553us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54004a88d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:24.788658 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1137652480}
I20260709 13:15:24.795418 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1469) took 2184 ms. Trace:
I20260709 13:15:24.796139  2393 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.796686  2569 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.800567 30630 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:24.803383 30632 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:24.806555  2600 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.807565  2421 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.795998 30556 rpcz_store.cc:276] 0709 13:15:22.611228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:22.611447 (+   219us) service_pool.cc:224] Handling call
0709 13:15:22.627709 (+ 16262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:22.890979 (+263270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.795126 (+1904147us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:24.795340 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603929344}
I20260709 13:15:24.754807  2530 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.812567  2358 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:24.814584 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.816788  2534 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.817132 30556 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:24.815096 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:24.816797 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.816277  2547 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.820807  2619 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.814734  2530 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:24.823146  2280 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:15:24.827239  2559 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.640s	user 0.014s	sys 0.000s
I20260709 13:15:24.831209 31367 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:15:24.823951  2549 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.676s	user 0.012s	sys 0.003s
W20260709 13:15:24.824335 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:24.825096  2529 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.595s	user 0.018s	sys 0.000s
W20260709 13:15:24.825798  2502 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.862s	user 0.008s	sys 0.007s
I20260709 13:15:24.835160  2549 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:24.838835 30631 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16.
I20260709 13:15:24.839932  2532 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.840301  2502 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:24.840696 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.840600  2532 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:24.840931  2502 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:24.843286  2532 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.843488  2502 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.843791  2431 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:24.845831  2619 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:24.835629 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:24.847219 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.848299  2280 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:24.848739  2280 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:24.849146  2280 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:24.849627  2639 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
I20260709 13:15:24.851527 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:24.852571 30556 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:24.854527  2308 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.855398  2549 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.855265  2308 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.856680 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:24.857560 30552 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:24.857681  2308 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.859277 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.831179  2559 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:24.910641 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2057) took 1734 ms. Trace:
I20260709 13:15:24.881687  2534 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:24.861459 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.913590  2534 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:24.913396 30557 rpcz_store.cc:276] 0709 13:15:23.176531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.178746 (+  2215us) service_pool.cc:224] Handling call
0709 13:15:23.179182 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.388546 (+209364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.910236 (+1521690us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:24.910529 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479343360}
I20260709 13:15:24.870179 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.878198  2477 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:24.904589 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1532) took 1081 ms. Trace:
I20260709 13:15:24.915035 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.887137 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:24.916213 31364 rpcz_store.cc:276] 0709 13:15:23.823504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.823883 (+   379us) service_pool.cc:224] Handling call
0709 13:15:24.904346 (+1080463us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54002b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:24.904511 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317680384}
I20260709 13:15:24.916929  2600 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:24.906790  2290 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:24.917192  2559 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:24.863085 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:24.882185 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 273347us
I20260709 13:15:24.920056 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.920671 30626 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:24.922096 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:24.882112 30550 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:24.868927 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.861873 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:24.923817 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55006.1us
I20260709 13:15:24.866112 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.932641 30700 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:24.932874 30555 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:24.933847 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.934583 30699 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 7.
I20260709 13:15:24.934459 30380 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 11, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:24.863401 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.936077 30707 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:24.936426 30380 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 7 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 11 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:24.886443  2619 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.878578 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.938964  2619 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:24.861086 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.940672 30552 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 8.
I20260709 13:15:24.941390  2619 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:24.879992  2621 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:24.941735 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54729.6us
I20260709 13:15:24.942795  2621 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:24.943219  2535 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:24.925782 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:24.926004  2421 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:24.864112 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.927649 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1436) took 1059 ms. Trace:
I20260709 13:15:24.930297 30628 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:15:24.963495 30628 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:15:24.963819 30628 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:15:24.964247 30628 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:24.931195  2194 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:24.925704  2598 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:24.879495 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:24.943475 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:24.945324 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.968305 30558 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:24.968212 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:24.969354 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.969484 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:24.938371 30380 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 4 to 5, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 5 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:24.862110 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.970417  2598 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:24.970805 30701 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:24.949550 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.946974 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.951053 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:24.957625 30554 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:24.949750 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:24.959184 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.959234 30552 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:24.963042 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:24.963430 30703 rpcz_store.cc:276] 0709 13:15:23.867962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.880737 (+ 12775us) service_pool.cc:224] Handling call
0709 13:15:23.881105 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:23.881354 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:15:23.979925 (+ 98571us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.865265 (+885340us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b18002743c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:24.865540 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.927012 (+ 61472us) log.cc:844] Serialized 14 byte log entry
0709 13:15:24.927209 (+   197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:24.927256 (+    47us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:24.927305 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.927384 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:24.927587 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":789548,"prepare.queue_time_us":1016276,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":135,"raft.queue_time_us":64127,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":8684,"spinlock_wait_cycles":6701952,"thread_start_us":236965,"threads_started":2,"wal-append.queue_time_us":186834}
W20260709 13:15:24.993315 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43646.9us
I20260709 13:15:24.955955 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:24.969691  2600 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:24.996141 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1487) took 1129 ms. Trace:
I20260709 13:15:24.992379 30706 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:15:24.996858 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45909.3us
I20260709 13:15:24.997748 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:24.993711 30702 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:24.998557 30627 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:24.972559 30707 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:24.970429 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:24.999874 30699 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:24.996712 30628 rpcz_store.cc:276] 0709 13:15:23.866979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:23.867132 (+   153us) service_pool.cc:224] Handling call
0709 13:15:23.873810 (+  6678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.964830 (+1091020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:24.995859 (+ 31029us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54004a92d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:24.996069 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2428753664}
I20260709 13:15:24.996483 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:24.964942 30700 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:25.008950 30698 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:25.010725 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51612.2us
I20260709 13:15:25.012840 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:25.014413 30701 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:25.018899 30707 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:25.024130 30706 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:24.924664  2518 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:24.956895  2529 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.009212 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:25.032357 30556 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:24.957278  2619 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:24.963059  2308 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:25.029004 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67880.3us
I20260709 13:15:25.026795 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:25.044445 30699 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:24.972976  2559 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.024243  2220 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:15:25.052870 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:25.060304 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51196.8us
I20260709 13:15:25.008424  2477 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:25.027472  2041 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:24.914865  2534 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.027570  2228 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:25.030973  2339 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:15:25.104833 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52056.3us
I20260709 13:15:24.978302  2421 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.023844  2502 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.068s
I20260709 13:15:25.023614  2441 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:15:25.028144  2597 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:15:25.027817  2520 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:15:25.031358  2560 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:15:25.031648  2600 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:15:25.031533  2471 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:15:25.092317  2549 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:25.092080  2220 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.101794  2228 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.096951  2041 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:25.103240  2531 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.049497  2529 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:25.107831  2477 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.059796  2308 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.114171  2131 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:25.114705  2378 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:25.109268  2339 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.053520  2619 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.157289  2652 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:15:25.150760  2549 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.160306  2480 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:25.158923  2581 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:15:25.169329  2378 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:25.168006  2477 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.178102  2378 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:25.174475  2480 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:25.179388  2619 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.179704  2480 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.181039  2480 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:25.182760  2477 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.174932  2549 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:25.165966  2531 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.244611 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 135) took 23630 ms (client timeout 15000 ms). Trace:
W20260709 13:15:25.248613 30522 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.017s	sys 0.082s
W20260709 13:15:25.249392 30522 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.018s	sys 0.082s
I20260709 13:15:25.118471 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:25.095760 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:25.106112 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:25.257757 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 151650us
W20260709 13:15:25.250174 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 525527us
I20260709 13:15:25.259387 30625 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 13:15:25.261061 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 142606us
I20260709 13:15:25.263113 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:25.257673 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 161834us
I20260709 13:15:25.154240  2220 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:25.167136  2131 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.295883  2131 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:25.296193 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:25.298065 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:25.268836 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 238951us
I20260709 13:15:25.311826  2290 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.312388 30671 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16.
I20260709 13:15:25.313107 30671 leader_election.cc:403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:25.313303  2290 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.271220 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.275264 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1768) took 1005 ms. Trace:
W20260709 13:15:25.279183 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.315068  2199 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16.
I20260709 13:15:25.317786  2290 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.319139  2290 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.319918  2290 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.320607  2290 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.160691  2041 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:25.241473 30380 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:15:25.267465 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 237220us
I20260709 13:15:25.159498  2228 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:25.355901  2571 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.356402 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.321372 31364 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:25.357653  2600 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:25.322208  2345 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:25.280076 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.012s	sys 0.022s
I20260709 13:15:25.323789 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.324765  2308 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.326233 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:25.328120 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65460.2us
I20260709 13:15:25.329140  2368 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:25.315085 30704 rpcz_store.cc:276] 0709 13:15:24.269900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.270022 (+   122us) service_pool.cc:224] Handling call
0709 13:15:24.270395 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.270658 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:24.870104 (+599446us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.274164 (+404060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.274341 (+   177us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:25.274490 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.274676 (+   186us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.274982 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":608620,"prepare.queue_time_us":599589,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":25640,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":391,"thread_start_us":403124,"threads_started":1,"wal-append.queue_time_us":327211}
I20260709 13:15:25.384485  2471 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.385663  2471 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:25.386562  2534 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.384493  2421 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.397406  2471 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.190165  2535 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.354756  2532 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.356726  2571 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.399770  2571 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.400239  2441 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:25.400871  2441 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.401275  2441 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:25.171727  2339 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:25.341394  2383 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.319559 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:25.362673  2345 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.363758 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.014s	sys 0.024s
I20260709 13:15:25.365746 30380 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 13, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 13 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:25.245698 30382 rpcz_store.cc:269] 0709 13:15:01.661372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:01.664683 (+  3311us) service_pool.cc:224] Handling call
0709 13:15:25.244188 (+23579505us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:16.323931 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:16.414756 (+ 90825us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:16.449857 (+ 35101us) write_op.cc:435] Acquired schema lock
0709 13:15:16.454570 (+  4713us) tablet.cc:662] Decoding operations
0709 13:15:16.541465 (+ 86895us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:16.541643 (+   178us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:16.541692 (+    49us) tablet.cc:685] Acquiring locks for 19 operations
0709 13:15:16.591197 (+ 49505us) tablet.cc:701] Row locks acquired
0709 13:15:16.591249 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:15:16.591495 (+   246us) write_op.cc:270] Start()
0709 13:15:16.591666 (+   171us) write_op.cc:276] Timestamp: P: 1783602916591395 usec, L: 0
0709 13:15:16.591708 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:15:16.596014 (+  4306us) log.cc:844] Serialized 7468 byte log entry
0709 13:15:17.702968 (+1106954us) op_driver.cc:464] REPLICATION: finished
0709 13:15:18.850522 (+1147554us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f180a913ef0
0709 13:15:19.039657 (+189135us) write_op.cc:301] APPLY: starting
0709 13:15:19.058748 (+ 19091us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:19.060122 (+  1374us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:19.060171 (+    49us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:19.062036 (+  1865us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:19.062545 (+   509us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=38
0709 13:15:19.102736 (+ 40191us) write_op.cc:312] APPLY: finished
0709 13:15:19.172218 (+ 69482us) log.cc:844] Serialized 131 byte log entry
0709 13:15:19.524914 (+352696us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.525200 (+   286us) write_op.cc:454] Released schema lock
0709 13:15:19.525324 (+   124us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:19.616831 (+ 91507us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:19.632289 (+ 15458us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3218807,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":1317082,"apply.run_cpu_time_us":25232,"apply.run_wall_time_us":559574,"num_ops":19,"prepare.queue_time_us":398818,"prepare.run_cpu_time_us":48547,"prepare.run_wall_time_us":517995,"raft.queue_time_us":218562,"raft.run_cpu_time_us":29710,"raft.run_wall_time_us":1570641,"replication_time_us":1111016,"spinlock_wait_cycles":37184896,"thread_start_us":2441272,"threads_started":5,"wal-append.queue_time_us":867965,"wal-append.run_cpu_time_us":53073,"wal-append.run_wall_time_us":3318532}]]}
I20260709 13:15:25.377190  2531 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.397588  2104 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.334007 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:25.414676 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.414803 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:25.417979 31389 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.418625  2535 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:25.419260  2598 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:25.419777  2598 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:25.419723  2535 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.421335  2598 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:25.424919 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.426980  2535 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.430992  2656 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:25.424896 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 397988us
I20260709 13:15:25.431880  2656 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.435554 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.436643 31396 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:25.437939 31389 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:25.354923  2547 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:25.440189  2529 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.360810  2600 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:15:25.455265 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 136054us
I20260709 13:15:25.455924  2345 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.456624 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:25.418522  2104 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:25.468598 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 134990us
I20260709 13:15:25.448138  2600 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:25.470675  2535 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:25.491626  2547 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:25.492018  2279 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:25.491695 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:25.535403 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62028.5us
I20260709 13:15:25.495038 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:25.495956 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:25.497844 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:25.500682 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.498636 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:25.513170  2358 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.700s	user 0.014s	sys 0.015s
I20260709 13:15:25.555362 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:25.509076  2634 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.007s
I20260709 13:15:25.509261  2598 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:25.577057  2634 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:25.580814 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.239s (SENT) [suppressed 15 similar messages]
I20260709 13:15:25.580770  2559 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:25.537839  2279 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.499447 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:25.519907  2656 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:25.591946 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 121390us
I20260709 13:15:25.593621 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:25.598965  2445 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.599849  2445 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:25.603623  2445 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:25.604243 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 174190us
I20260709 13:15:25.617882 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:25.605111  2600 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:25.608049  2421 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.619212  2421 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.611150  2668 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.618227 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:25.620692 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:25.620829  2668 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:25.619244 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:25.599017 30380 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:25.624526  2600 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:25.625017 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 139234us
I20260709 13:15:25.625101  2547 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:25.626935  2547 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:25.622525  2421 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.631047  2616 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:25.632112  2616 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.634114  2529 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:25.635131  2529 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:25.498466  2104 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:25.631294  2308 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:25.627722  2279 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:25.634232 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:25.640892  2445 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:15:25.637122 30380 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 9, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 9 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:25.676143 31367 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:25.671885 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.668129 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:25.674523 31389 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:15:25.722980 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 127213us
W20260709 13:15:25.686376 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93041.7us
I20260709 13:15:25.723920 30629 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:25.580997  2634 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.675393 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:25.665304  2280 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.009s
W20260709 13:15:25.680737  2378 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.017s
I20260709 13:15:25.680184 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:25.644826  2621 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.658066  2358 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:25.668471  2616 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.666850  2104 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:25.638365  2529 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.677245  2668 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.619491  2345 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.152s
I20260709 13:15:25.814466 30552 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:25.645146  2581 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:25.818022 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 191062us
I20260709 13:15:25.662022  2571 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.833649 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252904us
W20260709 13:15:25.831936 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 103379us
I20260709 13:15:25.810708  2532 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:15:25.851938 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.863499  2308 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:25.864239  2308 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:25.864701 31396 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:25.865553  2571 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.865479  2308 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:25.866779  2131 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.570s	user 0.021s	sys 0.000s
I20260709 13:15:25.815514  2378 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.867302  2131 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.869750  2131 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.870158  2378 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.872457 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:25.873111  2111 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:25.873859 30524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.027s	sys 0.058s
W20260709 13:15:25.874291 30524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.027s	sys 0.058s
W20260709 13:15:25.875300 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.866796  2228 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:25.876434 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.877899  2228 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:25.881243  2378 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:25.882122  2378 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.884099 30380 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:25.891389  2581 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:25.892114  2220 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.596s	user 0.014s	sys 0.007s
I20260709 13:15:25.892740  2220 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.811677  2280 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:25.797443  2621 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:25.827773  2358 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:25.896955  2621 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.897315  2358 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.898391 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:25.899628 30629 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:25.863330 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:25.901461 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 70889.5us
I20260709 13:15:25.861390 31394 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:25.884948  2378 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:25.833850  2616 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:25.894235  2581 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.910111 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:25.911127 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2069) took 1051 ms. Trace:
I20260709 13:15:25.912071  2280 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:25.912972 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 245052us
I20260709 13:15:25.916826  2220 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:25.898789  2339 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:25.938004  2041 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.547s	user 0.007s	sys 0.006s
I20260709 13:15:25.938632  2530 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:25.939489 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.940855 30625 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:25.942351 30627 raft_consensus.cc:2410] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:15:25.942531 31396 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:15:25.943181 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 98786.1us
I20260709 13:15:25.886065 30380 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 9, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:15:25.942508 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79883.2us
I20260709 13:15:25.849202  2532 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:25.949426 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:25.943717  2441 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.008s
W20260709 13:15:25.943388  2383 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.010s
I20260709 13:15:25.949249 30551 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:15:25.946569  2290 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.007s
I20260709 13:15:25.965482 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:25.962391  2199 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.965643  2532 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:26.001695 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:25.946300 30553 rpcz_store.cc:276] 0709 13:15:24.859588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.859765 (+   177us) service_pool.cc:224] Handling call
0709 13:15:24.860069 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:24.860312 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:24.860663 (+   351us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.721712 (+861049us) log.cc:844] Serialized 14 byte log entry
0709 13:15:25.826117 (+104405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:25.833171 (+  7054us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:25.899441 (+ 66270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:25.907621 (+  8180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:25.910876 (+  3255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":173598,"prepare.queue_time_us":451738,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":102,"raft.queue_time_us":177854,"raft.run_cpu_time_us":645,"raft.run_wall_time_us":23728,"thread_start_us":693187,"threads_started":2,"wal-append.queue_time_us":338850}
I20260709 13:15:26.010331 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:25.984483  2441 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.000959  2559 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.027225  2559 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.028213  2339 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.030033 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.001719  2656 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:26.031009  2549 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.821s	user 0.014s	sys 0.004s
I20260709 13:15:26.031137  2656 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.032053 30701 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:15:26.031790  2549 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.001385  2383 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:26.034427 30628 raft_consensus.cc:2410] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:26.034106  2199 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.017535  2104 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:25.960315  2041 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.020359  2547 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:25.997121  2290 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:25.957743  2616 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.026245  2441 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.035187 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.070775 30704 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.037065  2199 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.037617 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.075924 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.041847  2547 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.076969 30707 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.078253 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.078802 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:26.037845  2194 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.079741  2194 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.080610 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.040522  2041 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.036031  2104 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:26.040804 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45906.7us
I20260709 13:15:26.044739 30626 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:26.041276 30631 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:26.038586 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.066911  2559 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.040597 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.090384 30699 raft_consensus.cc:2410] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:15:26.031028 30624 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:26.039940 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:26.095124 30625 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:26.048964 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:25.962335  2534 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.003s
I20260709 13:15:26.093775 30706 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:26.104552 30698 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 8: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.017738  2668 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.048219 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.034545  2549 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.045325  2656 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.082544  2194 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.045099  2290 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.104748 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.073287  2529 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:26.121309 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.075209  2345 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.122696  2111 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:26.123131  2345 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.123744 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:26.127379 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.128001 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:26.128489 30699 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:26.129134  2194 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:26.129309 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:26.129777  2194 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:26.131013  2194 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.131759  2668 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.134029 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:26.134989 30553 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:26.135437 30705 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:26.075517  2584 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.083186  2675 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:26.134294  2668 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:26.142907  2111 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:26.153590 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.153975  2111 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:26.145172  2041 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:26.145366 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.082926  2441 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:15:26.082858  2581 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:15:26.154764 30704 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:26.167176 30552 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:26.157476  2041 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.126209  2345 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.138327 30380 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 6, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 6 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:26.169448 30559 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:26.046953  2670 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
I20260709 13:15:26.174758 30699 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:26.175494 30703 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:26.166543  2111 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.048148  2530 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:26.183172  2279 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.003s
I20260709 13:15:26.139272  2547 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:26.199523 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 214572us
I20260709 13:15:26.172915  2041 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:26.205004 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.170567  2104 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:26.213179 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.211277 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.215715 30704 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.216620 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.217444 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:26.218568  2547 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.216573 30699 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:26.220149  2675 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.220769  2675 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.228704 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.216519 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:26.231196  2480 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.050s	user 0.005s	sys 0.020s
I20260709 13:15:26.219359  2547 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.226217  2581 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.240080 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:26.247215 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 154383us
W20260709 13:15:26.247650 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44746.4us
I20260709 13:15:26.248200 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.248858  2675 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:26.249056 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:26.249250 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.247748 30703 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:26.249228 30704 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:26.160985  2584 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.197759  2279 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.248275  2480 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:26.214010  2445 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:26.241755 30553 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:26.281894 30380 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 7, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:26.292448 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.299741 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.300391 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.301003 30700 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:26.278743  2104 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:26.308897 30701 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.309945 30699 raft_consensus.cc:2410] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:15:26.301867 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.295135  2345 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:26.297084  2670 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:26.323940  2104 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:26.324383 30700 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:15:26.324815  2616 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.325228  2104 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.326025 30700 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:15:26.326717  2279 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.321020 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:26.338907 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89708us
I20260709 13:15:26.293865  2581 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.343564 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.341331  2345 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:26.353679 30699 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.344197 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.345561 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.346484 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:26.328831  2670 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:26.359685 30628 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:26.314694  2445 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:26.363819 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 105718us
I20260709 13:15:26.395825 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:26.434682 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:26.435540 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78535.7us
I20260709 13:15:26.361652  2670 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:26.439141 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.436522 30625 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:26.451517 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.397310  2529 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:26.457268 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.450237 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:26.456225 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:26.432719  2530 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.466900 30631 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:26.467304 30632 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:26.452590 30700 raft_consensus.cc:2376] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9. Re-sending same reply.
I20260709 13:15:26.482273 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:26.485183 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 174493us
I20260709 13:15:26.481273 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.489326 30699 raft_consensus.cc:2368] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:26.497040  2111 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:26.497745  2111 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:26.498162  2111 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:26.503017  2445 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:26.507551 30705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 682) took 1645 ms (client timeout 1676 ms). Trace:
W20260709 13:15:26.508167 30705 rpcz_store.cc:269] 0709 13:15:24.861686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:24.862756 (+  1070us) service_pool.cc:224] Handling call
0709 13:15:26.507217 (+1644461us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54005a32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:26.507434 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2744816512}
W20260709 13:15:26.511722  2681 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:15:26.513434  2682 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:15:26.517706  2111 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.522177  2383 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:26.522595 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:26.532688 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77505us
I20260709 13:15:26.533105 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:26.533413  2111 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:26.522886  2194 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.529036 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:26.537534 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50368.3us
I20260709 13:15:26.537348 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:26.521206  2571 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:26.480686  2529 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:26.549026 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.534272  2383 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:26.566129 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:26.544214  2656 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:26.548828  2530 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:26.566509 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.544871  2571 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:26.571543 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.555073  2445 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:26.568589  2290 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:26.568362  2559 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:26.580403 30552 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:26.582892 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:26.568766  2616 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.595757  2616 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.547991  2194 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.575860  2584 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:26.599313  2670 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.600696  2194 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:26.575312  2549 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:26.601509  2549 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.601934  2549 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:26.602056  2670 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:26.571996  2675 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:15:26.605965 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43724.8us
I20260709 13:15:26.608537 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:26.609362  2581 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:26.554600  2131 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:15:26.623135 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90216.3us
I20260709 13:15:26.579383  2445 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:26.625262 30552 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:15:26.626418 30552 pending_rounds.cc:85] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:26.598394  2616 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:26.613576  2532 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.587s	user 0.012s	sys 0.004s
W20260709 13:15:26.628108 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54054.8us
I20260709 13:15:26.629442 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:26.610296  2547 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:26.639423  2558 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:26.639937  2558 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.640327  2558 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:26.626770 30552 pending_rounds.cc:107] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:15:26.647048 30552 raft_consensus.cc:2889] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.654167 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:26.622414  2530 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.656071 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.660255 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:26.629267  2445 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:26.665374 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:26.593677  2041 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.667366 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:26.668149 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:26.674794 31389 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:26.565062  2345 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:26.637781  2534 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:26.620857  2290 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:15:26.672852 31396 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:26.682976  2656 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.683905 31361 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:26.686585 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:26.697382 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:26.698933 30552 raft_consensus.cc:2368] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
W20260709 13:15:26.691847 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83417.9us
I20260709 13:15:26.694422 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:26.696174 31361 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:26.627498  2471 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:15:26.727151 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.728618 30702 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:26.732026 30707 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:26.729192 31367 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:26.636629  2532 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:26.722029  2690 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:15:26.737071 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.752274 30706 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:26.736997 31361 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.759459 31361 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:26.740823  2534 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:26.764916 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:26.628477  2529 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:26.741619 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 103210us
I20260709 13:15:26.690393  2531 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:26.741888 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.723376  2656 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.742753 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.743692 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:26.745054  2692 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 13:15:26.745394  2571 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.747397 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:26.805299  2667 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:26.808163 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.748687 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:26.749502  2471 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:26.724344  2530 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.741029 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.771353 30553 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:26.773151 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:26.775619  2695 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.088s	user 0.001s	sys 0.002s
I20260709 13:15:26.775213  2534 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:26.799822 30559 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:15:26.668767  2383 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:26.632001  2670 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.135s
I20260709 13:15:26.810906 31364 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:26.810189 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 225008us
I20260709 13:15:26.819773 30706 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:26.818491 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:26.838762 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 198868us
I20260709 13:15:26.849303 30555 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:26.842552 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:26.812659  2471 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:26.856123 31394 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:26.857722 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 285078us
I20260709 13:15:26.858754 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.861871 31361 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 13:15:26.827684  2104 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:26.867331 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 187218us
I20260709 13:15:26.860532 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:26.870014  2680 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:26.874747 30551 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:26.875070 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:26.875897 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.877158 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.878445 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.861076  2502 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:26.878593 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.890259 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:26.890736  2584 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:26.891328  2584 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.891736  2584 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:26.892467  2345 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:26.879307  2131 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:15:26.895242 30703 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:15:26.879249 30629 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:26.882910 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.881353 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.885532 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.883033 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:26.890755 30702 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:26.880668 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:26.886026 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.910774 31361 pending_rounds.cc:85] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:26.911576 31361 pending_rounds.cc:107] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 13:15:26.912003 31361 raft_consensus.cc:2889] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:26.914438 30700 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:26.930069 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:26.920924 31364 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:26.964820 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:26.966070 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:26.967068 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 104594us
I20260709 13:15:26.820009  2531 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:26.967016 30627 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:26.889843  2502 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:26.885710  2679 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:26.968978 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:26.885601  2530 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:26.982540 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:26.986426 30629 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:26.985689 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 227281us
I20260709 13:15:26.968508 31367 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 13. Ops: []
I20260709 13:15:26.989199 30698 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:26.888845  2559 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:26.988878 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:26.990684 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 130561us
I20260709 13:15:26.991434 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:26.888702  2104 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 13:15:26.995359 30632 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:26.887887  2308 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 13:15:26.992727  2559 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:26.996960  2480 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:26.998008  2656 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:27.000227  2531 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:26.994223 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:27.007316 31361 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:26.889338  2471 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.034930 30629 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:27.038187 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.015377  2559 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:27.043959 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 192437us
I20260709 13:15:27.050282 31367 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:15:27.051385 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.020363  2675 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:27.022277  2532 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:27.034934  2529 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:27.034368  2700 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:27.063627  2531 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.009596  2679 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.039964  2421 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:27.053026  2616 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:27.071664 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.026677  2530 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:15:27.073249 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.073860 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.056157  2041 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.081485  2471 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:27.066044  2675 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.084337  2549 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.064030  2713 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.086269 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.080530  2699 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:27.093158 31361 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 8: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:27.091212 30555 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:27.089764  2558 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.109866 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.066795  2529 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:27.327143 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.075428  2421 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.091284  2534 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:27.327986  2558 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:27.328292  2421 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:27.329205 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.330504 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.328002  2529 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.331660 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.332441 31361 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:27.332858 30705 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:27.333231 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.333848 31361 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:15:27.334646  2703 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.266s	user 0.004s	sys 0.004s
I20260709 13:15:27.335093  2529 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:27.075188  2616 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.090595  2584 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.335779  2616 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:27.337453  2584 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.068933  2531 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.113560  2445 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.114215  2713 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.340184  2532 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:15:27.292771 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 303964us
I20260709 13:15:27.342511 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:27.106292  2502 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.313407 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.344614 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.345502  2713 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:27.315109 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.347090 31383 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:27.315519 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.316597 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.346390 31364 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:15:27.116034 31378 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:27.069564  2700 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:27.317615 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.217s (SENT)
I20260709 13:15:27.317073  2571 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:27.320561 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.327073 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:27.321919 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 312751us
I20260709 13:15:27.359817 31396 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 9: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:27.323467 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.323701 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:27.324805 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 229160us
W20260709 13:15:27.363519 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.327922 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
W20260709 13:15:27.338940 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 228042us
W20260709 13:15:27.293324 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 256773us
I20260709 13:15:27.350173 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.318431  2383 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:27.375824  2531 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.350881 31364 pending_rounds.cc:85] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:27.376921 31364 pending_rounds.cc:107] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:15:27.377265 31364 raft_consensus.cc:2889] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:27.354336  2700 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:27.317131 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:27.366062 30599 leader_election.cc:400] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:15:27.368485  2558 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:27.380635 30599 leader_election.cc:403] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:15:27.376478  2549 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:27.373576  2675 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.383733  2445 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:27.384145 31388 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:27.394979 30704 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
W20260709 13:15:27.396914 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:27.386934 31394 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.519s	user 0.015s	sys 0.000s
I20260709 13:15:27.398874 31394 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:27.398831  2131 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:27.399914  2531 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:15:27.400341  2131 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:27.400760  2131 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:27.386833 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.401656  2131 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:27.402585 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 298393us
I20260709 13:15:27.403707  2708 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.413700  2441 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:27.413805  2708 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.360071 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.404251 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.416739  2708 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:27.402704  2707 raft_consensus.cc:2764] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:15:27.428800  2421 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.431381 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:27.434921 31367 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 13:15:27.442124 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53368.8us
I20260709 13:15:27.442400 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:27.442512 31378 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:27.444772 30704 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:27.445734 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:27.450719 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.452041 30629 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:27.389967  2697 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.457147 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:27.454847 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.464360 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:27.463205 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:27.466310 30629 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:27.430011  2700 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.450160  2549 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.454079  2708 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:27.467931  2710 raft_consensus.cc:2764] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:15:27.470570  2131 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:27.440657  2558 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:27.470597 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90098.2us
I20260709 13:15:27.461738  2571 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:27.503082 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.503275 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.440850  2707 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.523643  2534 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:27.524073  2707 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.454517  2725 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:27.527096 31389 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 13:15:27.511320  2720 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.117s	user 0.000s	sys 0.002s
I20260709 13:15:27.512015 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:27.519114 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 404604us
I20260709 13:15:27.463744 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:27.484807 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.459636  2675 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:15:27.489145 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57880.2us
I20260709 13:15:27.490906 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.461028  2723 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.539707 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55006.8us
I20260709 13:15:27.488072 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.541213 31367 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:27.461716 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:27.497908 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.498903 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.454299  2670 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:15:27.504076 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.499826 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.490072  2445 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.511176  2279 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:27.511958  2718 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:27.490314  2708 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.490525  2549 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.512622 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.479794  2697 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.548249  2725 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:27.512699 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.514806  2724 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:27.516758 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.519268  2558 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.521417 30550 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:27.580950 30632 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:27.521615  2701 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:27.522250  2668 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:27.587432 30704 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:27.503608  2531 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:27.542678 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81050.9us
W20260709 13:15:27.591346 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.563477  2721 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:27.550711  2707 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.550452  2723 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.551370 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.549690 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:27.550755 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.551585 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:27.556246 30701 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:15:27.557152 30559 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:27.560009 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.560333 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.563501 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.564491 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:27.573534 30628 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:27.596858 30552 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:27.513923 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.551925  2571 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:27.552909  2532 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:27.557646  2445 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.587095 30624 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:15:27.522548  2700 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:27.587908 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.570919  2708 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:27.537744 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 74090.8us
I20260709 13:15:27.624354 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.625137 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.591846 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:27.594588 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.581151  2725 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:27.613642  2559 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.555s	user 0.011s	sys 0.005s
I20260709 13:15:27.595583 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.638746 31383 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:15:27.597543 30707 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:27.598129 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.598439 30556 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:27.600587  2723 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.601169 30633 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:15:27.639968 31378 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:27.600920  2713 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:27.603350  2697 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.595167  2531 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.604918  2549 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:27.607147  2699 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:27.608134  2441 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:15:27.608621 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58883.7us
I20260709 13:15:27.586112 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.580569  2724 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.627943 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.619354 30624 pending_rounds.cc:85] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 7
I20260709 13:15:27.540665  2675 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:27.626075 30554 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:27.636456  2532 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:27.596586  2679 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:27.657091 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.612919  2721 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:27.683701 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.636827  2383 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:27.686282 30552 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:27.686681 31389 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:27.687044 30627 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
W20260709 13:15:27.687657 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 101326us
I20260709 13:15:27.672837 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.635674  2725 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:27.645978 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.680552 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.674019 30624 pending_rounds.cc:107] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 11.8
W20260709 13:15:27.674553 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46748.6us
I20260709 13:15:27.694576 30624 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:27.698009 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:27.687371 30627 pending_rounds.cc:85] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 7
I20260709 13:15:27.699790 30627 pending_rounds.cc:107] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:15:27.706383 30627 raft_consensus.cc:2889] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:27.702059 31388 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:27.701113 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:27.707551 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61729.7us
I20260709 13:15:27.710043 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.666997  2699 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.710889 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.713117 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:27.670713  2571 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:27.686810  2502 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:27.725749 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.680721  2675 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:27.680455  2549 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:27.730960 30557 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:27.714085 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:27.681342  2724 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:27.699592  2679 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:27.667232  2616 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.693248  2383 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:27.650504  2559 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.709515  2471 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:27.713192  2441 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:27.738848 31378 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:27.715204  2421 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.739886 30555 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:27.730077  2697 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:27.758806 30629 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:27.741283 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.729190  2699 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.745620 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:27.746176 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 160160us
I20260709 13:15:27.715792  2696 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:15:27.759922 31388 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:27.730697  2502 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:27.764482  2571 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:27.843495  2571 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:27.844710  2571 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.764636 30699 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:27.764948  2531 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:27.762516  2721 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:27.766642 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.754207  2679 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:15:27.771363 31389 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:27.771154 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.743923  2383 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:27.772091 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.772996 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.773550  2441 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:27.757822 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:27.779251 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.861718 30707 raft_consensus.cc:606] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:27.758872  2702 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:27.782454 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.783490 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.791785 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.746922  2445 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:27.794765  2731 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:15:27.795943  2734 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:15:27.796352  2532 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:27.878404 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:27.803380  2730 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:15:27.804839  2733 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:15:27.811401 30557 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:27.818598  2279 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:15:27.822858 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.827587  2421 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:27.828645  2668 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.829469  2705 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:27.829695 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:27.833299 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:27.833967  2700 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:27.835314  2714 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:27.836282  2559 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:27.837507  2502 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:27.851156 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.815517 31367 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:27.917790 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.918305 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:27.918789 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:27.853971 31389 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:27.862165 30707 raft_consensus.cc:614] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:27.919649 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.868876 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:27.875039 31383 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:27.884600 31361 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:27.889947  2441 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:27.891227 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 120163us
I20260709 13:15:27.931335 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:27.900116  2279 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:27.933125 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:27.893153 31378 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:15:27.897547 31364 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:15:27.898543 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 188792us
W20260709 13:15:27.875640  2728 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.240s	user 0.002s	sys 0.002s
I20260709 13:15:27.871898  2670 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:27.922732 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:27.923454  2502 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:27.909150  2668 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:27.937603 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 224930us
I20260709 13:15:27.911947  2616 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.952307 30557 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:27.871980  2702 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:27.913295  2714 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:27.961392 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:27.951035 31389 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:27.980610 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:27.981132 30626 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:27.981491 30631 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:27.982967 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:27.984275 30625 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 7. Current term is 8. Ops: []
W20260709 13:15:27.986069 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:27.983299 30633 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
W20260709 13:15:27.992733  2737 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:15:27.994972  2279 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:27.996711  2735 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 13:15:27.996836  2279 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:27.998884  2616 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.999504  2616 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.001789  2616 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:27.951045  2529 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:28.012430 31389 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:27.951478  2559 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:27.954802  2667 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:27.956552  2716 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:28.030834 30557 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:27.959004  2534 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:28.002354  2616 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:27.971344  2131 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:27.953357  2679 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:15:28.089393 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.028080  2559 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:28.102517 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 269289us
I20260709 13:15:28.055898  2667 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.064738 30631 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:28.107492 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:28.121294 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.122586 30557 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:28.124960 30558 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:28.131146 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.083638  2716 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:28.134136 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:28.132388 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 177958us
I20260709 13:15:28.102918  2529 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.106276  2531 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:28.028865  2699 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.146040 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.106680  2559 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:28.008833  2714 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.144425 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.144825 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.145641 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.153812 30554 raft_consensus.cc:2410] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:15:28.119699  2667 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:28.151536 31389 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:15:28.146916 30704 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:28.154791 31389 pending_rounds.cc:85] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:28.155252 31389 pending_rounds.cc:107] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:15:28.157531 31389 raft_consensus.cc:2889] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:28.091285  2131 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.158835 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.123076  2471 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:15:28.163319 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.135640  2705 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.170046 30632 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:28.160594 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:28.163702 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 334119us
I20260709 13:15:28.165167 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.175104 30704 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:28.176163  2705 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.178938  2705 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.179008  2716 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.180896 30707 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:28.159963 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.158900  2721 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:28.097348  2616 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.173424 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.153373  2700 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:28.183902 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.184589 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.187299 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.187319 30556 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:28.199715 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.200845 30550 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:28.193559 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.193809 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.142014  2531 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:28.208405 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:28.207793 30551 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:28.186969 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.187745 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.207919 30553 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 9: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:28.207736 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:28.182571  2708 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.518s	user 0.008s	sys 0.000s
I20260709 13:15:28.215260 30558 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:28.215682 30552 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:28.216682 30554 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:28.199056  2736 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.396s	user 0.001s	sys 0.001s
I20260709 13:15:28.157423  2532 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.056s
I20260709 13:15:28.226831 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.194211  2697 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:28.237232 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 110174us
I20260709 13:15:28.145567  2529 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.255790 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.193222  2584 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:28.259542 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:28.249806 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56113.1us
I20260709 13:15:28.249076 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:28.261562 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.204651  2741 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.260879 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:28.265902 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 131874us
I20260709 13:15:28.271416 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:28.272107  2699 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.272176 30631 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:28.260308  2710 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:28.273324  2710 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.276391 30627 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:28.182583  2705 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:15:28.279632 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:28.284627  2750 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:15:28.285290  2549 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.551s	user 0.004s	sys 0.007s
I20260709 13:15:28.285563  2750 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:28.285826  2549 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:28.286684 30633 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:28.286772  2750 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.287932  2131 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:28.289342  2699 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:28.289968  2699 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.183065  2721 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.293226 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:28.297570 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.298909 30556 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:15:28.311264 30631 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
W20260709 13:15:28.313126 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56402.9us
I20260709 13:15:28.317813 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.236732  2708 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:28.319629 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.336548 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:28.290372  2699 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:28.296634  2714 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:15:28.275131  2584 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.486682 31396 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:28.376050  2675 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.004s
W20260709 13:15:28.491886 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.433229 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 618636us
W20260709 13:15:28.444226 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.445380  2745 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:15:28.446012 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.302635  2697 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:28.481494 31394 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:28.481842 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 220885us
I20260709 13:15:28.406414  2668 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.290340  2549 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:28.270409  2531 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.444422  2743 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:28.336014  2701 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.000s
W20260709 13:15:28.493352 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 232562us
W20260709 13:15:28.499226 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 196363us
I20260709 13:15:28.511631  2616 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:28.480926  2421 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:28.291224  2741 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.529667 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.531299 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:28.533191 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.536849 30556 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:28.551685 30551 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:28.549063 30699 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:28.551789 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.552703 30624 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:15:28.550531 30550 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:28.543511 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.514310  2549 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:28.557389 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.560262 30552 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:15:28.563372 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45993.4us
I20260709 13:15:28.568701 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.527667  2701 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:28.527288  2705 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:28.527237  2707 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:28.526847  2421 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:28.588259  2584 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.588495  2748 raft_consensus.cc:2764] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260709 13:15:28.589437  2421 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.590684  2421 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:28.507354  2675 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.577809  2740 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:28.586733  2719 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:15:28.595335 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 72754.2us
I20260709 13:15:28.535745  2279 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:15:28.567337  2549 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:28.617970 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:28.571295  2531 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:28.631129 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.572327  2743 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:28.635638 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.660028 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.577399  2705 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:28.628458  2707 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.667994  2707 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.673599  2707 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:28.622246  2383 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.739s	user 0.007s	sys 0.007s
I20260709 13:15:28.687546  2741 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:28.694142  2534 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 13:15:28.695282  2719 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:28.696225  2759 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.697579 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57336.8us
I20260709 13:15:28.698722  2383 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:28.703809  2757 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:28.672435  2584 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:28.649984  2668 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:28.729246 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:28.711714 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.712004  2710 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:28.714146  2699 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.620500  2740 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:28.659300  2725 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.944s	user 0.013s	sys 0.002s
W20260709 13:15:28.719331 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 175548us
I20260709 13:15:28.723029 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.744679 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.744679 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.745695  2679 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:28.746093  2741 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:28.746114  2679 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.747056  2719 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:28.747905  2679 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:28.748423  2719 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:28.747155 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:28.709630  2667 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.541s	user 0.010s	sys 0.000s
I20260709 13:15:28.711338 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.710425  2721 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:28.752770 30553 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:28.751310 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.754072  2741 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:28.754926 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:28.756153  2741 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:28.756601  2741 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.756999  2741 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:28.757535 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:28.737447 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 419739us
W20260709 13:15:28.740447 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75853.4us
I20260709 13:15:28.718645  2700 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:28.765010 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.735097  2742 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.298s	user 0.007s	sys 0.000s
I20260709 13:15:28.719579  2529 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:28.723438  2534 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:15:28.774319 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:28.775998 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.777053 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.777469  2741 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.778306  2668 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:28.778575  2746 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:28.742658  2725 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.761185  2740 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:28.787513 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.789378 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:28.779453 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:28.802253  2744 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.268s	user 0.000s	sys 0.000s
I20260709 13:15:28.749779  2719 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.803352  2747 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:28.805540  2383 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.703585  2743 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.730298  2759 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.809461  2759 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.811676  2725 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:28.752439  2667 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.781311  2745 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:28.820823 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.828682 30551 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:28.831486  2741 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.770378  2530 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:15:28.834240 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:28.834689 30553 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:28.835189 30633 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:15:28.837715  2559 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.676s	user 0.009s	sys 0.003s
I20260709 13:15:28.838984  2702 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.839737 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.840407 30705 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:28.840948  2708 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:28.827499  2724 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.084s	user 0.015s	sys 0.007s
I20260709 13:15:28.843082  2667 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.842998  2724 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.845505  2724 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.846271 30626 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:28.847486  2699 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:28.847976  2771 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:28.848536  2771 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.848902  2771 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:28.854281 30552 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:15:28.819497  2716 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.015s	sys 0.022s
I20260709 13:15:28.816587  2668 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:28.863443  2675 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.863276 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.865720 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.838233  2559 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:28.859716  2716 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.015s	sys 0.022s
I20260709 13:15:28.869562 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.881915 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.881646 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:28.882066 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52732.8us
I20260709 13:15:28.783700  2705 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.881733  2768 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.837729  2679 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.887154  2559 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.889235 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:28.893924 30705 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:15:28.894976 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115643us
I20260709 13:15:28.911161 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.912563 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:28.913830 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.914937 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.915781 30699 raft_consensus.cc:2368] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
W20260709 13:15:28.916002 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.916606 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.918916 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.895356 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.896406  2480 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:28.921952  2761 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.931142  2761 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.932842  2559 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:28.916079 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.933543  2761 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:28.934361  2749 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:28.933543  2559 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.925731 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.901022  2530 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:28.936748  2530 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.937234 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.938841  2584 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:28.939699 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:28.864746  2502 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.907s	user 0.008s	sys 0.007s
I20260709 13:15:28.920742  2768 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.934257 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.926939 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:28.958837 30706 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:28.919283 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.939740  2584 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:28.919734 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:28.922317 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.953574 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.919064  2679 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:28.962770 30559 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:28.949471 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:28.950240 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:28.964030 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.951920 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.951887 30626 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "907e974a334846208c70f0b813199df0"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:56732
I20260709 13:15:28.955601 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:28.965389 30626 raft_consensus.cc:606] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:28.965494 30705 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:28.966005 30626 raft_consensus.cc:614] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:28.957005 30554 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:15:28.957600 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:28.958956 31389 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:28.938725  2559 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:28.967526 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.922556  2668 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.935055  2749 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:28.899245  2700 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:28.974363 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:28.974921  2549 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:28.977790 30523 leader_election.cc:400] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:28.977603  2667 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:28.978847  2667 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:15:28.979218  2667 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:28.979447 30557 raft_consensus.cc:2410] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:15:28.980549  2530 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.971099 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:28.982211 30699 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:28.978183 30523 leader_election.cc:403] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:15:28.963621 30624 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:28.980062  2667 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:28.984490 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.985257 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:28.988001 30706 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:28.992542 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:28.987586 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:28.986856  2748 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:28.996270  2748 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:28.996675  2471 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:28.997248  2471 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:28.997575  2667 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:15:28.996090 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76474.8us
I20260709 13:15:28.996774  2748 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:29.003672 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.997642  2471 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:29.003954 30559 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:28.921958  2767 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:29.002904 30699 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:28.954784  2279 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:28.922228  2675 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 13:15:29.036253 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72333.7us
I20260709 13:15:28.977512  2549 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:28.987480  2529 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:28.963477  2713 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:28.977489  2714 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:28.987089  2725 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:28.963044  2708 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:15:29.039561 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:28.994643  2531 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:29.006482  2679 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:29.045580 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.057498  2697 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:29.058125  2697 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.002832  2480 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:29.058735  2749 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.059229  2697 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.060978  2753 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:15:29.003590  2530 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:15:29.063267  2759 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:29.064083  2767 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:29.064738  2767 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:29.065230  2767 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:29.069959 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:29.071449 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67882us
I20260709 13:15:29.075541  2759 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:29.078523  2679 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.081382  2679 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.081990 30559 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:29.050036  2531 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.084666  2531 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:29.085054 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 107457us
I20260709 13:15:29.086301  2714 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.086969  2710 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:29.087438  2710 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.087677  2714 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.088300  2710 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.089890  2710 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:29.090016 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.044142  2532 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:29.093683  2480 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:29.094007  2725 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.094574  2480 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:29.058821 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:29.095214  2725 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.096031 30699 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:29.096313  2670 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:29.096887  2670 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.097827  2670 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.098042 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.006863  2773 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:15:29.040251  2279 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.037665  2719 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:29.041201  2745 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:29.110421 30523 leader_election.cc:400] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
W20260709 13:15:29.111362 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.111056 30523 leader_election.cc:403] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:15:29.107753 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 137181us
I20260709 13:15:29.111549  2279 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:29.137277 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77370.7us
I20260709 13:15:29.133888 30707 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:29.147604 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:29.147782 30626 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:29.093703  2768 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:29.149729 30701 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:29.150576 30624 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:29.146831 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:29.155653 30633 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.108968  2549 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:29.157998 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.110867  2708 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.108894  2584 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.108223  2719 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.162891 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:29.164276 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.164835 31388 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:29.166478 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:29.167327 30706 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:29.168722 30698 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:15:29.051015  2771 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.178860 30705 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:29.181300 30699 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:29.152827  2713 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:29.149343  2670 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:29.160856  2549 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.125855  2753 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 13:15:29.203532  2777 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
W20260709 13:15:29.204365  2778 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:15:29.206125 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.179795  2716 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:29.220165  2707 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:29.222744 31396 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:29.228225 31383 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:29.180835  2719 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.234000 30633 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:29.180398  2708 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.246377 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.246634  2549 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.169176  2782 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:15:29.277133 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.187013  2584 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.276515 31361 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:29.279788 31396 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:29.279772 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:29.281602 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.282308 31388 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:29.268543  2759 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:15:29.284987  2795 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:29.285967  2795 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:29.201258  2740 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.310946 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.300685  2753 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:15:29.319546  2279 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.328498  2584 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:29.329274  2279 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:29.329952  2713 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10.
I20260709 13:15:29.331517  2795 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.332080  2795 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.331095 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.335690  2795 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:29.339395 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.345116  2131 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:29.345634  2131 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.350699  2740 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.352555 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:29.353358 30551 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:29.275856  2716 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.356436 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 790) took 1120 ms. Trace:
I20260709 13:15:29.357049  2716 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:29.363584  2767 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.365437  2529 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:29.365967  2529 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:29.366899  2529 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.368083  2529 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.341221  2708 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:29.386770  2531 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.389727  2531 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.366308  2767 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.390275 30707 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:29.395428  2771 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:29.400725 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:29.401710 30633 raft_consensus.cc:2368] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
W20260709 13:15:29.403877 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.404009 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:29.407155 30633 raft_consensus.cc:2368] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:15:29.408012 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:29.408741 30706 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:29.409361 31388 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:29.374151  2719 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.340550 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:29.411883 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71418.4us
I20260709 13:15:29.414575 30633 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:29.410554 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.415982 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:29.417593 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:29.356912 30554 rpcz_store.cc:276] 0709 13:15:28.235835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.236357 (+   522us) service_pool.cc:224] Handling call
0709 13:15:29.356177 (+1119820us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b540013a954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:29.356328 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1601141120}
I20260709 13:15:29.420339 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.423537 31383 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:29.424748 30633 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:29.442456 30626 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [15.8-16.9]
I20260709 13:15:29.444412 30705 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:29.438982 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:29.448603 30707 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:29.418197  2771 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:29.453356 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.460053 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.420558 31388 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18. Re-sending same reply.
I20260709 13:15:29.466101 30624 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:15:29.466264 30626 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:29.464865 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.470180 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.468896 31383 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:29.463442 30706 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
W20260709 13:15:29.473570 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 193878us
I20260709 13:15:29.420117  2740 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:15:29.450692  2531 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.482935 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:29.480324 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.486518 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:29.486677 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.461611  2679 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.464564  2530 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:29.346557  2131 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.490478 30626 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 13:15:29.480063 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.488641 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:29.461628  2707 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:29.494151 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.471510  2701 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:29.492110 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.506951 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:29.499267  2740 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:29.501813 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:29.508262 30628 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:29.472862  2792 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:29.518534 30707 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:29.521327 31292 leader_election.cc:400] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
I20260709 13:15:29.491560  2708 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:15:29.522195 31292 leader_election.cc:403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:15:29.489617  2719 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:29.514998  2445 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:29.524466  2796 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:29.515408  2707 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:15:29.525203  2713 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:29.468015  2797 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:15:29.483052  2675 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:15:29.531023 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.555558 31383 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
W20260709 13:15:29.532294 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45870.4us
I20260709 13:15:29.558382 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:29.544235  2471 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.004s
I20260709 13:15:29.519882  2795 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:15:29.502976  2584 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:29.540354  2748 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.505137  2782 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:29.564325  2791 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:29.564450 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.488384  2531 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.516484  2751 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.505645  2701 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:29.570174 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52388.3us
I20260709 13:15:29.571502 30551 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:29.600993 31292 leader_election.cc:400] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:15:29.579836  2708 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:29.497139  2679 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.595520 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:29.566745  2750 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.516866  2714 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:15:29.580498 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.601365 31292 leader_election.cc:403] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 13:15:29.581765  2765 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.994s	user 0.003s	sys 0.002s
I20260709 13:15:29.602526 31367 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:29.530455  2707 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:15:29.611521 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47192.5us
I20260709 13:15:29.630865 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:29.612656  2800 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:29.582193  2799 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:29.633168 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:29.580462  2471 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.643283 30626 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:29.644622 30624 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:29.653585 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:29.652456 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.655885 30628 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:29.658128 30631 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:29.605492  2771 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:15:29.666814 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.633854  2549 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.708534 30633 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:29.695928 30706 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:29.703787 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.711640 31361 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:29.672499  2701 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.716442 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:29.717489 30705 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:29.632315  2750 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.722550 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.727072 31383 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:29.731542  2697 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:29.733201 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.736099 30706 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:29.707813  2549 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.746153 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.728466  2701 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:29.753243  2710 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.663s	user 0.006s	sys 0.011s
I20260709 13:15:29.758999  2805 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:29.761884  2805 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:29.738749  2697 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.764398  2805 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:29.764118  2697 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.768138 31361 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:29.768980 30705 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:29.772048 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.676942  2740 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.785822 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.803714 31361 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:29.805636 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.793433 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:29.788048 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49837.9us
W20260709 13:15:29.782128 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.787568  2800 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.813274  2532 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:29.813959 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.816895 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.813292 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:29.823657 31388 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:29.823941 30553 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:15:29.825625 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:29.830335 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.831074 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:29.832078 30699 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:29.834821 31388 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:15:29.838048  2803 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 13:15:29.832603  2532 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:29.844465 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:29.780706  2806 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:15:29.851434 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:29.842944  2748 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:29.843703  2531 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:29.861558 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:29.864537  2768 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:29.865207  2768 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:29.867501  2807 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:15:29.868033 31364 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:29.869510 31364 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:29.869884 31364 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:29.870187  2751 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:29.875255  2716 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:29.875844  2716 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.876905  2716 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.883226 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.885334  2795 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:29.886914  2800 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.890288  2716 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:29.886103  2795 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:29.896530  2795 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.900122 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:29.901007 30553 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:29.901186 31383 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:29.848644  2679 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.855860  2749 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:29.852749  2471 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:29.899603 31364 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.912851 31364 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:29.914119 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:29.919955 30558 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:29.926664 30553 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:29.927917 30550 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:15:29.928596 30550 pending_rounds.cc:85] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:15:29.928900 30550 pending_rounds.cc:107] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:15:29.929167 30550 raft_consensus.cc:2889] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:29.932847  2776 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.926s	user 0.000s	sys 0.009s
I20260709 13:15:29.935060  2131 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:29.939633 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1816) took 1681 ms. Trace:
I20260709 13:15:29.940209 30704 rpcz_store.cc:276] 0709 13:15:28.258182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.258360 (+   178us) service_pool.cc:224] Handling call
0709 13:15:28.258780 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:28.718274 (+459494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:28.718317 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:28.721877 (+  3560us) log.cc:844] Serialized 26 byte log entry
0709 13:15:28.800642 (+ 78765us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:28.800695 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:28.800804 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:29.932060 (+1131256us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:29.939250 (+  7190us) raft_consensus.cc:1685] finished
0709 13:15:29.939283 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:29.939539 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242445,"raft.queue_time_us":243496,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":6189,"spinlock_wait_cycles":36161792,"thread_start_us":267167,"threads_started":2,"wal-append.queue_time_us":162854}
I20260709 13:15:29.912122  2814 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:29.947706  2814 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:29.950168  2814 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:29.954121 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:29.955148 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1572) took 1027 ms. Trace:
I20260709 13:15:29.916092  2531 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:29.866293  2768 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.955487 31378 rpcz_store.cc:276] 0709 13:15:28.927949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.930456 (+  2507us) service_pool.cc:224] Handling call
0709 13:15:29.953938 (+1023482us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:29.955075 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4459904}
W20260709 13:15:29.944928  2529 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.003s
I20260709 13:15:29.960800 30553 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:29.961236 31364 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:29.966470 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:29.852442  2740 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:29.928191  2749 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:29.970372 31389 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:15:29.976940 31389 pending_rounds.cc:85] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:29.976783 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 777) took 1829 ms. Trace:
I20260709 13:15:29.971078  2751 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:15:29.958685  2480 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:15:29.969283 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.976058 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:29.977958 31389 pending_rounds.cc:107] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:15:29.983325 31389 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:29.983693 31389 pending_rounds.cc:107] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:15:29.983850 31389 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:29.977486 30557 rpcz_store.cc:276] 0709 13:15:28.147557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.147706 (+   149us) service_pool.cc:224] Handling call
0709 13:15:29.976474 (+1828768us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:29.976696 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1866170752}
I20260709 13:15:29.959234  2750 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:29.985365 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:29.974313  2792 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:29.982658  2480 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:29.991824  2480 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:29.993351  2480 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:29.994068 31391 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:29.999547 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.000558 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:29.976632  2768 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:29.978515  2531 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:30.003880 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1571) took 1068 ms. Trace:
I20260709 13:15:29.986193  2675 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:30.005357 31391 rpcz_store.cc:276] 0709 13:15:28.927627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.931179 (+  3552us) service_pool.cc:224] Handling call
0709 13:15:29.993616 (+1062437us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:15:29.993879 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:29.994566 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:29.995958 (+  1392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4451200}
I20260709 13:15:30.012897 30707 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:30.013948 30699 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:30.014652 30705 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:30.015060 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:30.014467 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.015204 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.012998 30701 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:30.016814 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.017153 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.017939 30698 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:30.019378 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.024395 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.020985 30698 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:30.031296  2549 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.032075  2549 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.023969  2679 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 13:15:30.040295  2787 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 13:15:30.024691  2675 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:30.043593  2675 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:30.047127  2815 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:15:30.049477  2708 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
I20260709 13:15:29.986472  2812 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:15:30.052714  2798 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:30.054461  2798 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:30.055054  2798 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.054862  2749 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:30.058152 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 462790us
I20260709 13:15:30.059424 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.059585  2531 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:30.061548 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:30.062115 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.062947 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:30.066716 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:30.074045 31361 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:30.078712 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:30.083828 30556 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:30.087953 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1540) took 1177 ms. Trace:
I20260709 13:15:30.088366 30556 rpcz_store.cc:276] 0709 13:15:28.910400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.969826 (+1059426us) service_pool.cc:224] Handling call
0709 13:15:30.087866 (+118040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:30.057977  2798 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:30.093632 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:30.098028 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:30.099094 30556 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:30.107733 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:30.108731 30559 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:30.063795  2549 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:30.086321  2753 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:30.114379  2700 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.114506 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:30.114944  2753 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.115597 30700 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:30.116091  2753 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.117465 30557 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:15:30.118533  2530 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.615s	user 0.014s	sys 0.001s
I20260709 13:15:30.119010  2530 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:30.119763  2530 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10.
I20260709 13:15:30.121752  2808 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:30.104808  2534 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:15:30.122152  2808 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:30.124455 30698 raft_consensus.cc:606] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:30.124847 30698 raft_consensus.cc:614] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:30.113201 30624 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:15:30.134420 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.084794  2810 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:15:30.111747  2791 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:30.156281 30628 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
W20260709 13:15:30.156117 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 93573.5us
I20260709 13:15:30.113615  2799 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:30.086061  2795 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:15:30.172263 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.192862 30555 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:15:30.179412 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:30.089243  2670 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:15:30.193665 30552 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:15:30.190348 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:30.127683  2534 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.181393 30558 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 13:15:30.168877 31396 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: [15.9-17.10]
I20260709 13:15:30.119638  2771 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:15:30.197275 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.198019 30556 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:30.198997 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.139940  2792 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.199703 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.201713 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:30.203048 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.115029  2700 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.204241 31383 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:30.205556 30559 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:30.210101 31378 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:30.164397  2584 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:30.217746 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1434) took 1087 ms. Trace:
I20260709 13:15:30.200155  2791 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.260015 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.261173 31383 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:30.267674 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.268568 31367 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:30.209270  2800 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:30.247941 31394 rpcz_store.cc:276] 0709 13:15:29.130530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.130905 (+   375us) service_pool.cc:224] Handling call
0709 13:15:29.153108 (+ 22203us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:29.289130 (+136022us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:29.289182 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:29.295270 (+  6088us) log.cc:844] Serialized 52 byte log entry
0709 13:15:29.554939 (+259669us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:29.583546 (+ 28607us) log.cc:844] Serialized 14 byte log entry
0709 13:15:30.048787 (+465241us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b18001af8c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:30.206269 (+157482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.206395 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:30.206466 (+    71us) raft_consensus.cc:1685] finished
0709 13:15:30.217467 (+ 11001us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.217662 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21501,"prepare.queue_time_us":25995,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":61,"raft.queue_time_us":656990,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":215,"spinlock_wait_cycles":28338432,"thread_start_us":403367,"threads_started":3,"wal-append.queue_time_us":325562}
W20260709 13:15:30.272481 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 671636us
I20260709 13:15:30.275009 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:30.277540 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.280308 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.278409 31391 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:30.287082 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.201153  2532 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.202319  2534 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.296444 31367 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:30.297278 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.299312 31389 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:30.270166  2818 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:30.273311  2800 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.287418  2810 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.287007  2799 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:30.326512 30553 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:30.323671 31389 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:30.323858 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.250478  2792 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.345149 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:30.348701 31378 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 13:15:30.350840 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:30.288139  2771 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:30.356709 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:30.368369 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:30.369032 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.330940  2701 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:30.371168  2532 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:30.371578  2701 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:30.371728  2750 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:30.372057  2532 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:30.372247  2750 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.372150  2701 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:30.372993  2750 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:30.373052  2701 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:30.374432  2534 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:30.376924  2822 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:30.315392  2584 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:15:30.316218  2782 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:15:30.361466  2812 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:30.361143  2753 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:30.372818  2532 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:30.381273  2710 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:30.422948 30702 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:30.425827 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.428237  2700 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:30.430848 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.331516  2810 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.440022 31378 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 8.
I20260709 13:15:30.432221  2710 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.431756  2532 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:30.430459  2798 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:30.457801  2670 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:30.460027  2782 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:15:30.460837  2820 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:15:30.204787  2808 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.466895 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1462) took 1088 ms. Trace:
I20260709 13:15:30.414001  2814 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:30.475584 30550 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:30.478269 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:30.367730  2791 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.292665  2823 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 13:15:30.326418  2821 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 13:15:30.353335  2824 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 13:15:30.484431 30633 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
W20260709 13:15:30.513447  2131 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.575s	user 0.014s	sys 0.002s
W20260709 13:15:30.486653 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 162857us
I20260709 13:15:30.490121 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.516605 30551 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:30.491300 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.489593  2710 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:30.500161 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.500114 30700 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:15:30.493254 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:30.501035 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 135564us
I20260709 13:15:30.502770 30702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
W20260709 13:15:30.472934 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.520956 30702 raft_consensus.cc:606] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
W20260709 13:15:30.521341 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.521498 30702 raft_consensus.cc:614] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:30.490914  2559 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:30.522933  2559 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.525033 30702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:30.502940 31361 rpcz_store.cc:276] 0709 13:15:29.378111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.805205 (+427094us) service_pool.cc:224] Handling call
0709 13:15:30.466606 (+661401us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54008b15d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:30.466794 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557576448}
I20260709 13:15:30.322343  2818 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.502146  2782 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:30.429464  2753 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:30.507711 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:30.510332  2534 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:30.517427 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.395273  2800 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.505546 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:30.525758  2559 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:30.592713 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.593429  2814 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:30.621806 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:30.594348  2829 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 13:15:30.517940  2700 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
W20260709 13:15:30.595252  2826 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 13:15:30.596230  2768 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.591s	user 0.007s	sys 0.008s
I20260709 13:15:30.596184  2708 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:30.596251  2797 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:15:30.598011  2716 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.707s	user 0.009s	sys 0.005s
I20260709 13:15:30.601486  2705 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:30.602373  2480 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.015s
I20260709 13:15:30.604281  2714 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:15:30.605481 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 84479.5us
I20260709 13:15:30.525835 30702 raft_consensus.cc:606] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.691172 30702 raft_consensus.cc:614] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:15:30.619647 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.620191 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:30.693538 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:30.591951 30633 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:15:30.626325 30626 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:30.627593 30624 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:15:30.695458 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 202370us
I20260709 13:15:30.644904 30703 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:30.602406  2813 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:30.655977 30628 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:30.670995 30629 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:30.621209  2534 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:30.648120  2800 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:30.683310 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 176358us
I20260709 13:15:30.614953  2670 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:15:30.696352 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.612593  2812 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:30.709097 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 192635us
I20260709 13:15:30.710570 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.676296  2797 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:30.721290 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:30.659391  2559 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 13:15:30.730355 30628 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:30.719450 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:30.732407 31383 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:30.690659  2714 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.658934  2745 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:15:30.682585  2792 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:30.731808 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:30.738893 31378 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:30.720789  2480 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:30.739362 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.735323 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:15:30.739867 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:30.724324 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 100446us
I20260709 13:15:30.728282  2823 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:30.703084  2795 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:15:30.752090 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.753023 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:30.751804 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58371.2us
I20260709 13:15:30.707746  2748 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:15:30.757402 30628 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:30.761039 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1578) took 1429 ms. Trace:
I20260709 13:15:30.743732  2675 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:30.761749 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.750149  2480 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.766634  2480 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:30.771109  2534 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:30.774183  2819 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.430s	user 0.000s	sys 0.002s
I20260709 13:15:30.775511  2818 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:30.777758  2471 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:30.778363  2471 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:30.779325  2835 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:15:30.779825 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:30.781159  2795 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:30.781618  2795 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.782661  2795 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:30.783123  2667 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:15:30.784868 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45652.7us
I20260709 13:15:30.783296  2810 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:30.790305 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.761571 31388 rpcz_store.cc:276] 0709 13:15:29.331191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.835764 (+504573us) service_pool.cc:224] Handling call
0709 13:15:29.836119 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:30.122867 (+286748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.122900 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:30.123562 (+   662us) log.cc:844] Serialized 52 byte log entry
0709 13:15:30.209274 (+ 85712us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:30.392537 (+183263us) log.cc:844] Serialized 14 byte log entry
0709 13:15:30.430018 (+ 37481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:30.430125 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:30.760782 (+330657us) raft_consensus.cc:1685] finished
0709 13:15:30.760805 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:30.760964 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359059,"prepare.queue_time_us":271543,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":3295,"raft.queue_time_us":325063,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":47566,"thread_start_us":217899,"threads_started":2,"wal-append.queue_time_us":229780}
I20260709 13:15:30.727277  2825 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:15:30.756731  2697 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:30.792522 31367 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:15:30.804481 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72756.8us
I20260709 13:15:30.805186 31367 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:30.806456 31367 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.808030 31367 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:30.760560  2792 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:30.765048  2740 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:30.795142 30554 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:30.783967  2827 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.275s	user 0.002s	sys 0.000s
W20260709 13:15:30.803519 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:30.788167  2667 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:30.754669  2700 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:15:30.788806  2833 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:30.838956 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:30.839809 30704 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 7.
I20260709 13:15:30.840490  2701 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.842283 30704 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:30.794848 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:30.822156 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.846315 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:30.835906  2700 consensus_queue.cc:1243] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:30.844182 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.850171 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.782429  2710 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:30.853493 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.854264 30552 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:30.815716  2792 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:30.857697 30700 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:30.865758  2792 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.867316  2792 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:30.861207 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:30.868764  2821 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:30.869381 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79215.7us
I20260709 13:15:30.828135  2795 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:30.867136 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66946.5us
I20260709 13:15:30.871079 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.831228  2847 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:30.875779 30557 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:15:30.779421  2471 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:30.843562  2745 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:30.863062 31367 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:15:30.882268 31367 pending_rounds.cc:85] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:15:30.787295  2812 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.882669 31367 pending_rounds.cc:107] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:15:30.886209 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:30.886246 31367 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:30.890908 30706 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:30.869477  2821 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:30.896452 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:30.871731  2753 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:30.898491 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:30.900195 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.870663  2833 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:30.948007 30626 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:30.904646 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:30.897291 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.831574  2797 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:30.960964 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63767.6us
I20260709 13:15:30.911142  2700 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:30.883801  2667 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.919621  2840 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:30.920269  2848 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:30.921586  2849 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:30.880028  2751 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:15:30.903196  2841 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:30.913488 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:30.924661  2806 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:30.927227  2845 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:30.851291  2846 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:15:30.932806  2710 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:30.934017  2701 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:30.936686  2750 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.002s
I20260709 13:15:30.937839  2805 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:30.942255 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:30.989509 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47389.6us
I20260709 13:15:30.986078  2667 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:30.937916  2768 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:30.918782  2843 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:30.993644  2843 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:30.993762 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:30.995048 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.995687  2534 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:30.996006 31364 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:30.996336  2534 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:30.988785  2818 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:15:30.997416  2818 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:30.998008  2818 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:30.998283 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.999053 31394 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
W20260709 13:15:30.966924 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66813.1us
I20260709 13:15:31.002362  2805 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:31.002859  2846 consensus_queue.cc:1243] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:15:30.966216 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 182592us
I20260709 13:15:30.959395 30558 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:30.953431  2847 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:30.978785 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65437.8us
I20260709 13:15:30.997323 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:30.958388  2471 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:31.007455 31396 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:31.012501 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:31.017091 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:30.972863  2675 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:30.968622  2725 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:15:31.043641 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.042268  2833 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:30.988304  2750 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:30.968364  2700 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.005403  2856 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.018352  2847 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.038136  2745 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:31.019129  2845 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:31.034880  2848 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.040195  2846 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:30.985780  2841 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.033111  2480 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.065819  2847 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:30.997434  2534 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.081955  2855 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:31.083061  2857 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:15:31.103017 30553 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:31.114603  2750 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:31.079478  2853 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.153s
I20260709 13:15:31.118669  2821 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.064823  2856 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.121166  2782 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.115864  2771 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:31.123566  2771 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:31.124715  2771 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.070895  2848 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:31.126789 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.127082  2782 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:31.128364  2848 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:31.129787 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 124751us
I20260709 13:15:31.071152  2845 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:31.130384 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113439us
I20260709 13:15:31.130874  2840 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:31.131784 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:31.132253  2845 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.091703  2855 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.133989  2841 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.134081 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:31.134586  2805 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:31.135949  2855 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:31.129993  2700 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.168941 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.140045  2480 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:31.140435  2530 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:15:31.177031 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.181592 30698 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:31.169804 30630 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:31.140607  2851 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:31.141439 31396 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:31.143844 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.145481  2856 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:31.138491  2860 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:15:31.150144 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.152808 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.157882 30629 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:15:31.160540 30633 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:31.163537 30628 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:31.126360  2771 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:31.167872 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.171346 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.191911 31388 pending_rounds.cc:85] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 5
I20260709 13:15:31.192466 30632 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:31.173187  2745 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.149317  2837 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:31.163226  2805 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:31.141527  2818 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.177878 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.184819 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:31.198446 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47947.3us
I20260709 13:15:31.167991 30627 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:31.192505 31388 pending_rounds.cc:107] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 13:15:31.185663 31396 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:31.209136 31388 raft_consensus.cc:2889] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.206336 30703 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 8.
I20260709 13:15:31.211362 30702 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:31.206573 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.213410 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:31.216596 31396 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:31.238018 31396 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:31.092080  2870 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:15:31.206439  2805 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:31.199388  2837 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.260399 31367 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:31.233466  2806 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:15:31.276487 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 226808us
I20260709 13:15:31.283547 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:31.286412 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:31.202061  2851 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.259698  2837 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:31.284889  2532 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.776s	user 0.013s	sys 0.002s
I20260709 13:15:31.291944 31383 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:31.294462 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:31.316731  2851 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.291664  2871 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.311460  2818 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.321467 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.262099  2705 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:15:31.302312  2532 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:31.331813 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:31.405787  2881 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:15:31.307626 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.411324 31394 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:31.412930  2787 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:31.413489  2787 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:31.301339 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.338306 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.361547 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:31.419411 30707 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:31.361632 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:31.362403 30702 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:31.362998  2768 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:31.363025 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:31.442412 30701 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:31.442852 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 81314.4us
I20260709 13:15:31.364334 30703 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:31.365470  2839 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:31.448760 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.449569 30707 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:31.451320 31396 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:15:31.451713 31396 pending_rounds.cc:85] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:31.451951 31396 pending_rounds.cc:107] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:15:31.452214 31396 raft_consensus.cc:2889] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.452077 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:31.366369 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71997.4us
I20260709 13:15:31.452750 30555 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:31.365474 30700 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:15:31.367437 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.370874  2864 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:31.459506 30554 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:31.455538 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:31.371380 31383 pending_rounds.cc:85] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:31.465736 31383 pending_rounds.cc:107] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:15:31.466184 31383 raft_consensus.cc:2889] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.294543  2701 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:15:31.379263  2876 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:15:31.385911 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.344753  2856 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:31.354693  2697 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:31.388533  2855 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.295025  2847 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.395350 31367 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:31.414402  2787 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:31.456805  2818 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:31.447891  2471 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:31.467500  2679 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.487263 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:31.464067  2745 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.499907 30555 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:31.471860  2713 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.470521  2701 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.507083 30707 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:31.461357  2531 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:15:31.510944 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.397774  2871 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.513168 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:31.511035  2471 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.510937  2787 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:31.517670 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.501396  2818 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:31.488807  2697 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.519081  2534 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.514277  2679 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.470757  2549 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:15:31.519652  2745 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.522563  2837 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.527899 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.518175 31378 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:31.519797  2879 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:15:31.561859  2534 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.613349  2837 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:31.615155  2534 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:31.549446 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46087us
I20260709 13:15:31.495074  2855 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.527726  2701 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:31.523805  2713 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:31.642166  2713 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.552991  2818 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:31.642796  2855 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.644333  2871 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:31.605708  2679 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:31.607577  2887 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:15:31.608244  2697 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.610435  2805 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.618026  2745 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:31.620546 31388 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 13:15:31.647514 31388 pending_rounds.cc:85] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:31.647818 31388 pending_rounds.cc:107] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:15:31.648075 31388 raft_consensus.cc:2889] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.477756  2856 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.651036  2856 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.652658  2837 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:31.656032 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:31.624501  2878 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 13:15:31.558382  2888 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:31.667786  2888 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:31.635392  2886 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.119s	user 0.004s	sys 0.000s
I20260709 13:15:31.662149  2471 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.679131  2888 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:31.683840  2750 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:15:31.685210  2892 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.690726  2892 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.693866  2892 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.705916 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:31.681197  2679 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:15:31.708025  2805 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:31.716147 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.723183  2808 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:31.706051  2708 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:15:31.727077  2714 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.735492  2532 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:31.739017 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 83127.3us
I20260709 13:15:31.800004 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:31.801013  2837 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:31.752110 30703 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 8. Current term is 9. Ops: []
I20260709 13:15:31.802780  2837 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:31.803364  2837 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:31.804345  2837 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.751922  2874 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:31.753043 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.753309 30700 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:31.807585  2852 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:31.808126  2852 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:31.809384  2852 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.753618 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:31.754802 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:31.784860  2707 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:31.811054  2707 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:31.811132 30701 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:15:31.811472  2707 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:31.755419  2675 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.755748  2797 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:31.759114 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.816275 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:31.816759  2799 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.758976  2697 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:31.817417  2799 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.818192  2697 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:31.814343  2675 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.819252  2697 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.819855  2799 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:31.820564  2675 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:31.761045 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.821532 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.761904  2860 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:31.829586  2851 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:31.763175 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.832067 30632 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:31.764223 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:31.765187 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:31.766353 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.834333 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.835022 30706 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:31.835182 30705 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 9.
I20260709 13:15:31.767426 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.836539 30624 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:31.768617 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.837405 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.769147  2714 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:31.769893  2471 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:31.770776  2713 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:31.772581 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:31.773480 30702 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:31.774026 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:31.775197 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:31.776522 30551 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:31.916472 30557 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:31.777220  2549 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:31.784646 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 329240us
I20260709 13:15:31.918376 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:31.919284 30703 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:31.786201 30555 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:31.787236  2795 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.788030  2805 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:31.788290  2740 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:31.791121  2667 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:31.791863 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.937866 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:31.792357 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.794039  2823 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.942955  2823 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:31.945400  2823 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.946276  2824 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:31.946911  2824 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.949797  2824 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:31.951730 31367 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:15:31.953213 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.760919 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1079) took 2079 ms. Trace:
I20260709 13:15:31.765087  2847 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:31.820855  2799 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:31.956261  2799 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:31.957494  2799 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:31.955664 30631 rpcz_store.cc:276] 0709 13:15:29.663995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:29.678522 (+ 14527us) service_pool.cc:224] Handling call
0709 13:15:29.713485 (+ 34963us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:30.858330 (+1144845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:30.858377 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:30.865614 (+  7237us) log.cc:844] Serialized 52 byte log entry
0709 13:15:30.959078 (+ 93464us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:30.968994 (+  9916us) log.cc:844] Serialized 14 byte log entry
0709 13:15:30.981645 (+ 12651us) log.cc:844] Serialized 14 byte log entry
0709 13:15:31.098135 (+116490us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b180041a1c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:31.127627 (+ 29492us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b180041a1c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:31.127944 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:31.742397 (+614453us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54004aa6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:31.742473 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:31.742545 (+    72us) raft_consensus.cc:1685] finished
0709 13:15:31.742566 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:31.742779 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":201080,"prepare.queue_time_us":135113,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":94,"raft.queue_time_us":191464,"raft.run_cpu_time_us":419,"raft.run_wall_time_us":12108,"spinlock_wait_cycles":2553281920,"thread_start_us":92987,"threads_started":1,"wal-append.queue_time_us":191126}
I20260709 13:15:31.839075 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.959391 30628 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:31.840142 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.844080 30633 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:15:31.848356 31361 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:31.961812 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.962635 31361 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:31.963724 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.964576 31361 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:15:31.855737 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.857551 31396 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:31.967131 31394 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:31.967803 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.968397 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:31.969316 31394 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
W20260709 13:15:31.813973 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60430.9us
I20260709 13:15:31.863250  2701 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.973024  2701 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:31.919811 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.921222 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:31.925261 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:31.927093 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:31.930497 30554 pending_rounds.cc:85] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:31.981673 30554 pending_rounds.cc:107] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 12.6
I20260709 13:15:31.982089 30554 raft_consensus.cc:2889] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:31.983378 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:31.985196 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45550.3us
I20260709 13:15:31.995631 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.987068 30557 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:31.933183 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:31.934725 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.998778 30627 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:31.936378 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:31.968578 31396 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:31.857738 31378 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:31.983716 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:31.932703 30701 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
W20260709 13:15:31.999210 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66594.6us
I20260709 13:15:32.006985 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.007968 30551 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:32.016678 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.018755 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:32.019630 30625 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
W20260709 13:15:32.021343 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44382.4us
I20260709 13:15:32.024106 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:31.925706  2532 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:31.916257  2839 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:32.026397 31378 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:32.029104 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:32.030371 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:32.017729 30555 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:32.042119 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:31.805570  2888 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.047667 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.053901 31378 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:32.055984 31378 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:32.058533 30555 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:32.048022 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:31.998165  2877 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:32.071249 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:32.018114  2852 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:31.918969  2471 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.931401  2713 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:31.931752  2795 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:31.937955  2549 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.010274  2870 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:15:32.002074  2667 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.066114  2855 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:32.110376 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:32.112802 30703 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:32.113931 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:32.089941  2882 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:15:32.123317 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.037590  2751 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:15:32.156427  2751 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:32.156975  2751 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:32.050163  2532 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.093288  2708 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.163086  2708 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.164371  2532 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.165264  2532 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.165968  2708 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.084439  2877 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:32.181330  2818 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.538s	user 0.004s	sys 0.007s
I20260709 13:15:32.091576 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.125713 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:32.187577 30624 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:32.165889  2532 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.117182 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:32.086628  2871 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:15:32.103238  2895 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:15:32.121178  2855 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.046137  2805 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:15:32.135852 30702 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:32.140618  2471 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.144816 31361 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:32.205559 31394 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:15:32.206151 31394 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:32.206440 31394 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:15:32.206768 31394 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.206563 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:32.065623  2806 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:15:32.093516  2851 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:15:32.149128  2713 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.078058  2700 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:15:32.150007  2549 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.152274  2795 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.153053 30553 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:32.200425  2708 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.119743  2860 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:15:32.125084 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.210067 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:32.211330 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94244.1us
W20260709 13:15:32.218937 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 105186us
W20260709 13:15:32.242406 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.245412 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.197068  2871 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.158167  2751 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.153740  2839 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:32.247483  2480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.019s	sys 0.021s
I20260709 13:15:32.204113  2855 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:32.284292 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.291464 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:32.292255 30558 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:32.296281 30629 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:32.297716 30627 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:32.297634 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:32.193115  2818 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.265954  2894 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:32.298945  2705 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:32.309057 30551 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:32.344707 30624 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:32.300976 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:32.310016 31383 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:15:32.371276 31383 pending_rounds.cc:85] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:32.371630 31383 pending_rounds.cc:107] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:15:32.371877 31383 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.301600 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:32.315446 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:32.250380  2870 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:32.319581  2700 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.263355  2713 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.321004  2667 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.328281 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:32.302996  2808 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.007s
I20260709 13:15:32.302220  2532 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.454517 31389 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:15:32.454952 31389 pending_rounds.cc:85] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:32.455179 31389 pending_rounds.cc:107] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:15:32.455417 31389 raft_consensus.cc:2889] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:32.337141 30702 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 9.
I20260709 13:15:32.202111  2707 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.368854 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:32.459195 31367 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:15:32.459388 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90696.9us
I20260709 13:15:32.459913  2823 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.460346 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:32.460467  2823 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.323817  2871 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:32.446234  2892 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:15:32.465435  2892 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:32.465795  2892 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:32.393268 30558 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:15:32.347005  2480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.019s	sys 0.022s
I20260709 13:15:32.447084 30555 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:32.389882  2870 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.451459  2713 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.368212  2894 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.368486  2705 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:32.295368  2708 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:32.497290 30705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.017s	sys 0.045s
W20260709 13:15:32.498067 30705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.017s	sys 0.045s
W20260709 13:15:32.479571 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.013s
I20260709 13:15:32.386934  2534 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:32.498857 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.006s	sys 0.013s
I20260709 13:15:32.499207  2534 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.502452  2534 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:32.467828  2707 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:32.469520  2787 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:32.519992 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.481554  2870 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:32.479283  2808 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:32.521276 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.482645  2894 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:32.432103  2897 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 13:15:32.532915 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:32.446708  2700 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.537544 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:32.522444  2532 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.544854 30524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.017s
I20260709 13:15:32.541242 30707 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:15:32.502832 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.018s	sys 0.004s
W20260709 13:15:32.530764  2714 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.664s	user 0.014s	sys 0.000s
W20260709 13:15:32.549220 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.018s	sys 0.004s
I20260709 13:15:32.551194 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.551508 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:32.548623 30524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.008s	sys 0.018s
I20260709 13:15:32.546496 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:32.557543 30556 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:15:32.538944  2795 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:32.494395  2892 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:32.559188 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.534478  2480 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:15:32.561865 31364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.006s	sys 0.022s
I20260709 13:15:32.543879 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.564047 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.555686 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:32.615478 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59898.5us
W20260709 13:15:32.571309 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.559363 30554 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.020s
I20260709 13:15:32.550068  2667 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:32.563295  2714 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.562748  2795 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.600725 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:32.611618 31391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.024s	sys 0.029s
W20260709 13:15:32.631623 31391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.024s	sys 0.029s
I20260709 13:15:32.632969 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:32.618736 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.581950  2892 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:32.551276  2532 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.658631 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:32.564909 30555 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
W20260709 13:15:32.620925 30554 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.009s	sys 0.020s
W20260709 13:15:32.681443 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:32.719439 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.624194  2667 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:32.651168  2795 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:32.687316 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.550s (SENT) [suppressed 18 similar messages]
I20260709 13:15:32.660969  2818 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:32.695904  2899 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 13:15:32.716651  2903 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:15:32.701470 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.659823  2799 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:32.707216 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:32.711709  2532 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.651172  2714 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.667038  2907 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:32.690932  2713 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:32.720690 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:32.731294 30555 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:15:32.723896  2667 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:32.697813  2913 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:15:32.715309  2480 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.746650 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.754081 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:32.760193 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.751777  2907 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:32.774835 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.016s	sys 0.055s
W20260709 13:15:32.775400 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.016s	sys 0.055s
I20260709 13:15:32.755606  2700 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.763428  2532 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:15:32.771203  2907 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:32.776988  2906 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:32.788865 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:32.777426  2913 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:32.784485  2805 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.771965  2480 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.784870  2740 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:32.793884  2708 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.752456  2799 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.817899  2797 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.824563  2532 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:32.826735  2906 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:32.806591  2804 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:32.795120  2675 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
W20260709 13:15:32.841074 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 224101us
I20260709 13:15:32.817782  2787 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.797974  2713 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:15:32.847723 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.848054 31388 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:15:32.840230  2909 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:15:32.811472  2812 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:15:32.869429  2714 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.827877  2740 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:32.817763  2805 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.873649  2740 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.875382  2805 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:32.876438 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:32.880098  2679 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:32.857926  2906 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:32.883679 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:32.883365  2906 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.884433 30707 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:32.886524 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1869) took 1050 ms. Trace:
I20260709 13:15:32.888027  2889 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:32.888527  2889 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:32.889400  2889 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.862360  2787 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:32.893965 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.895468 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2152) took 1122 ms. Trace:
I20260709 13:15:32.894693  2913 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:32.895920 30553 rpcz_store.cc:276] 0709 13:15:31.773536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.773707 (+   171us) service_pool.cc:224] Handling call
0709 13:15:32.895273 (+1121566us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540052b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:32.895400 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544112896}
I20260709 13:15:32.897418  2480 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.887204 30704 rpcz_store.cc:276] 0709 13:15:31.835994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.990056 (+154062us) service_pool.cc:224] Handling call
0709 13:15:31.990677 (+   621us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.033931 (+ 43254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.033986 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.076877 (+ 42891us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.623296 (+546419us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:32.623370 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.623494 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:32.885023 (+261529us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:32.886203 (+  1180us) raft_consensus.cc:1685] finished
0709 13:15:32.886227 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:32.886447 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40712,"raft.queue_time_us":47767,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5700,"spinlock_wait_cycles":90173824,"thread_start_us":422309,"threads_started":1,"wal-append.queue_time_us":682566}
I20260709 13:15:32.842990  2797 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:32.902235  2705 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.889667  2713 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:32.907609 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.908668 31383 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:32.846033  2804 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.890540  2889 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:32.911845  2705 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:32.914458  2911 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.001s
I20260709 13:15:32.894008  2787 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:32.923782 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:32.922380  2804 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:32.933862 30707 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:15:32.926014  2855 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.537s	user 0.015s	sys 0.000s
W20260709 13:15:32.915621 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 228340us
I20260709 13:15:32.886463  2906 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:32.921725  2679 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:32.946048 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.926524  2871 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:32.964475  2871 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:32.918949  2471 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:15:32.968902 30703 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:32.937736  2925 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:32.971885  2925 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:32.972285  2925 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:32.973106 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:32.973796 31394 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:32.987583 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1404) took 2814 ms. Trace:
I20260709 13:15:32.988142 31389 rpcz_store.cc:276] 0709 13:15:30.172809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.343127 (+170318us) service_pool.cc:224] Handling call
0709 13:15:30.345200 (+  2073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.455907 (+2110707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.987223 (+531316us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b54008b2754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:32.987487 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4849956736}
I20260709 13:15:32.989674 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.977903  2797 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:32.993345 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:32.994320 31389 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:32.990342  2864 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:33.154537 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:33.066231  2916 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:15:33.030157  2714 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:15:33.017742  2920 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:15:33.020255 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 726) took 1515 ms. Trace:
I20260709 13:15:32.946431  2679 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.035889  2823 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.018925 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1562) took 1172 ms. Trace:
I20260709 13:15:33.046783  2808 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:32.807469 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:33.051628 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1558) took 1213 ms. Trace:
I20260709 13:15:33.068518  2805 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.025276 30558 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:33.191515 30559 rpcz_store.cc:276] 0709 13:15:31.838388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.838528 (+   140us) service_pool.cc:224] Handling call
0709 13:15:31.913420 (+ 74892us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.191980 (+278560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.192035 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.203456 (+ 11421us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.752180 (+548724us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:32.752224 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.889580 (+137356us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:32.900287 (+ 10707us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b48004506d8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:15:33.051182 (+150895us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:33.051255 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.051327 (+    72us) raft_consensus.cc:1685] finished
0709 13:15:33.051344 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.051552 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112388,"raft.run_cpu_time_us":2595,"raft.run_wall_time_us":596165,"spinlock_wait_cycles":718519168,"thread_start_us":468551,"threads_started":1,"wal-append.queue_time_us":594565}
I20260709 13:15:33.025456  2745 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.189313 30550 rpcz_store.cc:276] 0709 13:15:31.846114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.004428 (+158314us) service_pool.cc:224] Handling call
0709 13:15:32.054579 (+ 50151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.127781 (+ 73202us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:32.128177 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 13:15:32.817644 (+689467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:32.817713 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:32.817767 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.018648 (+200881us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b54000208d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:33.018706 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.018856 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":132,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":66,"raft.queue_time_us":120802,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":15769,"spinlock_wait_cycles":167406464,"thread_start_us":687888,"threads_started":2,"wal-append.queue_time_us":623431}
W20260709 13:15:33.049718  2912 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.232s	user 0.000s	sys 0.003s
I20260709 13:15:33.038790  2888 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.019280 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1424) took 1163 ms. Trace:
W20260709 13:15:33.201506 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:32.943121  2855 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.051494  2812 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:33.091266 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48754.1us
I20260709 13:15:33.207785 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:32.998579  2902 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.397s	user 0.000s	sys 0.001s
I20260709 13:15:33.054953 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1590) took 1113 ms. Trace:
W20260709 13:15:33.009196 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.211120 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:33.212457 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.214017 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:33.043790 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2135) took 2223 ms. Trace:
I20260709 13:15:33.210425 31367 rpcz_store.cc:276] 0709 13:15:31.941108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.997493 (+ 56385us) service_pool.cc:224] Handling call
0709 13:15:32.036631 (+ 39138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:32.459660 (+423029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.053004 (+593344us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54003377d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:33.054867 (+  1863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971061120}
I20260709 13:15:33.018308  2799 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:33.047164  2837 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:32.998930  2814 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:15:33.107321 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.225106 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:33.077383 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.158406  2667 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:33.124151 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.122244 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.230325 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.030815  2700 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:33.233204  2700 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.064059  2860 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:33.024575 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:33.147435 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 363405us
I20260709 13:15:33.000710  2870 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:33.150185 31394 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:33.247669 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:33.162422 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:33.249014 31389 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:33.159466 30698 rpcz_store.cc:276] 0709 13:15:31.505069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.505278 (+   209us) service_pool.cc:224] Handling call
0709 13:15:33.020026 (+1514748us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400342454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:33.020189 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816354176}
W20260709 13:15:33.183301 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 304867us
W20260709 13:15:33.194079 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 382795us
I20260709 13:15:33.198436 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1559) took 1353 ms. Trace:
I20260709 13:15:33.167577  2864 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:33.201406 31364 rpcz_store.cc:276] 0709 13:15:31.856138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.856253 (+   115us) service_pool.cc:224] Handling call
0709 13:15:31.919857 (+ 63604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.920253 (+   396us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:31.920284 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:31.933005 (+ 12721us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.616776 (+683771us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:32.616845 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.616953 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.019037 (+402084us) raft_consensus.cc:1685] finished
0709 13:15:33.019067 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.019216 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":258,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":55566,"thread_start_us":541322,"threads_started":1,"wal-append.queue_time_us":749060}
I20260709 13:15:33.160106  2714 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:33.263455 30555 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:33.200207  2805 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.220659 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:33.226965 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 204466us
W20260709 13:15:33.116604  2894 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.003s
W20260709 13:15:33.227563 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.228060 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:33.220393  2874 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:33.269783 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:33.204154  2745 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:33.253712 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.225160  2814 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:33.200430  2808 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.222076  2892 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:33.214746 30556 rpcz_store.cc:276] 0709 13:15:30.820376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:30.823022 (+  2646us) service_pool.cc:224] Handling call
0709 13:15:30.823415 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.558155 (+1734740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.043419 (+485264us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54002d69d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:33.043674 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3984212224}
I20260709 13:15:33.068732 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:33.266067 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:33.256958 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.224834  2839 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:33.270987 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1495) took 1256 ms. Trace:
I20260709 13:15:33.250450 30554 rpcz_store.cc:276] 0709 13:15:31.845057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.845202 (+   145us) service_pool.cc:224] Handling call
0709 13:15:31.864161 (+ 18959us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:32.719192 (+855031us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.719232 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:32.719914 (+   682us) log.cc:844] Serialized 52 byte log entry
0709 13:15:32.919471 (+199557us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:33.197738 (+278267us) log.cc:844] Serialized 14 byte log entry
0709 13:15:33.198055 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.198144 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.198203 (+    59us) raft_consensus.cc:1685] finished
0709 13:15:33.198216 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.198368 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10151,"raft.queue_time_us":279081,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":41818,"thread_start_us":1135833,"threads_started":3,"wal-append.queue_time_us":286996}
I20260709 13:15:33.272851 30550 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:15:33.274811 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:33.277786 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1423) took 1422 ms. Trace:
I20260709 13:15:33.241490  2799 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:15:33.265946  2795 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.007s
I20260709 13:15:33.290109 31391 rpcz_store.cc:276] 0709 13:15:31.855820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.856145 (+   325us) service_pool.cc:224] Handling call
0709 13:15:31.982081 (+125936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:31.982511 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:31.982543 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:32.014334 (+ 31791us) log.cc:844] Serialized 26 byte log entry
0709 13:15:32.632113 (+617779us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:32.632169 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.632250 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.048896 (+416646us) spinlock_profiling.cc:243] Waited 58.1 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:33.266998 (+218102us) raft_consensus.cc:1685] finished
0709 13:15:33.267025 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.277715 (+ 10690us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":23338,"spinlock_wait_cycles":133707648,"thread_start_us":544243,"threads_started":1,"wal-append.queue_time_us":713557}
W20260709 13:15:33.282018 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.299710 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:33.286280 31378 rpcz_store.cc:276] 0709 13:15:32.014274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.065348 (+ 51074us) service_pool.cc:224] Handling call
0709 13:15:32.066282 (+   934us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:32.067020 (+   738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.067064 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:32.097021 (+ 29957us) log.cc:844] Serialized 52 byte log entry
0709 13:15:32.609847 (+512826us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:32.609913 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:32.610033 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:33.065191 (+455158us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:33.263795 (+198604us) raft_consensus.cc:1685] finished
0709 13:15:33.263818 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.270893 (+  7075us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7253,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":4434,"spinlock_wait_cycles":203881216,"thread_start_us":401207,"threads_started":1,"wal-append.queue_time_us":647429}
I20260709 13:15:33.260087  2888 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:33.261646  2864 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:33.220659  2812 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.228817  2837 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:33.311968 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 243342us
W20260709 13:15:33.324740 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44959.7us
W20260709 13:15:33.330318 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109770us
I20260709 13:15:33.334013 31389 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:33.337986 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:33.330761 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82090.9us
I20260709 13:15:33.282313  2797 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.339380  2797 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.340301 31389 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:33.341876  2797 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:33.298280  2707 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:33.269107  2713 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:15:33.258466  2924 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 13:15:33.249147  2860 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:33.344414 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.259370  2907 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.264925  2913 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:15:33.297271  2877 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.304729  2705 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:33.353325 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:33.355355 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:33.353627 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70449.6us
I20260709 13:15:33.292542  2740 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:15:33.349905  2707 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:33.332986  2864 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:33.293020  2806 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:15:33.360751 31389 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:33.390183 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:33.336613  2888 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:33.352347  2531 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:33.284060  2894 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:33.369035 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:33.306658  2795 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.354303  2856 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:33.386505 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:33.394976 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.362727  2705 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:33.350521  2906 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:15:33.376446  2874 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:33.381078  2707 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:15:33.407271 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61255.4us
W20260709 13:15:33.438932 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52542.7us
I20260709 13:15:33.439056 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.378883  2839 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.380990  2892 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:33.440089 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.351369  2714 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.448836 30557 raft_consensus.cc:2368] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:15:33.410135  2705 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:33.462976 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94066.7us
I20260709 13:15:33.391278  2701 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:15:33.437157  2679 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.448019  2892 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.456230  2823 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:33.369578  2814 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:33.485651 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 138023us
I20260709 13:15:33.546782 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:33.455010  2839 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.547761 30701 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:33.459468  2713 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:15:33.481019  2667 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:33.480868  2892 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:33.481151  2870 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:33.482879  2856 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:33.538916  2797 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:33.584156 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 145249us
I20260709 13:15:33.548305  2814 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:33.547097  2894 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:33.552256  2922 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:15:33.557053  2942 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:33.536429  2740 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:15:33.590703 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.557452  2906 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:33.617595 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.582365  2745 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:33.616729 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 267821us
I20260709 13:15:33.503368  2823 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.581892  2805 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.579878  2808 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.620918 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:33.614818 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:33.483363  2888 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.577597  2950 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:33.596894  2870 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.593741  2797 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.589239  2812 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:33.620675  2932 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
I20260709 13:15:33.594223  2837 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.607928  2856 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.634598 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:33.639648  2700 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.627808  2667 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:33.617261  2480 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:33.593879  2701 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:15:33.642822  2812 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:33.643090 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1504) took 1161 ms. Trace:
I20260709 13:15:33.643504  2812 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:33.644126  2812 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:33.643697  2939 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
W20260709 13:15:33.647137  2940 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:15:33.649812  2751 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:33.620258  2860 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:33.663486 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:33.643647 31396 rpcz_store.cc:276] 0709 13:15:32.482111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.773666 (+291555us) service_pool.cc:224] Handling call
0709 13:15:32.774042 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:32.849244 (+ 75202us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:32.994365 (+145121us) log.cc:844] Serialized 14 byte log entry
0709 13:15:33.206992 (+212627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:33.207086 (+    94us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:33.207162 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.299680 (+ 92518us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b1800024be0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419
0709 13:15:33.642778 (+343098us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400372dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:33.642855 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:33.643028 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99,"prepare.queue_time_us":258307,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":2036,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":8900,"spinlock_wait_cycles":187585536,"thread_start_us":354580,"threads_started":2,"wal-append.queue_time_us":369346}
W20260709 13:15:33.671479  2925 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.699s	user 0.015s	sys 0.006s
I20260709 13:15:33.628341  2808 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.626888  2907 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.585718  2874 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:33.644227  2795 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:33.669718  2937 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:15:33.819408 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 340912us
W20260709 13:15:33.822564 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139650us
W20260709 13:15:33.851413 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 387314us
I20260709 13:15:33.664529  2856 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.667729  2870 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.821013  2860 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.853835  2923 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.911813 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:33.911119 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.767534  2925 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:33.866767  2531 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:33.888370  2837 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:33.935151 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:33.853008  2871 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:15:33.830927  2808 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:33.897169  2961 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:33.943761 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 282459us
I20260709 13:15:33.946986 30559 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:33.948415 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.949849 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:33.895341  2924 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.091s
I20260709 13:15:33.954855 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.954873 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.955789 30559 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:33.935936  2962 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:33.966641  2962 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:33.967468  2962 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.968894  2531 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:33.956944  2961 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.970559  2856 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.972314  2808 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:33.972880  2961 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:33.973126  2808 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:33.973577 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:33.950037  2837 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:33.974545 30553 raft_consensus.cc:2410] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:15:33.975909  2949 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:15:33.956912  2924 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.008s	sys 0.092s
I20260709 13:15:33.976608  2837 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.018960 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.020058 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:34.020222 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.020491 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.020797 30627 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:33.950399  2532 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.027172 30628 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:33.933234  2860 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.040216 30627 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:34.085664 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:33.938632  2923 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:34.039522  2804 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.096s	user 0.011s	sys 0.009s
I20260709 13:15:34.023218  2839 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:33.978397  2910 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
W20260709 13:15:34.101049  2889 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.153s	user 0.006s	sys 0.018s
I20260709 13:15:34.132496 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.140744 30628 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:34.143826 30555 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:15:34.024730  2946 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:15:34.161572 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.041270  2532 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.169687 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:34.159021  2965 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:15:34.178704 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1609) took 1171 ms. Trace:
I20260709 13:15:34.179616  2925 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.142127  2823 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.180497  2962 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.180495  2870 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.144804  2967 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.179234 31367 rpcz_store.cc:276] 0709 13:15:33.003482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.328043 (+324561us) service_pool.cc:224] Handling call
0709 13:15:33.350293 (+ 22250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.362442 (+ 12149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.362500 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.363120 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.853462 (+490342us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:33.853552 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.853817 (+   265us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.174355 (+320538us) raft_consensus.cc:1685] finished
0709 13:15:34.174382 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.174567 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":458,"raft.queue_time_us":11747,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":10947,"thread_start_us":489772,"threads_started":1,"wal-append.queue_time_us":583033}
I20260709 13:15:34.186496 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:34.149991  2797 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:34.182602  2751 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 13:15:34.190269 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:34.192688 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:34.137457  2804 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.244530 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.244590 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.251387 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.251619 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.182296  2967 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.257352 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.260572 30698 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:34.282423 30624 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:34.239522  2943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.000s	sys 0.012s
I20260709 13:15:34.242624  2954 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.288283 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 334634us
I20260709 13:15:34.308511 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:34.303459 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.008s	sys 0.017s
W20260709 13:15:34.314443 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.008s	sys 0.018s
W20260709 13:15:34.315416 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 777647us
I20260709 13:15:34.316810 30698 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:34.286859  2943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.000s	sys 0.013s
I20260709 13:15:34.312803 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.325649 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:34.284233  2740 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.305282  2952 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.021s
I20260709 13:15:34.345903 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:34.339819  2864 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.944s	user 0.013s	sys 0.003s
W20260709 13:15:34.392207  2705 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.917s	user 0.015s	sys 0.000s
I20260709 13:15:34.391501  2864 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.392964  2705 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.394989 30702 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:34.396659  2864 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.398237  2705 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.341434  2740 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.318158  2973 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:34.310580  2944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.004s	sys 0.022s
I20260709 13:15:34.350760 30706 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:15:34.338953  2952 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.004s	sys 0.022s
W20260709 13:15:34.395090  2976 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:15:34.375885 30700 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:15:34.375046 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:34.344017  2892 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.753s	user 0.015s	sys 0.000s
I20260709 13:15:34.340418  2531 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:34.420728 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:34.382824 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.311882  2797 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:34.383876 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1550) took 2318 ms. Trace:
W20260709 13:15:34.388216  2970 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:15:34.388646  2818 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:34.389125  2894 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.390664  2967 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.351342  2923 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.374019 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.312275  2954 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.373817  2860 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.418169 30701 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:34.426574  2818 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.427539  2894 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.449137  2894 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.423094  2892 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.453116  2892 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.455349 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1608) took 1452 ms. Trace:
W20260709 13:15:34.456317 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:34.457305  2532 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.460471  2954 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:34.460871  2935 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.113s	user 0.004s	sys 0.006s
I20260709 13:15:34.453117  2818 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.431704  2531 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:34.417847  2944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.008s	sys 0.022s
I20260709 13:15:34.464517  2531 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.470549  2855 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:34.472250  2814 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.878s	user 0.014s	sys 0.000s
W20260709 13:15:34.472285  2707 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.023s	user 0.015s	sys 0.000s
I20260709 13:15:34.474064  2707 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.477270  2707 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.434860 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.480523 30705 raft_consensus.cc:2410] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:15:34.481555 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:34.482175 30705 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:34.440034 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:34.483342 30632 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:34.442543 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.487073  2975 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.487840  2975 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.490399 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.491503 30701 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:34.423296 30698 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:34.479168  2714 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:34.422335 30625 rpcz_store.cc:276] 0709 13:15:32.056667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.056871 (+   204us) service_pool.cc:224] Handling call
0709 13:15:32.088321 (+ 31450us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:32.159259 (+ 70938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:32.159312 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:32.890804 (+731492us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:32.891459 (+   655us) log.cc:844] Serialized 52 byte log entry
0709 13:15:33.194991 (+303532us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:33.195041 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.351542 (+1156501us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:34.374591 (+ 23049us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.374668 (+    77us) raft_consensus.cc:1685] finished
0709 13:15:34.374683 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.374908 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":140,"spinlock_wait_cycles":205836800,"thread_start_us":303091,"threads_started":1,"wal-append.queue_time_us":413428}
I20260709 13:15:34.495298 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.456064 31378 rpcz_store.cc:276] 0709 13:15:33.003099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.301440 (+298341us) service_pool.cc:224] Handling call
0709 13:15:33.301841 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.302205 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.302232 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:33.302753 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:15:33.678006 (+375253us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:33.679176 (+  1170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:33.679263 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.454988 (+775725us) raft_consensus.cc:1685] finished
0709 13:15:34.455029 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.455245 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":831,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":10871,"spinlock_wait_cycles":1379200,"thread_start_us":330392,"threads_started":1,"wal-append.queue_time_us":915643}
I20260709 13:15:34.503563 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.504391 31383 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:34.502166 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.424862  2797 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.510674 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.519641 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.521198  2797 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.522179 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.523639 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.524160  2707 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.524936 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.524999  2707 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.528137 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.529188  2471 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:34.529999  2471 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:34.530185  2705 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.530695  2892 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.531324  2892 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.531098  2705 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:34.535219  2812 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.890s	user 0.015s	sys 0.000s
I20260709 13:15:34.535468  2962 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.535885  2812 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.536340  2962 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.538847  2823 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.540132 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.521339 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.541646 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:34.531581 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:34.533757  2892 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.539702  2823 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.551170  2797 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:34.548352 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.550510 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.565404 31389 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:34.551642 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.530062 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.567453 31364 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:34.512993  2856 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.502539  2860 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.490403  2975 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.558462 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.560850 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.544961  2705 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.559208 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.528221  2707 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.544935  2808 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.544965  2864 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.580202 31361 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:34.598737 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.599722 30698 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:34.601059 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:34.548013  2839 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.561681  2962 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.553498  2713 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:34.573894 31367 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:34.578567 31383 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:34.570528 30705 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:34.573859  2471 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:34.607316 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.581033  2812 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.610587  2713 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.582468  2797 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:34.602576  2864 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.623723 30698 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:15:34.597299  2808 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.629185 31389 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:34.610265  2967 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:15:34.615677  2962 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.608309  2839 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.577147  2860 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:34.649364 30707 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:15:34.653723  2975 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:34.573508  2856 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.619415  2532 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:34.640950  2992 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:34.625716  2713 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:34.630640 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82707.5us
I20260709 13:15:34.690531 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:34.623631 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:34.654531  2975 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.666241  3000 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:34.671422  2808 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.704888  2808 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:34.705093  2975 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.705600  2808 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:34.711078  3000 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:34.712033 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 735) took 1502 ms (client timeout 1606 ms). Trace:
W20260709 13:15:34.712486 30702 rpcz_store.cc:269] 0709 13:15:33.209883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.210798 (+   915us) service_pool.cc:224] Handling call
0709 13:15:34.711784 (+1500986us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400222ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:34.711960 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2720489216}
I20260709 13:15:34.713930 30699 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 13:15:34.717336 30557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 810) took 2166 ms (client timeout 1618 ms). Trace:
W20260709 13:15:34.717767 30557 rpcz_store.cc:269] 0709 13:15:32.550721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:32.550906 (+   185us) service_pool.cc:224] Handling call
0709 13:15:34.717076 (+2166170us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b540019b754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:34.717224 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2062254848}
I20260709 13:15:34.721338 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.672847  2860 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.674237 30698 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:15:34.646057  2962 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:34.692876 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69328.5us
I20260709 13:15:34.688987  2799 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:34.729851 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:34.733852 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.696966  2532 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:34.732537  2910 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:34.743990 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.752946 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.754112 31383 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:34.737589  2928 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:34.759963 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 101058us
I20260709 13:15:34.739413  2973 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:34.762042 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70069.7us
I20260709 13:15:34.746874  2532 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:34.746079  2823 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.767989 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:34.772928 31389 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:34.775460 31367 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: [15.9-18.11]
I20260709 13:15:34.776726 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.778388 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:34.780946 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.782351  2967 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.739001  2799 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.783572 31383 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:34.784119 31378 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:15:34.789855 31378 pending_rounds.cc:85] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:15:34.790140 31378 pending_rounds.cc:107] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:15:34.790416 31378 raft_consensus.cc:2889] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:34.790433 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.785471 31367 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:34.724664  2808 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.760491  2910 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.791414 31383 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:34.767091  2532 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:34.795169 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.796710 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.811539 31367 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:34.813045 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.818244 31389 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:34.801961 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 734) took 1592 ms. Trace:
I20260709 13:15:34.787829  2973 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:34.785920  2823 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:34.836333 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.841393 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:34.804759 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.826562 30703 rpcz_store.cc:276] 0709 13:15:33.209541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.210717 (+  1176us) service_pool.cc:224] Handling call
0709 13:15:33.226178 (+ 15461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:33.357413 (+131235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.357451 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:33.357499 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:33.357536 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.801660 (+1444124us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540078bc54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:34.801719 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.801878 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301260160}
I20260709 13:15:34.810890  3000 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.843660 31367 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:34.846057 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.846161 31383 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:34.848960 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.836020  2928 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:34.864673 31396 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:34.882041 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:34.853334 31361 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:34.840835  3005 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:34.837962  2823 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:34.859483 31367 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:15:34.900933 31383 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:34.845536  3003 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:34.888316 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 107513us
I20260709 13:15:34.797756  2967 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.906522 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:34.844849  2864 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.910400 31383 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:15:34.895910  2995 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
W20260709 13:15:34.914911  3002 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 13:15:34.918820  2997 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:15:34.920848  2936 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.485s	user 0.012s	sys 0.006s
I20260709 13:15:34.922088  3009 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:34.923197  2998 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:15:34.924101  2962 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:34.924693 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2162) took 1692 ms. Trace:
I20260709 13:15:34.924855  2856 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.812019  2910 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:34.922868  3009 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.910827  2823 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:34.935356  2928 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.927951  2740 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:34.953493 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1582) took 1043 ms. Trace:
I20260709 13:15:34.949431  2913 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:34.958490  2933 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:15:34.961690 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55264.2us
I20260709 13:15:34.911610  3005 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:34.965807  2962 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:34.962556 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:34.925133 30558 rpcz_store.cc:276] 0709 13:15:33.231971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.237956 (+  5985us) service_pool.cc:224] Handling call
0709 13:15:33.238378 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:33.243054 (+  4676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:33.243091 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:34.489376 (+1246285us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:34.489982 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 13:15:34.675154 (+185172us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:34.675193 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:34.675270 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:34.924345 (+249075us) raft_consensus.cc:1685] finished
0709 13:15:34.924371 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:34.924620 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11584,"raft.queue_time_us":7105,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":14923,"spinlock_wait_cycles":49685760,"thread_start_us":184675,"threads_started":1,"wal-append.queue_time_us":250006}
I20260709 13:15:34.974892  3007 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:35.012712  3007 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.013232  3007 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.016311  3009 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:34.983740  2913 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:35.017757  2913 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:34.859869  2839 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:34.981786 30555 rpcz_store.cc:276] 0709 13:15:33.909601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:33.910825 (+  1224us) service_pool.cc:224] Handling call
0709 13:15:34.953239 (+1042414us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54003809d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:34.953418 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494287104}
I20260709 13:15:34.985484 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:35.006747 31388 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:34.962638  2973 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:34.908478  3012 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:34.912384  2956 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.001151  2933 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:35.010684  3014 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:35.011924  3006 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:15:34.981381  2714 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.037037 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.038892 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:35.039323 30698 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:35.042176 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.047858 30703 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:35.018031  2823 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.054947 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.065096 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.065300 30555 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:35.066428 30699 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:35.010805  2910 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.035686  2913 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:35.032531  2973 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.079033 31378 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:35.038192  3013 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:35.087206 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.086066 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:35.088902 30555 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:35.045997  3003 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.050467  2860 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.030997  2962 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.103856 31367 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:35.050101  2839 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.109191 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:35.110109 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.079202  2910 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.098536  3013 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.111317 30555 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:35.037606  3012 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.116629 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.037652  3014 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.123654 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.124678 30627 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:35.117887 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.044941  2714 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.099444  2707 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.120124 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.130772 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.123324 30628 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:35.080672  2973 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:35.120781 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:35.125639 30632 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:35.063441  2956 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.127516 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.134305 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.156957 30628 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:15:35.139446  2938 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.779s	user 0.021s	sys 0.015s
I20260709 13:15:35.112046  3003 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:35.168521  3011 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:15:35.170282 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.171234 31388 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:35.151096 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.172984 30625 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:35.152552 30553 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:35.124080  2910 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.163614 30628 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:15:35.165365  3012 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.165227  2532 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.155454  2707 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.146142  2714 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.167183  2823 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.162916  2855 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:35.156795 30555 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:35.177089 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:35.172080 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:35.186210 30628 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:35.113379  2839 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.115676  2860 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.128300  2799 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.125725  3013 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:35.236603 30556 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:35.099112  2933 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.282559 30552 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:35.182742  2952 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.285846  2714 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.294982  2952 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.295233  2714 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.184615  2855 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.298008  2855 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:35.298993 30556 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:35.299139  2967 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.245023  3007 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.180177  2701 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:15:35.311040 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:35.286556 30628 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:35.291052 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:35.313264 30632 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:35.292479  3005 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.293799  2962 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.321394 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.235136  2888 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:35.235385  3003 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.328460 30628 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:15:35.334816 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:35.335505 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 292564us
I20260709 13:15:35.337155 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:35.337996 31391 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19. Re-sending same reply.
I20260709 13:15:35.342121 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:35.342288 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.343309 30628 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:15:35.345598 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.327564  2888 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.348723 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.329663  2933 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:35.343866 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:35.349261 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.349431 30627 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:35.351342 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.361780 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.361281 30624 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:35.367168 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.366559 30557 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:35.367923 30699 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:35.355362 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:35.371444 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:35.351300  2860 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.352564  2888 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:35.348935  3003 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.380925 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.382360 31391 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:35.386021  2707 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.387403  3014 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.388569  2531 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:35.348181  2740 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:15:35.392527  2910 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.391388 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.401037 31389 raft_consensus.cc:2368] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17.
I20260709 13:15:35.358909  2946 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:35.380441  2697 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:35.396595 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:35.402412 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:35.407974 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:35.419701 31391 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:35.418192 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:35.419210 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 232712us
I20260709 13:15:35.390288  2531 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:15:35.422475 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.423661 30699 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:35.424693 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:35.395862  2707 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.431206 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:35.406843  2713 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.710s	user 0.015s	sys 0.000s
I20260709 13:15:35.402034  2860 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.433512 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:35.410508  2697 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.438856 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:35.442268 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.443789 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.426694  2910 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.405133  2946 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:35.444082 30702 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:35.446058 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.450821 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.451408 30705 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:35.397684  3003 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.446647 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.435137  2928 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:35.451889 30707 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:35.436539  2929 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:35.454267 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:35.462227 31391 raft_consensus.cc:2368] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:35.450456 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:35.446576  3015 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.001s	sys 0.004s
I20260709 13:15:35.456789 30699 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:35.432716  2856 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.454291 30698 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:15:35.463588 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:35.464771 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:35.455289  2946 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:35.447830 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:35.484474 30705 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:35.441104  2697 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:35.439399  2713 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.458720  2928 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.440974  2860 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.439241  2707 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.461378  2929 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:35.471364  3007 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:35.453799  2910 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.519181  2701 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:35.527616 30632 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:35.483132  2925 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:35.534482 30557 raft_consensus.cc:2368] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:15:35.511408  2928 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:35.553192 30628 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:35.479341  2856 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.553249  2925 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.501983  3012 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:15:35.501578  2967 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:15:35.533007  2888 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.548849  2701 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.538709  3014 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:35.533891  3007 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:35.552346  2839 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.587858  2925 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:35.595567  2701 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:35.617853  2532 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.599313  3007 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:35.642674  3026 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.136s	user 0.001s	sys 0.001s
W20260709 13:15:35.647758  3028 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 13:15:35.648651  3025 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:15:35.649595  2962 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:35.667986  3024 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.206s	user 0.001s	sys 0.001s
I20260709 13:15:35.657200  2804 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.648913  3008 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.592243  2929 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.687209  2910 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.687731  2929 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:35.690439  2856 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.690919  2888 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.675386  2967 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:35.694926  2967 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:35.648828  3016 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.180s
I20260709 13:15:35.695909  2967 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.696907  3016 consensus_queue.cc:1243] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:15:35.697327  2967 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:35.699048  3008 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.700042  3027 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:35.701661  3008 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.697472  3016 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:35.702854  3016 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:35.703439  3033 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:35.703890  3016 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:35.699537  2956 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:35.705567  2913 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.004s
W20260709 13:15:35.710765  3029 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:15:35.711695  2707 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:35.682278  3030 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
W20260709 13:15:35.714146  2973 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
I20260709 13:15:35.714717  2973 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.715900 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:35.694406  2714 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:35.731637  3003 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.723016  2713 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.683941  2855 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.741948  2952 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:35.743485  3040 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:15:35.742082  2855 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.682818  2962 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:35.734380  2860 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:35.720372  3013 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.784386  2962 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.796780  3013 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:35.799528  2973 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.769405  2946 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.803947  2946 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:35.859706 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.859706 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:35.863968 30550 raft_consensus.cc:2368] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17.
W20260709 13:15:35.867446 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 478028us
I20260709 13:15:35.891708 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:35.892520 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.893690 31361 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:35.898415 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:35.906663 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.907104 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.889549 30596 leader_election.cc:400] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17.
W20260709 13:15:35.879762 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 431889us
I20260709 13:15:35.882439 30704 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:35.909044 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2170) took 1745 ms. Trace:
I20260709 13:15:35.912909 30552 rpcz_store.cc:276] 0709 13:15:34.163716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.163955 (+   239us) service_pool.cc:224] Handling call
0709 13:15:34.955472 (+791517us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:35.289050 (+333578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.907484 (+618434us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54003809d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:35.908499 (+  1015us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54003809d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:35.908789 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407023872}
I20260709 13:15:35.876897 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.918944 30596 leader_election.cc:403] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:35.919133 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.920624 30632 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:35.922796 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.924618 30624 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:35.931259 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:35.898245 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:35.940845 31388 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:35.941327 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1615) took 1107 ms. Trace:
W20260709 13:15:35.944391 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46690.7us
I20260709 13:15:35.947666 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:35.941959 31378 rpcz_store.cc:276] 0709 13:15:34.834113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.883230 (+ 49117us) service_pool.cc:224] Handling call
0709 13:15:34.883720 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:35.079752 (+196032us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:35.940536 (+860784us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54003377d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:35.941025 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":432238848}
I20260709 13:15:35.949952 31396 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:35.953744  2531 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:35.957590  2532 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:35.960366  2679 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:35.961839  2675 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17.
I20260709 13:15:35.860463 30378 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:35.975850  2679 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:35.988591  2714 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:35.995237  2531 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:35.999965  3010 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:36.001299  3010 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:35.990010  2714 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.001112  2952 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:36.004969  2923 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:36.005615  2923 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:36.007223  2923 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.009914  2923 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.011708  3010 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.015812 30378 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:35.974529  2532 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:36.021409 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.035166 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.032173 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.009960  2531 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:36.041849 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94445.8us
I20260709 13:15:36.043556 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.048328 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:36.060907 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.062139 30629 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:36.063586  2714 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:36.063812  2839 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:36.069468  2797 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.069830  3012 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:36.070688  3012 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:36.070578  2797 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.071343 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:36.072230 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.072438  3012 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.073328 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:36.073567 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:36.073972  2797 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:36.074427 30553 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.074352 30552 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:36.075294 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:36.080029 30557 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.070204 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.085685 30624 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:36.086410  2928 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 13:15:36.087105  2928 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.087698  2679 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:36.088493 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:36.090039  3009 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:15:36.100518  3051 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.101965 30629 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:36.101630  3051 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.106505 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.107610 31389 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:36.108330  2928 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:36.106063 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.058173  2701 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.115564 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.109259 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.124738 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.126014 30629 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.144425  3051 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:36.145493 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.029s	sys 0.056s
I20260709 13:15:36.145625 31391 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:15:36.146121 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.030s	sys 0.056s
I20260709 13:15:36.146306  2928 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:15:36.146625 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 639208us
I20260709 13:15:36.146940  2928 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:36.115703  2532 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.150740 31388 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:36.137081 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.153584  2714 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:36.157930  2701 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:36.158169  2532 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:36.151196  2928 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.160952  2928 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:36.167006 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.166989 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.168433 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.168823 30628 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.169623  3051 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:36.170097  3051 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.154855 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.177956 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.180312 30557 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:36.173116  3051 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:36.171357  3007 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.521s	user 0.013s	sys 0.000s
I20260709 13:15:36.193321 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.173291  2714 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:36.198900  3009 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:15:36.199234 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.201335 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.194990 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.211522  2714 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.212190 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.211623 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:36.215402  3045 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.217942 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.219273 31383 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:15:36.219162 31391 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:36.222298 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.221652 31396 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:36.216821 31378 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:36.239025 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.218194  3045 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.216681  2701 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:15:36.242509  2697 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.732s	user 0.013s	sys 0.013s
I20260709 13:15:36.243120 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.246802  3008 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.247565 30378 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 8, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 8 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.228118 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.253060 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.259706  3016 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.263192 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.255465 30553 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:36.203917  3007 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.269585 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:36.271338 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88304us
W20260709 13:15:36.273794 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 74738.5us
I20260709 13:15:36.258230  3045 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:36.271109 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.272709 30629 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:36.280300 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68847.3us
I20260709 13:15:36.275084  2701 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:36.282023 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 74906.2us
I20260709 13:15:36.289862 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.291895 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.292210 30378 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 6 to 8, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:36.293270 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.294584 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.295109 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.295516 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.295853 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.297151 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:36.297886 30632 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:36.298069 30555 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 9.
I20260709 13:15:36.294127 30378 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 12. New cstate: current_term: 12 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.309424 30378 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 11 to 13, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 13 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.315239 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.317185 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.318228 30629 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:36.319797  2797 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.320025 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.320647 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:36.331921 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.334216 30554 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:36.335389 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54006.7us
I20260709 13:15:36.285497  3009 raft_consensus.cc:2764] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:15:36.272493  2697 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:36.341044 30630 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.336510 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:36.338346 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49960.3us
I20260709 13:15:36.289043  3012 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:15:36.355389 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 70552us
I20260709 13:15:36.356186 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:36.357272  3053 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:15:36.357743  3052 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:15:36.357946  3053 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.358574  3053 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:36.359280  3049 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.303s	user 0.007s	sys 0.000s
I20260709 13:15:36.363013  2963 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.364086  2963 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:36.354724 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:36.370191  3013 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:36.370457 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43509us
I20260709 13:15:36.372252 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.373391  3013 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:36.373888  3013 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.373956 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.374372  3013 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:36.374755 30628 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:36.375129 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.320986  2797 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:36.384949 30555 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:36.387171  2892 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:36.387725  2892 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:36.377933 30380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 70) took 39526 ms (client timeout 15000 ms). Trace:
I20260709 13:15:36.390103  2892 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:36.390771  2973 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:36.391253  2973 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.391690  2973 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:36.352246  3012 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.398463 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2178) took 1042 ms. Trace:
I20260709 13:15:36.399042 30554 rpcz_store.cc:276] 0709 13:15:35.355991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:35.858563 (+502572us) service_pool.cc:224] Handling call
0709 13:15:36.398182 (+539619us) spinlock_profiling.cc:243] Waited 469 ms on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:36.398379 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1079480320}
I20260709 13:15:36.400817 30551 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:36.400997 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.402011 30633 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.290597  2701 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.385753  2714 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:36.394670  2697 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:36.417724  2925 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.784s	user 0.013s	sys 0.004s
I20260709 13:15:36.408797 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.399098  3012 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:36.419034 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.421754 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.346259  3008 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:36.425388 30625 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.409624  2714 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.412051  3010 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:36.433911 30632 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:36.476433  3010 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.476925  3010 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.477507 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.433772 31378 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:36.434492 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:36.438439 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 118010us
I20260709 13:15:36.441006  3008 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:36.443306 30630 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:36.444645 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:36.430753 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94351us
I20260709 13:15:36.444417 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.447503 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.447633 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.449090 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.451052 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.451910 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.444908  2925 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.454221 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.487162 31394 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:36.488268 30553 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:36.488140 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.489509 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.489543 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:36.492388 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.493409 30628 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:36.455164 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1510) took 1680 ms. Trace:
I20260709 13:15:36.455387  2714 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:36.497115 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.456570 30703 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:36.457827  3041 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:36.458294 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:36.463575 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:36.498147 31383 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.458425 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.454281 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:36.455734 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.460067 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:36.464931 30555 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:36.460431 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.465886 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:36.468120 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:36.461153 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:36.461243  3055 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
I20260709 13:15:36.457108 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:36.468176  2797 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:36.471506 30378 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 6 to 9, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:36.474023  2963 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:36.475860  3058 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:15:36.478466 30632 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:36.491071  2925 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:36.494721 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:36.454806 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:36.494132 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:36.456228 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:36.389118 30380 rpcz_store.cc:269] 0709 13:14:56.776382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:14:56.778533 (+  2151us) service_pool.cc:224] Handling call
0709 13:15:36.302374 (+39523841us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:26.735582 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:26.742052 (+  6470us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:26.742113 (+    61us) write_op.cc:435] Acquired schema lock
0709 13:15:26.742134 (+    21us) tablet.cc:662] Decoding operations
0709 13:15:26.755995 (+ 13861us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:26.756110 (+   115us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:26.756157 (+    47us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:15:26.757517 (+  1360us) tablet.cc:701] Row locks acquired
0709 13:15:26.757571 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:15:26.757790 (+   219us) write_op.cc:270] Start()
0709 13:15:26.758012 (+   222us) write_op.cc:276] Timestamp: P: 1783602926757704 usec, L: 0
0709 13:15:26.758064 (+    52us) op_driver.cc:348] REPLICATION: starting
0709 13:15:26.770337 (+ 12273us) log.cc:844] Serialized 4384 byte log entry
0709 13:15:27.333852 (+563515us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b54000f32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed8e28 00007f180a913581 00007f180a912d58 00007f180a9179f1 00007f180a9179a9 00007f180a917939 00007f180a917901 00007f180a916b2d 000000000038f354 000000000038f199
0709 13:15:27.379568 (+ 45716us) op_driver.cc:464] REPLICATION: finished
0709 13:15:27.448010 (+ 68442us) write_op.cc:301] APPLY: starting
0709 13:15:27.448127 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:27.802524 (+354397us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:27.802582 (+    58us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:28.286055 (+483473us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:28.491830 (+205775us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:15:28.517085 (+ 25255us) write_op.cc:312] APPLY: finished
0709 13:15:28.579773 (+ 62688us) log.cc:844] Serialized 104 byte log entry
0709 13:15:28.849043 (+269270us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b
0709 13:15:29.038756 (+189713us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.056738 (+ 17982us) write_op.cc:454] Released schema lock
0709 13:15:29.073309 (+ 16571us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:29.077556 (+  4247us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:29.077609 (+    53us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3500408,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":68061,"apply.run_cpu_time_us":65741,"apply.run_wall_time_us":1629472,"cfile_cache_hit":42,"cfile_cache_hit_bytes":43995,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6222,"cfile_init":2,"lbm_read_time_us":93689,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":10,"num_ops":11,"prepare.queue_time_us":310356,"prepare.run_cpu_time_us":39417,"prepare.run_wall_time_us":658042,"raft.queue_time_us":294322,"raft.run_cpu_time_us":8641,"raft.run_wall_time_us":86624,"replication_time_us":621270,"spinlock_wait_cycles":88050816,"thread_start_us":1111857,"threads_started":5,"wal-append.queue_time_us":797008,"wal-append.run_cpu_time_us":13483,"wal-append.run_wall_time_us":2261785}]]}
I20260709 13:15:36.494282 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:36.501142 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66799.4us
I20260709 13:15:36.501904 30699 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
W20260709 13:15:36.504495 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48932.7us
W20260709 13:15:36.460814  2929 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.772s	user 0.011s	sys 0.009s
I20260709 13:15:36.504695  2797 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:36.516395 30707 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:36.508579 30378 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 10 to 11, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.520375 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:36.521261 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:36.522600  2860 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:36.523324  2860 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.523840  2860 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:36.524161 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.524926 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:36.526893 30706 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:36.527779 30705 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:36.530910 31396 raft_consensus.cc:2376] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14. Re-sending same reply.
I20260709 13:15:36.532436  2864 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:36.533161  2864 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.533695  2864 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:36.514670  2797 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.535383  2797 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:36.542184 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:36.547505 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57059.7us
I20260709 13:15:36.553661 30627 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:36.506675  2532 raft_consensus.cc:2764] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:15:36.494227 31364 rpcz_store.cc:276] 0709 13:15:34.775081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:34.778785 (+  3704us) service_pool.cc:224] Handling call
0709 13:15:34.795962 (+ 17177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:34.797245 (+  1283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:34.797295 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:35.704047 (+906752us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b180049c660. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:15:35.704711 (+   664us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b2c00056c38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:35.705539 (+   828us) log.cc:844] Serialized 26 byte log entry
0709 13:15:36.087303 (+381764us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:36.087404 (+   101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:36.087549 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:36.403715 (+316166us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b180049c660. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:36.454744 (+ 51029us) raft_consensus.cc:1685] finished
0709 13:15:36.454772 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:36.454950 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1267,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":28192,"spinlock_wait_cycles":93468544,"thread_start_us":340052,"threads_started":1,"wal-append.queue_time_us":352812}
I20260709 13:15:36.573506 30700 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:36.573719  2839 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:36.574558  2839 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.575317  2953 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.557919 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:36.581285 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:36.585880 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 131482us
I20260709 13:15:36.588016  2888 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:36.588371  3059 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:15:36.591228  2925 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.592000  2925 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.561029  3008 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:36.595208  3008 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:36.595728  3008 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:36.598598 30378 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:36.607973 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45971us
I20260709 13:15:36.609465 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.629921 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.629307 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:36.602918  3007 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:36.625445 30378 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 10 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.610935  2925 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:36.637153 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59611.3us
I20260709 13:15:36.637703 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.642154 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:36.636152  2888 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:36.646915 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.581233  3041 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.648849 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:36.576098  2839 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.666054 31364 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:36.671779 31378 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:36.675177 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.677172 31388 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:36.662454 30378 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:36.679767 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 159720us
I20260709 13:15:36.664332  3065 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:15:36.678764  3042 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.682570  3044 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:15:36.682977  2701 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:15:36.683821 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96193.6us
I20260709 13:15:36.700865 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:36.690171  2923 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.005s
W20260709 13:15:36.703616 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 188132us
I20260709 13:15:36.705191  2856 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:36.705710  2701 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.685391  3065 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:36.697744 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0633037669999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
I20260709 13:15:36.679107  2888 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:15:36.720475 30378 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:36.695531  2973 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.719240  2701 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.726821  3031 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:15:36.728621  3031 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:36.729143  3031 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.710842  3065 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:36.721798 30378 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 3 to 6, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 6 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:36.737090  2933 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:36.707994  3042 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:36.749593 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.748026 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:36.717687  2856 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.751350 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:36.756903 30378 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 9 to 11, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:36.761585 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 132361us
I20260709 13:15:36.763206  3064 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:36.720208  3053 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:36.776232  3061 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:15:36.730324  3031 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.778666  2855 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:36.779469  2855 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.779247  2856 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.780570  2855 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:36.779628  3031 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:36.783149 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82471.8us
I20260709 13:15:36.785015 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:36.786751  2929 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:36.787519  3062 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:15:36.776583  3053 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:36.791934  3027 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:36.793627  3066 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:36.794787 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:36.792493  3027 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:36.799389  3027 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.800945  3027 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:36.750769  3041 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:36.804467  2973 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:36.763762  3064 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:36.810672  3064 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:36.812273  3064 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:36.832712  3042 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:36.845525 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:36.847479 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 115561us
I20260709 13:15:36.848726 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:36.850832 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:36.853636 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67062.8us
W20260709 13:15:36.858497 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63804.3us
I20260709 13:15:36.859524 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:36.861588 30555 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:15:36.874500 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.878168 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:36.879552 30555 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:36.871102  3042 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:36.865681 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:36.888994 30707 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:36.868577  3007 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:36.868309  3041 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:36.898154 30670 leader_election.cc:400] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:36.907970 30670 leader_election.cc:403] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.919333 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:36.919981 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.881980  3042 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:36.922289 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76889us
I20260709 13:15:36.895783  3053 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:36.932513 31383 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:15:36.933319 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46390.9us
W20260709 13:15:36.934777 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46930.5us
I20260709 13:15:36.765398  3010 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.882108  2923 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:15:36.873219  2892 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:36.902191  2928 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:36.952993 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:36.902189  3007 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.958384 31364 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:36.937906  3053 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:36.938385  3042 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:36.972913 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.977355 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:36.980810  3041 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:36.981323 31389 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:36.986083 30705 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:36.986737 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:36.988191 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 105805us
I20260709 13:15:36.970165  3007 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:36.993175 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:36.997768 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.001066 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.002269  2864 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:37.002753 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70989.5us
I20260709 13:15:37.003342  3066 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:36.975351  3053 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.003913  2697 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:37.005873  2697 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:15:37.006834  2953 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:37.006963 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.008288  2888 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:37.008955  2892 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:37.013118 30599 leader_election.cc:400] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17.
I20260709 13:15:37.001125 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.014113 30707 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:37.027245 30553 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:15:37.027590 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.028317  3010 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.028770 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:37.029958 31383 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:36.986449  3027 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:15:37.030506 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.031484 31396 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:37.031786 30599 leader_election.cc:400] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:15:37.032140 30599 leader_election.cc:403] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:15:36.975620  2928 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.037847  2928 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.039434 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.039863 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:37.040273 30553 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
W20260709 13:15:37.041441  3051 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.834s	user 0.012s	sys 0.004s
I20260709 13:15:37.042091 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.042668 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.043509 30702 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:37.043862 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.045253 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.045521 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.045876 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.046378 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.047169 30552 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:37.051175  2797 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.007s
I20260709 13:15:37.051947  2797 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.055292 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.056799 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.064760 30552 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:37.042111  3051 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.057715 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.067941  3069 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:37.068557  3069 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:37.068969  3069 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:37.070186  2963 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:37.058931 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47414.2us
I20260709 13:15:37.060274  2864 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:37.063163  2714 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.004s
I20260709 13:15:37.077252 30554 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:37.046780 30557 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
W20260709 13:15:37.067830 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44721.1us
W20260709 13:15:37.058171  2860 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 13:15:37.077436 30705 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:37.065647  2797 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.003994  3066 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.018043  2888 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:15:37.111090 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.088975 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:37.092376 30555 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:37.083706 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.104889 30557 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:15:37.106606 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.122836 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.119812 30557 pending_rounds.cc:85] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 6
I20260709 13:15:37.130223 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.131311 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.131901 30550 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
W20260709 13:15:37.129916 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74902.8us
W20260709 13:15:37.135937 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66337.4us
I20260709 13:15:37.126554 30557 pending_rounds.cc:107] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:15:37.143784 30557 raft_consensus.cc:2889] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:37.140007 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.135519 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.084672  2953 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:15:37.098649  3076 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:37.151232 30556 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:37.147894 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 109823us
I20260709 13:15:37.130793  2923 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:37.091267  2714 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.154215 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:37.133383  2973 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.158246 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:37.150405  3012 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.706s	user 0.015s	sys 0.000s
I20260709 13:15:37.166096  3012 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.091813  2860 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.168964  3012 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.127717  2933 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:15:37.170970 30700 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:15:37.172428  3051 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:37.173534 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 97841.4us
I20260709 13:15:37.173785  3042 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.133150  2928 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:15:37.176400  2973 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.143162  3053 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.188828 30557 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:37.152138  2923 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:37.179574 30556 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:37.180927 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.182165 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154687us
I20260709 13:15:37.132891  3044 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:15:37.194270 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:37.194757  3075 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.136s	user 0.002s	sys 0.000s
I20260709 13:15:37.162137  2531 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:37.189065 30553 raft_consensus.cc:2368] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
I20260709 13:15:37.195735  2531 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:37.196069  3078 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:37.196908  2531 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.176525 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.157510  2864 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:37.198349 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.198588  2864 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:37.199740  2864 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.201143  2864 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:37.201364 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.191164 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.203080 31294 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:37.203078 30552 raft_consensus.cc:2368] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:15:37.205374 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:37.206246 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98847.6us
I20260709 13:15:37.206790 30550 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:37.211588 30555 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:37.162298  2929 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.202381 31396 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:37.216375 30556 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:37.217310 30554 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:37.216452 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.171103  2860 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.222385 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.207386  2923 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:37.214046  3008 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.006s
I20260709 13:15:37.233652 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.179806  3069 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.238279 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:37.228715  3013 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.850s	user 0.015s	sys 0.001s
I20260709 13:15:37.219758  3007 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.234257 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:37.244401 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.247646 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.249949 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.242413 30700 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:37.246225 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.252534  2714 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.252831  3007 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:37.239346  3008 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.253957 30551 raft_consensus.cc:2368] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:15:37.256683 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.257665 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.258687 30706 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:37.260056 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.260735 31378 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:15:37.262477  2797 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:37.262846 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59965us
I20260709 13:15:37.263538  3083 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:37.263327  2797 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.264039  3083 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:37.228245  3027 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.265076 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:37.265971 30550 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 10: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:37.266453  3042 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.267277  3042 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:37.267750  3042 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.232398  2531 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:37.265089  3083 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.268981  3042 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.269862  3083 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:37.257682 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.265147  3027 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.271354  2973 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:37.272101  3066 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:37.275316 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.281543 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59225.4us
I20260709 13:15:37.273826 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.283062 30704 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:37.287616 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.286279 30702 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:37.288477 31388 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:15:37.294727 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.295300 30625 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:37.295929 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.296173 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.296685 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.296980 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.297417 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.297902 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.298192 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.298533 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.299214 30630 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:37.299391 30625 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:15:37.299791 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.300380 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.299422 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.300486 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.301448 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.300716 30628 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:15:37.303053 30628 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:37.301138 30630 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:37.304497  3008 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:37.304586 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54702.1us
I20260709 13:15:37.305380 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.305617 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.306113 31391 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:37.306860  3084 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:37.312879  3084 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.313524  3084 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:37.305380 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.318917  3066 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:37.319301 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45406.9us
I20260709 13:15:37.321252  3066 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:37.321463 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:37.322418  3066 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:37.303053 30631 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:37.322875  3066 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:37.311868 30624 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:37.323451 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:37.304054 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74096.4us
I20260709 13:15:37.323740  2973 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:37.327438 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.324642 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.324633 30556 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:37.327320 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.329239 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.327826  2973 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.343022  2973 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:37.347292  3066 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.349072  3066 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.349599  2963 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
W20260709 13:15:37.309315  3065 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.553s	user 0.014s	sys 0.000s
I20260709 13:15:37.354590 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.354980 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.354590 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.356515 30702 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:37.306861  2856 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:37.361181 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.362380  2888 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:37.367182 30556 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:37.368733 30704 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:37.365281 31378 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:37.369395 30699 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:37.371459 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.372172 31383 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:37.268246  3013 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.370007 30703 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:15:37.381115 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57780.4us
I20260709 13:15:37.329367  2860 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:37.329209  2933 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:37.383104 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.392714 30556 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:37.324760  3078 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:15:37.387915 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:37.403967 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:37.407728 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102183us
W20260709 13:15:37.392654 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 88285.3us
I20260709 13:15:37.353699  3027 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:37.435995 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 68477.8us
I20260709 13:15:37.395984 30704 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:37.426292 30551 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:37.428660 31383 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:37.366863  2856 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.416025 30633 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:15:37.439741 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51939.2us
I20260709 13:15:37.453197  3069 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.454355 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.456579  2963 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:37.458371 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.395342  2860 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.476732  2963 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:37.476934  2860 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.391499  2888 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:37.473121  2855 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:15:37.367091  3065 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.446885  3012 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:37.443202  2856 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:37.495333  2888 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.496207  3053 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.475102  2797 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.498831 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.499471  2797 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:37.500048  2797 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:37.500483  2797 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.500731  3067 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:37.501704  2929 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:37.503335 30701 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:37.504963 30703 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:37.506577 30702 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 13:15:37.485975 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.490633  2963 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:15:37.500524 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.493320  3084 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.493665  2933 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:37.512511 30632 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:37.474962  3031 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:15:37.514763 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186343us
I20260709 13:15:37.520038 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:37.481577  3045 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:37.516147 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 322309us
I20260709 13:15:37.524350 30707 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:37.525004 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.536053 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.474668  2701 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:15:37.539085 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:37.541862 30699 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: [5.3-9.5]
I20260709 13:15:37.545946 30629 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:37.548110 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.542025 30627 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:15:37.551162 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50690us
I20260709 13:15:37.552735 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.564035 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.523521  3012 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.566119 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.567093  3012 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.535953  2933 consensus_queue.cc:1243] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:15:37.570640 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:37.576083 31388 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:37.587924  3031 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:37.590087 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.591598 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.588152  3091 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:37.592929  3091 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.593483  3091 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:37.597963 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.540515  3045 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.571601  3053 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:15:37.602675 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.604251 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:37.610076 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:37.617712 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:37.578446  3067 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:37.639889 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:37.641681 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50175.4us
I20260709 13:15:37.545881  2923 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.602497  3097 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:37.606854  3031 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:37.660326 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 256453us
I20260709 13:15:37.660132 30556 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:37.661402 31294 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:15:37.607105  3044 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:37.607215  2963 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:37.607265  3084 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:37.612968  3053 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:37.684036 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:37.688930 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 173030us
I20260709 13:15:37.690088 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:37.692899 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:37.660094  3097 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:37.670486  2714 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:37.678010  3013 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.641227  3008 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:15:37.671180  2963 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:37.659644  3069 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:15:37.674494  3100 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:37.651260  2928 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:15:37.685580  3007 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:37.699579 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.730759  2714 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.736512  2963 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:37.737119 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47164.8us
I20260709 13:15:37.738672  2923 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.694947  3095 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:37.741707 30707 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:37.706408  3082 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.747443 30550 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
W20260709 13:15:37.748912 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 131316us
W20260709 13:15:37.768767 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128981us
I20260709 13:15:37.739634  3009 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:15:37.800774 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61167.2us
I20260709 13:15:37.801430  3064 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:15:37.736631  3012 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:37.826570 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:37.757261  3078 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:37.812482 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:37.757766  2929 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:37.819082 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:37.819080 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 134842us
I20260709 13:15:37.852547 31396 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:37.853075 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:37.831763 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:37.811030 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:37.833395 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.746194  3100 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.834717  3045 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.813516  3084 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:37.834820 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 135346us
I20260709 13:15:37.862174 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:37.826013  3013 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:15:37.797480  3007 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.810127  2531 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:15:37.838153 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.839196 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:37.864919 31383 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:37.839263  3065 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:37.740522  2973 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.747265  3099 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:37.842248 31367 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:37.821655  3082 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.815798  3106 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:37.815419  3009 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.827896  2933 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:15:37.849617 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:37.827291  3044 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.858340 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:37.858829 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46472.1us
I20260709 13:15:37.893497 31391 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:37.854954 31396 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
W20260709 13:15:37.859735 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:37.876724 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66956.6us
I20260709 13:15:37.851747  3078 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.820362 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:37.853505  3012 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.826120  3031 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:15:37.886919 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76195.3us
I20260709 13:15:37.862780  3051 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:15:37.881702 31292 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.574s (SENT) [suppressed 37 similar messages]
I20260709 13:15:37.896142 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:37.891920  2923 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:37.873615  3100 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:37.861343  2839 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:37.889204  3101 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:37.913156 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51090.5us
I20260709 13:15:37.923060 31383 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:37.828959  3095 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.902397  3078 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:37.874801  3094 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:15:37.903048  2963 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:37.867708  3084 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.926501  3069 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:37.933022 31388 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:37.933743 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:37.907290  2929 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:37.946030 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:37.954994 31383 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:37.912302  3065 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:37.948016 31294 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:37.953620  3069 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:37.964282  3069 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.965791  3069 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:37.966595  3113 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:37.968499  2797 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:37.970229  3066 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.002s
I20260709 13:15:37.954048 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:37.855085  3097 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.978938 30702 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:37.889498  3106 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:37.963404 31294 leader_election.cc:403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:15:37.925870  2928 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:37.981877 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48232us
I20260709 13:15:37.971369  2797 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:37.971909  3083 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.701s	user 0.002s	sys 0.014s
I20260709 13:15:37.935170 30557 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:37.934707  2923 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.888621  2714 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:37.936885  2697 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:37.982402 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
W20260709 13:15:37.986330 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 104688us
I20260709 13:15:37.961879  2929 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:37.988771 30705 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:37.997609 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:37.977118  3084 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.002040 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:37.991192 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 87240.4us
W20260709 13:15:37.990377 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56165.5us
I20260709 13:15:37.990150 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.991298 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:37.989888 31383 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:37.991663  2928 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.009987  3115 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:38.001417  2697 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.014742 30703 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:38.000382  3107 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
I20260709 13:15:38.032334  3106 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.031975  2923 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.032929 30631 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:15:37.987654  3114 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:37.989121  3044 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:37.977872  3009 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:37.992367  3091 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.037184 30698 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:38.024752 30633 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:15:37.992632  3066 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:38.056161 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.056250 30631 pending_rounds.cc:85] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 7
I20260709 13:15:38.075062 30631 pending_rounds.cc:107] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:15:38.075515 30631 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:38.055545 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.002149  2929 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:38.062028 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.063073 31294 leader_election.cc:400] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
I20260709 13:15:38.077368 31294 leader_election.cc:403] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:38.043248 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:38.072665 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.073148 30633 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 10
I20260709 13:15:38.078864 30633 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:15:38.079315 30633 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:38.083673 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44681us
W20260709 13:15:38.060958  2856 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.008s
W20260709 13:15:38.092088 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74941.7us
I20260709 13:15:38.059265  3114 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.093559 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.100672 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.101786 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.095670  2714 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:38.104364 31396 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:38.104600  3128 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:38.106142  3113 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:37.989897  3007 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.057497  3094 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:15:38.058449  3027 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:15:38.097863  2860 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.617s	user 0.012s	sys 0.004s
I20260709 13:15:38.095075  2856 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.104355  2714 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:38.111613 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:38.126392  3091 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:38.112464 31394 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:38.131546 30550 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:38.131717  2928 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:38.122314 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.157552 30704 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:38.099845  3045 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:38.088092  3084 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:15:38.032967  3118 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:15:38.128324 30556 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:38.166035 31361 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:38.120059  2697 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:37.988041  3012 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:38.101402  2864 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.900s	user 0.015s	sys 0.003s
I20260709 13:15:38.139276  3076 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:15:38.100309  3114 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.154466  3117 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.177s
W20260709 13:15:38.011251  3092 consensus_queue.cc:1175] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 16 status { last_received { term: 14 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 13:15:38.168064 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45847.3us
I20260709 13:15:38.111395  2797 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
W20260709 13:15:38.171198 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 108268us
W20260709 13:15:38.177613  3123 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:38.164662  3045 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:38.211840 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 137455us
I20260709 13:15:38.152079  3128 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:15:38.215391 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 117442us
I20260709 13:15:38.208679 31294 leader_election.cc:400] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:15:38.166947  2928 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:38.181190  3095 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.216195 31294 leader_election.cc:403] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 13:15:38.217010 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 115353us
I20260709 13:15:38.175199  3115 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:38.146631  2860 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.185199  3091 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:38.195430  2856 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.231832 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:38.239547 30556 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:38.209105  3027 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:38.241457 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.244971 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.217208  3045 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:38.215548  3128 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.251516 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.252946 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.280053 31292 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:38.280191  3053 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.281244 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:38.220721  3066 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:38.236153  2923 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:38.282255  3066 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.256379 30699 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:38.242954  2797 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:38.262317  3096 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:38.249337  3027 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:38.267072 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.267897 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.270732  3115 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.250787  3134 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:38.273696 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.224990  2864 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:38.281296  2860 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.281474  3053 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:38.170497  3113 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.208055  2892 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:15:38.282523 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.274199  3078 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:38.286540 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45224.4us
I20260709 13:15:38.286913 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.293377 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.292560 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.302668 31294 leader_election.cc:400] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:15:38.288581 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1620) took 1024 ms. Trace:
I20260709 13:15:38.303707 31294 leader_election.cc:403] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:15:38.311852 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66976.6us
I20260709 13:15:38.313321 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.303990 30557 rpcz_store.cc:276] 0709 13:15:37.263882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.267244 (+  3362us) service_pool.cc:224] Handling call
0709 13:15:37.267644 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:37.983460 (+715816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.288277 (+304817us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b5400242554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:38.288503 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534703872}
I20260709 13:15:38.322346 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:38.323555 30626 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:38.324617 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.284085  3084 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:38.294469  3091 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:38.261271  3095 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:38.282960  3101 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:38.352079 31389 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:15:38.334224 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.284718  3066 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.335583 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.333040 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.335068 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.339234 30701 raft_consensus.cc:2368] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:38.356843 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:38.355685 30628 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18. Re-sending same reply.
I20260709 13:15:38.358523 30630 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:38.358765 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45633.2us
I20260709 13:15:38.360011 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:38.360885 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.361316 31361 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:38.362118 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.332476  3069 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:38.335485  3044 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:38.361591 30626 raft_consensus.cc:2376] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17. Re-sending same reply.
I20260709 13:15:38.333201  3100 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:15:38.291180  3096 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.339823  3095 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:38.292012  3027 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:38.387854 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49149.9us
W20260709 13:15:38.388820 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66433.4us
I20260709 13:15:38.354971  3134 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:38.389768  3012 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:38.390760 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.428104  3134 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.432154 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.375785  3091 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:38.388147  2928 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:38.437831 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:38.439137 31367 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:38.447147 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.375280  2860 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:15:38.430763  3101 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:38.414999  3012 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.395481  3095 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:38.450606 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59980.6us
I20260709 13:15:38.451987  3136 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 13:15:38.453606  3142 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:15:38.495733 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.497313 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.497879 30626 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.393389  3007 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:15:38.392030  3130 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:15:38.499718 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:38.441391  3084 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:38.500870 31367 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:38.459277 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.502099 30625 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:38.461254  2855 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:38.462206  3078 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.508280 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.462942  2860 consensus_queue.cc:1243] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 16.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:38.463626 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:38.449129  2929 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.464505 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:38.515128 31394 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:38.516003  3114 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.516882  2929 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.518116  3097 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:38.464963  2714 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:38.467929  2973 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:38.470831 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 122155us
I20260709 13:15:38.439018  3044 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:38.477264 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:38.540944 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63830.4us
I20260709 13:15:38.351045  3128 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.542785  3084 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.543375  3128 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:38.484725  2797 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:15:38.486843  2531 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:38.546919  2531 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.476112  3012 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:38.488703  3136 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:38.377019  3069 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.505403  3092 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:15:38.503568  2855 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:38.478116  3101 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:38.520174 30555 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:15:38.518596  3114 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:38.528549 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77684.8us
I20260709 13:15:38.539400 30552 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:38.549432 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:38.561489 31394 raft_consensus.cc:2410] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:38.528985 30626 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:38.565198 31383 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:38.565829 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.566710 30626 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:38.568357 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.569279 30626 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:15:38.557976 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1903) took 1046 ms. Trace:
W20260709 13:15:38.577956 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.570034 30704 rpcz_store.cc:276] 0709 13:15:37.501599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.508543 (+  6944us) service_pool.cc:224] Handling call
0709 13:15:37.527092 (+ 18549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.158274 (+631182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:38.548246 (+389972us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540008be54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:38.548436 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449396608}
I20260709 13:15:38.584270 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.578128 30625 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:38.587602 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.590118 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:38.590929 30700 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.591743 30629 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:38.592574 30703 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:38.594251 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.595920 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.596954 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.597285 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.598436 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:38.598866 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.599184 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.599517 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.599529 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.600468 30626 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:38.552731  3044 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:38.608531 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.606640 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:38.609517 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.607748 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:38.605065 31391 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:38.552170  3136 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:38.613472 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.618355 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.619488 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:38.657375 31367 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.570523  3101 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:38.621762 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.628557 30702 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:38.667903 30702 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:38.590093  3078 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.602459  3114 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.547914  2531 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.627216  3007 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:38.648000 30624 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:38.647281 30625 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:38.681761 30625 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.640491 31383 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:38.651927 30556 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:38.633870  3045 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.660494 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.643092  3094 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:38.643224  3135 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.622686  2855 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.687367 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.681644 30702 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.683375 30625 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.696288 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:38.701462 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.701807  2855 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:38.703482 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.707595  2797 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:15:38.709663  3069 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:38.693917  3104 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:38.710908  3104 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.711846  3104 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.707335 30702 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:38.713032  3104 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.714702  3135 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.717425  3135 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.654877  3097 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:15:38.719283 30625 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:38.719906  3097 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:38.720376  3097 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.721300  3097 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.723024  3158 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:38.684639  3114 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.723726  3158 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.724298 30551 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:38.725617  3114 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.725926  3158 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:38.681087  3007 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.731091  3007 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:38.694022  3078 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.731328 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:38.732494 30707 raft_consensus.cc:606] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:38.733594  3078 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.698572  3094 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.718350  3136 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.740716  3136 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:38.699786  2973 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:38.744308 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:38.745952 30628 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:38.747614 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:38.748374 30707 raft_consensus.cc:606] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:38.748723 30707 raft_consensus.cc:614] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:38.750329 30632 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:15:38.750838 30632 pending_rounds.cc:85] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:15:38.751117 30632 pending_rounds.cc:107] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:15:38.751346 30632 raft_consensus.cc:2889] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:38.752238  3094 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:38.755661 30702 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:15:38.760129 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.761603 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.762734 30704 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:38.763870 30704 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 9. Current term is 10. Ops: []
I20260709 13:15:38.764962 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.765591 30704 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:15:38.765957 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 69767.5us
I20260709 13:15:38.766556 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.776525 31396 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:38.777773  2892 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:15:38.778831  3042 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:38.779431 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:38.767457 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64136.8us
I20260709 13:15:38.770287 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:38.775931 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.770128 30630 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:38.774780 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:38.778167 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.769804  3045 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.787949 31383 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:38.788060 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72181014679a4c538f96f2d305a5474f"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:38.793876 30707 raft_consensus.cc:606] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:38.788496 30700 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:38.790284 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:38.790354 30702 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:38.771564  3027 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:38.769704  3031 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:15:38.766664  3082 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:38.774902  3119 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.814s	user 0.001s	sys 0.004s
I20260709 13:15:38.803056 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.804409  3097 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:38.804939 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1648) took 1600 ms. Trace:
I20260709 13:15:38.806384 30628 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:38.769543  2860 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:15:38.807401  3069 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:38.772784  3090 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:38.809219  3090 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.810753 30703 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [13.9-16.11]
I20260709 13:15:38.767683  2797 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:38.805388 31364 rpcz_store.cc:276] 0709 13:15:37.204083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.204263 (+   180us) service_pool.cc:224] Handling call
0709 13:15:37.204682 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:37.533950 (+329268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:37.534017 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:37.542067 (+  8050us) log.cc:844] Serialized 26 byte log entry
0709 13:15:37.902236 (+360169us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:37.902306 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:37.902418 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:38.774240 (+871822us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:38.804582 (+ 30342us) raft_consensus.cc:1685] finished
0709 13:15:38.804608 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:38.804868 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":57117,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":38123904,"thread_start_us":600818,"threads_started":2,"wal-append.queue_time_us":363391}
I20260709 13:15:38.789335  3053 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:38.815173 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:38.786996 30625 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:15:38.832991 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:38.833664 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.839102 31367 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:38.849470 30702 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:38.851034 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72947.7us
W20260709 13:15:38.851677 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61501.6us
I20260709 13:15:38.815448  3100 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:38.814975  2697 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:15:38.823640  3083 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.816031  2797 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:38.858071 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.843245  3045 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:38.867211 30553 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:38.868752 30702 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:38.848510  2923 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:38.810503  3090 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.873428 30702 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:38.875733 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.855183  2933 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:38.855183  3053 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:38.814522  3095 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.878548 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:38.864833  2973 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:38.876400 30702 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:15:38.867100  3013 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.012s	sys 0.015s
I20260709 13:15:38.860807  2697 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.890170 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.870416  3045 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:38.887979  3013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.013s	sys 0.015s
I20260709 13:15:38.896809 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.865048  3094 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:15:38.883211  3042 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:15:38.910025  3099 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.996s	user 0.010s	sys 0.022s
I20260709 13:15:38.884738  2973 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:38.901190  3013 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:38.871102  3083 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.892738  3135 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:38.899296  3045 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:38.922220 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107204us
I20260709 13:15:38.912881  3158 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:38.929639  3094 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:38.930307  3094 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:38.931555  3094 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.933415  3094 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:38.934638  3096 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:38.936394 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.938727 31389 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:38.939827  3053 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:38.940438 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 43754.7us
I20260709 13:15:38.941289 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.941512 30551 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:38.942942  3116 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:38.944149 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.944334 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.895244  3148 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:38.919906  2973 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:38.947053  2928 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:38.954869  2797 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.957893  3135 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:38.958539  3135 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:38.958984  3130 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:15:38.959103  2697 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.960768  2697 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:38.963231  3013 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:38.964075  3130 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:38.966876  3113 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:38.971424  2714 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:38.967597  3012 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:38.976728  3012 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:38.977617 30556 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:38.879694  2923 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:38.981896  2923 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:38.983603 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:38.984596 30553 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:38.987109 31294 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:38.988531  3066 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:38.988966 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:38.989068  3066 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:38.989732  3066 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:38.992151  3083 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:38.997118  3007 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:38.999095 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:38.999943 30705 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:38.997547 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.004067 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:39.005570 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.006824 30625 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:38.920559  3099 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.010s	sys 0.023s
I20260709 13:15:39.002555  3007 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.028115 30698 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:39.018731  2933 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.030437  2933 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:39.031973  3082 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:39.032598  3082 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.033056  3082 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:39.035393  3167 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:39.035907 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.037708 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.038553  3100 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:39.038815  3160 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:39.054811 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:39.063738 30627 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:39.037891  2933 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:39.066708 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.067101 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1911) took 1094 ms. Trace:
I20260709 13:15:39.069133 30699 rpcz_store.cc:276] 0709 13:15:37.957534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.957698 (+   164us) service_pool.cc:224] Handling call
0709 13:15:37.974662 (+ 16964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.285346 (+310684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.051414 (+766068us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400341f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:39.051683 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644454656}
I20260709 13:15:39.071967 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.074813 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:39.077628 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:39.079355 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:38.912211  3101 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:39.019387  3099 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:39.090055 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.085294 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.058899  3097 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.100080  3160 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:39.100451 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.101037  3097 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.101554 30625 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:39.102759 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.103375 30625 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:39.106359  3135 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:39.110172  3135 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:39.111532 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:39.103535  3097 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.113962  3114 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:39.114414  3114 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.114794  3114 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:39.087237  3007 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:39.119575 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1603) took 1081 ms. Trace:
I20260709 13:15:39.119833  3007 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:39.120666  3007 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:39.122943 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 763) took 1136 ms. Trace:
I20260709 13:15:39.123461 30701 rpcz_store.cc:276] 0709 13:15:37.986736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.987088 (+   352us) service_pool.cc:224] Handling call
0709 13:15:39.122597 (+1135509us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400435654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:39.122839 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781636864}
I20260709 13:15:39.124220  3095 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.125334  3095 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:39.133451 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.134204 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.120268 30632 rpcz_store.cc:276] 0709 13:15:38.037760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.038100 (+   340us) service_pool.cc:224] Handling call
0709 13:15:38.061963 (+ 23863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.749938 (+687975us) spinlock_profiling.cc:243] Waited 97 ms on lock 0x7b540064dfa0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:15:38.751839 (+  1901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.119122 (+367283us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b54004abfd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:39.119448 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325346432}
W20260709 13:15:39.135874 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.136072 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71729.3us
I20260709 13:15:39.141726 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.142251 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.080579  2864 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:39.143455 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.186023 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.172035 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.185474 31391 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:39.187568 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.188879 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.189815 31383 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:39.126240  3095 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:39.221925 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:39.222458 30625 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:39.244494 30701 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:39.098976  3136 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:39.137378  3113 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.219354  3042 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:39.099291  2933 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
W20260709 13:15:39.284287 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.224314  3167 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:15:39.240707  3100 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:39.259846 30625 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:39.258062  2860 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:39.265537 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 123886us
I20260709 13:15:39.294013 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.276708 31383 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:39.294764 30625 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:39.257037 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58267.7us
I20260709 13:15:39.131994  3045 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.218891  2839 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.186961  3066 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:39.298946 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.300217 31394 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:39.300626 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:39.303637 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.305217 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:39.248162  2973 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:39.306298 30705 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:39.307058 30625 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:39.287813  3064 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.308719 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:39.313594 30699 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:39.314146 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:39.310245 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.266117  2855 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:15:39.316504 30702 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 10: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:39.318487 30632 raft_consensus.cc:2368] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:39.317565 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:39.320497  3170 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:15:39.322770 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.324437 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.310245  2839 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:39.329003 31396 raft_consensus.cc:2410] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:39.330051  3066 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:39.333310 30625 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:39.333819 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.335163 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.335531 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.345002 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.347508  2797 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.347961 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.341521 30631 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:39.349357 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.349941 30631 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:39.350796 30632 raft_consensus.cc:2368] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:15:39.348157  2797 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:39.353752 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.354446  2797 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.355273  2933 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.355906  2933 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.356261  3095 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:39.359702  2933 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:39.360716  3168 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:15:39.198390  2864 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:39.263695  3136 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:15:39.367353 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2201) took 1128 ms. Trace:
W20260709 13:15:39.367447 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 172841us
I20260709 13:15:39.369844 30632 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:39.299789  2860 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.385294 31383 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:39.392674 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.388800 30555 rpcz_store.cc:276] 0709 13:15:38.239296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.239883 (+   587us) service_pool.cc:224] Handling call
0709 13:15:38.240403 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.564514 (+324111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.366986 (+802472us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400174b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:39.367262 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642654464}
I20260709 13:15:39.386301 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.387570 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.398170 31364 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:39.394317 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.362917  3082 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.368350  2531 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:39.292075  3113 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:39.402032 31383 raft_consensus.cc:2376] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16. Re-sending same reply.
I20260709 13:15:39.378141  3097 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.415586 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.416497 31383 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:39.407292 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.419361 31394 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:39.419842 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.421100 31391 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:39.343387  3104 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:15:39.318238  3064 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:39.423879 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.390549  3066 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:39.438205 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 101522us
I20260709 13:15:39.313674  3099 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:39.447589 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.408802  2933 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:39.451134 30555 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:39.448462  3101 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:39.524485  2697 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.005s
W20260709 13:15:39.530187 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:39.422868  3097 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.420935  3135 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:15:39.530716 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.603863 31391 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:39.615013 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.618252 30701 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:39.447136  3066 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:39.658350  3174 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 13:15:39.661754 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 272207us
I20260709 13:15:39.677893 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:39.703384  3182 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:15:39.707530  3177 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:15:39.619315  3045 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.574373  3136 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:15:39.721740 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.614319  3114 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:39.580926 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.746876 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.600759 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 268602us
I20260709 13:15:39.727559 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:39.728962  2860 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:39.756763 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.761613 31391 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:39.710361  3135 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:39.763862 31292 leader_election.cc:400] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:39.683203  3042 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:39.773087 31292 leader_election.cc:403] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:15:39.559099 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.732847 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 276933us
I20260709 13:15:39.724439  3064 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:39.723985  2973 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:39.799727 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:39.800554 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1528) took 1227 ms. Trace:
I20260709 13:15:39.801136 30698 rpcz_store.cc:276] 0709 13:15:38.572685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.572849 (+   164us) service_pool.cc:224] Handling call
0709 13:15:39.800349 (+1227500us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54005a2dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:39.800482 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":553934208}
I20260709 13:15:39.766203  3135 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:39.771828  3167 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:39.767495  3184 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:39.823755 30628 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:15:39.825220 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:39.828330  3122 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.868s	user 0.000s	sys 0.016s
I20260709 13:15:39.835170 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.802865  2864 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:39.838600 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.807156  3160 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:39.839546 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:39.846897 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:39.823668  3095 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:39.847733 30629 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:39.851512 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 85648us
W20260709 13:15:39.849872 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 186694us
I20260709 13:15:39.824064  3082 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:39.855131 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:39.856505 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:39.858366 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.859462 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.869647 31391 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:15:39.904629 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45263.9us
W20260709 13:15:39.905601 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 99031.9us
I20260709 13:15:39.905920 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:39.907712 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68310.4us
I20260709 13:15:39.909258 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:39.866827 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:39.861647  3184 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:15:39.867859  3179 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.260s	user 0.002s	sys 0.000s
I20260709 13:15:39.862071  3160 raft_consensus.cc:2764] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:15:39.861708  3167 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:39.868816  2860 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:39.907286  2973 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:39.906505  3082 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:15:39.881686  3042 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:39.909042  3135 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:39.916458 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:39.924273 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1479) took 1556 ms. Trace:
W20260709 13:15:40.042152 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 127877us
W20260709 13:15:39.980432 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113694us
W20260709 13:15:39.952006 30627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 889) took 1688 ms (client timeout 1609 ms). Trace:
I20260709 13:15:39.980542 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1598) took 2396 ms. Trace:
I20260709 13:15:40.047139 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:39.980584 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:39.910171  3113 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:40.047446 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 140696us
I20260709 13:15:39.948464 30629 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:40.048254 30624 rpcz_store.cc:276] 0709 13:15:37.584419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.584827 (+   408us) service_pool.cc:224] Handling call
0709 13:15:37.585268 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.681331 (+1096063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.980220 (+1298889us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54007aa654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:39.980455 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2436769024}
I20260709 13:15:39.947386 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:40.003284  3173 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.263s	user 0.000s	sys 0.002s
W20260709 13:15:40.057907  3007 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.936s	user 0.018s	sys 0.000s
W20260709 13:15:40.049680 30627 rpcz_store.cc:269] 0709 13:15:38.261481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.300067 (+ 38586us) service_pool.cc:224] Handling call
0709 13:15:39.949766 (+1649699us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540063ced4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:39.949969 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310629760}
I20260709 13:15:39.923828  3097 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:40.058439  3007 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:40.060196 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1482) took 1622 ms (client timeout 2134 ms). Trace:
W20260709 13:15:40.060678 31378 rpcz_store.cc:269] 0709 13:15:38.437338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.437506 (+   168us) service_pool.cc:224] Handling call
0709 13:15:40.059927 (+1622421us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b5400717854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:40.060098 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056234368}
I20260709 13:15:40.061123  3009 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:40.054299 30625 raft_consensus.cc:2368] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:39.916769  2864 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.140s
I20260709 13:15:40.056867  3187 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:39.918260  3095 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:40.048493 31388 rpcz_store.cc:276] 0709 13:15:38.367861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.368117 (+   256us) service_pool.cc:224] Handling call
0709 13:15:38.368553 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.393144 (+ 24591us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:38.393182 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:38.415028 (+ 21846us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:38.415086 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.923939 (+1508853us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:39.924006 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:39.924207 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55988480}
I20260709 13:15:39.951148  3114 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:39.971436  3053 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:39.940243  3069 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:39.923069  3194 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:39.928603  3066 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.076288  3007 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:39.947610  3191 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:40.050993  2860 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:40.088441  3135 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:40.091845  3100 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.001s
I20260709 13:15:40.047077  2973 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.100518 30626 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:40.098405  3100 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:40.115593 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68592.1us
I20260709 13:15:40.118049 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:40.118945 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 213163us
I20260709 13:15:40.119679 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:40.100275  3202 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:40.110117  3194 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.140316  3202 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.140461  3194 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.143043  3200 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:40.144723  2973 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.131243  3184 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:15:40.099998  3191 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.150184  3125 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:40.150743  3125 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.150924  3100 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:40.152138  3185 raft_consensus.cc:993] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:15:40.151913  3125 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.152736  3185 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:40.153142  3185 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:40.153380  3125 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:40.153744  3190 raft_consensus.cc:1081] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:15:40.112977  3133 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:40.155223  3133 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.083786  3187 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.140929  3202 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:40.172658 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:40.158048 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.173431 30550 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:40.159224 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.159626 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2210) took 1305 ms. Trace:
I20260709 13:15:40.160893 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.133108 31292 leader_election.cc:400] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
W20260709 13:15:40.177268 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44289.9us
I20260709 13:15:40.161310 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1647) took 1625 ms. Trace:
I20260709 13:15:40.178932 30557 rpcz_store.cc:276] 0709 13:15:38.535614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.536954 (+  1340us) service_pool.cc:224] Handling call
0709 13:15:38.548611 (+ 11657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:38.558215 (+  9604us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:38.685188 (+126973us) log.cc:844] Serialized 14 byte log entry
0709 13:15:39.017685 (+332497us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:40.141145 (+1123460us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:40.141678 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 13:15:40.141856 (+   178us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:40.141898 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.141975 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.160995 (+ 19020us) raft_consensus.cc:1685] finished
0709 13:15:40.161091 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.161246 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30900,"prepare.queue_time_us":90007,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":57743,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":33007,"spinlock_wait_cycles":75056512,"thread_start_us":233636,"threads_started":2,"wal-append.queue_time_us":450738}
I20260709 13:15:40.174381 30556 rpcz_store.cc:276] 0709 13:15:38.854488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.129748 (+275260us) service_pool.cc:224] Handling call
0709 13:15:39.130262 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:39.263692 (+133430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.263728 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:39.263779 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:39.263812 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.159344 (+895532us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54002d69d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:40.159401 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.159549 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306239232}
W20260709 13:15:40.138712  2531 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.006s
I20260709 13:15:40.186882 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.188589 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:40.191597 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73631us
I20260709 13:15:40.157030  3104 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:40.150455  3191 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.154035  3185 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:40.197382 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 122914us
I20260709 13:15:40.172046  3200 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.194195 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:40.195647 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.193455 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.198547 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:40.202301 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 120632us
I20260709 13:15:40.203332 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.204309 30558 raft_consensus.cc:2410] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:15:40.205426 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.207435 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.203899 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:40.210299  3100 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:40.213336  3100 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:40.215343 31292 leader_election.cc:400] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:15:40.216806 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.193584  3104 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.220948 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.221307 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.222772 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.225512  3205 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:40.226769  3196 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:40.227748  3198 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.218077 30559 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:40.121531  2860 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.156445  3133 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.199112  3194 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.229954  2973 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:40.201421  3190 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.242013  3216 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.243875  3187 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.216058  3082 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:40.246382  3213 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:40.246829  2531 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:40.249748  3207 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.251149  3104 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.252182  3196 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:40.203737  3200 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.259935 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.259581 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.262123  3205 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:40.264958 30550 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:40.270545 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.283535 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.270789 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.271476 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.285068 30698 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.267604  3082 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.272810 30702 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:40.273938 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.226438  3193 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.277669 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.289999 31292 leader_election.cc:400] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:40.292217 31292 leader_election.cc:403] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:15:40.293365 30559 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:40.286474 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.272184 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.295325 31367 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:40.267256  2973 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:40.294577  3013 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:40.308571  3013 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:40.308852  2973 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.294734 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.296856 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.297396 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.297544 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.312685 31396 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:40.297859 31367 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:40.272131  3211 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:40.298997  3104 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:40.299436  3095 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:40.291270  3099 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:15:40.300211 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.325348 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.277318  3210 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.298997  3212 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.307307 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.351280 30702 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 9.
I20260709 13:15:40.280756  2839 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.319577 30698 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:40.320379 30704 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:40.321838 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.286466  3100 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:40.264562  3190 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.292230  3082 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:40.291867  3187 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:40.326120 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.326563 30554 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 10.
I20260709 13:15:40.266480  3216 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.329891  3208 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:40.375451  3208 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:40.375986  3208 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.376361  3208 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:40.376736  3216 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.313565  3125 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:40.309623  3013 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.368592  3092 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.323966  3205 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.380703 30703 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:40.379441  3194 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:40.384816 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:40.278573  3207 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.402034  3200 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.362924  3100 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.295156  3193 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.407372  3100 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:40.408259  3207 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.397464  3125 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.416343  3125 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.380424  3181 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.419025  3181 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:40.419697  3095 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.420306  3196 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.420688  3095 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:40.364240  3212 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.326196  3211 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.424198  3212 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.381057  3187 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.425091  3187 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:40.425258  3211 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.426894 30707 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:40.397379  3092 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.420938  3196 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:40.430121 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45420.8us
W20260709 13:15:40.440268 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 241904us
I20260709 13:15:40.441484 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.442032 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:40.445542 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.446630 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:40.447448  3190 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.447608  3031 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:40.450121 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.448232  3190 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:15:40.471462 30670 connection.cc:256] RPC call timeout handler was delayed by -1.0648946020000949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2134s for a response
I20260709 13:15:40.450752  3200 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:40.360975  3210 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:40.472661 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 149102us
W20260709 13:15:40.484037 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:40.468602  3135 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:40.466261  3215 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:40.452560  3031 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:15:40.494261 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.402856  3205 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:40.505033 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 136303us
I20260709 13:15:40.499836  3135 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.525892 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.536583  3082 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.554302 30626 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:15:40.555548  3092 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.556604  3082 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.556252  3135 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.568091  3196 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.580452  3084 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:40.581107  3084 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:40.581243 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.583537 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.582039  3084 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.584290 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.585078  3084 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:40.563340 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.586848 30706 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:40.545531  3210 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.550806  3095 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.593284 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.594687 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.553040  3066 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.595624 30632 raft_consensus.cc:2410] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:40.595510  3066 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:40.596177 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1489) took 2034 ms. Trace:
I20260709 13:15:40.596632 31389 rpcz_store.cc:276] 0709 13:15:38.561421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.561607 (+   186us) service_pool.cc:224] Handling call
0709 13:15:38.938317 (+376710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:38.939265 (+   948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.595986 (+1656721us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400113d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:40.596107 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":851232000}
I20260709 13:15:40.552737  3100 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.600841  3078 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.601603  3078 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:40.603075 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.604130 30707 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:40.608831 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.609803 30701 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:40.613265  3210 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:15:40.619170  3083 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:40.619954  3083 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.621178  3083 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:40.625123 30554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 856) took 2019 ms (client timeout 1668 ms). Trace:
W20260709 13:15:40.625829 30554 rpcz_store.cc:269] 0709 13:15:38.605531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:38.687787 (+ 82256us) service_pool.cc:224] Handling call
0709 13:15:40.624771 (+1936984us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:40.624965 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3735503104}
I20260709 13:15:40.635159  3130 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:40.639660 30699 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:40.640458  3069 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:40.643618  3069 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:40.644796  3069 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:40.651160 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:40.651625 30556 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:40.652060 30551 raft_consensus.cc:2368] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
I20260709 13:15:40.652762  3027 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.653410  3027 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.656556 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.659943 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:40.669930 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:40.671521 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 78391.4us
W20260709 13:15:40.673398 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 87241.4us
I20260709 13:15:40.676460 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.678895 30625 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:40.681255 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.682487 30625 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:40.670354 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.657487 30559 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:40.646243  3069 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.684787 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.685135 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.685793 30628 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:40.686113 30627 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:40.642012  3130 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:40.689342 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 163559us
I20260709 13:15:40.691012 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.691802 30625 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:40.692200  3012 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:40.692605  3224 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.682817  3190 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.704063  3193 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:40.704357 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.704599  3190 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:40.704931 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.705643 31389 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:40.694828 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:40.706872  3100 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.707177 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.696382 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.707873 31361 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:40.695930 31367 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:40.710156 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:40.709008 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:40.697966 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:40.712865 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.726174 30554 raft_consensus.cc:2368] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:40.728376 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.724881 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:40.741134 30551 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:40.683621  3008 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.747376 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:40.748543 30627 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:40.743145 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.755793 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:40.705931  3175 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.256s	user 0.014s	sys 0.000s
I20260709 13:15:40.706890  3185 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.764588 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.775810 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:40.683039  3101 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.765533 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.692107  3097 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.767874 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.776348 30703 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.799664 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:40.771806 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:40.800419 31378 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:40.800531 30707 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:40.781383 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.801333 31367 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:40.785924 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.781090 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:40.797375 30627 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:40.729601  3027 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:40.728981  3078 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:40.814318 31378 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:40.730190  3094 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:40.760474  3113 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.799232 30559 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:40.802379 30554 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:40.706897  2928 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 13:15:40.763356  3182 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:40.760131  3193 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:40.808460 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.776283  2697 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:40.817931 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:40.763139  3096 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:40.763293  3012 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:40.766865  3195 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:15:40.862187 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 81173.7us
I20260709 13:15:40.844069 31378 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:40.860548 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:40.865715 30704 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:40.817704 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:40.868085 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50301.4us
I20260709 13:15:40.704443  3224 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:40.818924 30705 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:40.760902  3008 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:40.802459  3104 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.942349 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1158) took 1942 ms. Trace:
I20260709 13:15:40.943833 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.871402 30627 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:40.955386 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 284275us
I20260709 13:15:40.974401 30633 rpcz_store.cc:276] 0709 13:15:39.000427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:39.000593 (+   166us) service_pool.cc:224] Handling call
0709 13:15:39.001108 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:39.001580 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:39.001632 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:39.002324 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 13:15:39.221276 (+218952us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:39.221342 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:39.221455 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:40.638249 (+1416794us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:40.901938 (+263689us) raft_consensus.cc:1685] finished
0709 13:15:40.901978 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:40.942248 (+ 40270us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12001,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":13326,"spinlock_wait_cycles":7166720,"thread_start_us":218459,"threads_started":1,"wal-append.queue_time_us":404384}
I20260709 13:15:40.802690  3100 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:40.898350  3208 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.006s
I20260709 13:15:40.926702 30624 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:40.787485  2531 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:40.956926  3135 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.971414 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:40.989187 31367 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:40.872493 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:41.032130 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 159516us
I20260709 13:15:40.963832  3064 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.992792 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.869747  3193 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:40.807982  3148 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.869931  3181 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:40.944965  3190 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.002274  3196 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.871729  2697 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:40.958068  3096 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:40.819149  3094 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:40.898896  3013 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:41.038347 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:40.816179  3097 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.002493  3095 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.991533 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.001915  3099 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.996577  2973 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:40.978163  3130 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:40.803867  3101 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:41.071717 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 144846us
I20260709 13:15:41.101718 30625 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.103087 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:40.928217  3113 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.130580  3196 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:41.106961 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.112849 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.264185 30699 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:41.024820 31389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50024
I20260709 13:15:41.128842 31383 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:40.974501  3195 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.049870  3231 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:40.985215  3104 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.273840  3195 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.274272  3104 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.274835  3230 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:41.276791  3224 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.277798  3231 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.278787  3066 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.279592  3205 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.140028  3095 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.280455  3231 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.102351  3199 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:41.281723  3199 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:41.271461  3233 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:41.131045 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:40.900512  3027 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:41.283901 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1541) took 2143 ms. Trace:
I20260709 13:15:41.284111  3027 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.047233  3135 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.284332 30700 rpcz_store.cc:276] 0709 13:15:39.140311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.595198 (+1454887us) service_pool.cc:224] Handling call
0709 13:15:40.595535 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:40.596857 (+  1322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.596904 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:40.597474 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:15:40.868880 (+271406us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:40.868924 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:40.868996 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.283669 (+414673us) raft_consensus.cc:1685] finished
0709 13:15:41.283692 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.283828 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":931,"raft.queue_time_us":1681,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"spinlock_wait_cycles":1792,"thread_start_us":270982,"threads_started":1,"wal-append.queue_time_us":499277}
I20260709 13:15:41.285531  3196 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.286401  3135 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:41.139212  3202 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.950s	user 0.014s	sys 0.006s
I20260709 13:15:41.286881 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.287254  3202 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.287850  3101 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.289762  3202 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.270961  2856 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:15:41.175287  3078 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:15:41.270737 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.294117 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.137907  3012 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.294641 30700 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:41.294889  3009 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.295454  3009 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.297573  3009 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.127193  3096 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:41.299474  3069 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.003s
I20260709 13:15:41.300208  3113 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.302845 30551 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
I20260709 13:15:41.143671  3099 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:41.302896 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71144us
I20260709 13:15:41.305712 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:41.308292  3239 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:41.309369  3181 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:41.271647 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.310117  3181 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:41.310352 30699 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:15:41.310959  3237 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:15:41.311455 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.311133  3181 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.312026 30699 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:41.312418 30631 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:41.315289 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:41.117750  2697 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.131453  3182 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:41.144295  3187 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.712s	user 0.012s	sys 0.002s
W20260709 13:15:41.144788  3198 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.910s	user 0.017s	sys 0.000s
I20260709 13:15:41.025731  3208 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:41.155154  3232 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:15:41.014896  3100 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:41.228580 30706 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.641s	user 0.017s	sys 0.003s
I20260709 13:15:41.217100  3008 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.103688  3064 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.127548  3094 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.271945  3097 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.052654 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.116976  3148 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.282160  3199 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:41.091464  3193 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:41.295696 30700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:41.333674  3198 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:41.283901 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:41.270682 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.399971 31378 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:41.316639 30707 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:41.322650 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:41.281096  3231 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:41.281970  3095 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.333549 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.335059 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:41.282001  3084 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.005s
I20260709 13:15:41.337414 30706 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:41.305635  3099 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.419281 31361 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:41.419454  3204 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.284538  3027 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:41.420120  3204 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.332052  3187 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.425029  3187 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.426832 30630 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:41.428447 30628 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:41.428622 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:41.414450  2856 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:41.430231 30630 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:41.432179  3064 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.433966 30706 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:41.435513 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:41.437161 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.404948  3231 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:41.438037 30706 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:41.362510 30625 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:41.378103 31391 raft_consensus.cc:2410] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
W20260709 13:15:41.367169 30703 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.000s
I20260709 13:15:41.392802 31367 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:41.205410 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.440935 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:41.394060  3237 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:15:41.404587 31383 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:41.334708  2973 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:41.408324 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73406.6us
I20260709 13:15:41.405638 30699 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:41.409644 30707 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:41.412247  3100 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.462534 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.463320 30705 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:41.414314  3182 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.417721  3067 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:41.441967 30703 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:41.467344 30704 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:41.467717 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.468475 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:41.461167 31378 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:41.458029 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:41.474169 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:41.455874  3185 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.004s
I20260709 13:15:41.481491 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:41.483189  3190 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:41.483788  3190 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.484297  3190 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:41.492743 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:41.494238  3195 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:41.495481 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:41.496857  3195 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:41.497454  3195 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.497953  3195 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:41.500146  3192 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:41.509496  3190 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.511616 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:41.512820 30630 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:41.514734 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:41.518013  3190 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.454648  2929 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:41.527549 31394 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:41.467015  3213 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:15:41.534228 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.535380 31383 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:41.474771  3078 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:41.438547  3231 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.543030  3245 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:41.559064  3245 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.559794  3148 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.542542  3078 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.560950  3078 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:41.561724  3245 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.540910 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:41.539670 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.567029 31383 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:41.567464 30706 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
W20260709 13:15:41.568053  3210 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.954s	user 0.017s	sys 0.000s
I20260709 13:15:41.568573  3210 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:41.539562 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:41.569681 30698 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:15:41.570191 30698 pending_rounds.cc:85] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 10
I20260709 13:15:41.540319  2929 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:41.570992 30701 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:15:41.571354  2929 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.573820  3042 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:41.574565  3042 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.526199  2856 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:41.593865 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:41.602084 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 507642us
I20260709 13:15:41.594226 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.604561 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.604496 30553 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:41.570580 30698 pending_rounds.cc:107] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:15:41.575604  3090 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:41.609786 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.593382 30550 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:41.618440 30550 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:41.620764 30706 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:41.619612 30550 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.576858  3187 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:41.605821  3210 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:41.556391  2973 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.622089 30550 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:41.630847 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 324681us
I20260709 13:15:41.576463 30700 raft_consensus.cc:606] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:41.632643 30700 raft_consensus.cc:614] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:41.557117  3208 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.605502  2933 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.597702  3064 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.556600  3204 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:41.540223 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.582026 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.627605 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:41.669749 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.672788 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:41.627087 30632 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.008s	sys 0.002s
I20260709 13:15:41.628551 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.683059 31396 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:41.611852 30698 raft_consensus.cc:2889] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:41.635672 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:41.641357 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:41.641966 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.685698 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.686103 30706 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:41.684976 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49408us
I20260709 13:15:41.685474  3012 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:15:41.647614 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 341953us
I20260709 13:15:41.649503 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:41.687180 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.643707 31294 leader_election.cc:400] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16.
W20260709 13:15:41.699580 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56107.9us
I20260709 13:15:41.596988  3096 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.700202 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.702631  3096 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:41.662052 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.702916 30629 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:41.629693  3187 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.690871  3012 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.598279  3193 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.688699 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:41.705734 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:41.706485  3193 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.684394 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.708575  3157 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:41.708900 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:41.709332  3157 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.624401  3200 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.686314 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 915) took 1014 ms. Trace:
I20260709 13:15:41.709853  3157 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:41.710348  3200 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.710870  3238 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:41.711687  3238 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:41.703799  3187 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:41.630254  3210 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:41.630714  3084 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.686553 30698 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:41.691890 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.714180 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.716635 30558 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:41.732971 30550 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:41.717381 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.694801 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:41.734654 30551 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:41.709756 31396 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:41.683540 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:41.719990 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 70605.7us
W20260709 13:15:41.721498 31364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.014s	sys 0.022s
I20260709 13:15:41.710327 30632 rpcz_store.cc:276] 0709 13:15:40.671446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.671618 (+   172us) service_pool.cc:224] Handling call
0709 13:15:40.672057 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:40.682965 (+ 10908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.683002 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:40.703500 (+ 20498us) log.cc:844] Serialized 52 byte log entry
0709 13:15:40.964294 (+260794us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:41.102078 (+137784us) log.cc:844] Serialized 14 byte log entry
0709 13:15:41.684193 (+582115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.684787 (+   594us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:41.685000 (+   213us) raft_consensus.cc:1685] finished
0709 13:15:41.685177 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:41.686154 (+   977us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48841,"prepare.queue_time_us":310370,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":59174,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":65406,"thread_start_us":935122,"threads_started":3,"wal-append.queue_time_us":334559}
W20260709 13:15:41.655365  3013 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.006s	sys 0.003s
W20260709 13:15:41.722851 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.726591 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:41.742187 31364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.015s	sys 0.022s
I20260709 13:15:41.703953  3012 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:41.756966 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.757282 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.758112 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:41.769984 30523 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:41.713687  3194 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:41.779685 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79304.2us
I20260709 13:15:41.780544 31389 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:15:41.781098 31389 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:41.778568 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:41.781983 30550 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:41.782100 31294 leader_election.cc:400] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
W20260709 13:15:41.779844 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43751.9us
I20260709 13:15:41.783449 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:41.782538 31294 leader_election.cc:403] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:15:41.654935  3243 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.113s
I20260709 13:15:41.785130 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.640826  3009 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 13:15:41.731552  3210 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:41.794036 30707 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:41.797657 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:41.799549 30632 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:41.752238  3013 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:41.656499  2933 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.757014 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.801208 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.808377 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:41.812965 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.817598 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:41.821445  3190 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:41.823405  3083 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:41.823766  3250 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:15:41.713575  3064 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.819727 31364 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:41.826150 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:41.773770  3253 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:41.833292 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.769711  3101 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:41.771561  3008 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:15:41.839617 30550 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:41.825475  3246 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:41.856240  3246 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:41.856771  3246 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:41.858832  3200 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.859292 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:41.825088  3113 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:15:41.862731  3042 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.863538  3253 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.866811  2973 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:41.867502  2973 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.870198  2973 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.792562  3244 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:41.871208  3249 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 13:15:41.871032  3244 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.873881  3244 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:41.779989  3238 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.875363  3254 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.876134  3254 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.880801  3246 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:41.899008  3246 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.806713  3185 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:41.848193 30624 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:15:41.838253  3199 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.886796  3066 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:15:41.887264  3013 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:41.890858  2933 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:41.976298 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1168) took 1175 ms. Trace:
W20260709 13:15:41.892434  2697 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.004s
I20260709 13:15:41.760250  3084 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.989229  3259 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:15:41.990694  3084 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:41.789211  3196 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:15:41.991210 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 775) took 1280 ms. Trace:
I20260709 13:15:41.991663 30701 rpcz_store.cc:276] 0709 13:15:40.710716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.729344 (+ 18628us) service_pool.cc:224] Handling call
0709 13:15:40.729676 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.571445 (+841769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.990910 (+419465us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54000c9754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:41.991135 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934034176}
I20260709 13:15:41.846220 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:41.909816 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1671) took 1017 ms. Trace:
W20260709 13:15:41.931551 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 161655us
I20260709 13:15:41.938392 31396 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:41.994716 31396 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:15:41.995263 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:41.757009  2714 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.153s
I20260709 13:15:41.857937  3195 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:41.941375 31389 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.791095  3094 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.977089 30629 rpcz_store.cc:276] 0709 13:15:40.800573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.802867 (+  2294us) service_pool.cc:224] Handling call
0709 13:15:40.866999 (+ 64132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.703653 (+836654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:41.975885 (+272232us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:41.976196 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921767296}
I20260709 13:15:42.012081 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.788132  3027 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.013936 30630 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:42.013774 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 287276us
I20260709 13:15:41.992782 31361 rpcz_store.cc:276] 0709 13:15:40.893038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.893289 (+   251us) service_pool.cc:224] Handling call
0709 13:15:41.909448 (+1016159us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54003377d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:41.909702 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895930112}
I20260709 13:15:41.866505  3253 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:41.920058  3095 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:41.921638  3244 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:41.943050  3008 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:42.043286 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.006834 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:42.047042 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 222489us
I20260709 13:15:42.007501 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.051163 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:42.009668 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:42.054111 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.050375 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:42.055840 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46272.5us
I20260709 13:15:42.056795 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:41.943496  3185 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.004258  3042 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:42.014564 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 206268us
I20260709 13:15:42.017518 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.032735  3259 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.069828 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:41.999940  2697 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:41.923648  3182 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 13:15:42.037586 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:41.984697  2973 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.041960 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:42.014088  3009 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.058261  3084 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:42.032701  3205 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:42.068411 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1511) took 1069 ms. Trace:
I20260709 13:15:42.023870 30551 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:42.039831  3253 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.070634 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.070053 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.015872 30554 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:42.102689 30628 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:42.076098 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:42.076602 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.047781  3095 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.083200 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.088845 30701 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:42.011461 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:42.093633 30698 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:42.094350 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.061684  3244 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:42.068666  2714 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:42.101059 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 86093us
I20260709 13:15:42.103509 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.103871 30552 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:42.098512 31378 rpcz_store.cc:276] 0709 13:15:40.998738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.478972 (+480234us) service_pool.cc:224] Handling call
0709 13:15:41.479662 (+   690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:41.480185 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.480215 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.542782 (+ 62567us) log.cc:844] Serialized 26 byte log entry
0709 13:15:41.752702 (+209920us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:41.788876 (+ 36174us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.067953 (+279077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.068066 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.068121 (+    55us) raft_consensus.cc:1685] finished
0709 13:15:42.068136 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.068328 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29763,"prepare.queue_time_us":22661,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":7722,"raft.queue_time_us":132382,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":6687,"thread_start_us":442558,"threads_started":2,"wal-append.queue_time_us":336911}
I20260709 13:15:42.038233  3254 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.169562  3067 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:42.097383  3009 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.184475  3238 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.185130  3238 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.106871  3253 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:42.160743  3264 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:15:42.159638  3270 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:15:42.165395  3245 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.186837  3009 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.094935  2973 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.187327  3245 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.107705  3255 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:42.188310  3255 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.188710  3255 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:42.189270  2973 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.189674  3245 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.140359  3027 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.085304  3259 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.169157  3094 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:42.179589 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 168247us
I20260709 13:15:42.191267 31378 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:42.085741  3246 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:42.108779 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:42.110641 30632 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:42.195514 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.113013  3193 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:42.113214  3204 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.113066  3198 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.122784  3276 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.074941  3130 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.100762  3205 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.136935  3199 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:42.094082  3078 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.530s	user 0.013s	sys 0.001s
I20260709 13:15:42.205592  2973 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:42.147310 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.164613 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1509) took 1167 ms. Trace:
W20260709 13:15:42.166993 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60593us
I20260709 13:15:42.155042  3066 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 13:15:42.104285 30633 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:42.171945  3083 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:15:42.187638  3253 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.164915  3096 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.191942 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:42.190387 30557 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:15:42.108741  3273 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.107882 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:42.197506 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 147271us
I20260709 13:15:42.121829  3084 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:42.102964  3101 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:42.164156  2697 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.138574  3064 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.074513  3042 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.145823  3095 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.238598 30629 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:15:42.198544  3246 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.195492  3245 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:42.144584 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61613.9us
I20260709 13:15:42.243166  3276 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.253141 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.226387  3185 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.225577  3252 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:42.269358 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 272214us
I20260709 13:15:42.256749 31391 rpcz_store.cc:276] 0709 13:15:40.997136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.477721 (+480585us) service_pool.cc:224] Handling call
0709 13:15:41.531856 (+ 54135us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:15:41.538121 (+  6265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:41.538157 (+    36us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:15:41.566720 (+ 28563us) log.cc:844] Serialized 78 byte log entry
0709 13:15:41.785799 (+219079us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:42.029817 (+244018us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.048343 (+ 18526us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.048764 (+   421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.048891 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.164253 (+115362us) raft_consensus.cc:1685] finished
0709 13:15:42.164288 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.164524 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9016,"prepare.queue_time_us":83632,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":10876,"raft.run_cpu_time_us":537,"raft.run_wall_time_us":28251,"spinlock_wait_cycles":8249600,"thread_start_us":423063,"threads_started":2,"wal-append.queue_time_us":432522}
I20260709 13:15:42.251214  3205 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:42.254288  3090 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.632s	user 0.014s	sys 0.000s
I20260709 13:15:42.224495  3193 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:42.367321  3012 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.000s
I20260709 13:15:42.377283  3193 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:42.287253 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.167905  3244 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.283145 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:42.258924  3259 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.264475  3083 consensus_queue.cc:1243] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:15:42.228143  3198 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.384548 30625 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:42.384752  3198 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.314990 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:42.306533  3008 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:42.385556  3190 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:42.386255  3190 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:42.386833  3190 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:42.386546  3008 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.387631 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:42.321843 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47090us
W20260709 13:15:42.322295 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 216533us
I20260709 13:15:42.275094  3009 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:42.244202  3130 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.282650  3253 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:42.325578 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 133720us
I20260709 13:15:42.326664 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.169245  2714 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.258643  3078 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:42.329708 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76729.4us
I20260709 13:15:42.332218 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:42.333707 30550 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:42.255532  3275 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:42.347527 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.347419  3241 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:15:42.353025  3042 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.330718  3245 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:42.361442  3210 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.531s	user 0.008s	sys 0.006s
I20260709 13:15:42.304334  3273 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:42.364600  3187 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 13:15:42.283674  3096 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.268467  3027 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:42.371455  3095 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.317754  3271 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:42.375912  3012 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.269599  3051 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 13:15:42.225387  3204 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.446266 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.451678 30630 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:15:42.453044 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.326148  3246 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:42.395339 30626 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:42.397032 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:42.412247 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.337071  2697 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:42.417263  3282 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:42.465699 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49486.5us
I20260709 13:15:42.418035  3078 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.418418 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1514) took 1274 ms. Trace:
I20260709 13:15:42.419168  3278 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.424160 30629 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:15:42.432966 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.427652  3231 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:42.343851  3252 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.471195 30558 raft_consensus.cc:2368] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11.
I20260709 13:15:42.467103 31394 rpcz_store.cc:276] 0709 13:15:41.144419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.293784 (+149365us) service_pool.cc:224] Handling call
0709 13:15:42.418211 (+1124427us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54008b2754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:42.418353 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535247232}
I20260709 13:15:42.443675  3130 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.475147 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.476320 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1169) took 1668 ms. Trace:
I20260709 13:15:42.476933 30627 rpcz_store.cc:276] 0709 13:15:40.807521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.810885 (+  3364us) service_pool.cc:224] Handling call
0709 13:15:40.871207 (+ 60322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:40.976393 (+105186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.475963 (+1499570us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b54002637a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0a02 00007f1806edcdd6 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:42.476233 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4867840}
I20260709 13:15:42.398236  3009 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.486920  3255 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.488245 31367 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:42.488618  3009 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.444106  3096 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.391115 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.492411  3198 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:42.493333  3198 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.493803  3198 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:42.494196  3157 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.773s	user 0.010s	sys 0.003s
I20260709 13:15:42.496130  3204 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.460656  3191 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.443259 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.497215  3191 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.497761 30553 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:15:42.490293 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 99324.5us
I20260709 13:15:42.499501 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:42.473901 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76969.2us
I20260709 13:15:42.440045  3027 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.444746  3276 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.475132 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:42.502960 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.503552 30630 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:42.445150  3271 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.480692 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:42.435622  3210 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.453135  2714 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.513345 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.437407  3187 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.472034  3282 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:42.468361  3241 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:15:42.482849  3286 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:42.482995 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.484407  3252 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.475397  2697 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:42.485913 30628 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:42.496181  3198 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.414683  3253 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.502717 30705 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:42.500993 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.505837 30700 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:42.549086 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.506109 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.506851 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.552134 30707 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:42.517172 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.440086  3095 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.507411  3096 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:42.533442 30629 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:42.484818  3245 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:42.539600 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 212569us
I20260709 13:15:42.544144 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.559352 30624 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:42.559904 30699 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:42.559414 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58494.5us
I20260709 13:15:42.572769 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.572890 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.565188 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.574738 30704 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:42.576061  3281 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.576362  3271 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.576792  3281 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:42.566886 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:42.536278  3241 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:42.473059  3278 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.540189  3204 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:42.532948  3286 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:42.583396 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.533002  3231 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:15:42.595914 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.597854 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.618851 30699 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:42.631274 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 208457us
I20260709 13:15:42.592098 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.620956 30707 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:42.510377  3157 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.626226 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:42.641764 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.642733 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:42.564873  3198 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:42.587630  3244 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:42.624507 30704 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:42.569340  3096 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.643654 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.616343  3271 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.588836  3241 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:42.646433 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.647505 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:42.648835 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:42.648893  2531 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:15:42.669232 31361 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:42.668455 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 97483.8us
I20260709 13:15:42.669023 30556 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:42.617710  3193 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:15:42.635520 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.675417 30707 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:42.676982  3246 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.652850 30553 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:42.597991  3245 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:42.654583 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.608196  3204 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.659793 30627 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:42.659740 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.652065  3255 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:42.684285 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.660146 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.631233  3286 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:42.667023 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.683214 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:42.670274 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:42.669185 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.670769 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.671221  3083 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:42.671640  3012 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.723738 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.672024 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.672251  3027 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.652905  3198 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.661721  3096 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:42.692314 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.700778 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.740278 30625 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:42.741945 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.704350 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.743110 30554 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:42.705731 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.744542 30627 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:42.688175 30707 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:42.747218 30625 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:42.748988 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.713541 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.694972  3230 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:42.685626  3204 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:42.715649  3254 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:42.716238 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 922) took 1008 ms. Trace:
W20260709 13:15:42.718921  3301 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
I20260709 13:15:42.720683  3241 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.657722  3296 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:15:42.724716 30704 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
W20260709 13:15:42.725726 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55035.2us
I20260709 13:15:42.732972  3205 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.735922 30630 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:42.707996 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.790410 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1627) took 1592 ms. Trace:
I20260709 13:15:42.712882 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.669955  3271 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.758718 30632 rpcz_store.cc:276] 0709 13:15:41.708846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.815968 (+107122us) service_pool.cc:224] Handling call
0709 13:15:42.715972 (+900004us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400052154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:42.716157 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270326144}
I20260709 13:15:42.754413 30705 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:42.761802 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.700448  3278 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.779121  3273 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.784353 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.804670 30631 rpcz_store.cc:276] 0709 13:15:41.197748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.197954 (+   206us) service_pool.cc:224] Handling call
0709 13:15:41.198404 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:41.313018 (+114614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.789823 (+1476805us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54004ab5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:42.790280 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261716224}
W20260709 13:15:42.757735  3292 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.002s
I20260709 13:15:42.782344  3282 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.725224  3067 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.821895  3067 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:42.803107  3205 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.824796 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1518) took 1003 ms. Trace:
W20260709 13:15:42.804021 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96075.4us
I20260709 13:15:42.760558  3230 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.805685 30707 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:42.774259  3157 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.804968 31394 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:42.796074 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.832597 30704 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:42.725225  3190 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.807678 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.808118 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:42.833777 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:42.834008 30630 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:42.723227  3253 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.735462  3104 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:42.799796 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.838440 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.839056 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.839924 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.738981  3198 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:42.800577  3245 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.735215  3083 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:42.806363  3148 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:42.819357 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.827788 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:42.800237 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.819245 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:42.670082  2697 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:42.825366 31396 rpcz_store.cc:276] 0709 13:15:41.821220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.101121 (+279901us) service_pool.cc:224] Handling call
0709 13:15:42.112247 (+ 11126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.319044 (+206797us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.319080 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:42.327755 (+  8675us) log.cc:844] Serialized 26 byte log entry
0709 13:15:42.526629 (+198874us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:42.625414 (+ 98785us) log.cc:844] Serialized 14 byte log entry
0709 13:15:42.637931 (+ 12517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.638070 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:42.715259 (+ 77189us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b48000f3fd8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:15:42.824391 (+109132us) raft_consensus.cc:1685] finished
0709 13:15:42.824532 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:42.824711 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15528,"prepare.queue_time_us":113404,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":141140,"raft.run_cpu_time_us":3465,"raft.run_wall_time_us":530684,"spinlock_wait_cycles":17307520,"thread_start_us":460005,"threads_started":3,"wal-append.queue_time_us":311266}
I20260709 13:15:42.858258 30632 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:42.842269 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.822252  3271 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.797017  3195 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.863031  3296 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:42.861244 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.877404 31391 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:42.840844 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.844797 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:42.850117 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.834247  3241 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:42.849031 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:42.831854  3230 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:42.880191 30700 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:42.846143 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:42.885171 31394 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:42.845805  3193 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:42.902264 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:42.894761 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 302736us
I20260709 13:15:42.906353 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.846846  2531 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:42.846515  3205 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:42.846040  3294 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:42.889709  3246 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:42.899174  3195 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:42.947891 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:42.925231 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.852161  3148 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.926764 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:42.928682 30627 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:42.878270  3095 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:42.929267 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.930066 31391 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:42.959148 30700 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:42.938578 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.849130  3187 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:42.901311  3193 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.846098  3185 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:15:42.948752 30550 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:42.939045  3205 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.949529 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.969429 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:42.852901  3245 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.955461  3195 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:42.963239 30632 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:42.964583 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:42.960430 31394 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:42.960739  3148 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:42.956873 30705 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:42.976361 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.971112  3205 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:42.986696 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 789) took 1357 ms. Trace:
I20260709 13:15:42.992372 30698 rpcz_store.cc:276] 0709 13:15:41.628979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.789871 (+160892us) service_pool.cc:224] Handling call
0709 13:15:42.986377 (+1196506us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54005a32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:42.986593 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603298048}
I20260709 13:15:42.988831  3253 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:42.956063  3309 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:42.961278  3294 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:43.001384 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.036123 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.069363 30700 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:43.003360 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:43.071624 30559 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:43.004282 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1625) took 1827 ms. Trace:
I20260709 13:15:43.074111 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.073100 30633 rpcz_store.cc:276] 0709 13:15:41.177221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.178165 (+   944us) service_pool.cc:224] Handling call
0709 13:15:41.178591 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.257733 (+1079142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.004013 (+746280us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54001fbb54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:43.004215 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2128396032}
I20260709 13:15:43.008797 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1519) took 1187 ms. Trace:
I20260709 13:15:43.079124 31378 rpcz_store.cc:276] 0709 13:15:41.821561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.183582 (+362021us) service_pool.cc:224] Handling call
0709 13:15:42.184107 (+   525us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.191726 (+  7619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.008528 (+816802us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b5400717ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:43.008732 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15536512}
I20260709 13:15:43.011237 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:43.006860 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.081046 30701 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:42.996588 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.026311 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:43.083804 31394 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:42.924153  3104 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:42.999884  3187 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:42.969291  3193 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:43.025525 31294 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.668s (SENT)
I20260709 13:15:43.096449 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:42.998698 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.061528 30705 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:43.025997 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:43.098336 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72457.2us
I20260709 13:15:43.101001 30550 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:43.099356 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.102690 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1681) took 1066 ms. Trace:
W20260709 13:15:43.103821 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78345.7us
I20260709 13:15:42.990842  3195 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:43.089139 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:43.113945 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.000314  3318 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:43.017177  3311 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:43.047108 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 70841.9us
I20260709 13:15:43.127811 30701 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:43.158334 30701 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:43.147678 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.161363 31394 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:43.154331 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:43.164660 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.168042 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.109036 30700 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:43.024574  3066 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.103109 30550 rpcz_store.cc:276] 0709 13:15:42.036545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.101787 (+1065242us) service_pool.cc:224] Handling call
0709 13:15:43.102157 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.102351 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.102377 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.102421 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:43.102457 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.102510 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.102606 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.173791 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.114521 30559 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:43.114907 30633 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:43.093277 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.177809 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:43.103593 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:43.179550 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76032us
I20260709 13:15:43.094396  3294 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.102898  3252 raft_consensus.cc:2764] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 13:15:43.118631  3312 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:15:43.119938  3315 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:15:43.122292  3253 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.018812  3210 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:43.057117  2973 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:43.002429  3184 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:43.101125  3187 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:42.972654  3083 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.193670  3099 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:43.068868  3309 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.184458 30705 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:15:43.113837  3296 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:15:43.123759  3208 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:43.192541 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:43.192987 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.151978  3190 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:43.152277  3318 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.195881 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:43.197793 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.197824 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:43.148727  3238 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.200565 30550 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:43.224035 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 881) took 1182 ms. Trace:
I20260709 13:15:43.122545  3096 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.123678  3213 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:15:43.175383  3311 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:43.183630  3185 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:43.248494  3185 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.173039  3244 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:43.213467  3101 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:43.251142  3101 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:43.016896  3204 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:43.188225  3196 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:43.254210  3196 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.206286  2973 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.254736  3196 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:43.224764  3318 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:43.256562  2973 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:43.259056 31388 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1504) took 2254 ms (client timeout 1658 ms). Trace:
W20260709 13:15:43.259862 31388 rpcz_store.cc:269] 0709 13:15:40.992389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.992528 (+   139us) service_pool.cc:224] Handling call
0709 13:15:43.246831 (+2254303us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.247083 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530359552}
I20260709 13:15:43.261816 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:43.209394 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 364702us
I20260709 13:15:43.196355  3224 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:43.262948  3224 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.263391  3224 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:43.264469 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.265444  3316 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:43.194803  3066 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:43.266033  3316 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.268697  3316 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.268777  3066 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:43.269431  3316 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:43.153879  3286 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.195472  3113 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:43.272390 31389 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.273051 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50024 (request call id 1501) took 2281 ms. Trace:
I20260709 13:15:43.272908  3113 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.273468 31389 rpcz_store.cc:276] 0709 13:15:40.991431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.993581 (+  2150us) service_pool.cc:224] Handling call
0709 13:15:42.420446 (+1426865us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b54004b05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed4a3d 00007f180bea4cd0 00007f1805429254 00007f18054291e2 00007f1805429111 00007f180542908c 00007f1805428342 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.271783 (+851337us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b4c003db700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18051a9a20 00007f18051baa29 00007f18051bb13f 00007f180517f2ab 00007f18051acf3c 00007f18051ad3fe 00007f180543a88b 00007f1806e39495 00007f1806e8a30f 00007f1806ed4a55 00007f180bea4cd0
0709 13:15:43.272949 (+  1166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199622784}
I20260709 13:15:43.274004  3309 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.274458 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:43.275264  3113 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:43.269836  3297 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.497s	user 0.003s	sys 0.000s
I20260709 13:15:43.276479 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1626) took 2099 ms. Trace:
I20260709 13:15:43.276764 30629 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:43.224519 30550 rpcz_store.cc:276] 0709 13:15:42.041833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.175627 (+1133794us) service_pool.cc:224] Handling call
0709 13:15:43.200197 (+ 24570us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.223683 (+ 23486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.223967 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.277204  3316 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:43.278025 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1686) took 1422 ms. Trace:
I20260709 13:15:43.135377  3104 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.215478  3099 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.278281  3316 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.278561 31364 rpcz_store.cc:276] 0709 13:15:41.855786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.209956 (+354170us) service_pool.cc:224] Handling call
0709 13:15:42.210369 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:42.210826 (+   457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.210864 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:42.240044 (+ 29180us) log.cc:844] Serialized 26 byte log entry
0709 13:15:42.579537 (+339493us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:42.579589 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.579668 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.269133 (+689465us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:43.277691 (+  8558us) raft_consensus.cc:1685] finished
0709 13:15:43.277716 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.277960 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31817,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":15069,"spinlock_wait_cycles":70827264,"thread_start_us":232777,"threads_started":1,"wal-append.queue_time_us":413176}
I20260709 13:15:43.281225 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:43.283953 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1677) took 1265 ms. Trace:
I20260709 13:15:43.284446 30551 rpcz_store.cc:276] 0709 13:15:42.018440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.171105 (+152665us) service_pool.cc:224] Handling call
0709 13:15:42.179010 (+  7905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.179320 (+   310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.179360 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.179455 (+    95us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:42.179505 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.179587 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.283727 (+1104140us) spinlock_profiling.cc:243] Waited 7.38 ms on lock 0x7b54000bf9d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.283881 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16975616}
I20260709 13:15:43.288703  2856 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:43.247620 31383 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:43.277004 30626 rpcz_store.cc:276] 0709 13:15:41.177531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.178839 (+  1308us) service_pool.cc:224] Handling call
0709 13:15:41.179149 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.460322 (+1281173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.276151 (+815829us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54004a88d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:43.276401 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2795917056}
I20260709 13:15:43.291427 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 871) took 1283 ms. Trace:
I20260709 13:15:43.291940 30557 rpcz_store.cc:276] 0709 13:15:42.007870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.008122 (+   252us) service_pool.cc:224] Handling call
0709 13:15:43.291154 (+1283032us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540028a5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.291356 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":335367040}
I20260709 13:15:43.293210 30626 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:43.293831 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1684) took 1231 ms. Trace:
I20260709 13:15:43.245424  3190 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:43.295967 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1682) took 1249 ms. Trace:
I20260709 13:15:43.279201 31394 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:43.298094 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.299100 31389 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:43.271422  3187 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.213608 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.217077  3208 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.277968 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:43.200212  3253 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.269829 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.227870 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.300184 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:43.301040 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1683) took 1227 ms. Trace:
I20260709 13:15:43.245537  3311 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:43.272303  3101 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:43.276427  3096 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.324473 30703 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [13.9-15.10]
I20260709 13:15:43.324983  3245 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.324751  3196 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:43.326159  3205 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.327095  3269 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:43.327566  3269 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:43.327718 31394 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:43.328713  3205 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.328620  3269 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:43.329831  3269 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:43.309890  3198 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.331780  3230 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.332392 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.333400 31394 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:15:43.334054  3230 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.334415 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:43.296586 30553 rpcz_store.cc:276] 0709 13:15:42.046672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.282545 (+1235873us) service_pool.cc:224] Handling call
0709 13:15:43.295250 (+ 12705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.295512 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.295551 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.295633 (+    82us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:43.295690 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.295763 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.295892 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.336037 30626 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:15:43.336388 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:43.302312 31396 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:43.204286  3184 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:43.304877 30550 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:15:43.289294  2856 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:43.294306 30557 rpcz_store.cc:276] 0709 13:15:42.062513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.292937 (+1230424us) service_pool.cc:224] Handling call
0709 13:15:43.293281 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.293481 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.293508 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.293558 (+    50us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:43.293594 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.293663 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.293770 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.246207  3285 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:43.308260 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 80497.2us
I20260709 13:15:43.306679  3208 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:43.310148 30551 rpcz_store.cc:276] 0709 13:15:42.059429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.285660 (+1226231us) service_pool.cc:224] Handling call
0709 13:15:43.286036 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.286267 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.286296 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.286348 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:43.286386 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.286445 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.286563 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.356067 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.282244 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:43.317975 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48251.8us
I20260709 13:15:43.318957  3311 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:43.303306  3099 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:43.274181  3286 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.338359 31361 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:43.377609 30551 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:43.337131  3298 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:43.382999 31389 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:43.383826 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1689) took 1046 ms. Trace:
I20260709 13:15:43.371517 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:43.336629  3187 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:43.333411  3198 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.363615 31364 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:15:43.376807 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.326395 30703 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:15:43.278244 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1181) took 1060 ms. Trace:
I20260709 13:15:43.384137 31389 rpcz_store.cc:276] 0709 13:15:42.337530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.355337 (+1017807us) service_pool.cc:224] Handling call
0709 13:15:43.382786 (+ 27449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.383519 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.383749 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.386255 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1174) took 1173 ms. Trace:
W20260709 13:15:43.392901 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.339923  3190 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:15:43.374034 30558 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 867) took 1657 ms (client timeout 1668 ms). Trace:
I20260709 13:15:43.339365  3252 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.340003  3096 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:43.392611 30629 rpcz_store.cc:276] 0709 13:15:42.217734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.794522 (+576788us) service_pool.cc:224] Handling call
0709 13:15:42.810722 (+ 16200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.277299 (+466577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.278027 (+   728us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54004ab0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:43.278188 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1071241088}
I20260709 13:15:43.393291 30625 rpcz_store.cc:276] 0709 13:15:42.212735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.774956 (+562221us) service_pool.cc:224] Handling call
0709 13:15:42.775475 (+   519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.775810 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:42.925886 (+150076us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.385949 (+460063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.386015 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:43.386053 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.386104 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.386203 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58853,"prepare.queue_time_us":157625,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":634,"thread_start_us":533747,"threads_started":2}
I20260709 13:15:43.397295 30557 pending_rounds.cc:85] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 4
W20260709 13:15:43.379161  3303 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.659s	user 0.000s	sys 0.003s
I20260709 13:15:43.404846 30557 pending_rounds.cc:107] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 13:15:43.405210 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:43.407713 30557 raft_consensus.cc:2889] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:43.398350 30558 rpcz_store.cc:269] 0709 13:15:41.716745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.756610 (+ 39865us) service_pool.cc:224] Handling call
0709 13:15:43.356863 (+1600253us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.373932 (+ 17069us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1600258816}
I20260709 13:15:43.409411 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:43.397099 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.424670 30550 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:43.382311  2864 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:43.383236  3298 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.448629 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.444505 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.445866 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.408027  2856 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.453787  2856 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:43.446940  2864 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:43.457803  2864 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:43.460479 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:43.406543  3078 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:43.463337  3313 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.387s	user 0.004s	sys 0.000s
I20260709 13:15:43.376047  3213 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:15:43.469754 30629 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:43.378554  3210 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:15:43.378078  3275 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:15:43.404071  3096 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:43.404906  3187 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:43.442678  3192 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:43.484648 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.467661  3298 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:43.402666  3082 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:15:43.408694  3252 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.475898  3245 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:43.494572 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.499969 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.504052 30705 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:43.506683 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.483552  3096 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:43.508522  2697 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:43.527024 30629 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:43.538254 30629 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:43.517467  2697 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:43.554499 30559 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:43.511890  3321 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:15:43.539981  3198 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:43.575528  3253 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.577134 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1559) took 1070 ms. Trace:
I20260709 13:15:43.571506 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:43.588049 30705 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:43.591849 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:43.659245 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67532.9us
I20260709 13:15:43.584116  3078 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:43.657886 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:43.657619 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.815s (SENT) [suppressed 16 similar messages]
I20260709 13:15:43.603770  3238 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:15:43.676887 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1182) took 1216 ms. Trace:
I20260709 13:15:43.678720 30628 rpcz_store.cc:276] 0709 13:15:42.460056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.872734 (+412678us) service_pool.cc:224] Handling call
0709 13:15:42.873173 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.873508 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:43.222957 (+349449us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.676425 (+453468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.676510 (+    85us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:43.676569 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.676653 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.676819 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7888,"prepare.queue_time_us":96871,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"thread_start_us":791376,"threads_started":2}
I20260709 13:15:43.582717  3198 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.685500 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.688889 30706 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:43.671352  3078 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:43.705710  3191 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.708032  3096 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.709332  3196 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:43.732863 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 335841us
I20260709 13:15:43.691280  3157 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:43.690918 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:43.699687  3078 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:43.681401  3318 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.712819  3253 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:43.757002 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:43.771097  3252 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.687505  2697 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.794445 30625 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
W20260709 13:15:43.774176  3195 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
W20260709 13:15:43.777079  3224 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.006s
I20260709 13:15:43.758311  3157 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:43.836705 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1678) took 1817 ms. Trace:
W20260709 13:15:43.780736  3148 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.756s	user 0.021s	sys 0.000s
I20260709 13:15:43.839227 30552 rpcz_store.cc:276] 0709 13:15:42.019439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.178566 (+159127us) service_pool.cc:224] Handling call
0709 13:15:42.800664 (+622098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.800871 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:42.800907 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:42.800950 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:42.800983 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:42.801050 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.836401 (+1035351us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54001aa250. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.836615 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426271488}
I20260709 13:15:43.775753  3198 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.852824 31388 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:43.853892 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1696) took 1384 ms. Trace:
I20260709 13:15:43.891994 31388 rpcz_store.cc:276] 0709 13:15:42.469793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.852196 (+1382403us) service_pool.cc:224] Handling call
0709 13:15:43.852598 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.853492 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.853810 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.788558  3253 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:43.897228  3253 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:43.899956 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 797) took 1333 ms. Trace:
W20260709 13:15:43.847679 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 1.34545e+06us
I20260709 13:15:43.903708 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.844460 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1558) took 1337 ms. Trace:
I20260709 13:15:43.790377  3275 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:43.881760  3318 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:43.915903 31388 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:43.904920 30704 rpcz_store.cc:276] 0709 13:15:42.506806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.844173 (+337367us) service_pool.cc:224] Handling call
0709 13:15:42.986574 (+142401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.986929 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:42.987325 (+   396us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.209957 (+222632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.210064 (+   107us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:43.210139 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.210247 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.844170 (+633923us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54005a32d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:43.844369 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2903,"prepare.queue_time_us":789,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":162,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6598,"spinlock_wait_cycles":328380672,"thread_start_us":222035,"threads_started":1,"wal-append.queue_time_us":488717}
I20260709 13:15:43.910264 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.900511 30701 rpcz_store.cc:276] 0709 13:15:42.566556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.159382 (+592826us) service_pool.cc:224] Handling call
0709 13:15:43.159750 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.161861 (+  2111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.161896 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.169820 (+  7924us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.660807 (+490987us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:43.660867 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.660948 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:43.899695 (+238747us) raft_consensus.cc:1685] finished
0709 13:15:43.899719 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.899884 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1713,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9666,"thread_start_us":451926,"threads_started":1,"wal-append.queue_time_us":643274}
I20260709 13:15:43.917665 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.918160 30706 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:43.919713 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1699) took 1386 ms. Trace:
I20260709 13:15:43.920460 31388 rpcz_store.cc:276] 0709 13:15:42.533228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.908260 (+1375032us) service_pool.cc:224] Handling call
0709 13:15:43.908742 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.916663 (+  7921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.919623 (+  2960us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:43.920042 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:43.774659  3096 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:43.928900 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.929747 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.929868 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:43.934167 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:43.936175 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.941963 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:43.943679 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.836302  3195 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:43.839435  3224 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.874248  3298 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:43.985414 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.853243  3148 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:43.932461  3198 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:43.928714  3252 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:15:43.948520 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:44.032652 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90792.6us
I20260709 13:15:44.034556 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:44.033468 30701 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:43.934501  3316 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:43.947490  3096 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:44.086758 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.078428 30700 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:43.934365  3321 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:44.091096 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.091619 31378 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:44.093703 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:43.929137  3275 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:44.094660 30701 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:43.944834  3285 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:43.939152  3191 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 13:15:44.088934 31394 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:44.031121  2864 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.573s	user 0.016s	sys 0.000s
I20260709 13:15:44.098757 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.100313 30701 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:44.017504  3252 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:44.031733  3190 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:44.103978 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:43.991060  3245 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:44.052341  3198 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:44.107491 30706 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:44.104070 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:44.110945 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 453338us
I20260709 13:15:44.052213  3324 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:44.097658  3311 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.712s	user 0.005s	sys 0.008s
I20260709 13:15:44.086444  3196 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:44.092033  3083 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:44.101316  2864 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.104975  3252 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:44.116720  3099 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.005s
I20260709 13:15:44.116015  3316 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:44.122192  3193 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.990s	user 0.015s	sys 0.001s
I20260709 13:15:44.140347 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.144034 30701 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:44.115123  3321 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:43.990962  3157 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.137959  3341 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:15:44.149371  2864 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.101008  3285 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.150797  3192 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:44.157254 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 263372us
I20260709 13:15:44.158375 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:44.167164 30701 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:44.122800  3078 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.142334  3083 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:44.176229 30630 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.169579  3192 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:44.164348  3330 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.294s	user 0.002s	sys 0.000s
I20260709 13:15:44.138504  3311 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.191553 30705 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:44.138257  3324 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.141283  3196 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:44.196885 31364 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.146922  3099 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.152611  3193 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:44.185526  3208 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.005s
I20260709 13:15:44.183547  3191 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.154165  3321 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:44.196834  3269 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.867s	user 0.012s	sys 0.006s
I20260709 13:15:44.185006  3341 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.198480  3195 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.233150 30633 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.236395 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 921) took 2532 ms. Trace:
I20260709 13:15:44.237067 30624 rpcz_store.cc:276] 0709 13:15:41.705813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.944795 (+238982us) service_pool.cc:224] Handling call
0709 13:15:41.994994 (+ 50199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.571474 (+576480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.235970 (+1664496us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b54009434d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:44.236307 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284633472}
I20260709 13:15:44.259891 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1688) took 1922 ms. Trace:
I20260709 13:15:44.260553 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1694) took 1890 ms. Trace:
I20260709 13:15:44.198300  3245 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.260725 31396 rpcz_store.cc:276] 0709 13:15:42.337191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.341695 (+1004504us) service_pool.cc:224] Handling call
0709 13:15:43.385203 (+ 43508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.385668 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.385710 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.400388 (+ 14678us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.911351 (+510963us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:44.258938 (+347587us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.259358 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.259505 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.259579 (+    74us) raft_consensus.cc:1685] finished
0709 13:15:44.259598 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.259799 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2639,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":756031,"threads_started":2,"wal-append.queue_time_us":702593}
I20260709 13:15:44.265232 31389 rpcz_store.cc:276] 0709 13:15:42.370526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.393301 (+1022775us) service_pool.cc:224] Handling call
0709 13:15:43.420358 (+ 27057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.421036 (+   678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.421085 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.496022 (+ 74937us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.991894 (+495872us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:44.258360 (+266466us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.260149 (+  1789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.260251 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.260296 (+    45us) raft_consensus.cc:1685] finished
0709 13:15:44.260313 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.260485 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150,"raft.queue_time_us":7609,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":25629,"thread_start_us":620723,"threads_started":2,"wal-append.queue_time_us":616910}
I20260709 13:15:44.216259  3208 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.186554  3190 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:44.248838  3148 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.184077  3198 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:44.254431  3298 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.256179  3224 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:44.281123 30701 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:44.282013 30706 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
W20260709 13:15:44.283267 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 248856us
I20260709 13:15:44.223788  3341 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:44.220981  3191 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.292996 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:44.283440  3328 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:15:44.298139 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.299010 30628 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.286003  3275 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.303738 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:44.294886  3334 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
W20260709 13:15:44.311228  3337 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:15:44.313921  3067 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:44.260793  3083 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.319808  3311 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.322695  3285 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:44.331015  3322 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.620s	user 0.004s	sys 0.000s
I20260709 13:15:44.332890  3324 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.334834  3224 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:44.365931  3224 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:44.366438  3224 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:44.367188 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1548) took 3597 ms. Trace:
I20260709 13:15:44.367643 30702 rpcz_store.cc:276] 0709 13:15:40.769633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:40.769820 (+   187us) service_pool.cc:224] Handling call
0709 13:15:40.770213 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:40.770674 (+   461us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:40.770707 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:41.617180 (+846473us) spinlock_profiling.cc:243] Waited 8.12 ms on lock 0x7b18002fbb40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:15:43.017025 (+1399845us) spinlock_profiling.cc:243] Waited 7.6 ms on lock 0x7b2c00285df8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:43.017634 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 13:15:43.395385 (+377751us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:43.395439 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.420663 (+ 25224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.330298 (+909635us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:44.366895 (+ 36597us) raft_consensus.cc:1685] finished
0709 13:15:44.366923 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.367095 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6949,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":18644,"spinlock_wait_cycles":63017984,"thread_start_us":360935,"threads_started":1,"wal-append.queue_time_us":647239}
I20260709 13:15:44.369731 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:44.336688 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:44.371012 30629 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:44.372786 30702 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:15:44.338886 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1570) took 2712 ms. Trace:
I20260709 13:15:44.374796 31383 rpcz_store.cc:276] 0709 13:15:41.626647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.897868 (+271221us) service_pool.cc:224] Handling call
0709 13:15:43.247274 (+1349406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.289861 (+ 42587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.338532 (+1048671us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400521cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:44.338756 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3102393216}
I20260709 13:15:44.376325 30702 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.376765 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1522) took 1355 ms. Trace:
I20260709 13:15:44.377172 31383 rpcz_store.cc:276] 0709 13:15:43.021570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.375956 (+1354386us) service_pool.cc:224] Handling call
0709 13:15:44.376249 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.376442 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.376468 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.376511 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:44.376542 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.376599 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.376706 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.378104 30702 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:44.339222  3078 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.340591  3196 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:44.301023  3345 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
W20260709 13:15:44.305485  3333 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.158s	user 0.001s	sys 0.001s
I20260709 13:15:44.346884  3344 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:44.381108  3344 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:44.382046 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2239) took 1954 ms. Trace:
I20260709 13:15:44.382550 30557 rpcz_store.cc:276] 0709 13:15:42.427559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.372735 (+945176us) service_pool.cc:224] Handling call
0709 13:15:43.397041 (+ 24306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:43.408593 (+ 11552us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.408641 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:43.409328 (+   687us) log.cc:844] Serialized 52 byte log entry
0709 13:15:43.790854 (+381526us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:43.790906 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.791003 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.381831 (+590828us) raft_consensus.cc:1685] finished
0709 13:15:44.381854 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.381987 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4062,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":19956,"thread_start_us":381119,"threads_started":1,"wal-append.queue_time_us":706306}
I20260709 13:15:44.383627  3344 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:44.347146  3193 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:44.348395 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:44.350915 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 935) took 1437 ms. Trace:
I20260709 13:15:44.384608 30626 rpcz_store.cc:276] 0709 13:15:42.913838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.294381 (+380543us) service_pool.cc:224] Handling call
0709 13:15:43.301410 (+  7029us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.336481 (+ 35071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.350569 (+1014088us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b540063ced4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:44.350839 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78985088}
I20260709 13:15:44.385313  3192 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:44.385842 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67473.5us
I20260709 13:15:44.386978 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:44.358220 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:44.359160 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1637) took 2085 ms. Trace:
I20260709 13:15:44.392760 31389 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:44.334841  3341 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:44.314357  3191 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.340819  3208 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.391057 30631 rpcz_store.cc:276] 0709 13:15:42.273297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.815279 (+541982us) service_pool.cc:224] Handling call
0709 13:15:42.815925 (+   646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.071697 (+255772us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:43.094820 (+ 23123us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.480685 (+385865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.480774 (+    89us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:43.480856 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.358863 (+878007us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54004abfd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:44.358921 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.359098 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24288,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":196,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":167,"spinlock_wait_cycles":607524992,"thread_start_us":384836,"threads_started":1,"wal-append.queue_time_us":585872}
I20260709 13:15:44.362885 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.385924 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:44.352154  3317 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.327s	user 0.008s	sys 0.000s
I20260709 13:15:44.352629  3321 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.389189 30633 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.407063 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:44.411458 30626 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:44.411931 30702 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:15:44.413547 30702 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:44.415987 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:44.399343 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:44.426831 30633 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:44.402724 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.428035 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:44.396574  3196 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.429337 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:44.405588 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.393539 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1960) took 1554 ms. Trace:
I20260709 13:15:44.435783 30701 rpcz_store.cc:276] 0709 13:15:42.837333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.300660 (+1463327us) service_pool.cc:224] Handling call
0709 13:15:44.391170 (+ 90510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.391450 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.391491 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.391556 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:44.391606 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.391689 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.391838 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.404585  3344 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.441823 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:44.405673  3285 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:44.456818 30633 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:44.448575 30626 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:44.437045  3311 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.440747  3148 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:44.469421 30633 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:15:44.447391  3272 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:44.442468  3298 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:44.475651 30624 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:44.485756 30633 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:44.486080 30629 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:44.440528  3013 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:15:44.442807  3196 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:44.498147 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1525) took 1475 ms. Trace:
I20260709 13:15:44.499645 30624 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.491190  3285 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:44.500679 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:44.502465 31383 rpcz_store.cc:276] 0709 13:15:43.022272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.378117 (+1355845us) service_pool.cc:224] Handling call
0709 13:15:44.378418 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.396092 (+ 17674us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:44.415216 (+ 19124us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.497730 (+ 82514us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.497799 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:44.497853 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.497929 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.498076 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18879,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"spinlock_wait_cycles":2426880,"thread_start_us":80912,"threads_started":2,"wal-append.queue_time_us":79802}
I20260709 13:15:44.449260  3344 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:44.508249 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66502.1us
I20260709 13:15:44.515583  3157 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:44.526712 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1526) took 1485 ms. Trace:
I20260709 13:15:44.527439 31389 rpcz_store.cc:276] 0709 13:15:43.022498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.507336 (+1484838us) service_pool.cc:224] Handling call
0709 13:15:44.507724 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.507948 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.507981 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.508028 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:44.508073 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.508134 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.508332 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.485709  3311 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.531860  3311 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.538195 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1530) took 1426 ms. Trace:
I20260709 13:15:44.539078 31389 rpcz_store.cc:276] 0709 13:15:43.112373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.528637 (+1416264us) service_pool.cc:224] Handling call
0709 13:15:44.537507 (+  8870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.537813 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.537842 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.537887 (+    45us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:44.537924 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.537982 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.538110 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.505076  3285 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:44.545468  3252 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:44.546983 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60125.3us
W20260709 13:15:44.561923  3353 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:15:44.563712 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1521) took 1542 ms. Trace:
W20260709 13:15:44.564378  3360 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:44.565281  3344 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:44.566242  3359 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:15:44.564388 31388 rpcz_store.cc:276] 0709 13:15:43.021269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.929909 (+908640us) service_pool.cc:224] Handling call
0709 13:15:43.930352 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:43.946217 (+ 15865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.946263 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:43.985375 (+ 39112us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.282558 (+297183us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:44.282605 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.282713 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.563297 (+280584us) raft_consensus.cc:1685] finished
0709 13:15:44.563338 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.563613 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7275,"thread_start_us":184008,"threads_started":1,"wal-append.queue_time_us":382636}
W20260709 13:15:44.567924  3361 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:44.568223 31388 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:44.568930 31388 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:44.568441  3252 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.570130 31388 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.576953 31388 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:44.557337  3285 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:44.607008  3358 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:15:44.608198  3364 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:15:44.608311  3281 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.611346  3185 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.616456 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1690) took 2247 ms. Trace:
I20260709 13:15:44.659169 31364 rpcz_store.cc:276] 0709 13:15:42.369063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.391183 (+1022120us) service_pool.cc:224] Handling call
0709 13:15:43.391598 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.199484 (+807886us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.616153 (+416669us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b54000113d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:44.616384 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1851280256}
I20260709 13:15:44.661540 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1532) took 1497 ms. Trace:
W20260709 13:15:44.617516  3356 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:15:44.622905  3365 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:44.627791 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.644124 31388 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:15:44.596870  3192 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:44.674340 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1680) took 2650 ms. Trace:
I20260709 13:15:44.675104 30554 rpcz_store.cc:276] 0709 13:15:42.024355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.100329 (+1075974us) service_pool.cc:224] Handling call
0709 13:15:43.297798 (+197469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.298009 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:43.298034 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.298212 (+   178us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:43.298248 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.298315 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.673997 (+1375682us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540028a5d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:44.674240 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401624832}
I20260709 13:15:44.648360  3196 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:44.623097  3253 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:15:43.581023 30698 rpcz_store.cc:276] 0709 13:15:42.507110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.075267 (+568157us) service_pool.cc:224] Handling call
0709 13:15:43.075652 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.075894 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:15:43.076251 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.576707 (+500456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.576779 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:43.576838 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:43.576923 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:43.577065 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12802,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":164151,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":113,"thread_start_us":327546,"threads_started":1,"wal-append.queue_time_us":499963}
I20260709 13:15:44.688660 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:44.689867 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.690960 30629 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:44.654760 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:44.540655  3157 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.662072 31364 rpcz_store.cc:276] 0709 13:15:43.164011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.660280 (+1496269us) service_pool.cc:224] Handling call
0709 13:15:44.660786 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.661071 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.661112 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.661172 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:44.661220 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.661293 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.661460 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.666893 31388 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:44.698777 31388 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:15:44.699249 31388 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:44.700361 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1705) took 1551 ms. Trace:
I20260709 13:15:44.700804 31388 rpcz_store.cc:276] 0709 13:15:43.149420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.567509 (+1418089us) service_pool.cc:224] Handling call
0709 13:15:44.568036 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.699929 (+131893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.700271 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.646672  3078 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:44.667647 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1531) took 1552 ms. Trace:
I20260709 13:15:44.705370 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:44.708248 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.708797 30554 raft_consensus.cc:2368] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:44.707674 31389 rpcz_store.cc:276] 0709 13:15:43.115840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.540480 (+1424640us) service_pool.cc:224] Handling call
0709 13:15:44.542930 (+  2450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.543610 (+   680us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:15:44.544627 (+  1017us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.667077 (+122450us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.667168 (+    91us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:44.667246 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.667371 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.667556 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":245,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":23348,"thread_start_us":121902,"threads_started":1}
I20260709 13:15:44.710536 31364 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.711241 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1535) took 1355 ms. Trace:
I20260709 13:15:44.711689 30702 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:44.711817 31389 rpcz_store.cc:276] 0709 13:15:43.355375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.710070 (+1354695us) service_pool.cc:224] Handling call
0709 13:15:44.710493 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.710777 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.710815 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.710874 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:44.710935 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.711014 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.711165 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.717100 30624 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:44.718966 30701 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:44.725664 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.726804 30629 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.666599  3190 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:44.728048 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1533) took 1555 ms. Trace:
I20260709 13:15:44.685343  3298 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:44.732054 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:44.731210 31364 rpcz_store.cc:276] 0709 13:15:43.173042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.698846 (+1525804us) service_pool.cc:224] Handling call
0709 13:15:44.699358 (+   512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.711240 (+ 11882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.727950 (+ 16710us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:44.733208  3224 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.686348  3365 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.734033  3190 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:44.751783 30629 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.737607 30633 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:15:44.742084 30624 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
W20260709 13:15:44.743988 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.753892 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:44.756376 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:44.757308 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.730391  3148 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:44.763608 30523 connection.cc:256] RPC call timeout handler was delayed by -1.6212757969999529s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 13:15:44.731415  3196 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:44.730680  3272 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:44.760229  3325 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.090s	user 0.000s	sys 0.015s
I20260709 13:15:44.723534  3078 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:44.755182  3345 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:44.781134 30556 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1686) took 2691 ms (client timeout 2398 ms). Trace:
W20260709 13:15:44.781863 30556 rpcz_store.cc:269] 0709 13:15:42.084133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.108496 (+ 24363us) service_pool.cc:224] Handling call
0709 13:15:44.775082 (+2666586us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:44.775229 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072245504}
I20260709 13:15:44.769687  3196 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:44.821172 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1958) took 2020 ms. Trace:
I20260709 13:15:44.821751 30706 rpcz_store.cc:276] 0709 13:15:42.800833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.292214 (+1491381us) service_pool.cc:224] Handling call
0709 13:15:44.295445 (+  3231us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:15:44.296354 (+   909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.296386 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:15:44.392584 (+ 96198us) log.cc:844] Serialized 104 byte log entry
0709 13:15:44.489871 (+ 97287us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:15:44.489938 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.490056 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:44.820779 (+330723us) raft_consensus.cc:1685] finished
0709 13:15:44.820813 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:44.821077 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":103,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":3502,"thread_start_us":47692,"threads_started":1,"wal-append.queue_time_us":121156}
I20260709 13:15:44.786329  3294 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:44.826800 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 299156us
W20260709 13:15:44.842864 31361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1578) took 2324 ms (client timeout 1765 ms). Trace:
W20260709 13:15:44.843593 31361 rpcz_store.cc:269] 0709 13:15:42.518891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.671694 (+152803us) service_pool.cc:224] Handling call
0709 13:15:44.836577 (+2164883us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b540002e754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:44.842740 (+  6163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406538624}
I20260709 13:15:44.807900  3345 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.829492  3350 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:44.854801 31391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1577) took 2336 ms (client timeout 2205 ms). Trace:
I20260709 13:15:44.855502 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:44.859521 30523 connection.cc:256] RPC call timeout handler was delayed by -1.3494757969999682s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 13:15:44.856283 31391 rpcz_store.cc:269] 0709 13:15:42.518511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.996335 (+477824us) service_pool.cc:224] Handling call
0709 13:15:44.854478 (+1858143us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b540057b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:44.854687 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439877376}
W20260709 13:15:44.860283 30523 connection.cc:256] RPC call timeout handler was delayed by -1.3193618010000137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:15:44.860764 30523 connection.cc:256] RPC call timeout handler was delayed by -1.3193618010000137s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:15:44.861337 30523 connection.cc:256] RPC call timeout handler was delayed by -1.1304286050001338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1501s for a response
W20260709 13:15:44.862599 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:44.863400 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.863729 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.864331 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:44.866163 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.867115 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.867337 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:44.867864 30559 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:44.870299 30559 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:44.871089 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.872067 30551 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:44.874286 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.876806 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:44.877660 30551 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:15:44.905905 30556 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:44.906831 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:44.907318 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:44.909962 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:44.910246 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:44.910765 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.911471 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:44.914057 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:44.914822 31396 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:44.936585 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:44.832579  3294 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.035472 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.031153 30557 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:44.856336  3311 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:45.032138 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:44.836063  3374 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:15:45.036293 30552 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:45.036839 30551 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:45.037782 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:45.034282  3368 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:15:45.040349  3365 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.059504 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1534) took 1759 ms. Trace:
I20260709 13:15:45.059708 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1563) took 2468 ms. Trace:
I20260709 13:15:45.060833  3344 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:15:45.062441  3363 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.534s	user 0.002s	sys 0.000s
I20260709 13:15:45.065692  3241 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:45.136771 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.135773 31388 rpcz_store.cc:276] 0709 13:15:43.300246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.701951 (+1401705us) service_pool.cc:224] Handling call
0709 13:15:44.702450 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:44.709818 (+  7368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:44.709855 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:44.731744 (+ 21889us) log.cc:844] Serialized 26 byte log entry
0709 13:15:44.907082 (+175338us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:44.907162 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.907275 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:45.059255 (+151980us) raft_consensus.cc:1685] finished
0709 13:15:45.059281 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.059440 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11840,"raft.queue_time_us":48200,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":4715,"thread_start_us":144986,"threads_started":1,"wal-append.queue_time_us":300571}
I20260709 13:15:45.136437 30700 rpcz_store.cc:276] 0709 13:15:42.591427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.176450 (+585023us) service_pool.cc:224] Handling call
0709 13:15:43.177052 (+   602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.088183 (+911131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.059444 (+971261us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:45.059642 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2072180096}
I20260709 13:15:45.138576 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.139072 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.139734 31388 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.140007 30555 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.139524  3184 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.140726  3184 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:45.141072 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.067306 30703 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:45.067715 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.142803 30703 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:45.143447 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75828.4us
I20260709 13:15:45.143652 30703 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.144723 30703 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.145078 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.067498 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.068946 31389 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:45.146879  3344 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:45.147269 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79850.7us
I20260709 13:15:45.070492 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.148090  3365 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.148257 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:45.148576 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78199us
I20260709 13:15:45.148689  3365 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.150028  3013 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:45.040752 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.150398 31361 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:45.150867 31361 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:45.150914 30556 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:45.040480  3187 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.071398 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:45.152244 30627 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:45.151831 31361 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.042299 31396 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:45.153053 31361 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.153752  3241 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:45.041762 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.072636 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:45.043376  3373 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:15:45.155309 30629 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:45.149771 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:45.043771  3355 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:15:45.156680  3355 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:45.156737 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.157280  3355 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:45.042538 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 183189us
I20260709 13:15:44.846470  3376 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.129s
W20260709 13:15:45.074980  3310 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 2.120s	user 0.000s	sys 0.007s
I20260709 13:15:44.832480  3298 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.077512  3294 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:45.078341  3099 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:45.079286 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.080303  3345 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:45.043923  3375 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 13:15:45.273643  3252 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:45.276404 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 144) took 37026 ms (client timeout 15000 ms). Trace:
I20260709 13:15:45.048935  3366 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:45.279701  3366 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:44.833256  3148 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.049609 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.281759 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.050581  3078 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.047179 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:45.085412 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1562) took 2500 ms. Trace:
I20260709 13:15:45.283417 30699 rpcz_store.cc:276] 0709 13:15:42.585279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.174526 (+589247us) service_pool.cc:224] Handling call
0709 13:15:43.175069 (+   543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.251556 (+ 76487us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:15:43.251979 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 13:15:43.747734 (+495755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:43.747802 (+    68us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:43.747860 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.085125 (+1337265us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b540032a854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:45.085177 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.085336 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3324,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"raft.queue_time_us":16447,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2635,"spinlock_wait_cycles":175048960,"thread_start_us":480068,"threads_started":1,"wal-append.queue_time_us":803737}
I20260709 13:15:45.050979  3082 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:45.285300  3082 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:44.848093  3272 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.286185  3082 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:44.905720  3350 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:44.957985  3157 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:44.916989  3196 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.088748 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:45.268167  2929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.563s	user 0.008s	sys 0.009s
W20260709 13:15:45.091462  3326 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.389s	user 0.004s	sys 0.001s
I20260709 13:15:45.094591 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 925) took 2684 ms. Trace:
I20260709 13:15:45.095395  3190 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:45.097909  3224 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.058295  3243 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:45.043200 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.301537 30630 rpcz_store.cc:276] 0709 13:15:42.410673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:42.842438 (+431765us) service_pool.cc:224] Handling call
0709 13:15:42.842910 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:44.177075 (+1334165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.094168 (+917093us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54004aabd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:45.094492 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3065844096}
W20260709 13:15:45.099562 30625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 945) took 1747 ms (client timeout 2160 ms). Trace:
W20260709 13:15:45.307085 30625 rpcz_store.cc:269] 0709 13:15:43.352353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.404834 (+ 52481us) service_pool.cc:224] Handling call
0709 13:15:45.099241 (+1694407us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b54004aa1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.099468 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888301952}
I20260709 13:15:45.077605 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:45.308699 30633 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:45.052379 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.052784 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.310114 31391 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:45.101209 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1679) took 3948 ms. Trace:
I20260709 13:15:45.114229 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1640) took 2315 ms. Trace:
W20260709 13:15:45.115425 30705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 806) took 1725 ms (client timeout 1596 ms). Trace:
W20260709 13:15:45.116377 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1706) took 1966 ms (client timeout 1813 ms). Trace:
I20260709 13:15:45.117002 31383 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:45.119428 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:45.313740 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 194410us
I20260709 13:15:45.120254  3192 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.120893  3185 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.057775 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 949) took 1347 ms. Trace:
I20260709 13:15:45.041392 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.129190  3253 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:45.137681 30551 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:45.322672 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.325189 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.068450  3204 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.147612  3311 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.074272  3370 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.196290 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.158197  3355 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.191064  3371 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:45.150935  3113 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:45.147624  3344 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.198014  3269 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:45.171144  3377 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:45.190963  2928 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:45.180176  3241 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:45.183741  3238 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:45.141110 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.200788 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.193972 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1586) took 2644 ms. Trace:
I20260709 13:15:45.147785  3205 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:45.189765  3365 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:45.184497 30382 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 14, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:45.208818 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.362543 30554 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 13:15:45.211076 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:45.243218  3285 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.006s
I20260709 13:15:45.244690 30552 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:45.236953 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.243357 30556 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:45.364470 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.364845 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.365136 30552 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:45.365372 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.366037 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.367393 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.236123 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:45.268777 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 189588us
W20260709 13:15:45.269204 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76963.2us
W20260709 13:15:45.047518 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.369149 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:45.369691 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.369747 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:45.370345 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.287810 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.290030 30704 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:45.290200  3013 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:45.258476 30703 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
W20260709 13:15:45.376434 30703 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 807) took 1986 ms (client timeout 1559 ms). Trace:
I20260709 13:15:45.292761 30706 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:45.090004 30707 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:45.290759  3082 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:45.312706 31378 rpcz_store.cc:269] 0709 13:15:43.150019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.192570 (+ 42551us) service_pool.cc:224] Handling call
0709 13:15:45.116137 (+1923567us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400680454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.116295 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1997327104}
I20260709 13:15:45.311089 31367 rpcz_store.cc:276] 0709 13:15:41.153174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:41.776492 (+623318us) service_pool.cc:224] Handling call
0709 13:15:42.422728 (+646236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:42.488714 (+ 65986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:44.425832 (+1937118us) spinlock_profiling.cc:243] Waited 65.3 ms on lock 0x7b54008b2754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:45.100915 (+675083us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54008b2750. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.101124 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635378304}
I20260709 13:15:45.382464 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.382303 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:45.376849 30703 rpcz_store.cc:269] 0709 13:15:43.390178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.471331 (+ 81153us) service_pool.cc:224] Handling call
0709 13:15:45.067149 (+1595818us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54005319a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:15:45.376346 (+309197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281015168}
I20260709 13:15:45.383481 31378 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.383230 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.331877  3185 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.320484 30628 rpcz_store.cc:276] 0709 13:15:43.711667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.756627 (+ 44960us) service_pool.cc:224] Handling call
0709 13:15:45.057502 (+1300875us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54004abad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.057693 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191598336}
I20260709 13:15:45.332535 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:45.333654 30551 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:15:45.334075 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 227867us
I20260709 13:15:45.402459 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.403210 30551 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
W20260709 13:15:45.403302 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 70885.2us
W20260709 13:15:45.403685 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.404090 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.404870 30551 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.405637 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:45.406064 30704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 805) took 2016 ms (client timeout 2096 ms). Trace:
W20260709 13:15:45.406474 30704 rpcz_store.cc:269] 0709 13:15:43.389524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.935811 (+546287us) service_pool.cc:224] Handling call
0709 13:15:45.405894 (+1470083us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400050854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.406006 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3108617472}
I20260709 13:15:45.339505  3370 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:45.340482 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.340747 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.341775 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.314159  3243 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:45.279781 30378 rpcz_store.cc:269] 0709 13:15:08.249907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:08.271284 (+ 21377us) service_pool.cc:224] Handling call
0709 13:15:45.274447 (+37003163us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:37.244646 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:37.245414 (+   768us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:37.245472 (+    58us) write_op.cc:435] Acquired schema lock
0709 13:15:37.245493 (+    21us) tablet.cc:662] Decoding operations
0709 13:15:37.247648 (+  2155us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:37.247777 (+   129us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:37.247805 (+    28us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:15:37.249120 (+  1315us) tablet.cc:701] Row locks acquired
0709 13:15:37.249176 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:15:37.249375 (+   199us) write_op.cc:270] Start()
0709 13:15:37.249555 (+   180us) write_op.cc:276] Timestamp: P: 1783602937249295 usec, L: 0
0709 13:15:37.249600 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:15:37.350867 (+101267us) log.cc:844] Serialized 5539 byte log entry
0709 13:15:37.605234 (+254367us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1803dd30ac 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b
0709 13:15:38.284248 (+679014us) op_driver.cc:464] REPLICATION: finished
0709 13:15:38.465649 (+181401us) write_op.cc:301] APPLY: starting
0709 13:15:38.465783 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:38.487449 (+ 21666us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:38.487494 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:38.749108 (+261614us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:38.749508 (+   400us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:15:38.749580 (+    72us) write_op.cc:312] APPLY: finished
0709 13:15:38.750078 (+   498us) log.cc:844] Serialized 128 byte log entry
0709 13:15:38.750325 (+   247us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.750534 (+   209us) write_op.cc:454] Released schema lock
0709 13:15:38.780100 (+ 29566us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:38.864561 (+ 84461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:38.867249 (+  2688us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5214905,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":181001,"apply.run_cpu_time_us":23212,"apply.run_wall_time_us":382770,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62787,"num_ops":14,"prepare.queue_time_us":107078,"prepare.run_cpu_time_us":30752,"prepare.run_wall_time_us":601102,"raft.queue_time_us":157061,"raft.run_cpu_time_us":45852,"raft.run_wall_time_us":108370,"replication_time_us":1034446,"spinlock_wait_cycles":11390848,"thread_start_us":737254,"threads_started":4,"wal-append.queue_time_us":539426,"wal-append.run_cpu_time_us":13146,"wal-append.run_wall_time_us":1853516}]]}
W20260709 13:15:45.309307  3181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.601s	user 0.011s	sys 0.021s
I20260709 13:15:45.319013  3190 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:15:45.305576  3099 consensus_queue.cc:1175] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 11 status { last_received { term: 9 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: [10.7-10.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:45.351239 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.355914 31394 rpcz_store.cc:276] 0709 13:15:42.549667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.334105 (+784438us) service_pool.cc:224] Handling call
0709 13:15:45.159367 (+1825262us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b54003377d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.193878 (+ 34511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733071232}
I20260709 13:15:45.413152  3205 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.418704 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.418833  3205 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.371716 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.372967 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:45.451756 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 80149.1us
W20260709 13:15:45.374317 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 137417us
I20260709 13:15:45.452682 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.378712 31292 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:15:45.379225 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.454044 31292 leader_election.cc:403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:15:45.327075  3192 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:45.460577 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82112.5us
I20260709 13:15:45.461566 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.463994  3311 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.464926  3311 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:45.465373  3311 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.465757  3311 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:45.451411  3204 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:45.469743 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.412608  3082 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:45.470852 31389 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.471940 30552 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:45.475620  3185 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:45.476287 30553 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:45.477898 30553 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:45.478240 30553 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:45.478546  3192 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:45.381165 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 145150us
W20260709 13:15:45.311908 30705 rpcz_store.cc:269] 0709 13:15:43.389871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.656511 (+266640us) service_pool.cc:224] Handling call
0709 13:15:45.115177 (+1458666us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400342e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.115339 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195772672}
I20260709 13:15:45.361462 31391 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:45.311833 30632 rpcz_store.cc:276] 0709 13:15:42.798651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:43.265857 (+467206us) service_pool.cc:224] Handling call
0709 13:15:43.266291 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:43.271561 (+  5270us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:15:43.271881 (+   320us) log.cc:844] Serialized 14 byte log entry
0709 13:15:44.111754 (+839873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:44.111856 (+   102us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:44.111939 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.113924 (+1001985us) spinlock_profiling.cc:243] Waited 5.01 ms on lock 0x7b54005f41d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:45.113986 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:45.114153 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13223,"prepare.queue_time_us":173,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":39543,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":52,"spinlock_wait_cycles":11549184,"thread_start_us":825468,"threads_started":2,"wal-append.queue_time_us":579648,"wal-append.run_cpu_time_us":1402,"wal-append.run_wall_time_us":1001419}
I20260709 13:15:45.376214 30382 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:45.384222 30550 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 10.
W20260709 13:15:45.302412  2929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.599s	user 0.009s	sys 0.009s
I20260709 13:15:45.378006  3224 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:45.387905  3113 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:45.389618  3365 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:15:45.376471  3285 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.520758  3285 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:45.391836  3350 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:45.392328  3294 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:45.357918  3377 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.355345  3238 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:45.351754  3344 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.330058  3253 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.400557 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:45.339723 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1527) took 2316 ms. Trace:
I20260709 13:15:45.542945 31383 rpcz_store.cc:276] 0709 13:15:43.022822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:44.508411 (+1485589us) service_pool.cc:224] Handling call
0709 13:15:45.116794 (+608383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:45.312970 (+196176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:45.339495 (+ 26525us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b5400680450. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:45.339656 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398109056}
I20260709 13:15:45.544544 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:45.402070 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:45.375897  3386 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:15:45.410220 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.411963 31364 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:45.413520 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79121.4us
I20260709 13:15:45.424705 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.446919 30559 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:45.408604  3148 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:45.449067 30557 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:15:45.550465 30557 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:15:45.550909 30557 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:15:45.551236 30557 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.425695  3243 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.476418  3184 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:45.418167  3181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.710s	user 0.012s	sys 0.022s
W20260709 13:15:45.387104  3210 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:15:45.379813 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.562569 31383 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:45.480904 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.563184 31396 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:45.482693 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.473781 30552 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:45.488773 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:45.564683 31396 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:45.480840 30553 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.485049  3371 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:15:45.498894  3082 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:45.523407 30382 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 14, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:45.453300  3190 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:45.529039 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:45.529332 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:45.571487 30632 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:45.476262  3370 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:45.529855 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 80058.9us
W20260709 13:15:45.501633  3374 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:15:45.547151 31388 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:45.537694  3391 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:45.562786 31364 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:45.490537 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:45.495191 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:45.579502 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:45.544030  3294 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:45.578163 30382 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 9 to 12, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 12 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:45.564713 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82172.6us
I20260709 13:15:45.584911 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:45.567976 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79362.9us
I20260709 13:15:45.580333 30558 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:45.591841  3204 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:45.566200 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.498932  3341 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 13:15:45.592572  3204 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:45.593736  3204 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.581184 31378 raft_consensus.cc:2376] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18. Re-sending same reply.
I20260709 13:15:45.579519 31364 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:45.565438 30553 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:45.599107 31388 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:45.581274 31394 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:45.600805 30382 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 14, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:45.604308 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 123506us
I20260709 13:15:45.604347 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:45.606006 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:45.568557 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 115814us
I20260709 13:15:45.567673  3224 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:45.607223 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.611938  3380 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:45.612056  3190 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.567253  3148 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:45.569512  2929 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:45.631630 30553 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:15:45.632253 30553 pending_rounds.cc:85] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:45.632602 30553 pending_rounds.cc:107] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:15:45.632956 30553 raft_consensus.cc:2889] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.633719 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:45.579418  3391 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:45.572238  3082 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:45.613340 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.624786 30632 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:45.614765  3253 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:45.609606  3090 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:15:45.619024  3181 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:45.628273  3366 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.646677 31367 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:45.629843  3399 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:45.599371 31364 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.650707 31364 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:45.650887 31367 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:45.652894 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.618451 30382 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 12, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 12 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:45.598778  3243 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:45.563725  3184 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.617560 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:45.620658  3380 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:45.658219 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:45.661509 31378 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:45.663578  3078 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:45.666363 30625 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
I20260709 13:15:45.632004  3205 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:45.608361  3187 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:15:45.636142  3224 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:45.671687 30382 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 15, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:45.682312 31388 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:15:45.665985  3253 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:45.665915  3395 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:45.643354  3350 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:45.647349  3082 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:45.577229  3238 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.666152  3239 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:45.692049 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:45.695784 30382 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:45.666338  3382 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:15:45.677275  3366 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:45.706969  3238 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.706537  3380 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.708470  3380 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.694047  3294 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.714430  3285 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:45.715065  3285 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:45.715481  3285 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:45.715839  3184 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:45.716461  3391 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:45.717450 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.717952  3391 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.677593  2928 raft_consensus.cc:2764] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:15:45.718590  3224 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.720012  3148 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.722486  3395 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:45.724022  3395 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:45.724624  3377 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:45.729655 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.728833 30382 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 11, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 11 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:45.685627  3181 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:45.733083 30382 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 15, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 15 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:45.674054  3113 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:15:45.694188  3350 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.698292  3253 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:45.672657 31383 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:15:45.748813 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 146784us
W20260709 13:15:45.738353 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 172221us
I20260709 13:15:45.751183 30382 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 14, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:45.764153 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.775350  3253 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.786012 31364 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:15:45.818243 31364 pending_rounds.cc:85] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:45.818598 31364 pending_rounds.cc:107] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:15:45.818884 31364 raft_consensus.cc:2889] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:45.820562 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.737099  3181 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:45.822593  3181 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:45.825220 31361 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:15:45.825743 31361 pending_rounds.cc:85] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:15:45.825968 31361 pending_rounds.cc:107] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:15:45.826249 31361 raft_consensus.cc:2889] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:45.795359 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 198052us
I20260709 13:15:45.810426  3090 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:15:45.827486  3090 consensus_queue.cc:1243] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:15:45.741503  3350 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:45.827683  3408 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.828403  3408 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:45.829747  3090 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:45.830246  3090 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:45.822027  3254 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:15:45.832537  3205 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:15:45.834865 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.836197 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.837095 31391 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:45.837918 30382 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 17, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:45.844558 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 213770us
I20260709 13:15:45.783831  3184 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:15:45.867823 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.872236 30707 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:45.867609 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.868618 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.873754 30705 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:45.873821 30704 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:45.869158 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.870352  3406 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:45.871237 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.873044 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:45.883823 30703 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:45.874935 30699 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:45.875949 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:45.875471 30382 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:45.881989 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 250972us
I20260709 13:15:45.891801 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:45.893174 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:45.892856 30704 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:45.890408  3090 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.895475 30707 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:15:45.881541  3408 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:45.874413  3311 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:45.906519  3311 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:45.895674  3090 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:45.852605  3260 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.916544 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:45.917629 30701 raft_consensus.cc:2376] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15. Re-sending same reply.
I20260709 13:15:45.925159 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.929527 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:45.934032 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:45.940753 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 307103us
I20260709 13:15:45.944979 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:45.951175 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.984459 30382 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 12, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 12 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:45.987481 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53612.9us
I20260709 13:15:45.989184 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:45.890297  3405 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:45.992488 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:45.886744  3294 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:45.996762 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.983016 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:45.983075  3190 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:45.984519  3311 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:46.008867 30382 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:45.991398  3372 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:45.990259  3113 raft_consensus.cc:2764] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:15:45.997696  3370 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:46.018428  3372 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:46.019009  3372 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:46.019368  3241 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:15:46.020946  3241 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:45.991789  3380 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:46.018437  3370 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.022971  3370 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:46.008348  3366 raft_consensus.cc:2764] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:15:46.006098  3405 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.012642  3190 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:46.007789  3294 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.026597  3082 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.047102 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:46.027724  3350 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.048552 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:46.029243  3241 consensus_queue.cc:1243] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:15:46.051070 30632 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:46.050560 30382 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 13, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 13 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.051604  3082 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.055182  3350 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:46.029541  3285 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:45.995692  3272 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.061362  3285 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:46.031188  3311 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:46.062458  3311 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:45.984954  3260 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:46.064810  3260 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:46.040091 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:45.995061 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 167536us
I20260709 13:15:46.080819 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:46.081095 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:46.083343 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:46.086220 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:46.088963 30382 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:15:46.095346 31389 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:46.095129  3370 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.099995  3370 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:46.111871 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 195283us
I20260709 13:15:46.111522 30382 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:46.114627 30673 leader_election.cc:400] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:46.115607 30673 leader_election.cc:403] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:15:46.115309 30382 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.117553 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:46.043431  2928 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:46.036762  3294 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.124256 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:46.141423 30699 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.035357  3341 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:15:46.123034 31294 leader_election.cc:336] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.815s (SENT)
I20260709 13:15:46.147531 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:46.032591  3405 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.028133  3406 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.134991 31396 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:46.138994 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:46.161367 30703 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:15:46.134305  3285 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:46.090751  3204 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:15:46.142275 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:46.142525 30704 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:46.109169  3269 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:46.118057  3241 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:46.152074 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:46.140339  3185 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12.
I20260709 13:15:46.173950 30701 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:46.143658  3252 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:46.121567 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.160713  3365 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:46.168654  3253 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:15:46.147357  3341 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:46.132396  3260 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:15:46.131851  3090 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:15:46.184638 30699 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:46.195923 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43984.7us
I20260709 13:15:46.223449 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:46.273082 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49784.5us
I20260709 13:15:46.198925  3395 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:46.034153  3190 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.175639  3269 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:46.275161  3404 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:46.275388  3413 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.274478 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:46.293206 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:46.213383 30703 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:46.164371  3406 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.216511  3285 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:15:46.319530 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.276713  3254 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.275743  3415 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.283124  3269 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:46.220763  3148 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 13:15:46.351058  3429 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:15:46.352058  3420 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 13:15:46.320310  3365 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:46.280362  3372 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.332027  3285 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:46.358565  3418 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:15:46.330159  3366 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:46.290211  3395 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:46.363998  3425 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:15:46.367965  3365 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:46.331498  3415 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.301579  3404 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.376943  3395 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.372136  3372 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:46.308063  3413 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.388136  3413 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:46.389634  3415 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.391304  3406 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:46.391884  3366 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:46.400695  3404 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:46.401677  3366 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.443866 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:46.451742 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:46.455968 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 14, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:46.457507 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:46.465067 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:46.465812 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.470031 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 8, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:46.483790 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 415749us
W20260709 13:15:46.485754 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:46.486137 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 404946us
I20260709 13:15:46.493216 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:46.494969 30707 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:46.495253 30555 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:46.493167 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.501000 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:46.507001 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.507571 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:46.510295 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.511694 30632 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:46.515121 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.515861 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.517261 30632 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:46.518733 30628 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:15:46.535668 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.537495 30630 raft_consensus.cc:2410] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:15:46.551540 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:46.553339 31361 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:46.327921  3181 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.554924  3285 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.526486 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1724) took 1048 ms. Trace:
I20260709 13:15:46.550127 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:46.573513  3205 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.574747  3205 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:46.575796 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1721) took 1098 ms. Trace:
I20260709 13:15:46.576578 31389 rpcz_store.cc:276] 0709 13:15:45.477189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.594451 (+117262us) service_pool.cc:224] Handling call
0709 13:15:45.594802 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:46.096068 (+501266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.575084 (+479016us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54000113d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:46.575519 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150471424}
I20260709 13:15:46.579408  3224 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:46.580643  3224 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.583474  3252 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:46.588817  3404 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:46.589550  3404 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.590054  3404 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:46.591251  3405 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:46.591843 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:46.335453  3341 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.571568  3204 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:15:46.564095 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:46.596222  3181 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:46.620344  3382 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:15:46.627497 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 120297us
I20260709 13:15:46.644361  3382 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:46.645037  3090 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:46.645354  3341 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:46.645853 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:46.648661  3190 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:46.560856 31394 rpcz_store.cc:276] 0709 13:15:45.478090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.600854 (+122764us) service_pool.cc:224] Handling call
0709 13:15:45.601242 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.602261 (+  1019us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:45.709443 (+107182us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b1800025b40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:45.820276 (+110833us) log.cc:844] Serialized 14 byte log entry
0709 13:15:46.058729 (+238453us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:46.059480 (+   751us) log.cc:844] Serialized 26 byte log entry
0709 13:15:46.059637 (+   157us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:46.059671 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.059744 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:46.525409 (+465665us) raft_consensus.cc:1685] finished
0709 13:15:46.525667 (+   258us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:46.526092 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9423,"prepare.queue_time_us":138274,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":2098,"raft.queue_time_us":10230,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":9768,"spinlock_wait_cycles":7907712,"thread_start_us":297980,"threads_started":2,"wal-append.queue_time_us":400267}
I20260709 13:15:46.607997 30382 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 12, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 12 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:46.581411 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1714) took 1209 ms. Trace:
I20260709 13:15:46.648705  3252 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:15:46.652256 30382 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 13, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 13 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:46.599601 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:46.629165  3431 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:15:46.655771  3445 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:46.656960  3445 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:46.657379 31361 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
W20260709 13:15:46.626328  3430 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.306s	user 0.000s	sys 0.001s
I20260709 13:15:46.594518  3405 raft_consensus.cc:2764] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:15:46.645744  3090 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.660355  3090 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:46.661092 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:46.602383  3365 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.633250  3184 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:46.665956  3184 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:46.650957 31396 rpcz_store.cc:276] 0709 13:15:45.372263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.566456 (+194193us) service_pool.cc:224] Handling call
0709 13:15:46.134692 (+568236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:46.157813 (+ 23121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.580897 (+423084us) spinlock_profiling.cc:243] Waited 568 ms on lock 0x7b54002b07d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:46.581198 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1305987328}
I20260709 13:15:46.668888  3344 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:46.670560 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:46.671768 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:46.682144 31396 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.682288 31383 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:46.672868  3405 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:46.683728  3405 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:46.684365  3405 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:46.605101  3241 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:46.685410  3241 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:46.631276  3224 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:46.674003  3205 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:46.687067 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:46.688762  3241 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:46.690102  3241 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:46.658846  3445 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:46.610723 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:46.693511 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50848.4us
I20260709 13:15:46.693936 30630 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 11: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:46.695353 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 120809us
W20260709 13:15:46.644557 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 157345us
I20260709 13:15:46.747018  3238 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:46.748765  3238 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:46.749212  3238 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:46.759614 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98874.6us
I20260709 13:15:46.723354  3196 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:46.766747 30523 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.002s	sys 0.014s
W20260709 13:15:46.767611 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 626046us
I20260709 13:15:46.750530  3238 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.788666  3238 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:46.794859 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.004s	sys 0.016s
W20260709 13:15:46.795452 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.005s	sys 0.016s
W20260709 13:15:46.797057 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 675573us
I20260709 13:15:46.738408  3181 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:15:46.798288 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.805423 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.734827  3382 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:46.808212 31396 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:15:46.815050  3269 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.815764  3243 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:46.816334  3243 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:46.817952  3269 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:46.817727  3243 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:46.825304 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.835417  3447 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:46.836257  3447 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:46.846547 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:46.830045 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:46.852931  2929 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:46.852622  3444 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:46.855753  3382 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:46.858174 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.858554 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.859526 30630 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:15:46.861704 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:46.862651 30632 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:46.863925 30630 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:15:46.867553 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.870930 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:46.871690 30699 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:15:46.837611  3447 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:46.913592 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:46.921500 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 123416us
I20260709 13:15:46.833822  3184 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:46.926301 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.922680 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:46.929162 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:46.940055 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 63101.6us
I20260709 13:15:46.932832  3243 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:46.945713 30706 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:46.949103 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 164527us
I20260709 13:15:46.949203 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:46.950062 30555 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:46.950085 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66645.7us
I20260709 13:15:46.950898 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:46.953748 30700 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:46.954365 30630 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:15:46.958693 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 181071us
I20260709 13:15:46.960332  3181 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:46.960737  3254 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.637s	user 0.015s	sys 0.005s
I20260709 13:15:46.961452  3254 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:46.961174  3181 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:46.964074 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.964177  3444 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:15:46.964490 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 154302us
I20260709 13:15:46.966743  3444 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:46.968495  3444 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:46.971868 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.973672 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:46.952775  3187 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:46.975564 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:46.977051 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:46.981719 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:46.983217 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:46.985251  3450 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:15:46.987718  3184 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:46.962980  3341 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:46.991888 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.003837 30702 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
W20260709 13:15:47.018673 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.001995 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.006513 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.020020 30552 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:47.020128 30703 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:15:47.004567 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.021580 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:47.022011 30554 raft_consensus.cc:2376] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13. Re-sending same reply.
I20260709 13:15:46.991067  3341 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:47.023854 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.000512 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.024464 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:47.024202  3341 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.001302 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.026077 30558 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:46.999158  3439 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:47.027479 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:46.992426  3272 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.932s	user 0.007s	sys 0.018s
I20260709 13:15:47.029080 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.004106 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:47.002420 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.031054 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:47.031512 30559 raft_consensus.cc:2410] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:47.031682  3204 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.032387  3204 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.033311  3365 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:47.032783 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.034041  3365 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:47.032580  3374 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:47.035097 30559 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:15:47.030470 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:47.006682  3446 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.086s	user 0.002s	sys 0.000s
I20260709 13:15:47.003358 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:47.040387 30556 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:47.043015 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:46.998814  3099 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:47.002705 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.001742 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:47.070340 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.078253 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.044502  3224 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:47.078658 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.081846 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.081482 30706 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
W20260709 13:15:47.026386  3311 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.963s	user 0.009s	sys 0.012s
I20260709 13:15:47.029124  3272 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:47.094041 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:47.095503 30551 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:47.073302  3374 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:47.071633  3099 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.002905  3090 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:46.990492  3405 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.104844 30558 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:47.004374  3187 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:47.112841 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.108072 30557 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:47.108430 30550 raft_consensus.cc:2410] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:47.096887  3205 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:47.127619  3355 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.128336  3355 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.129612  3391 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:15:47.130764  3372 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.131572  3372 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.134716  3366 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:47.135521  3366 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.136057  3406 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:15:47.138180  3366 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:47.139107 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 167245us
I20260709 13:15:47.140218 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:47.142899 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 207872us
I20260709 13:15:47.143934  3252 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:47.144331 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:47.142894  2929 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:47.147351  3294 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:47.147722  3204 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.151041  3365 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.152662  3090 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.155812  3405 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:47.159428 30702 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:47.160815 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.160872 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.125653 30703 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:15:47.118759 31294 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.560s (SENT)
I20260709 13:15:47.114297  3099 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:47.175046 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.179530 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1725) took 1448 ms. Trace:
I20260709 13:15:47.179509 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:47.180154  3447 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:47.180357 31367 rpcz_store.cc:276] 0709 13:15:45.731134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.731300 (+   166us) service_pool.cc:224] Handling call
0709 13:15:45.731703 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:45.732114 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:45.732149 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:46.572519 (+840370us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b2c00056c38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:15:46.573610 (+  1091us) log.cc:844] Serialized 26 byte log entry
0709 13:15:46.806943 (+233333us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:15:46.807028 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:46.810275 (+  3247us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:47.178888 (+368613us) raft_consensus.cc:1685] finished
0709 13:15:47.178946 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:47.179296 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":360,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":3668,"spinlock_wait_cycles":6460160,"thread_start_us":210009,"threads_started":1,"wal-append.queue_time_us":257968}
I20260709 13:15:47.134495  3372 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.183657 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:47.184655 30705 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:47.184753  3187 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.187062  3181 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.094878  3224 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.197294 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:47.172152 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:47.192672  3224 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:47.179782  3454 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:15:47.214119 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:47.200083  3404 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.526s	user 0.014s	sys 0.003s
I20260709 13:15:47.172012  2929 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:47.226402 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:47.230397 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 69698.4us
I20260709 13:15:47.111469  3311 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.241668 31367 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:15:47.235590 30554 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:47.224061  3253 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.249488  3190 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:47.258392 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.274024 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.230829  3204 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 13:15:47.252048  3406 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:47.275792 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:47.281332 30550 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:47.286087 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.284255 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:47.242709  3404 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.290867  3239 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:47.294576 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:47.296424 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:47.299136 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 176287us
I20260709 13:15:47.298835  3205 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:47.250976  3187 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:15:47.320516 30555 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:47.300063 30703 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:47.302923 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 201612us
I20260709 13:15:47.303582 31364 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:15:47.305644  3148 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:47.307418 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 161501us
I20260709 13:15:47.251626  3224 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:15:47.311030 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:47.311088 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.282193  3374 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.266808  3447 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:15:47.343556 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66327.6us
I20260709 13:15:47.335312 30705 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:15:47.355407 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.268354  3253 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.320112  2929 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.300866  3252 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:15:47.379082  3253 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.381520 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.331722  3090 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:15:47.386850 30555 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
W20260709 13:15:47.386300 31294 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.596s (SENT)
I20260709 13:15:47.394961  3404 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.397130 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:47.397504 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.393886 30555 pending_rounds.cc:85] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 8
I20260709 13:15:47.399256 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:15:47.407745 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.411748 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.412599 31396 raft_consensus.cc:2410] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:15:47.358328  3447 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.416806  3447 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:47.418040  3370 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.418296 30558 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:47.418663  3370 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.420058  3311 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.421542 30553 raft_consensus.cc:2410] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:15:47.423311 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56513.9us
I20260709 13:15:47.426286  3190 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:15:47.401414 30555 pending_rounds.cc:107] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:15:47.432334 30555 raft_consensus.cc:2889] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:47.416935  3205 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:47.410919  3241 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.357332  3239 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:47.474722 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.410458  3253 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:47.475210 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84122.1us
I20260709 13:15:47.410250  3184 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:47.463671 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:47.476315 30706 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:47.487092  3082 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:47.487573 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:47.488562 30701 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:47.488897  3466 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:15:47.488404  3099 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.489132  3355 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:47.490031  3444 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.004s
I20260709 13:15:47.490471  3444 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.492028 30704 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:47.492782  3370 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:47.493067  3444 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.385344  2929 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.495939  2929 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.487149 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.475674  3238 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.497162 30705 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:47.497457  3238 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.499815  3239 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:47.489871 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.487794  3082 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:47.501283  3082 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:47.490864 31383 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:47.440097  3294 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:15:47.505651 30700 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:47.461530  3205 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:47.513108 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 151922us
I20260709 13:15:47.528172 31361 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:15:47.528882 31361 pending_rounds.cc:85] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 6
I20260709 13:15:47.532327 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.527086 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.499833  3238 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.536835 31361 pending_rounds.cc:107] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:15:47.538472 31361 raft_consensus.cc:2889] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:47.540094 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.542342 31396 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:47.525368 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.522246  3205 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:47.545614 30706 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:15:47.546774 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:47.537303 31364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1553) took 1674 ms (client timeout 2207 ms). Trace:
I20260709 13:15:47.550513 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:15:47.553270 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.553730 31388 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:47.554649 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:47.553017 31364 rpcz_store.cc:269] 0709 13:15:45.856231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:45.864569 (+  8338us) service_pool.cc:224] Handling call
0709 13:15:47.508720 (+1644151us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400717854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:47.530143 (+ 21423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3097505664}
I20260709 13:15:47.567242  3190 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:47.567579  3478 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:15:47.567904 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:47.570996  3444 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:47.571403  3444 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:47.571775  3444 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:47.522500  3241 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.573001  3374 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.573690  3374 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.575088  3241 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.575969  3374 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.556676  3344 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:47.578912 31294 leader_election.cc:336] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.654s (SENT)
I20260709 13:15:47.579638  3238 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:15:47.580318 30706 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:47.580747  3099 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:47.584299 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.585598 31367 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:47.587471 31396 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:47.560400 31389 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:47.597797 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.586766 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:47.605357 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:47.605650 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 78687.2us
I20260709 13:15:47.611513 30706 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [13.9-17.11]
I20260709 13:15:47.610105 30556 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:47.616971 31383 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:15:47.605072  3341 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.592201  3406 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:47.623481 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.624982 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:47.632892 30550 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:15:47.635032 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50894.2us
W20260709 13:15:47.645893 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 278328us
W20260709 13:15:47.659816  3480 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:15:47.680474 30702 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:47.681751 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.682217 30702 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:47.665868  3476 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:47.684425  3444 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.638887  3470 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:15:47.680442 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.635447  3238 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:15:47.667335 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 106362us
I20260709 13:15:47.676275 31388 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:15:47.690850 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 95993.7us
I20260709 13:15:47.691536 30706 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 14. Ops: []
I20260709 13:15:47.686865  3238 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.694585 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:47.697091  3482 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 13:15:47.698108  3470 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:47.625276  3374 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.698717  3470 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:47.699048  3374 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.700021  3470 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:47.698346  3479 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:15:47.692584  3238 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:47.697243 30704 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:47.701939 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.709110 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:47.710103  3470 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:47.710139  3391 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.707685  3190 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:47.711570 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.709492 30700 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:47.725514 30701 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:47.632004  3341 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:47.728366 30699 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:15:47.728757 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:47.735769 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60008.5us
I20260709 13:15:47.736819 30706 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
W20260709 13:15:47.737284 31294 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.717s (SENT)
I20260709 13:15:47.737025 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.738431 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.736438  3391 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.740535 30702 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:47.725993  3445 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:47.717203  3090 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:15:47.765336 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.727119  3374 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:47.773674 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.748667  3370 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:47.777910 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.764088  3269 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.728382  3477 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:15:47.771665  3344 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:15:47.782754 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44436us
I20260709 13:15:47.749905  3444 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:47.762426  3090 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.774586  3445 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:47.713316  3487 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:15:47.770850  3484 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.789414 30702 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:47.757252  3391 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.777956  3239 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.791803 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.784715  3090 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:47.791270  3269 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.785043  3355 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:15:47.796393  3341 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:47.785223  3252 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.812292 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:47.839308 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83749.1us
I20260709 13:15:47.798012 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.783393  3370 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.798359  3204 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.804121  3184 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:47.806872 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 81747.3us
I20260709 13:15:47.805121  2928 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.829430  3476 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.829609  3082 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.783300  3344 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.790997  3484 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.832067 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.832405 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.833540 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.835927 30624 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:15:47.797915 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.828420 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.878396 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:47.888353 30630 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:47.828420 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:47.846289 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:47.811916  3445 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.890784 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:47.854221 31388 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
W20260709 13:15:47.859095 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:47.861332 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.865693  3494 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:47.855950  3205 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.869122 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.809782  3269 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:47.872459  3477 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:47.876834 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.908253 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:47.903558 30705 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:15:47.797240  3239 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.920560 30702 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:47.925549 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:47.886158  3344 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:47.927597 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.930763  3493 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:47.909837  3224 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:47.934295 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.894359  3341 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:47.942775 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.948589 30556 raft_consensus.cc:2410] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:47.944937 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:47.950814 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154267us
I20260709 13:15:47.953397 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:47.946451 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.906901  3252 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.873899  2928 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.955040 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.955832 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:47.955101 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.958420 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.959721 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:47.931505  3493 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:47.895324  3476 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:47.911146  3204 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.972311 30551 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:47.922847  3494 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.983373 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.950404  3099 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:15:47.963011 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:47.979676 30558 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:47.987025 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:47.969404 30633 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:15:47.970593 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:47.983418 30553 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:47.932304  3477 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:47.991642 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.972007 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:47.984851 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:47.998425 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:47.955827  3294 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:48.007694 30631 raft_consensus.cc:2368] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:47.986310 30707 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:48.008072 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63272.6us
I20260709 13:15:48.013944 31391 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:48.018774 31378 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:48.019706 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.020887 30705 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:15:48.021870  3204 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:47.965324  3341 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.021930 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.030550 30706 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:48.023000 30628 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:48.038924  3294 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:48.044480 30552 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:47.990417  3253 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:48.048800 30627 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:15:48.050226 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.022567  2928 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.023787  3205 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.013933 30559 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:48.052122  3476 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.053026 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:48.055142 31294 leader_election.cc:336] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 2.135s (SENT)
I20260709 13:15:48.067658 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:48.055925  3341 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.061735 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.025655  3204 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:48.072286 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56765.3us
W20260709 13:15:48.070190 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 93403.2us
I20260709 13:15:48.083849 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.084887 30631 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:48.087447 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.072367 30707 raft_consensus.cc:2376] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11. Re-sending same reply.
I20260709 13:15:48.107805 31378 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:15:48.109320 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.109323 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.094512  3382 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.110400 30552 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:48.075546  3497 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:48.086531 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.099213  3205 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:48.122954 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:48.127125 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 236493us
I20260709 13:15:48.137753 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.138422 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:48.128921 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53391.9us
I20260709 13:15:48.139380 30558 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:48.139932 30558 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:48.138736 30554 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:48.146878  3269 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:48.147441  3370 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.125824  3224 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:48.125360  3082 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.125363  3497 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.126302  3374 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:48.172623  3238 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:48.172144 30558 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.173839  3224 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:48.175205 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.139120  3391 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:15:48.177975 30551 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:15:48.174293 30554 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:48.125555  3341 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:15:48.177309  3253 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:48.148571  3205 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.188715  3252 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.189040  3205 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.191833 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.192624 30554 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:48.193631 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:48.194832 30631 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:48.195530 30553 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:15:48.195923 30553 pending_rounds.cc:85] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:48.196146 30553 pending_rounds.cc:107] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:15:48.196192  3477 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:48.196376 30553 raft_consensus.cc:2889] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:48.196857 30553 pending_rounds.cc:107] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:15:48.197178 30553 raft_consensus.cc:2889] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:48.198130  3341 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:48.198590  3341 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:48.199985 30558 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:48.199615  3341 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:48.201119  3341 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:48.129065  3382 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.156221  3239 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:48.152307  3463 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:15:48.170126  3497 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:48.213790 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58948.8us
I20260709 13:15:48.193141  3090 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:48.214732 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68410.5us
W20260709 13:15:48.217618 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 79325.1us
W20260709 13:15:48.242375 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.183915  3484 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.219753 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.219103 30554 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:48.242870 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.244966 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.249866 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:48.255039  3499 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:15:48.258105 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:48.259255  3503 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:15:48.259771 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.260811 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.261090 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.262125 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.263646 31391 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:48.265396 30632 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:48.244520  3099 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:48.218454  3509 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:48.271276 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 216199us
I20260709 13:15:48.269143 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:48.269923 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.270563 30631 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:48.262293 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.277077 30555 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:48.247443  3476 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:48.281685 30558 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 13:15:48.282327 30558 pending_rounds.cc:85] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 5
I20260709 13:15:48.283335 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.267494  3494 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:48.290644 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.286307 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.308701 30705 raft_consensus.cc:2393] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:48.303188 30558 pending_rounds.cc:107] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:15:48.309598 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.307231 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:48.311987 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.309656 30558 raft_consensus.cc:2889] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:48.313865 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:48.322149 30555 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:15:48.327019 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:48.287470  3495 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.227s	user 0.000s	sys 0.002s
I20260709 13:15:48.316062 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:48.338798 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54860.9us
I20260709 13:15:48.237462  3253 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:48.340082 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.273864  3099 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:48.340184 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.286912  3513 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.284598  3493 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.329509  3391 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:15:48.346798 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.286582  3382 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:48.275560  3509 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:48.392099 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.393826  3509 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.324719  3512 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:48.394707  3512 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:48.395244  3512 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:48.396344  3355 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:48.397009  3355 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:48.400337  3344 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.401888 30624 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:48.403774 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.404368  3344 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.349383 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:48.406399 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65057.4us
I20260709 13:15:48.407816 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.359066 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.409040 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.409425 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.409433 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:48.373590 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59864.6us
I20260709 13:15:48.411005  3490 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.411536 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.411677  3487 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:48.359383 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.412612  3487 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:48.413053 31367 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:15:48.412987 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.411763  3490 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:48.376017 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.414999  3190 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:48.415913  3490 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.416186 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.352813 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.352087  3269 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.419770  3269 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:48.363912  3511 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:15:48.364253  3406 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:48.378854  3481 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.718s	user 0.000s	sys 0.005s
I20260709 13:15:48.364287 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.368402 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:48.345408  3494 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:48.372648  3517 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:15:48.372798  3514 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:48.374015  3513 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:48.340483  3470 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:15:48.382725 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:48.448627 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72738.7us
I20260709 13:15:48.391152  3238 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.325014  3374 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:15:48.455377 30632 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:48.408062  3439 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:48.411098 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1632) took 1142 ms. Trace:
I20260709 13:15:48.362582 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.362732 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.359179 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.404847 30630 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:48.466588 30523 leader_election.cc:400] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:48.473094 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.487291 30700 raft_consensus.cc:2410] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:15:48.427686 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.435989 30633 raft_consensus.cc:2376] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12. Re-sending same reply.
I20260709 13:15:48.491261 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.492758 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.442265 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.481575 30557 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:48.493870 30699 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:48.446652 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:48.453563 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.495441 30629 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 10.
I20260709 13:15:48.475426 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.484768 30523 leader_election.cc:403] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:15:48.425163 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:48.476066 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 924) took 1061 ms. Trace:
I20260709 13:15:48.487849 31383 rpcz_store.cc:276] 0709 13:15:47.269132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.269321 (+   189us) service_pool.cc:224] Handling call
0709 13:15:48.410851 (+1141530us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54002b1e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:48.411033 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467130240}
I20260709 13:15:48.498472 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.440557  3494 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:48.486021 30626 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:48.472241 31391 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:48.497810 30550 rpcz_store.cc:276] 0709 13:15:47.414250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.419233 (+  4983us) service_pool.cc:224] Handling call
0709 13:15:47.618695 (+199462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:47.633467 (+ 14772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:48.475780 (+842313us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000f0350. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:48.475957 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451095296}
I20260709 13:15:48.398327  3355 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.475550  3241 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:48.511868  3355 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:48.512161  3241 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:48.513203  3241 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.480218  3493 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:48.513556  3494 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:48.480109  3269 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.470425  3513 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:48.517908 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.518054 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.506029 30625 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
W20260709 13:15:48.523159 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 108628us
W20260709 13:15:48.457479 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 165402us
I20260709 13:15:48.521873 30628 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:48.526810 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.499366 30633 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:48.518863 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:48.471009  3224 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:48.539637 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:48.540251 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90280us
I20260709 13:15:48.538478 30633 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
W20260709 13:15:48.543273 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.544763 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:48.498003  3406 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:48.552048 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.519452  3241 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:48.519166  3090 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:48.553457 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.551271 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:48.547375 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.550190 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:48.547411 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:48.585268 30555 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:48.586138 30700 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:15:48.559558 30625 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:48.587347 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:15:48.587839 31292 leader_election.cc:403] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:15:48.604833 30633 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:15:48.610750 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.611419 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:48.612200 30550 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:48.617745 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.605455 30633 pending_rounds.cc:85] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 10
W20260709 13:15:48.618604 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.618811 30633 pending_rounds.cc:107] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:15:48.619315 30633 raft_consensus.cc:2889] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:48.539789  3224 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:48.546676  3238 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 12. Current term is 13.
I20260709 13:15:48.553048  3445 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:48.627983 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.633791 30629 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:48.543901  3344 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:15:48.609998  3391 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:48.663261 30700 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:48.620194  3374 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:15:48.672629 30625 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:15:48.678194 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.622681  3517 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:48.682417 30632 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
W20260709 13:15:48.689353 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.655206  3190 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:48.632756  3501 raft_consensus.cc:2764] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 13:15:48.698036 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.701200  3190 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:48.704978  3190 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:48.626482  3406 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:15:48.662806  3445 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.716953  3445 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:48.719569 30633 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:15:48.714581 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109062us
I20260709 13:15:48.673857  3391 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:48.723306 30629 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:15:48.723528 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 182468us
W20260709 13:15:48.724478 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:48.726733 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 190595us
I20260709 13:15:48.739583 30555 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:48.742831 30625 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:48.691254  3243 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:48.753008 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:48.749133 30554 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:48.770763 30629 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:48.777209 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:48.744401  3341 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.005s
I20260709 13:15:48.728853  3391 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:48.780982 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:48.744781  3184 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:48.806847 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:48.808220 30555 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:15:48.782907  3294 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.008s
I20260709 13:15:48.780272  3501 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:48.819276 30628 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
W20260709 13:15:48.827098 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 287568us
I20260709 13:15:48.789578  3445 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.794265  3184 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:48.811038  3099 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:48.777922  3243 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.770778  3238 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
W20260709 13:15:48.839335 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 92586.1us
I20260709 13:15:48.820508  3508 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:48.831461  3497 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.582s	user 0.013s	sys 0.019s
I20260709 13:15:48.876906 30555 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:15:48.816308  3490 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 13:15:48.889545 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99621.2us
W20260709 13:15:48.843828  3204 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.752s	user 0.010s	sys 0.009s
I20260709 13:15:48.896003 30630 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:48.900789 30626 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:48.901364 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1677) took 1060 ms. Trace:
W20260709 13:15:48.901631 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.822937  3090 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:15:48.901888 30630 rpcz_store.cc:276] 0709 13:15:47.840929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.932418 (+ 91489us) service_pool.cc:224] Handling call
0709 13:15:48.403493 (+471075us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:48.901292 (+497799us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6045568}
I20260709 13:15:48.841611  3344 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.823439  3294 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.915474 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:48.919817 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 105893us
W20260709 13:15:48.923883 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:48.920660 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:48.927517 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:48.846818  3445 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:48.911190  3204 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:48.889083  3497 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:48.954962 30700 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 10. Current term is 11. Ops: []
W20260709 13:15:48.967887  3525 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:15:48.969295  3243 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:48.970093  3526 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:15:48.920377  3344 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:48.971560  3528 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:15:48.976505 30550 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:48.978339 30552 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:48.960124  3253 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:48.969369  3529 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:15:49.003539 30707 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:48.971577  3294 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:49.005402 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 100551us
I20260709 13:15:48.954599  3184 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:48.994799  3243 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:48.981766  3536 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:49.022341 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 188952us
W20260709 13:15:49.026530 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 374449us
W20260709 13:15:49.032475 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107720us
I20260709 13:15:49.033013 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:49.035424 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.037737 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.037393 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:49.050848  3539 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:15:49.060721 30699 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.005s
I20260709 13:15:49.061231 30699 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:49.036885  3184 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:49.063117 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:49.022272  3090 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 13:15:49.075611 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 48555us
I20260709 13:15:49.075999  3082 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:49.076712  3082 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:49.077843  3082 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:49.068404  3527 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
I20260709 13:15:49.035569  3536 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.086174 30627 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:49.087060  3536 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.087355  3344 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:49.062590  3099 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:49.096138  3445 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.026779  3243 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.099030  3243 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:49.102147  3537 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:15:49.103796 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:49.104627 30553 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:15:49.109774  3538 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:15:49.112367  3533 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
W20260709 13:15:49.114508  3522 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.647s	user 0.000s	sys 0.005s
I20260709 13:15:49.121901  3497 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:49.122937  3374 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.146708  3374 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.153812 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.154875  3370 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:49.071023  3501 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:15:49.071668  3470 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.156862  3470 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:49.157490  3552 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:15:49.159516  3470 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:49.129263  3530 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 13:15:49.131281 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1679) took 1266 ms. Trace:
I20260709 13:15:49.162086 30624 rpcz_store.cc:276] 0709 13:15:47.864731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.025843 (+161112us) service_pool.cc:224] Handling call
0709 13:15:48.065099 (+ 39256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.402388 (+337289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.130949 (+728561us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:49.131178 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774029056}
I20260709 13:15:49.163071 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1201) took 1290 ms. Trace:
W20260709 13:15:49.129000  3523 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.466s	user 0.000s	sys 0.001s
I20260709 13:15:49.163513 30626 rpcz_store.cc:276] 0709 13:15:47.872385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.518977 (+646592us) service_pool.cc:224] Handling call
0709 13:15:48.900589 (+381612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.909720 (+  9131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.162858 (+253138us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54007229d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:49.163010 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876792192}
W20260709 13:15:49.165180  3494 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.650s	user 0.001s	sys 0.029s
I20260709 13:15:49.134133 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 971) took 1116 ms. Trace:
I20260709 13:15:49.166047 30633 rpcz_store.cc:276] 0709 13:15:48.017304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.632077 (+614773us) service_pool.cc:224] Handling call
0709 13:15:48.632486 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.720060 (+ 87574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.133826 (+413766us) spinlock_profiling.cc:243] Waited 86.7 ms on lock 0x7b5400052154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:49.134039 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199409408}
I20260709 13:15:49.166134  3494 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:49.141119  3514 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.000s
I20260709 13:15:49.167716 30559 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:49.167929  3514 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:49.149899  3374 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:49.142930 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.169760  3551 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:15:49.171187  3512 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.775s	user 0.019s	sys 0.004s
I20260709 13:15:49.175383 31389 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:49.176208  3509 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:49.179160 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:49.099704 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:49.160223 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 89716.6us
I20260709 13:15:49.180689  3506 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:49.181177  3506 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:49.181247  3484 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:49.181838  3484 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.182228  3484 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:49.182332  3506 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:49.184062  3506 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:49.185230  3513 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:49.185696  3513 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:49.186748  3513 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.188179 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1205) took 1314 ms. Trace:
I20260709 13:15:49.188222  3513 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:49.188642 30628 rpcz_store.cc:276] 0709 13:15:47.873685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.528164 (+654479us) service_pool.cc:224] Handling call
0709 13:15:48.544529 (+ 16365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:48.822969 (+278440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.187839 (+364870us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b54000541d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:49.188096 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625928448}
W20260709 13:15:49.193223 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.194274 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 86516.5us
W20260709 13:15:49.174121 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 146206us
I20260709 13:15:49.195721 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:49.174079 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.155162  3545 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:49.157601 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.197647 30704 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:49.203039 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:49.203930 31391 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:49.175343  3512 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.198592  3545 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:49.207525  3555 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:49.208884 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:49.215044 30700 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
W20260709 13:15:49.213114 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.206408 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:49.220940 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.219022 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:49.225093 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.231775 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:49.234331 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:49.213732  3514 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.238442 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:49.232616 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:49.240239 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 965) took 1224 ms. Trace:
I20260709 13:15:49.222405  3366 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:49.240773 30627 rpcz_store.cc:276] 0709 13:15:48.015351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.391754 (+376403us) service_pool.cc:224] Handling call
0709 13:15:49.240046 (+848292us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b54001fbb54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:49.240173 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1355034496}
I20260709 13:15:49.247782 30632 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:49.245843 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:49.244608  3366 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:49.222491  3405 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:15:49.235522  3099 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:49.241575  3439 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.008s
W20260709 13:15:49.243667  3190 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.538s	user 0.013s	sys 0.002s
I20260709 13:15:49.249923 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:49.263247 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 105775us
I20260709 13:15:49.264348  3512 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:49.277446  3554 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:15:49.267002 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:49.278277  3532 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:15:49.280323 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:49.278306  3190 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.281936  3504 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:15:49.283118  3504 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:49.283509  3504 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:49.267058  3294 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 13:15:49.283790  3190 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.248039  3514 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.284577  3504 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.285495  3204 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:49.285977  3504 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:49.286702  3090 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:15:49.287744 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.288117  3494 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:49.266503  3496 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:15:49.289956 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.244390  3239 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:15:49.291704 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.241201  3447 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:49.287981 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.291982 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:49.293074  3447 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:49.293656 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:49.293737  3447 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:49.294107  2928 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:49.292943 31389 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:49.308903 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 77220.5us
I20260709 13:15:49.248497  3470 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:15:49.294931  3238 raft_consensus.cc:2764] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:15:49.320647  3496 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:49.321273  3496 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:49.295280 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:49.323431 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 980) took 1174 ms. Trace:
I20260709 13:15:49.324157 30700 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:49.324452 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.326110  3476 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:49.251127  3366 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:49.326552  3476 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.302663  3439 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:49.326923  3090 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:49.308117 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:49.328045  3090 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:49.328892  3090 consensus_queue.cc:1243] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 17.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:49.328883  3099 raft_consensus.cc:2764] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:15:49.290347  3391 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.330348  3512 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:49.331131  3447 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:49.332540  3391 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:49.333281  3391 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:15:49.291772  3239 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:49.217868  3187 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:15:49.335819  3562 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:49.248238  3508 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:49.336560  3562 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:49.336732  3508 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.337257  3508 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:49.337664  3562 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:49.338768  3501 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:49.234467  3445 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:15:49.342200 31391 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:49.268599  3502 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:49.240427  3374 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:49.286355  3514 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.326573 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.308713  2928 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:49.318063  3294 consensus_queue.cc:1243] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:49.343457 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:49.323938 30632 rpcz_store.cc:276] 0709 13:15:48.149158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.691563 (+542405us) service_pool.cc:224] Handling call
0709 13:15:49.241361 (+549798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:49.248245 (+  6884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.312544 (+ 64299us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54001fbb50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:49.323354 (+ 10810us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1219485440}
I20260709 13:15:49.358946 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:49.327016  3476 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:49.376293 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82696.6us
I20260709 13:15:49.377770 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.385679  2928 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.345818  3556 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 13:15:49.361727  3239 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:49.371450  3090 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:49.388074  3090 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.388685  3090 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:49.353259  3238 raft_consensus.cc:2764] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:15:49.349845  3374 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:49.394428  3374 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:49.270232  3545 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.398659  3545 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:49.369596  3502 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:49.399964  3502 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.366556  3187 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:49.387398  2928 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:49.418108 30700 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:49.420097 30701 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:15:49.428848  3239 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:49.430045 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:49.431007  3238 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:49.429224 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:49.429302 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:49.433105 30556 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:49.433845 30632 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:15:49.450771 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.449383 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:49.452670 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.445477 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2005) took 1494 ms. Trace:
I20260709 13:15:49.460721 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:49.322299  3496 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.496981 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:49.499667 30553 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:49.509516 30556 pending_rounds.cc:85] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 10
I20260709 13:15:49.510181 30556 pending_rounds.cc:107] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:15:49.510473 30556 raft_consensus.cc:2889] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:49.515357 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:49.516983 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:49.482601  3184 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.518769  3512 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:49.519493  3512 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.525743  3536 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:49.526438  3536 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:49.442559  3484 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.527699  3536 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.529253  3484 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.530020  3536 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:49.535300 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 157648us
I20260709 13:15:49.536550 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:49.537943  3253 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.002s
I20260709 13:15:49.538585  3253 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:49.539624 31396 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
W20260709 13:15:49.541596 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.448716  3294 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.518530  3184 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.447309  3487 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:49.447134  3555 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.482527 30703 rpcz_store.cc:276] 0709 13:15:47.911535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:48.055260 (+143725us) service_pool.cc:224] Handling call
0709 13:15:48.055726 (+   466us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:15:48.056244 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:48.056276 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:15:48.060811 (+  4535us) log.cc:844] Serialized 104 byte log entry
0709 13:15:48.536494 (+475683us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:49.133090 (+596596us) spinlock_profiling.cc:243] Waited 981 us on lock 0x7b18000a9a80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:49.361288 (+228198us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.371444 (+ 10156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.371582 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:49.371653 (+    71us) raft_consensus.cc:1685] finished
0709 13:15:49.405849 (+ 34196us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.406110 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":168660,"prepare.queue_time_us":278296,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":11679,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":10499,"spinlock_wait_cycles":2255232,"thread_start_us":652743,"threads_started":2,"wal-append.queue_time_us":602636}
I20260709 13:15:49.495291  3532 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:15:49.611876  3281 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:49.604058 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53695.1us
I20260709 13:15:49.608675 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 1991) took 2603 ms. Trace:
I20260709 13:15:49.517433  3445 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:49.445575  3344 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.112s
W20260709 13:15:49.649720 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.664844 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.498764  3470 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:15:49.557205  3181 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:49.640890 30553 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:49.677937 30632 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:49.620537  3496 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:49.684759 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 374934us
I20260709 13:15:49.691335 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.610756  3550 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:49.659022  3445 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:49.452175  3224 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.565687  3294 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.655279 30698 rpcz_store.cc:276] 0709 13:15:47.004882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:47.005159 (+   277us) service_pool.cc:224] Handling call
0709 13:15:47.005558 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:47.005940 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:15:47.106917 (+100977us) log.cc:844] Serialized 14 byte log entry
0709 13:15:47.425926 (+319009us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b1800388fc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:48.520438 (+1094512us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800388fc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:15:48.520746 (+   308us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.022275 (+501529us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.167947 (+145672us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b1800388fc0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:49.168405 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 13:15:49.607610 (+439205us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:49.607703 (+    93us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:49.607777 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.608170 (+   393us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:49.608387 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6092,"prepare.queue_time_us":1482747,"prepare.run_cpu_time_us":193,"prepare.run_wall_time_us":198,"raft.queue_time_us":283559,"raft.run_cpu_time_us":593,"raft.run_wall_time_us":14893,"spinlock_wait_cycles":30508928,"thread_start_us":1057915,"threads_started":5,"wal-append.queue_time_us":504566,"wal-append.run_cpu_time_us":2390,"wal-append.run_wall_time_us":1178118}
I20260709 13:15:49.719882 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:49.731557 30559 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:49.674276  3241 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:15:49.739909 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 203470us
I20260709 13:15:49.740885 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.747197 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:49.750963  2928 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:49.751860  3568 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:15:49.755982  3439 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:49.756843  3439 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.654328  3555 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.760697  3555 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:49.762538  3187 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:49.763298  3187 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:49.630951  3487 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.768203  3512 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:49.769497  3190 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:49.770117  3190 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:49.773171  3187 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.775310  3187 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:49.780088  3366 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.780884  3563 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:49.541407  3239 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:49.783011  3563 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:49.704527  3391 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:49.785058  3563 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:49.785828  3391 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:15:49.786063  3563 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:15:49.783185  3370 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:49.787427  3391 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:49.676618  3476 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.787847  3391 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:49.788816  3391 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.739490  3500 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:15:49.790175  3391 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:49.781415  3439 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:49.792194  3294 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.704803  3181 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:49.707654  3445 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:49.839388 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.840178 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.767s (SENT) [suppressed 37 similar messages]
I20260709 13:15:49.816303  3366 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:49.826448  3512 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:49.857000 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.857555  3512 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.823683  3494 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:49.860446 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:49.854938  3181 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:49.706482  3550 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:49.831741  3184 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:49.832221  3513 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.643s	user 0.013s	sys 0.000s
W20260709 13:15:49.852526 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 105402us
W20260709 13:15:49.864887 30705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.013s	sys 0.038s
W20260709 13:15:49.879117 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.879330 30705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.014s	sys 0.038s
W20260709 13:15:49.857035  3508 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.002s
W20260709 13:15:49.871827  3504 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.004s
W20260709 13:15:49.863761  3099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.030s	sys 0.021s
I20260709 13:15:49.836159  3090 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:49.919359 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:49.919775 30550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.007s	sys 0.008s
W20260709 13:15:49.927449 30550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.008s	sys 0.009s
W20260709 13:15:49.917079 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.001s	sys 0.015s
W20260709 13:15:49.958014 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.001s	sys 0.015s
W20260709 13:15:49.958421 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 623182us
I20260709 13:15:49.962035 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.963784 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:49.909751  3099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.031s	sys 0.021s
I20260709 13:15:49.903460  3508 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.979573  3513 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:15:50.002118  3224 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:50.004002  3505 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:50.012960  3190 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:50.028254 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 149471us
I20260709 13:15:50.030551 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.034839 31391 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:50.036404 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.037415 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:49.978731  3575 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:50.039053 31389 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:49.976305  3502 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:50.046464 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:50.027837  3494 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.025986  2928 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:50.050032 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.028321  3551 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:50.066735 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.040954  3238 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:50.048214  3190 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:50.042339  3575 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:50.051362  3497 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:50.052088  3487 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:50.092476 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252327us
W20260709 13:15:50.093037 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 311961us
W20260709 13:15:50.107877 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.088253  3404 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.002s	sys 0.015s
W20260709 13:15:50.140265 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45654.9us
W20260709 13:15:50.152093 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.068044  3241 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:50.107412 31396 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:50.142164 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.033103  3505 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:50.146567 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:50.146678 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 100318us
I20260709 13:15:50.155014 30632 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:50.086802  3569 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:50.062997  3494 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.068722  2928 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:50.094823  3575 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:50.151562  3404 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.002s	sys 0.016s
I20260709 13:15:50.186115 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:50.177201  3374 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.776s	user 0.016s	sys 0.000s
I20260709 13:15:50.078011  3551 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:49.919399 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:50.208027 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.208786 31367 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:50.219151 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.260562 30559 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.527s	user 0.002s	sys 0.010s
I20260709 13:15:50.192513  3509 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.210564  3366 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:50.277385  3550 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:50.197573  3355 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.073s
W20260709 13:15:50.278713 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.280092  3584 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
W20260709 13:15:50.281112  3577 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.238s	user 0.000s	sys 0.001s
W20260709 13:15:50.281592  3576 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
W20260709 13:15:50.282051  3586 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:15:50.204619  3487 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:50.192330  3583 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:15:50.234990  3532 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:15:50.289387  3570 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.305s	user 0.001s	sys 0.001s
I20260709 13:15:50.174918  3445 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:50.327498 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.360018 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.366922 30559 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:50.269399 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:50.392189 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.400099 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.470467 31378 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:50.485289 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.469146 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.486173 31378 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:50.390378 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:50.197736  3502 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:15:50.205698  3374 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.502740  3238 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.503523  3238 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.504298  3374 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:50.505110 31394 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:50.506178  3238 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.421681 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.199613  3569 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.510843 31364 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:50.453682 31361 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:15:50.511849 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1729) took 1252 ms. Trace:
I20260709 13:15:50.512665 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2030) took 1171 ms. Trace:
I20260709 13:15:50.513042  2928 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:50.510463 31388 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:50.410862  3509 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:50.512320 30550 rpcz_store.cc:276] 0709 13:15:49.259896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.260093 (+   197us) service_pool.cc:224] Handling call
0709 13:15:49.260618 (+   525us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:49.261136 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.261182 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:49.261962 (+   780us) log.cc:844] Serialized 26 byte log entry
0709 13:15:49.394561 (+132599us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b18001c51c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:49.929186 (+534625us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:50.197697 (+268511us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.198107 (+   410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.198218 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:50.511441 (+313223us) raft_consensus.cc:1685] finished
0709 13:15:50.511588 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.511772 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43367,"prepare.queue_time_us":85426,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":34,"raft.queue_time_us":1256,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":8893,"spinlock_wait_cycles":8264960,"thread_start_us":717124,"threads_started":2,"wal-append.queue_time_us":789970}
W20260709 13:15:50.515383  3496 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.797s	user 0.014s	sys 0.000s
I20260709 13:15:50.513144 30704 rpcz_store.cc:276] 0709 13:15:49.341635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.341815 (+   180us) service_pool.cc:224] Handling call
0709 13:15:49.342202 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:49.367843 (+ 25641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.367899 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:49.403831 (+ 35932us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b4800550458. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:15:49.419832 (+ 16001us) log.cc:844] Serialized 26 byte log entry
0709 13:15:50.028065 (+608233us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:50.028133 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.028352 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:50.512332 (+483980us) raft_consensus.cc:1685] finished
0709 13:15:50.512355 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:50.512597 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69529,"raft.queue_time_us":26145,"raft.run_cpu_time_us":2431,"raft.run_wall_time_us":32834,"spinlock_wait_cycles":47469312,"thread_start_us":490330,"threads_started":1,"wal-append.queue_time_us":907234}
I20260709 13:15:50.515841  3509 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.470204  3090 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.469974  3532 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.520001  3532 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.520915 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:50.401069 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 248295us
W20260709 13:15:50.492298 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 102066us
I20260709 13:15:50.490038 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.527624 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:50.528340 30550 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:50.528430 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:50.468091 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.431150  3508 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.498133 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:50.499091 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.541982 30557 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:50.500902 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.506561 30551 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:15:50.506973 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.543802 30553 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:50.422946 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:50.508147 30558 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:50.544703 30701 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:15:50.531162 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.491855 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:50.546797 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47627.3us
I20260709 13:15:50.549634 31388 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:50.545789 31378 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:50.557430 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.558372 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.560393 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.560808 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:50.561259 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:50.564949 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.525791  3588 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:50.567440 30629 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:15:50.567271 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.565341 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.569550 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.570057 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.572979  3588 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.573323 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.561182  2928 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.574258 30624 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:50.574381  2928 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.576709  2928 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.580009  3569 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.581365  3505 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:50.573486  3588 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:50.584263 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.584995 30556 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:50.585475 30556 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:50.585726  3583 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:50.586279  3583 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.587109 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.586302 30556 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:50.587944 31361 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:50.588135 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:50.588719 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.589349 31396 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:50.589815 30557 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:15:50.590459 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.591688  3494 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.585080 30559 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:50.590282 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.594658 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:50.595340 30553 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:50.597800 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.598505 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.598767 30631 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:50.588284 30556 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:50.599958 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.601786 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.597290 30701 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:50.605255 30550 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:50.606024 30629 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:50.466024  3487 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.550508  3536 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.601369 31367 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:50.611337 30555 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:15:50.612438 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:50.555639  3355 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.619552 30625 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:15:50.632040 30625 pending_rounds.cc:85] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:15:50.632395 30625 pending_rounds.cc:107] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:15:50.623929 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.629719 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:50.633680 30553 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:50.634493 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:50.632730 30625 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:15:50.617051  3224 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.004s
I20260709 13:15:50.637706 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:50.576743  3445 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.651129  3238 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:50.673856  3238 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:50.674428  3238 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:50.676044  3509 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.676661  3509 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:50.671114 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.637893  2928 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:50.680599 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.682996 30553 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:15:50.620776  3366 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:15:50.623162  3494 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:15:50.703012 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 221093us
I20260709 13:15:50.666110  2929 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.665625  3497 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:50.710943 30556 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:50.710140 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.713840 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.639861  3536 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.695955  3099 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:50.747665  3099 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:50.749204  3536 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:50.727439  2929 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.714133  3497 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:50.752000  3497 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:50.681598  2928 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.755347  2928 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:50.757560  3476 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.759245  3589 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.760113  3589 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.761587 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:50.762823  3569 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:50.763418  3569 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:50.736645 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.736397 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.737171  3583 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:50.738065 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.739044 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.766538 31391 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:50.739504 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:50.766947 31364 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:50.740046 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.730007 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:50.778275 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:50.741417 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:50.779876 31396 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:50.651129  3355 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:50.768301 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:50.765970 31378 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21. Re-sending same reply.
I20260709 13:15:50.750232  3099 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:50.787554 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:50.767923  3187 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.992s	user 0.015s	sys 0.001s
I20260709 13:15:50.791898 30699 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:50.726928  3494 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:50.791733 30702 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:50.801795 30626 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:50.807718 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:50.811457 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:50.773828  3496 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:50.808890 30702 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
W20260709 13:15:50.790066  3580 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.701s	user 0.001s	sys 0.016s
I20260709 13:15:50.810840 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:50.819365 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:50.819682 30707 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:50.829396 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:50.806398  3494 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:50.813900  3532 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:50.764472  3569 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:50.762138  3238 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:50.826143  3559 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:50.830435  3536 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:50.843585  3582 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:15:50.856834 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:50.865295 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:50.865904 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:50.866840 31396 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:15:50.848266  3509 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:50.901309 30702 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:50.825402  3496 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:50.910595 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 132462us
I20260709 13:15:50.830430  3476 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:15:50.912416 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:50.909350 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.854930  3599 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:50.918748 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:50.928025 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:50.926487 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 187102us
I20260709 13:15:50.930548 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:50.931236 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:50.931588 30702 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 11.
I20260709 13:15:50.933800 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:50.934497 30630 raft_consensus.cc:2376] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16. Re-sending same reply.
I20260709 13:15:50.937196 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:50.937808 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.938925 30702 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:15:50.936956 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:50.928803 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.858454  3559 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:50.941110 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:50.942369 30558 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:50.895350  3536 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:50.946408 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:50.947608 30625 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:50.926347  3609 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:50.924649  3556 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:50.951836 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.067315  3612 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:51.067750 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:51.068372  3612 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.068900  3612 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.069190 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:51.070490 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:51.071292  3604 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.232s	user 0.001s	sys 0.000s
I20260709 13:15:51.072045 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.072445 30625 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:15:50.968904 30630 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 10. Current term is 12. Ops: []
W20260709 13:15:51.073762  3603 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:15:50.969925 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.075120 30556 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:50.973721 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:51.014531 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.076033 30628 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.076375 30702 raft_consensus.cc:2410] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:15:51.038709 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.062287 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:51.077317 30629 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:51.077486 30558 raft_consensus.cc:2410] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:50.956616 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:51.071165 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119418us
I20260709 13:15:51.078478 30632 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:15:50.963258  3532 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:50.936154  3344 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:51.078981 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.072840 30551 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 10.
I20260709 13:15:51.038154 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.093667  3496 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:50.971145  3224 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:51.079378 31364 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:15:50.938690  3487 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.087841 30632 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:15:50.929629  3439 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
I20260709 13:15:51.097893 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:51.080571  3582 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:15:50.948309  3589 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.113446 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.112133 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:51.115651 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:15:51.080011 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 136833us
I20260709 13:15:51.064584  3556 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.124782  3556 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:51.125336  3605 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:15:51.126920 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2303) took 1691 ms. Trace:
W20260709 13:15:51.103713 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 65706.9us
I20260709 13:15:51.127548 30554 rpcz_store.cc:276] 0709 13:15:49.435708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.459216 (+ 23508us) service_pool.cc:224] Handling call
0709 13:15:49.507315 (+ 48099us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:49.507928 (+   613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.507975 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:49.524860 (+ 16885us) log.cc:844] Serialized 26 byte log entry
0709 13:15:49.911963 (+387103us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:50.151732 (+239769us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.157248 (+  5516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:50.157358 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.126606 (+969248us) raft_consensus.cc:1685] finished
0709 13:15:51.126660 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.126843 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120973,"prepare.queue_time_us":102366,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":68,"raft.queue_time_us":12219,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":39062,"thread_start_us":562523,"threads_started":2,"wal-append.queue_time_us":1137872}
I20260709 13:15:51.106594  3496 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:51.130717 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.131531  3496 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:15:51.132642  3487 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:51.133080  3487 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:51.134516  3190 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.043s	user 0.014s	sys 0.001s
W20260709 13:15:51.136080  3575 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.923s	user 0.015s	sys 0.000s
I20260709 13:15:51.136607  3575 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.137212  3512 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.119540  3559 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.137825 31389 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:51.102129  3224 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.110646  3238 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.107198  3355 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:51.123360  3574 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.963s	user 0.002s	sys 0.011s
I20260709 13:15:51.105444  3619 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:15:51.134081  3487 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.099668  3366 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.088459  3532 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.140453 30707 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:51.146188 30632 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:15:51.148869 30628 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:51.160054 30632 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:51.147861  3224 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.152078 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:51.156919 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:51.167876 30699 raft_consensus.cc:2410] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:15:51.169225  3612 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.158773  3366 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.176791  3366 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.155427  3355 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:51.180434 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1574) took 1034 ms. Trace:
I20260709 13:15:51.180991 31389 rpcz_store.cc:276] 0709 13:15:50.146192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.146578 (+   386us) service_pool.cc:224] Handling call
0709 13:15:50.146929 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:51.138384 (+991455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.180103 (+ 41719us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:51.180357 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278328192}
W20260709 13:15:51.185313 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 95375.5us
I20260709 13:15:51.188705 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:51.192515 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:51.195168 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.202325 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.204792 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:51.204927 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.206589 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:51.210760 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:51.216053 30555 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:51.152597  3559 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:15:51.168687  3616 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:51.220775 30557 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:15:51.222568 30550 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:15:51.194595  3366 raft_consensus.cc:2764] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:15:51.222184 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.168278  3238 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:15:51.230670 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:51.232041 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.201421  3613 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:51.204857  3512 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:51.212074  3355 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:51.216240  3190 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:51.222196  3619 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:51.215997  3612 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:51.222357  3099 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.247395 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2023) took 2058 ms. Trace:
I20260709 13:15:51.228236  3622 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:51.218212  3563 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:15:51.251058 30705 rpcz_store.cc:276] 0709 13:15:49.188853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.192298 (+  3445us) service_pool.cc:224] Handling call
0709 13:15:49.192672 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:49.193055 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.193081 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:49.193714 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 13:15:49.879942 (+686228us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:49.880007 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.880117 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.122554 (+1242437us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:51.241309 (+118755us) raft_consensus.cc:1685] finished
0709 13:15:51.241338 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.247281 (+  5943us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7227,"raft.queue_time_us":4957,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":9056,"spinlock_wait_cycles":7397632,"thread_start_us":685757,"threads_started":1,"wal-append.queue_time_us":856680}
I20260709 13:15:51.237354  3613 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.241837  3512 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.261905 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:51.267112 30555 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:51.230052  3589 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:51.247481  3190 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:51.257107  3575 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.285945 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:51.261263  3622 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.283871  3253 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:15:51.261333  3099 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.284050  3374 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:15:51.290118  3591 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:15:51.239420  3497 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.270946  3613 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.274906  3512 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.280926  3366 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:51.260933  3391 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:15:51.279510  3536 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.284706  3190 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.298247  3622 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:51.336294 30556 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:51.331598 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.284279  3583 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:15:51.343345  3563 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:51.351483  2928 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.596s	user 0.015s	sys 0.000s
I20260709 13:15:51.326382  3575 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 13:15:51.377540 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 97339.7us
I20260709 13:15:51.380780 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.381203 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:51.381588 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.381892 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.383141 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.383798 31396 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:51.388957 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.391908 30702 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:51.393455 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.396199 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.398196 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.401182  3497 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.382893  3253 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:51.402771  3253 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:51.391196  3591 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:51.404070  3591 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.405496  3224 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.397889  3439 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:15:51.411216  3623 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:15:51.411103  3439 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.412029  3439 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.413854  3190 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.421401  3253 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.423144 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.427958 30550 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:51.428474 30704 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:51.428996 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.380704  2928 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.437271 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.394766  3391 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:51.436012 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:51.439299 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:51.447296 30555 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:15:51.458678  3575 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:15:51.380366  3563 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:51.467934  3588 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.885s	user 0.015s	sys 0.000s
I20260709 13:15:51.468780  3588 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.479900  3238 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.480759  3238 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.482028  3374 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:51.483402  3621 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:51.484009  3621 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:51.483750  3238 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.485212  3621 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.446189  3589 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:51.492625  3253 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.460970  3536 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.482599  3374 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:51.519660  3579 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.332s	user 0.020s	sys 0.024s
I20260709 13:15:51.521386  3609 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:51.524125  3575 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.528862  3563 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:51.535586  3630 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:15:51.536262  3632 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:51.540247  2928 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:51.509605  3626 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 13:15:51.461450  3391 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:51.548205  3494 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.697s	user 0.015s	sys 0.000s
I20260709 13:15:51.552526  3621 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.515484  3536 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.563107  3632 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.595323  3190 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.596323  3439 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.598245  2929 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.596678 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.599828 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:51.600298 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:51.600780 30704 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:15:51.563817  3556 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.601588 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.600955 31383 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.601863 30706 raft_consensus.cc:2368] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15.
I20260709 13:15:51.604120 31396 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:51.573810  3494 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:51.605749 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 350204us
I20260709 13:15:51.613816 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:51.614717 31396 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.612637 30553 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:51.620294  3391 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.559122  3575 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.624240 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.625609 31396 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:51.605743  3439 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.632948 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1222) took 1035 ms. Trace:
W20260709 13:15:51.630312 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.628748 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.636060 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:51.632100 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2304) took 2194 ms. Trace:
I20260709 13:15:51.600051  3532 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.596372  3536 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.586176  3632 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.622325  3588 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.638105 30552 rpcz_store.cc:276] 0709 13:15:49.436062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.437101 (+  1039us) service_pool.cc:224] Handling call
0709 13:15:49.437567 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:49.438008 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:49.438044 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:49.438750 (+   706us) log.cc:844] Serialized 26 byte log entry
0709 13:15:49.896401 (+457651us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:49.896473 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:49.896584 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.518887 (+1622303us) spinlock_profiling.cc:243] Waited 9.49 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:51.623612 (+104725us) raft_consensus.cc:1685] finished
0709 13:15:51.623639 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.630028 (+  6389us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":21816064,"thread_start_us":457119,"threads_started":1,"wal-append.queue_time_us":642282}
I20260709 13:15:51.633381 30624 rpcz_store.cc:276] 0709 13:15:50.597751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.598159 (+   408us) service_pool.cc:224] Handling call
0709 13:15:50.629172 (+ 31013us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:50.637377 (+  8205us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:15:50.798799 (+161422us) log.cc:844] Serialized 14 byte log entry
0709 13:15:50.921484 (+122685us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:50.941821 (+ 20337us) log.cc:844] Serialized 26 byte log entry
0709 13:15:50.942287 (+   466us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:50.947032 (+  4745us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.624141 (+677109us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b18001a51e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:15:51.624399 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.624530 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.632684 (+  8154us) raft_consensus.cc:1685] finished
0709 13:15:51.632718 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:51.632881 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":714219,"prepare.queue_time_us":443842,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":6110,"raft.queue_time_us":723680,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":18836,"spinlock_wait_cycles":242475776,"thread_start_us":228628,"threads_started":2,"wal-append.queue_time_us":317073}
W20260709 13:15:51.665758  3634 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:15:51.668190  3591 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.673036 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.650691  3344 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:51.674667  3224 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.675225 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.675992 30551 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
W20260709 13:15:51.675436 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.678979 30705 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:51.644577  3187 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.644044 31383 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:51.656121  3514 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:51.656101  3628 raft_consensus.cc:2764] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 13:15:51.694566 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86113.7us
I20260709 13:15:51.585889  3374 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.699350 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:51.705497 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.707216 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:51.721745 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:51.724920 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 778551us
I20260709 13:15:51.694655  3190 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.728199 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:51.729281 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:51.725231 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:51.734372 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.737722 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.738358 30554 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:51.740870 31364 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:51.741309 31364 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:51.741711 31389 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:51.741757 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:51.743781 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:51.744290 31396 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:51.717090  3405 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.742110 31364 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.687798  3501 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:15:51.750818 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:51.755417 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:51.756330 31383 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.724602  3514 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:15:51.729729  3569 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.026s	sys 0.036s
I20260709 13:15:51.750677 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:51.728365  3099 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.756217 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:51.746135  3500 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:51.762075 31364 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:51.765424  3638 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:51.705876  3187 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.730899  3631 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:51.763588  3405 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.761595 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1570) took 1727 ms. Trace:
I20260709 13:15:51.800179 31394 rpcz_store.cc:276] 0709 13:15:50.034392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.034524 (+   132us) service_pool.cc:224] Handling call
0709 13:15:50.048131 (+ 13607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:50.505586 (+457455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.761280 (+1255694us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400684054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:51.761528 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1050391296}
I20260709 13:15:51.747251  3374 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:51.778890 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:51.782675 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.675330  3344 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.737800  3635 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:51.752368  3536 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:51.767771  3569 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.027s	sys 0.037s
I20260709 13:15:51.752542  3589 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:51.752454  3439 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:51.803129 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.768651  3204 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:51.755848  3556 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.761152  3476 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.808557 31383 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
W20260709 13:15:51.788043  3630 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:15:51.769630  3484 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:51.767953  3514 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:51.768298  3504 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.771615  3447 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:51.771791  2929 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.743775  3190 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.832088  3204 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.743546  3622 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.896624  3622 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.814335  3635 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:15:51.844551  3366 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:15:51.814033  3631 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:15:51.869289 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.825462  3609 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.862917  3181 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.830940  3536 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:51.757813  3619 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:15:51.889420  3640 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:51.885718  3599 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:51.834391 30701 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:51.899219 30559 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:15:51.901700  3551 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:51.903391  3600 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:51.903985  3606 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.904538  3641 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:51.907483  3639 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:15:51.910154  3575 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.914964  3563 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:51.922055 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.924860 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:51.837801  3589 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:51.862529  3514 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:51.931922 30557 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:51.849398  3484 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:51.836042  3439 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.938387 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:51.910390  3204 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:51.851308  3476 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.836419  3494 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.932756  3239 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:51.956305 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:51.920416 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:51.914340  3635 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:51.957643 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.932971  3391 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:51.976608  3391 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:51.963738 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 190206us
I20260709 13:15:51.884584  3504 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.937466  3599 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:51.887322  2929 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:51.970099 31364 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:15:51.930912  3532 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.935915  3609 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:51.960469  3640 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:51.982478  3616 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.746s	user 0.017s	sys 0.000s
I20260709 13:15:51.982439 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:52.002594 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82315.7us
I20260709 13:15:52.002924 30626 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:51.983822 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:51.984408 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.004698 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:52.005841  3645 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:15:52.006470  3187 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.970919  3500 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:51.934414  3181 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:51.983608 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:51.968926  3600 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:51.955785  3551 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:15:51.956688  3606 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.071693  3500 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.072815  3181 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.073335  3606 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.968204  3563 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.075270  3551 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:52.075744  3551 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:52.076427  3563 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:51.986589  3599 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:52.076669  3551 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.078109  3551 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:51.960405  3641 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.021741 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.030715  3616 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:52.039208 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.080287 30553 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:51.971503  3405 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:52.060138  3642 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.177s	user 0.002s	sys 0.002s
W20260709 13:15:52.086819 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.060355 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:52.021840  3648 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:52.061480 30630 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:52.007257  3619 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:52.096226  3619 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:52.096822  3619 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:51.966099  3366 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:15:52.024420  3484 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.029469  3589 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.027648  3631 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:51.991808 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:51.987892  3536 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.116808  3391 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:51.978513  3506 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.117683  3506 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.119009  3439 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.119328  2929 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.069036 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.120180  3609 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.071098 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.084517 30554 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:52.120615  3190 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.121500 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.121680 30628 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:52.121744  3504 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:51.986742  3509 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.069300  3591 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:52.122864 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.123241  3509 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.123399  3591 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.124038 30628 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:52.125051 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.125550  3509 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.093684  3648 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:52.125788  3591 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.126919 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.126914  3648 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.128027 30628 raft_consensus.cc:2410] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:15:52.128870  3613 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.796s	user 0.010s	sys 0.006s
I20260709 13:15:52.129384  3613 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.130592 30702 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:52.094967  3641 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.095474 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:52.131824 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 140135us
I20260709 13:15:52.113687  3563 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:52.113368  3600 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.115381 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.152159 30552 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 11: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:52.152904 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.153198 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.153981 30551 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:52.116138  3476 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.071171  3640 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.132519 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:52.132714  3562 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:52.085150 30633 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:15:52.139025 31364 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:52.143700 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:52.089833 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:52.141852 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.141815 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:52.158396 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:51.980484  3239 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.145507 30626 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:52.145613  3563 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:52.146440  3609 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.110769  3588 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:52.128902 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:52.175577 30628 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:15:52.160221 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70515.7us
I20260709 13:15:52.120262  3506 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.162855  3616 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:52.153338  3621 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.574s	user 0.010s	sys 0.002s
I20260709 13:15:52.166493 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:52.134480  3625 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.794s	user 0.002s	sys 0.001s
I20260709 13:15:52.181135 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2044) took 1325 ms. Trace:
I20260709 13:15:52.183470 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 995) took 1230 ms. Trace:
I20260709 13:15:52.111506  3281 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.190917 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:52.197800 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54255us
I20260709 13:15:52.154812  3190 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.202795 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:52.176906  3632 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.504s	user 0.000s	sys 0.013s
I20260709 13:15:52.159538  3562 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.105683  3405 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:15:52.172238  3439 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.210568 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:52.180595  3642 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:15:52.208765 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:52.196823 30629 rpcz_store.cc:276] 0709 13:15:50.952821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.086253 (+133432us) service_pool.cc:224] Handling call
0709 13:15:51.086748 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.645236 (+558488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:51.645291 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:51.655686 (+ 10395us) log.cc:844] Serialized 26 byte log entry
0709 13:15:51.754305 (+ 98619us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:51.790406 (+ 36101us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.174162 (+383756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.174282 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:52.174356 (+    74us) raft_consensus.cc:1685] finished
0709 13:15:52.174380 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.183358 (+  8978us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":621431,"prepare.queue_time_us":36285,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":615289,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":2605,"spinlock_wait_cycles":28483840,"thread_start_us":422151,"threads_started":2,"wal-append.queue_time_us":126950}
W20260709 13:15:52.261157 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58073.9us
I20260709 13:15:52.261438 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:52.221670 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.186982  3512 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.849s	user 0.004s	sys 0.013s
I20260709 13:15:52.262128 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:52.222509 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.265864 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.277819 31364 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:52.174825  3607 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:52.174883  3532 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:52.285108 30630 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:52.286966 30629 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:52.287825 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.288085  3508 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:52.302187  3508 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.302870  3508 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:52.259618  3562 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:52.310086 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:52.203603  3391 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.311178  3391 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.312796 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1613) took 1124 ms. Trace:
I20260709 13:15:52.313282 30706 rpcz_store.cc:276] 0709 13:15:51.187999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.202019 (+ 14020us) service_pool.cc:224] Handling call
0709 13:15:52.312514 (+1110495us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b540021fa54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:52.312693 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":853496192}
I20260709 13:15:52.207607  3476 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:52.315801  3476 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:52.317065  3476 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.318445 30704 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:52.205840  3632 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.320608 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 867) took 1215 ms. Trace:
I20260709 13:15:52.321458  3632 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.207013  3616 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.327298 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 999) took 1370 ms. Trace:
I20260709 13:15:52.327859 30625 rpcz_store.cc:276] 0709 13:15:50.957064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.089820 (+132756us) service_pool.cc:224] Handling call
0709 13:15:51.090178 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.645955 (+555777us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:51.645989 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:51.665184 (+ 19195us) log.cc:844] Serialized 26 byte log entry
0709 13:15:52.067421 (+402237us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b18001a0aa0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:52.178573 (+111152us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:52.178657 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.178773 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:52.326906 (+148133us) raft_consensus.cc:1685] finished
0709 13:15:52.326936 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.327219 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":555333,"raft.queue_time_us":572114,"spinlock_wait_cycles":17142784,"thread_start_us":110864,"threads_started":1,"wal-append.queue_time_us":659416}
I20260709 13:15:52.337409 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1616) took 1138 ms. Trace:
I20260709 13:15:52.337918 30705 rpcz_store.cc:276] 0709 13:15:51.198776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.308975 (+110199us) service_pool.cc:224] Handling call
0709 13:15:52.337150 (+1028175us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54005a32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:52.337332 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762772864}
I20260709 13:15:52.289355  3550 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:52.338848  3550 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:52.339182 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.339452  3613 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.339859  3550 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.341472 30699 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:52.291087 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 854) took 1521 ms. Trace:
I20260709 13:15:52.291829 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:52.345294 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 53542.9us
I20260709 13:15:52.346220 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:52.292726 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.346851 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.347132 30550 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:52.321115 30704 rpcz_store.cc:276] 0709 13:15:51.105378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.601954 (+496576us) service_pool.cc:224] Handling call
0709 13:15:52.318161 (+716207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.319176 (+  1015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.320310 (+  1134us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b540021fa50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:52.320524 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646272384}
I20260709 13:15:52.348157 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.297140 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1614) took 1104 ms. Trace:
I20260709 13:15:52.348694 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:52.184319  3609 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.210309  3281 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.210522  3190 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.344961 30701 rpcz_store.cc:276] 0709 13:15:50.769320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.769534 (+   214us) service_pool.cc:224] Handling call
0709 13:15:50.769898 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:51.937853 (+1167955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.290788 (+352935us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54005a2dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:52.291005 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2447594752}
I20260709 13:15:52.355406 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.314256  3391 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.286384 30628 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 11.
I20260709 13:15:52.339936 30705 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:15:52.358207 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 175371us
I20260709 13:15:52.359597 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.360122 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:52.171512  3204 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.363207 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.405550 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.365476 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.348894 30703 rpcz_store.cc:276] 0709 13:15:51.193005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.667539 (+474534us) service_pool.cc:224] Handling call
0709 13:15:51.667993 (+   454us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:51.668510 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:15:51.668844 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.751162 (+ 82318us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:51.767921 (+ 16759us) log.cc:844] Serialized 52 byte log entry
0709 13:15:51.768120 (+   199us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:15:51.784335 (+ 16215us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.785732 (+  1397us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.863556 (+ 77824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.863653 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.904351 (+ 40698us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b1800464da0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:52.024030 (+119679us) spinlock_profiling.cc:243] Waited 5.64 ms on lock 0x7b1800464da0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:52.296841 (+272811us) raft_consensus.cc:1685] finished
0709 13:15:52.296871 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.297064 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4489,"prepare.queue_time_us":242156,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":136,"raft.queue_time_us":31460,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":19133,"spinlock_wait_cycles":20031744,"thread_start_us":155195,"threads_started":2,"wal-append.queue_time_us":143763}
W20260709 13:15:52.399068 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43771.6us
I20260709 13:15:52.411173 30523 leader_election.cc:400] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15.
I20260709 13:15:52.411742 30523 leader_election.cc:403] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:52.400738 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:52.413594 30550 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:52.414196 31394 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:52.415488 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.424769 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.425684 30628 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:52.416970 31364 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:52.267553  3621 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.186s
I20260709 13:15:52.397652  3494 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:52.431308 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.282919  3512 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.284020  3439 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.412860  3619 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.359213  3181 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:52.448009 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:52.449522 30624 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:52.453132 30699 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:15:52.430361 30628 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:52.369750  3532 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.471477 30628 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:52.370319  3613 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:52.492766 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:52.461159 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.396513  3641 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:52.413196  3241 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:52.423271  3294 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:15:52.427914  3625 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:15:52.465647 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.475656 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.476650  3204 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.477094 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.537292 30554 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:52.426541  3514 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.482666 30703 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:52.344349  3563 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:52.484308  3662 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.339s	user 0.001s	sys 0.003s
W20260709 13:15:52.485958  3635 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.005s
I20260709 13:15:52.462505  3599 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.487182  3487 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:52.456611  3638 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.457163 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:52.573684 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.447026  3622 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:52.506028  3611 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:52.499459 30705 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:52.507508 31383 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:52.538887  3551 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.539875 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:52.540472 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:52.546948 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.559793 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.494786  3239 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.488783  3181 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.565403 31364 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:15:52.570218 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:52.573206  3635 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.574201 30630 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:52.578557 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:52.624889 30554 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:52.507882 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.580631  3638 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.611001  3563 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:52.610997  3181 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:52.601068 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.627393 30705 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:52.619256 30555 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:52.584519 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1611) took 1548 ms. Trace:
I20260709 13:15:52.629637 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.521937  3532 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.539253  3641 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:52.595564 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.636092 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.544389  3666 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:15:52.627030 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:52.579804  3611 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.622517  3239 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.631451 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.616503  3559 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:52.617558 30553 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:52.675001  3281 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.635402  3508 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.664105  3611 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:52.679529  3599 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.679859  3439 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:52.680507  3671 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:15:52.622850  3551 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.636843 30555 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:52.681066 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:52.630296  3660 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:15:52.682387 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:52.683118 30555 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:15:52.683172  3241 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.646732 30706 raft_consensus.cc:2368] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
W20260709 13:15:52.620770 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 178649us
I20260709 13:15:52.683753  3241 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.684490 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:52.684752 30556 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:52.685900  3241 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.668233  3559 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:52.687712  3641 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.688103 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.687918  3559 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.688597  3675 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.689030 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.689239  3675 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.691681  3512 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.692243  3508 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.693310 30629 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:52.663928 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.624652  3494 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:52.667217 31364 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:52.680405  3599 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:52.628513 30700 rpcz_store.cc:276] 0709 13:15:51.036491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.131950 (+ 95459us) service_pool.cc:224] Handling call
0709 13:15:51.132374 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.132625 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:51.132649 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:51.133210 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 13:15:51.227641 (+ 94431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:51.240355 (+ 12714us) log.cc:844] Serialized 14 byte log entry
0709 13:15:51.249533 (+  9178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:51.249674 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:51.601215 (+351541us) spinlock_profiling.cc:243] Waited 8.35 ms on lock 0x7b18001c5b20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:15:52.133821 (+532606us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:52.584161 (+450340us) raft_consensus.cc:1685] finished
0709 13:15:52.584191 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.584407 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8635,"prepare.queue_time_us":12256,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":8827,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9368,"spinlock_wait_cycles":26535040,"thread_start_us":93998,"threads_started":1,"wal-append.queue_time_us":158553}
W20260709 13:15:52.706255 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:52.702894 30556 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:52.686465  3241 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.712742 30553 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:52.716158 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:52.707468  3665 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 13:15:52.724148 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:52.725569  3669 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:15:52.728566  3664 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 13:15:52.729799  3677 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:15:52.732606  3562 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.720305  3190 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.723856  3551 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.708071  3599 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:52.740175  3670 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:15:52.741053  3599 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.742998  3532 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:52.745368  3663 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 13:15:52.708413  3508 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:52.709224  3609 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.746181  3508 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:52.746191 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.749022 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.749534  3508 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:52.753613  3635 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.750422 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.758235 30706 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:52.759331 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:52.759981 30707 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:52.760905 30554 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:52.698518  3239 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.771750 30706 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:52.773304 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1620) took 1280 ms. Trace:
I20260709 13:15:52.773684 30706 rpcz_store.cc:276] 0709 13:15:51.492918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.764548 (+1271630us) service_pool.cc:224] Handling call
0709 13:15:52.771606 (+  7058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.773078 (+  1472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.773252 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:52.721210  3674 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:52.715629  3241 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:52.786259  3241 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:52.787254  3241 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:52.787876  3241 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:52.791677 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.770993  3622 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15.
I20260709 13:15:52.796388 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.798588 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:52.799592 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 110635us
W20260709 13:15:52.794642 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1615) took 1601 ms (client timeout 2067 ms). Trace:
I20260709 13:15:52.804441 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.703670  3494 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.800488 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.808494 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:52.802201 30702 rpcz_store.cc:269] 0709 13:15:51.193220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.393061 (+199841us) service_pool.cc:224] Handling call
0709 13:15:52.794379 (+1401318us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b54005a28d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:52.794543 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1692897664}
I20260709 13:15:52.823069 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.772569  3619 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:52.841202 30706 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:15:52.868098 30706 pending_rounds.cc:85] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 6
I20260709 13:15:52.868429 30706 pending_rounds.cc:107] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:15:52.868726 30706 raft_consensus.cc:2889] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:52.890902 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2048) took 1039 ms. Trace:
I20260709 13:15:52.894277 30706 rpcz_store.cc:276] 0709 13:15:51.851823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.781157 (+929334us) service_pool.cc:224] Handling call
0709 13:15:52.823649 (+ 42492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.890432 (+ 66783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.890805 (+   373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:52.833575 30700 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:15:52.811849  3099 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:52.819902  3501 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:52.819408  3281 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:52.831857 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.832269 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:52.896365 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2050) took 1021 ms. Trace:
I20260709 13:15:52.806944  3512 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.824120  3253 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:52.830749  3635 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:52.923375 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.831779  3514 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.889863  3487 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:52.888612  3239 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:52.903154 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71416.3us
I20260709 13:15:52.892585  3680 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:52.921656 30700 rpcz_store.cc:276] 0709 13:15:51.875199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.796080 (+920881us) service_pool.cc:224] Handling call
0709 13:15:52.821531 (+ 25451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.895950 (+ 74419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.896276 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:15:52.926795 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94623.8us
I20260709 13:15:52.938987 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:52.939924 31364 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:15:52.939760 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:52.933255 31388 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:52.902801  3675 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.923341  3682 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:52.949280 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 949) took 1569 ms. Trace:
W20260709 13:15:52.947258 30631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 991) took 2009 ms (client timeout 1976 ms). Trace:
W20260709 13:15:52.971033 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 170641us
I20260709 13:15:52.898375  3551 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:52.969839 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:52.923110  3619 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:52.974666  3239 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:52.934549  3099 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:15:52.944257  3672 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 13:15:52.938722  3680 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:52.944962  3678 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:15:52.974499  3391 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:52.975143  3619 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.971415 30559 rpcz_store.cc:276] 0709 13:15:51.379707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.379882 (+   175us) service_pool.cc:224] Handling call
0709 13:15:51.396171 (+ 16289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.938271 (+542100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.948902 (+1010631us) spinlock_profiling.cc:243] Waited 502 ms on lock 0x7b54000f05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:52.949192 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1155564928}
I20260709 13:15:52.926913  3512 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.935086  3638 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:52.934553  3487 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.973939 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:52.942634  3629 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:15:52.971485 30631 rpcz_store.cc:269] 0709 13:15:50.937432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.937575 (+   143us) service_pool.cc:224] Handling call
0709 13:15:52.946990 (+2009415us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54004a88d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:52.947150 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2380539648}
I20260709 13:15:52.946868  3631 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:15:52.978813  3239 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:52.997332 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:52.984462  3674 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:52.998597  3629 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:52.968739  3501 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:15:52.935602  3241 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.994863  3391 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:52.997224  3619 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:53.007266 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:52.985981  3099 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:52.973405  3682 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:52.979331  3551 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:52.988394  3680 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.013147 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:52.983275  3614 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:15:52.999131  3487 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:53.015771 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:53.007725  3239 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:53.063650 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47997.1us
I20260709 13:15:53.011758  3629 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:53.074570 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61492.2us
I20260709 13:15:53.075930 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:53.080993  3512 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:53.010775  3666 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:15:53.006594  3181 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.095009 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:53.063038  3536 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:53.014885  3391 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:53.021109  3099 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:53.107147 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.071606  3638 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:53.065064  3688 raft_consensus.cc:2764] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:15:53.105011  3684 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:15:53.128528 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.087875  3679 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:53.134047 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.134891 30702 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:53.093111  3631 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:53.011580  3611 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.142963  3508 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:53.162433  3508 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.145445  3682 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.132720  3691 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:52.195611 30698 rpcz_store.cc:276] 0709 13:15:50.855842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:50.856483 (+   641us) service_pool.cc:224] Handling call
0709 13:15:52.084878 (+1228395us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:15:52.085074 (+   196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.085329 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:15:52.085610 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.172150 (+ 86540us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.172731 (+   581us) log.cc:844] Serialized 14 byte log entry
0709 13:15:52.172931 (+   200us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:52.172993 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:15:52.173065 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:52.173170 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:52.181059 (+  7889us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19009,"prepare.queue_time_us":3724,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":38489,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":10414,"spinlock_wait_cycles":4496384,"thread_start_us":59859,"threads_started":1,"wal-append.queue_time_us":59514}
I20260709 13:15:53.147490  3690 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.100086  3685 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:53.148883  3638 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:53.159863  3611 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.170560 30698 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: []
I20260709 13:15:53.134334  3536 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:15:53.172405 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 192871us
I20260709 13:15:53.196177 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:53.200616 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:53.167492  3684 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:15:53.204425 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.205390 31394 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:15:53.218927  3696 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:53.197268  3374 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.220184  3696 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:15:53.220115  3374 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.221196  3696 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:53.226337  3551 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:53.228410  3181 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.198450  3501 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.231281  3501 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.236712  3508 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.166901  3674 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.249462 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.249676 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.250123 30552 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:53.174687  3690 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.252920  3690 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.238819  3631 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:53.234000  3501 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.253566  3631 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:53.241284 30702 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:53.254861  3638 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.257541 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:53.258463 30550 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:53.241892  3536 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.261638 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 870) took 1084 ms. Trace:
I20260709 13:15:53.253410 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:53.243959  3374 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.250447 30555 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:53.254559  3631 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:53.260533 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:53.274336 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.273432 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.278208  3551 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:53.281154  3551 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.281754  3551 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.277379  3697 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.245565 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:53.284621 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1623) took 1280 ms. Trace:
I20260709 13:15:53.284765  3697 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.285202 30706 rpcz_store.cc:276] 0709 13:15:52.004007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.895642 (+891635us) service_pool.cc:224] Handling call
0709 13:15:52.925995 (+ 30353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.078802 (+152807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:53.078857 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:53.078929 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:53.078983 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.284353 (+205370us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400342954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:53.284409 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:53.284560 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":350644608}
I20260709 13:15:53.285841  3512 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:53.199883  3591 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.286453  3512 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:53.286339  3374 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:15:53.262248 30700 rpcz_store.cc:276] 0709 13:15:52.177479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.973820 (+796341us) service_pool.cc:224] Handling call
0709 13:15:52.983644 (+  9824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:52.984087 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:52.984120 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:52.987873 (+  3753us) log.cc:844] Serialized 26 byte log entry
0709 13:15:53.166335 (+178462us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:53.166397 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.166505 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:53.261366 (+ 94861us) raft_consensus.cc:1685] finished
0709 13:15:53.261396 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:53.261575 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6533,"raft.queue_time_us":1623,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":8913,"thread_start_us":152904,"threads_started":1,"wal-append.queue_time_us":230906}
I20260709 13:15:53.279951  3501 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:53.279829  3619 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.267632 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:53.276492  3694 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:53.312218 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 147926us
I20260709 13:15:53.315448 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.315742  3690 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.316497  3690 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.319037  3690 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.321178 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:53.322054  3690 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:53.322672  3690 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.323137  3690 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:53.290447  3512 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.328514 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.329300 31389 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:15:53.330493 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:53.332520 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 130510us
I20260709 13:15:53.342984  3619 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.346575  3680 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.347292  3697 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.348009  3703 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:53.348523  3703 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:53.335387  3512 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:53.349509  3703 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.354851  3680 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.357849 31378 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:15:53.358281 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.362090 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:53.306702  3591 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:53.371326 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 95094.7us
I20260709 13:15:53.372745 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.373183 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:53.374648 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.363827 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.376667 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:53.376832 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.376598 31378 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:53.377931 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.372751 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:53.358595  3181 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:53.356885  3501 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.388293 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.389711 31388 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:53.389861 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.391323 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.372053  3619 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.393013 31367 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:53.393946 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.394579 31367 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:53.395092 31378 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:53.325927  3694 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.376997  3680 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:15:53.398363 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.387250  3697 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:53.401006 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.402185 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.374552  3508 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:15:53.418347  3508 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:53.418823  3508 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:53.419723  3508 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.397065  3619 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.424608  3619 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:53.402256  3697 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.426028  3697 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.419783  3487 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.439548  3691 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:53.440840  3487 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.405218 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.408905  3562 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.409750  3391 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.413024  3591 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.392624  3501 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.455639  3391 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.455896  3501 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.458133  3099 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.459668 31391 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:53.461958  3099 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.421137  3508 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.424424  3694 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.431452 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1716) took 1138 ms. Trace:
I20260709 13:15:53.470585 30629 rpcz_store.cc:276] 0709 13:15:52.293484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.473888 (+180404us) service_pool.cc:224] Handling call
0709 13:15:52.474238 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:52.693755 (+219517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.431102 (+737347us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54005f3cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:53.431368 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503279872}
W20260709 13:15:53.434716  3563 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.800s	user 0.012s	sys 0.003s
I20260709 13:15:53.471910  3631 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:53.476413  3551 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:53.478703 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.480892 30705 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:53.445874 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1600) took 1509 ms. Trace:
I20260709 13:15:53.498831 31383 rpcz_store.cc:276] 0709 13:15:51.936546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.936746 (+   200us) service_pool.cc:224] Handling call
0709 13:15:53.445640 (+1508894us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540057f154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:53.445806 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224244096}
I20260709 13:15:53.500371 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.501111 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:53.502012 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.444206 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:53.503059 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58982.8us
I20260709 13:15:53.374058  3239 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.446429 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:53.449175 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:53.449376 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.451421 30703 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:53.386554  3181 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:53.486104 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:53.515043 30627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1706) took 1530 ms (client timeout 1743 ms). Trace:
W20260709 13:15:53.515702 30627 rpcz_store.cc:269] 0709 13:15:51.984370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.122518 (+138148us) service_pool.cc:224] Handling call
0709 13:15:53.446237 (+1323719us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:15:53.514961 (+ 68724us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8005248}
I20260709 13:15:53.486919 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:53.487941 30559 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
W20260709 13:15:53.490706 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1617) took 2290 ms (client timeout 1775 ms). Trace:
I20260709 13:15:53.505342 30550 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:53.505720 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.492057  3629 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.403137 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.516924 31396 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:53.519685 30558 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:53.510505 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:53.517905 31378 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:53.545289 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.548246 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.546526 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:53.549212 31367 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:53.550264 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.551527 30703 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [15.9-18.11]
I20260709 13:15:53.552706 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.553450 30550 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:53.553325 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:53.547583  3712 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 13:15:53.556010 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.557250 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:53.517270 30707 rpcz_store.cc:269] 0709 13:15:51.200150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.203809 (+  3659us) service_pool.cc:224] Handling call
0709 13:15:53.490377 (+2286568us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540062ff54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:53.490565 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3489499520}
I20260709 13:15:53.592044 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.593729 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.596609 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.594647 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.559827 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.544531  3181 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.597759 31361 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:53.562124 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:53.600893 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:53.598943 31378 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:53.595885 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:53.596539 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:53.593535 30703 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:53.608014 31383 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:53.611371 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:53.594156  3600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.440s	user 0.012s	sys 0.015s
I20260709 13:15:53.609795 31367 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:53.547292  3683 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.544526  3569 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:53.621758 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.622470 30707 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:53.607964 30550 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:53.614567 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:53.618124 30698 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:53.618592 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:53.559938  3487 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:53.549875  3496 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:53.621104 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 243929us
I20260709 13:15:53.621347 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.633468 30552 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:15:53.622776 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:53.623976 31378 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:53.558204  3684 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:15:53.630720 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:53.624250 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:53.555940  3691 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:15:53.637733 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:53.640267  3629 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.641127  3680 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:15:53.621222  3600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.466s	user 0.013s	sys 0.015s
I20260709 13:15:53.595651  3562 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.643409 31383 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:53.632881  3619 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:53.619315  3694 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:53.619079  3638 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:53.618204  3591 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.651481 30550 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:53.619633  3099 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:53.651965 30556 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:15:53.632426  3487 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.657050 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:53.637925 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.674796  3562 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.652992  3551 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.652194  3684 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:53.651290  3391 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:53.882478 30551 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:53.878404 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:53.833030  3690 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.009s
I20260709 13:15:53.636174  3683 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.654484  3680 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:53.645706  3496 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:53.803889 30553 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:15:53.627959  3569 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.804582  3253 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.919533  3253 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:53.920217  3562 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.907889  3629 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:15:53.920982  3697 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:53.867846 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.921993  3253 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:53.875131 31383 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:53.888692 30523 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425): Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 1.784s (SENT)
I20260709 13:15:53.874840  3099 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.874390  3487 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:53.695497  3710 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:15:53.874351  3607 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:53.876030  3619 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.870570  3694 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:53.874599  3638 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.843413 30552 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:53.895654 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:53.934360 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:53.935166 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:53.895802 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:53.882248  3514 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.008s	sys 0.010s
I20260709 13:15:53.909778 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:53.913188 30551 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:15:53.918742  3569 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:53.903203  3501 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:53.831164 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:53.975523 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.926033 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:53.804662  3239 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.931809 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:53.937273 30627 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:53.883435  3600 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.933360  3619 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:53.982959  3600 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.984163  3550 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:53.985884  3600 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:53.945041  3253 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:53.928984  3099 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:53.945293 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49760.3us
I20260709 13:15:53.947171  3694 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.992108 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:53.993408  3253 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:53.903087  3690 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.926767  3496 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.896147  3494 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:53.948267 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 947) took 2728 ms. Trace:
I20260709 13:15:53.922071  3684 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:53.949397  3716 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:15:53.950274 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:53.950907 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:53.954897 31396 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1783) took 1597 ms (client timeout 1580 ms). Trace:
I20260709 13:15:53.955610  3622 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:53.958776  3607 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:53.960604 31383 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:53.962867  3697 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:53.963784  3680 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:53.964252 31367 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:53.964632  3616 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:53.958599 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2056) took 1663 ms. Trace:
I20260709 13:15:54.002033 30629 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:54.002241 30552 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:54.009116 30702 rpcz_store.cc:276] 0709 13:15:52.294760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.149708 (+854948us) service_pool.cc:224] Handling call
0709 13:15:53.150184 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.254868 (+104684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.958213 (+703345us) spinlock_profiling.cc:243] Waited 90.9 ms on lock 0x7b5400222ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:53.958511 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208955136}
W20260709 13:15:54.011198 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:54.002112 31396 rpcz_store.cc:269] 0709 13:15:52.357673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.357960 (+   287us) service_pool.cc:224] Handling call
0709 13:15:53.954605 (+1596645us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540044e654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:53.954789 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2468094208}
I20260709 13:15:53.996420 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.016649 30705 raft_consensus.cc:2368] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
W20260709 13:15:54.031385 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52861.3us
W20260709 13:15:54.031831 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 224547us
I20260709 13:15:54.032518 31361 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:53.994500 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:54.033986 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:53.997922 30557 rpcz_store.cc:276] 0709 13:15:51.220194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.229327 (+  9133us) service_pool.cc:224] Handling call
0709 13:15:51.927150 (+697823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:51.969274 (+ 42124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:53.947986 (+1978712us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b5400358750. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:53.948190 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603827072}
I20260709 13:15:54.042833 30625 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:54.042815 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.045047 31361 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:15:53.971338  3514 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.691s	user 0.008s	sys 0.011s
I20260709 13:15:53.972052  3638 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:53.878397  3591 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.065341 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.067914 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:54.030932  3501 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:53.996961  3690 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.004920  3607 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:54.099426 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.071276 30625 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:15:53.997704  3494 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.071228  3563 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:54.002460  3622 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.079628 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:54.081137 30555 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:15:54.048683  3562 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:15:54.087126 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:54.141741 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54727.7us
I20260709 13:15:54.088160  3703 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:54.088938 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.056083 30705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:54.090004 31396 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:54.144927 30705 raft_consensus.cc:606] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:15:54.093119  3253 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:54.093402 30624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1227) took 2112 ms (client timeout 2244 ms). Trace:
W20260709 13:15:54.149514 30624 rpcz_store.cc:269] 0709 13:15:51.981386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:51.983539 (+  2153us) service_pool.cc:224] Handling call
0709 13:15:54.093134 (+2109595us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b5400410fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:54.093305 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1023449984}
I20260709 13:15:54.094338  3708 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:54.151664 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:54.095861 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.097103 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.098313 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.106055 30698 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:15:54.103775 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.155058 30702 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:54.104451 30626 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:15:54.108537 31383 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:54.120621 31378 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:54.156665 31361 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:15:54.124264 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:53.995433  3619 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.143822  3512 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:54.152278  3253 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:54.161283 30555 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
W20260709 13:15:54.100018  3631 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.627s	user 0.016s	sys 0.001s
I20260709 13:15:54.135015 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:54.136184 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.072830  3697 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:54.109335 31394 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:54.105592  3691 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:54.144469 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.107362  3501 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.134713  3181 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:54.145464 30703 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:15:54.064497  3190 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:15:54.140614 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.146659 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.147732 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.145339 30705 raft_consensus.cc:614] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:54.180914 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:54.300945 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.165735 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:54.242035 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 319745us
I20260709 13:15:54.275761 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:54.328162 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.283295 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.329721 30633 raft_consensus.cc:2368] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14.
I20260709 13:15:54.325063 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:54.165838 30698 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:54.182186 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.164093 30701 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:54.334160 30631 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:54.169799 30699 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:54.337103 30625 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:54.302440 30552 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:54.325836  3607 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:54.340061 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:54.161757  3253 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:54.381915 31396 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:54.383278 30554 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:54.384003 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:54.384374 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.385076 30553 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:54.385617  3501 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.386451 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:54.387893 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:54.388469 30707 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:54.381196  3562 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.389267  3562 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.386360  3550 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:15:54.392025  3550 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:54.196941  3622 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.163654  3569 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.398043 30702 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:54.391595 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.392441  3550 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:54.348353  3638 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.174237  3099 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:54.339761  3551 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.165230  3696 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.257s
I20260709 13:15:54.167208  3619 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.220510  3439 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.402594  3607 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.424432  3439 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.173062  3494 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.169392  3697 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:54.426213  3607 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.425932 30703 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:54.426630  3697 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:54.439519 30631 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:54.407912 30702 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:54.440986 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:54.410825 30701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2059) took 2040 ms (client timeout 1757 ms). Trace:
W20260709 13:15:54.443027 30701 rpcz_store.cc:269] 0709 13:15:52.398068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.559389 (+161321us) service_pool.cc:224] Handling call
0709 13:15:54.410633 (+1851244us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400435654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:54.410749 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3554703104}
I20260709 13:15:54.437692 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:54.417351 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.445528 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.446297 30624 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:54.418248 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.220463  3616 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:54.418598 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.418984 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.436403 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:54.450489 31378 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:54.456913 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:54.436911 30554 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:15:54.460357 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.460989 30707 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:54.463626 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:54.465171 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.466804 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.467134 30700 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:15:54.250667  3391 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:54.415496 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.197021  3694 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:54.406293 30698 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:54.478063 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.416834 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.478570 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:54.474056 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.480022 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:54.382572  3588 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:15:54.483604 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.423085  3562 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.172972  3691 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.501585 30551 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:54.501668  3253 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.407173  3404 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.096s
I20260709 13:15:54.502822  3691 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.407411  3591 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.408609  3683 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.503368  3404 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:54.503851  3404 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:54.473395 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.405850  3631 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:54.488499 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.490067 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.492290 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.492699  3181 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.492795  3635 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:54.493176  3391 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:54.419938  3559 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:54.494379 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.410844  3550 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:54.494696  3487 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.523s	user 0.012s	sys 0.000s
I20260709 13:15:54.495649 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:54.477854  3719 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:54.499080  3439 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.487998  3588 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:54.482139  3619 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:15:54.481455  3659 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:54.419102  3724 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:15:54.500291 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.500626  3694 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.509104 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:54.508983 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.535856  3649 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:54.548525 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.544646  3619 consensus_queue.cc:1243] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:15:54.418866  3638 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.421236  3099 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:54.510509 31389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.018s	sys 0.022s
I20260709 13:15:54.507877 30627 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:54.508059  3613 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:54.512394 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.513343 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:54.504211  3253 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.521178 30630 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:54.422029  3551 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.475083 30703 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:15:54.529623 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.533692 31378 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:54.528214 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:54.510015  3496 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:54.536332 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:54.641261 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 113079us
I20260709 13:15:54.347901  3508 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 13:15:54.623119 30633 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:54.642741 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:54.507807  3648 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:15:54.482455  3616 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.521126  3635 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:54.509320 30553 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:54.527849  3536 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:15:54.553104 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.010s	sys 0.034s
I20260709 13:15:54.552523 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.545627  3550 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:54.576213 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:54.550902 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 447393us
I20260709 13:15:54.576095 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.577075 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.664640 30625 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:54.653858  3344 raft_consensus.cc:2764] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:15:54.528163  3487 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.537142  3404 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.670444  3404 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:54.671896  3487 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:54.555155  3684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.007s	sys 0.021s
W20260709 13:15:54.674888  3684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.008s	sys 0.022s
I20260709 13:15:54.675930 30706 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:15:54.677376 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 875) took 1129 ms. Trace:
I20260709 13:15:54.677757 30706 rpcz_store.cc:276] 0709 13:15:53.547899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.553138 (+  5239us) service_pool.cc:224] Handling call
0709 13:15:54.677140 (+1124002us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540032ad54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:54.677308 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479809920}
I20260709 13:15:54.545359  3659 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.679633 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.551647  3725 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.680392 30553 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:15:54.680804 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.607950  3694 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.540413  3719 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.540917  3439 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:54.718694  3439 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:54.611711 31389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.018s	sys 0.022s
I20260709 13:15:54.613183 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.720060  3719 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.720618 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1613) took 1277 ms. Trace:
I20260709 13:15:54.618117 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.721061 31389 rpcz_store.cc:276] 0709 13:15:53.443413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:53.804358 (+360945us) service_pool.cc:224] Handling call
0709 13:15:53.804708 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:53.805235 (+   527us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:53.875302 (+ 70067us) log.cc:844] Serialized 14 byte log entry
0709 13:15:54.720249 (+844947us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:54.720308 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:54.720359 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.720441 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:54.720561 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":278658,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"thread_start_us":913975,"threads_started":2,"wal-append.queue_time_us":785016}
W20260709 13:15:54.621531 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 108297us
W20260709 13:15:54.626874 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 471745us
I20260709 13:15:54.686336  3181 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:54.723819  3181 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.725301 30627 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:15:54.725770 30627 pending_rounds.cc:85] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:15:54.726015 30627 pending_rounds.cc:107] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:15:54.507326  3631 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:54.726258 30627 raft_consensus.cc:2889] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:54.643973 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.649356 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:54.723806 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:54.727833 30630 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:15:54.662093 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.010s	sys 0.034s
I20260709 13:15:54.610579  3569 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.551569  3649 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:54.681703 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:54.682233 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:54.731493 30554 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:54.732851 30553 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:54.684162 30625 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:54.685447 30556 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:54.652103  3600 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:54.735097  3344 raft_consensus.cc:2764] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:15:54.689782  3725 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.694352 30626 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:15:54.695047 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:54.664333  3253 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:15:54.737469 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55283.8us
I20260709 13:15:54.737838 30631 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:54.703276  3514 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
W20260709 13:15:54.538913  3556 consensus_queue.cc:1175] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 13 status { last_received { term: 12 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:15:54.599664  3588 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.717221  3616 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.662645  3635 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:54.719558  3696 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.729790 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:54.726305  3181 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:54.738440 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 861430us
I20260709 13:15:54.755508 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.755229 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.737926  3508 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:54.755229 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.758395 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.755198 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:54.739655  3253 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:54.737351  3649 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:54.763481 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:54.769421 31389 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:54.772290 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:54.773957 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:54.774914 31364 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:54.776046 30554 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:54.736708  3717 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:15:54.740775  3729 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:54.743726  3569 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:54.745605  3588 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:15:54.736574  3694 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.855903 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:54.857317 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:54.784089 31378 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:54.781081 31389 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:54.767750  3181 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.748420  3514 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:54.867122 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.868403 31389 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:54.741922  3691 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:15:54.860244 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.873008 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.778748  3607 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:54.877966 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.746373  3697 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:54.875514 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.882038 31389 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:15:54.865167 31292 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:15:54.884493 31292 leader_election.cc:403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:15:54.885053 30703 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:15:54.797072 31391 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:54.758948  3696 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:54.887001 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.892648 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:54.894804 31388 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:54.759397  3631 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.869621  3708 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:54.860721  3591 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:54.900287 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 430844us
I20260709 13:15:54.865846  3736 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.909842 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:54.761731  3512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.001s	sys 0.016s
I20260709 13:15:54.903479 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:54.948148 31367 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:15:54.948662 31367 pending_rounds.cc:85] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:15:54.948881 31367 pending_rounds.cc:107] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:15:54.949131 31367 raft_consensus.cc:2889] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:54.869441  3181 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.911614 30703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:54.964596 30703 raft_consensus.cc:606] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:15:54.901485  3704 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:54.764443  3719 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:15:54.968973  3703 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:54.757465  3616 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:54.970321  3616 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.973353  3616 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:54.881606  3374 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:54.975150  3374 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:54.978210  3374 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:54.875675  3514 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.012567  3616 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:54.947028  3496 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:54.960848 30704 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:55.014966  3616 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:54.962093 31388 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:54.907145  3708 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.015455  3616 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.015156  3607 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:55.015723  3708 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.016124  3607 consensus_queue.cc:1243] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:15:54.862730  3569 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.016458 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:55.018442  3569 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:54.983534  3727 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:15:55.021121  3496 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:54.957131  3648 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:55.022857  3648 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.022650  3496 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:55.002581 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.027930  3439 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:54.959021 30706 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.013808  3514 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:54.901871  3253 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.030871  3253 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:55.013584  3181 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.041358 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:54.910676 31391 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:15:55.042485 30707 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:55.043783 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.044713 30703 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:54.919857  3619 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:15:55.052454 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:54.967113  3512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.002s	sys 0.016s
I20260709 13:15:55.064885  3648 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.065887  3697 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:55.068274 30704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1629) took 2366 ms (client timeout 1680 ms). Trace:
I20260709 13:15:55.070276  3696 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:55.072404 30671 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.534s (SENT) [suppressed 13 similar messages]
W20260709 13:15:55.129098 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.054070 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:55.136893 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82939.9us
W20260709 13:15:55.120658 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 439986us
I20260709 13:15:55.062027  3616 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:54.994115  3725 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:55.073410  3694 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:55.134550 30704 rpcz_store.cc:269] 0709 13:15:52.702162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:52.715035 (+ 12873us) service_pool.cc:224] Handling call
0709 13:15:55.067972 (+2352937us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b540037d2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:55.068164 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5087991552}
I20260709 13:15:55.047643  3514 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:55.058147  3494 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:15:55.153090 31364 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
W20260709 13:15:55.147251 30550 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.000s	sys 0.013s
W20260709 13:15:55.145622 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.020s	sys 0.084s
I20260709 13:15:55.115255  3374 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:15:55.154277 31378 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
W20260709 13:15:55.154325 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.021s	sys 0.084s
W20260709 13:15:55.154851 30550 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.000s	sys 0.014s
I20260709 13:15:54.917635  3591 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.135701  3733 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.133958  3512 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:55.140498  3501 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:54.921049  3736 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.163738 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1770) took 1014 ms. Trace:
I20260709 13:15:55.186756 30550 rpcz_store.cc:276] 0709 13:15:54.148963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.150329 (+  1366us) service_pool.cc:224] Handling call
0709 13:15:54.150708 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:54.150958 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:54.553941 (+402983us) log.cc:844] Serialized 14 byte log entry
0709 13:15:55.156554 (+602613us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:55.156645 (+    91us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:55.156722 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.156830 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:55.163636 (+  6806us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":531229,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"thread_start_us":991133,"threads_started":2}
I20260709 13:15:55.191818 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.180348 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:55.193758 31396 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:55.149775  3691 consensus_queue.cc:1243] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:55.200062 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.201028 30703 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:55.198789 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.203747 30706 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.158558  3374 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.237389 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.190026  3501 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.244640 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.247253 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.171008  3514 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.239410  3619 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.239557  3374 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:55.265195 30624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.013s	sys 0.075s
W20260709 13:15:55.265702 30624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.013s	sys 0.075s
I20260709 13:15:55.244228  3501 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:55.240463  3691 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:55.272303 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.256783  3569 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:55.273394  3569 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:55.274024  3569 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:55.268503 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 498446us
I20260709 13:15:55.256439  3559 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:55.190476  3733 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.281293  3733 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.282781 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:55.285809 30550 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 12: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:55.257030  3439 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:15:55.302938 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:55.306782 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 340064us
W20260709 13:15:55.298887 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 409621us
I20260709 13:15:55.315016  3591 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.317201  3514 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:55.310097  3635 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.546s	user 0.007s	sys 0.007s
I20260709 13:15:55.319101  3635 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:55.286098  3691 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.289805  3559 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:55.270138  3619 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:55.332276 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 259910us
I20260709 13:15:55.334980 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:55.335368 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.336144 30706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:55.336807 30706 raft_consensus.cc:606] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:15:55.337206 30706 raft_consensus.cc:614] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:55.329689 30704 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.324913 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:55.318506 31396 pending_rounds.cc:85] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:15:55.337980 31396 pending_rounds.cc:107] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:15:55.338289 31396 raft_consensus.cc:2889] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:55.305454  3725 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:55.333446 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:55.329460 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 587655us
I20260709 13:15:55.340641 30551 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:15:55.321990  3635 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.310199  3344 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:55.285653  3512 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.348976 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:55.346894 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 202595us
I20260709 13:15:55.352490 30553 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:55.345060 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:55.345639 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.330899 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:55.359891 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 212833us
W20260709 13:15:55.364126 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.366345 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.366807 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.367358 30703 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:55.367774 30707 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:55.361042 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:55.369660 30706 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:55.371590 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:55.372346 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.383080 30553 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:55.387406 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.388484 30703 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.334879  3753 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:15:55.389819 30673 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.373713 30707 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.350492  3344 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.392359 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.393141 30703 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:55.395254 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.398370 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.369915  3752 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:55.346148  3725 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.400421 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.442881 30706 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
W20260709 13:15:55.447059 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46719.4us
W20260709 13:15:55.446909 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 170652us
I20260709 13:15:55.374150  3562 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:55.386520  3439 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:55.473403 30706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:55.474072 30706 raft_consensus.cc:606] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:15:55.474300 30706 raft_consensus.cc:614] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:55.374482  3616 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:15:55.453052  3745 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:15:55.452677  3760 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:55.453557  3753 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:55.396337  3648 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:55.454877  3754 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:55.456156  3344 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.388950  3635 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 13:15:55.490945 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 104625us
I20260709 13:15:55.464649  3729 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:55.465065  3704 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
W20260709 13:15:55.413429  3742 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:15:55.408618  3514 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:55.465901  3733 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:55.466848 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.467896  3659 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.468878  3694 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:55.470928  3691 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.471007 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1777) took 1221 ms. Trace:
I20260709 13:15:55.471626  3591 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:55.472775 30630 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:15:55.480108 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.403215  3099 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:15:55.483186 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 118605us
I20260709 13:15:55.391921  3559 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:55.451014  3740 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.202s	user 0.000s	sys 0.002s
I20260709 13:15:55.518406  3550 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:55.524070  3514 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.514945  3729 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:55.524459  3733 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.514782  3760 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:15:55.443475  3736 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.528664  3344 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:55.470213  3374 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.531306  3697 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:55.486963  3600 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:15:55.470222  3569 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:55.518858 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:55.524926 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58183.1us
I20260709 13:15:55.535094  3704 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:15:55.526897 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:15:55.571403 30698 raft_consensus.cc:606] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:15:55.571861 30698 raft_consensus.cc:614] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:15:55.528246 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:55.486603  3746 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:15:55.568230 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:55.557459 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 221247us
I20260709 13:15:55.577358 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:55.577847 30707 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:55.578132 30702 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:55.496162  3616 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.538552 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1627) took 1104 ms. Trace:
I20260709 13:15:55.536896 31388 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:55.532527 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:55.570302 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1249) took 1076 ms. Trace:
I20260709 13:15:55.538419  3514 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.540319  3631 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:55.569129 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:55.556648 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.527410 30556 rpcz_store.cc:276] 0709 13:15:54.249907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.339050 (+ 89143us) service_pool.cc:224] Handling call
0709 13:15:54.339472 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:54.734642 (+395170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.470652 (+736010us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400112954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:55.470925 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":782856448}
I20260709 13:15:55.594586 30703 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:55.570024 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51290.3us
W20260709 13:15:55.573794 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45672.6us
I20260709 13:15:55.596624 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:55.555359 31361 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:55.542712  3551 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.520913  3725 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:55.552209  3683 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.536468  3099 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.537947  3550 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.570586 31378 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:15:55.554183  3253 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.004s
W20260709 13:15:55.591995 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59569.9us
I20260709 13:15:55.514480  3754 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:55.581264  3756 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:15:55.548674  3374 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.554947  3619 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.528580  3591 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:55.649981 31364 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:55.660580  3099 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:55.657289 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.533964  3694 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.582605  3762 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:55.549401  3697 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.588690 30624 rpcz_store.cc:276] 0709 13:15:54.494071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.721855 (+227784us) service_pool.cc:224] Handling call
0709 13:15:54.722200 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.723969 (+  1769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:54.724006 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:54.724581 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.266204 (+541623us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:55.266266 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.266344 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:55.570069 (+303725us) raft_consensus.cc:1685] finished
0709 13:15:55.570096 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:55.570237 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5565,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8067,"thread_start_us":541168,"threads_started":1,"wal-append.queue_time_us":614447}
I20260709 13:15:55.625442 31396 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:55.672291 30523 leader_election.cc:400] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:55.693002 30523 leader_election.cc:403] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:15:55.641177  3753 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:15:55.697566 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51067.4us
I20260709 13:15:55.700830 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:55.675738 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:15:55.541157  3733 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.601459  3749 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:55.676206 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79750.7us
I20260709 13:15:55.590067  3648 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:55.583582  3719 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:55.599802  3404 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
W20260709 13:15:55.664321  3649 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.816s	user 0.010s	sys 0.005s
I20260709 13:15:55.712806  3550 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.712919  3649 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.539803  3729 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.643579  3611 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:55.586360 31394 rpcz_store.cc:276] 0709 13:15:54.434388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.434541 (+   153us) service_pool.cc:224] Handling call
0709 13:15:54.445663 (+ 11122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:54.446114 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:54.446142 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:54.446827 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:15:54.918746 (+471919us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:54.918801 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:54.918898 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:55.538274 (+619376us) raft_consensus.cc:1685] finished
0709 13:15:55.538301 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:55.538476 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":403779,"threads_started":1,"wal-append.queue_time_us":659061}
I20260709 13:15:55.589044  3704 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:55.715269  3649 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.716246  3719 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:55.716818  3719 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.716583  3704 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.686950  3591 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:55.544175  3767 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:15:55.596107  3616 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.704562 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:55.699383 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:55.708693 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80349.6us
I20260709 13:15:55.644942  3747 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 13:15:55.661520  3588 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.758s	user 0.009s	sys 0.006s
I20260709 13:15:55.659615  3683 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.715358  3729 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:55.712323  3569 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.724507 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:55.735612 30550 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:55.724993  3648 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:55.737977  3683 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.723183  3611 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.723205  3765 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:55.738930  3765 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:55.739305  3765 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:55.742877  3743 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:55.743412  3743 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.744068  3616 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.744366  3743 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.745941  3743 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:55.747252 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1251) took 1252 ms. Trace:
I20260709 13:15:55.747709 30630 rpcz_store.cc:276] 0709 13:15:54.494703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.728465 (+233762us) service_pool.cc:224] Handling call
0709 13:15:55.472374 (+743909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:55.528734 (+ 56360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:55.747003 (+218269us) spinlock_profiling.cc:243] Waited 743 ms on lock 0x7b54004abfd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:55.747178 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710001920}
I20260709 13:15:55.749434  3697 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.710764  3749 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.702807  3600 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:15:55.752040  3749 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:55.733811  3762 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:55.709126  3494 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:15:55.753561  3619 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.755155 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:15:55.755448  3694 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:55.710779  3753 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:55.756460  3717 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:15:55.756924  3717 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.756850  3753 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.757671 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:15:55.707871  3691 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:15:55.699548  3640 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:55.702483  3769 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:55.765144 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.709326 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.764210 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.764503 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.729449  3767 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:15:55.730685  3607 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:55.754098  3619 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:55.777107 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:55.777671 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49612.8us
I20260709 13:15:55.807169 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:15:55.777730 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49408.4us
I20260709 13:15:55.785800 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:55.773348 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 64108.6us
I20260709 13:15:55.791074 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:55.781150 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51258.5us
W20260709 13:15:55.742455  3708 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.001s
I20260709 13:15:55.760143  3569 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:15:55.792714 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.819922 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:55.776850  3694 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.785298  3600 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:55.758074  3717 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.821893  3696 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:55.827132 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:55.866286 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59142.6us
I20260709 13:15:55.827829 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:55.786089  3374 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:55.828472 31378 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:55.771076  3772 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:55.832670 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:55.783603  3536 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:15:55.829865 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.829133 31294 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:55.800067  3607 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.878679 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.725230  3754 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.839381 30705 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:15:55.839573 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:55.882503 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53481.9us
I20260709 13:15:55.847102 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.842308 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:55.825287 30550 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:55.883412 30700 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:55.859892 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:55.780653  3404 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:15:55.860908 30699 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:15:55.886308 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:55.886942 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.780324  3747 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:15:55.821266  3640 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:55.871917 30701 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:15:55.864450 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0329325780001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2006s for a response
W20260709 13:15:55.889498 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0140325780000694s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 13:15:55.875016 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.875322 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:55.799738  3767 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.879483 30707 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 11.
W20260709 13:15:55.865332  3501 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.579s	user 0.007s	sys 0.007s
I20260709 13:15:55.846527  3494 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:55.840359 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:55.848359  3600 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.884795 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:55.884465 30698 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:15:55.891089 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48858.7us
I20260709 13:15:55.883466 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:55.869752  3374 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:55.880961  3696 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:55.911080 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:15:55.899981 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59722.6us
I20260709 13:15:55.913166 30701 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:55.913285 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:55.865686  3649 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:15:55.861596  3694 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.838047  3708 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.885403  3622 raft_consensus.cc:2764] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:15:55.846777  3611 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.902860  3754 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:55.909421  3374 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:55.914500 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:55.892107  3640 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:55.889881  3697 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:15:55.871749  3765 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:55.891676  3762 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:55.922816 30698 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: [15.9-19.11]
W20260709 13:15:55.909981 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90195.5us
I20260709 13:15:55.897009  3775 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:15:55.916417  3496 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:55.918149  3404 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.918493  3344 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:15:55.907404  3494 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:55.942219 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:55.885639  3772 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:15:55.945233 31389 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:55.917086  3391 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:55.903697 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:55.946997 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56906.4us
I20260709 13:15:55.947471  3640 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:55.902288  3501 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.951149  3697 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:55.948771 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:55.919485  3607 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.952895  3344 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:55.989678 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:55.911180  3691 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:15:55.960417 30550 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
W20260709 13:15:55.966563 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:55.967571  3616 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:55.922043  3600 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.948691  3782 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:56.032850 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.033759 30555 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:55.926074  3719 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.941833  3746 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 13:15:55.951954  3619 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:55.937417  3747 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:56.049635 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:55.987444 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.050403 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:55.987888 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:55.990811 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:55.991541  3391 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:55.992265  3607 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:55.992417 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43747.6us
I20260709 13:15:55.924779  3767 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:55.954632  3775 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:55.998848  3536 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.061540  3536 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.000365  3708 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:56.001180  3508 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:15:56.002732  3771 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:15:56.062405  3391 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.064276  3659 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:56.064848  3659 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:56.067341  3778 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:15:56.075242  3775 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.076511  3404 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:56.004021  3717 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:56.003796  3344 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:56.006462 30671 leader_election.cc:336] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.743s (SENT)
W20260709 13:15:56.006783 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.086795 30555 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:56.087751 30702 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:55.959128  3496 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:56.065843  3659 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.010994 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:56.012302  3773 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:56.012487 31367 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:56.019150 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:56.024096 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.040028 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.033407  3600 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:56.044220 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 99829.7us
W20260709 13:15:55.986601 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82967.2us
I20260709 13:15:56.019435  3736 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:15:56.034823  3691 consensus_queue.cc:1243] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:15:56.056108 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:56.083490 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 242090us
I20260709 13:15:56.084860 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:56.085245 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:56.088689 30703 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:15:56.088791 31383 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:56.120900 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.086871 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.089808 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.090478 30631 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 10. Current term is 11. Ops: []
I20260709 13:15:56.091895 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.122992 30628 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:56.124155 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.092669 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:56.124847 30628 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:56.125615 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:56.093252 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.125974 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.009431 31388 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:15:56.017570  3774 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:15:56.096184 30555 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:56.088119 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:55.980057  3696 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.101827 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:56.103107 31292 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.976s (SENT)
I20260709 13:15:56.104497 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.104655 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.115329 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.145185  3747 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:56.146224 30633 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:15:56.038234  3616 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.089243 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.092432 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.147720 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:56.128312 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109300us
I20260709 13:15:56.149616 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:56.150235 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.123478 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.150269 30633 raft_consensus.cc:2376] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20. Re-sending same reply.
I20260709 13:15:56.151701 30631 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:15:56.151778 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.152982 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.136597 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.156177 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.138012 30551 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: []
W20260709 13:15:56.139107 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51104.6us
I20260709 13:15:56.140578 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.156800 30631 raft_consensus.cc:2376] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20. Re-sending same reply.
W20260709 13:15:56.140802 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 134394us
W20260709 13:15:56.136633 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 125788us
W20260709 13:15:56.141871 30523 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 2.244s (SENT)
I20260709 13:15:56.157739 30523 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.154239 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:56.137336 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.160526  3494 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.141093  3514 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.163514 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.166429 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.169124 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.101161  3501 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:56.135625  3767 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:56.119278  3736 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:56.222244 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.135731  3773 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:56.140348  3344 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:15:56.245376 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60709.4us
I20260709 13:15:56.231148 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.194211 30633 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:15:56.232164 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 90307.2us
I20260709 13:15:56.098328  3755 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:15:56.154951  3439 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:56.146518  3747 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.154713  3619 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:56.148061  3769 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:56.234848 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.249058 30557 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:15:56.094306  3640 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.246455 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:56.186908  3753 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.222316  3674 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:56.230016  3739 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:56.270295 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.100770  3697 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.239221  3736 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:56.288841 30625 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:56.250504 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:56.238763  3659 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:56.254176 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.222826  3494 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:15:56.279554  3739 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:56.280066  3099 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
W20260709 13:15:56.282310  3591 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.003s
W20260709 13:15:56.310292 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59868.7us
I20260709 13:15:56.252586  3619 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.278644  3753 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.257468  3769 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.314899 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.238356  3496 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:56.317171 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72796.1us
I20260709 13:15:56.283082  3773 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.319546 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:56.284852  3762 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:15:56.340430 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 113460us
I20260709 13:15:56.273765  3797 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:15:56.331633 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.318117  3775 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:56.313267  3694 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.355404 30551 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:56.313431  3765 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.315225  3782 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:56.359541 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:56.365542 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.330561  3719 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.331826  3755 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.315629  3648 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:15:56.363171  3508 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:15:56.368466  3788 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:15:56.329530  3619 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:56.398247  3611 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:56.401113 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:56.403748 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:56.404305  3536 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.357616  3494 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:15:56.369328  3674 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:15:56.370398 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:56.408223 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 261405us
W20260709 13:15:56.408747 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 301394us
I20260709 13:15:56.446278  3694 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.446970  3694 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.447602  3631 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:56.449927 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:56.412115  3787 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.328s	user 0.000s	sys 0.001s
W20260709 13:15:56.413839 31383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1813) took 1661 ms (client timeout 2064 ms). Trace:
W20260709 13:15:56.451187 31383 rpcz_store.cc:269] 0709 13:15:54.752762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.752892 (+   130us) service_pool.cc:224] Handling call
0709 13:15:56.413606 (+1660714us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400680454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:56.413762 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3042852096}
I20260709 13:15:56.416579  3404 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:56.351970  3691 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:56.323848  3099 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.334830  3769 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:56.326772  3591 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:56.425726  3793 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:15:56.438587  3753 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.438374 31364 raft_consensus.cc:2368] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:15:56.438367 31367 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:56.410244  3391 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:15:56.411379  3514 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:56.450351  3512 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:56.422883  3439 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:56.429033  3508 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:56.484128 31396 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:56.368531  3374 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.447574 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:56.449702  3694 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.458725  3691 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:56.461393 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90568.1us
W20260709 13:15:56.426383  3795 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:15:56.469599 30551 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:56.479059  3559 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:15:56.459841  3616 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:56.480273  3099 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:56.480728 30626 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 13:15:56.459352  3765 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:56.481166 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.481508 31361 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:56.481921 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.457911  3404 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:56.442698  3802 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:15:56.467357  3494 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:56.487576 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.493921  3691 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:56.476038  3253 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:56.496208 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.494212 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:56.481969  3512 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:56.500984 31389 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:56.504828 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50486.2us
W20260709 13:15:56.495395  3729 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.007s
I20260709 13:15:56.437021  3775 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:56.490252  3514 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:56.493340  3640 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.512136 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:56.514139 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.519244 30558 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:56.531347 30706 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:15:56.426162  3755 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.526211  3514 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:56.524370  3753 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:15:56.510883  3494 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:56.542697  3439 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.536787  3512 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.543653  3494 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.545135  3512 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:56.534555  3755 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.548099 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:56.506575  3739 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.548362 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:56.549459 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:56.552886 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.513126  3099 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:56.560140  3391 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:15:56.562757  3704 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:56.563095  3391 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:56.519488  3775 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:56.570665 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:15:56.536886  3743 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.791s	user 0.016s	sys 0.000s
I20260709 13:15:56.526433  3674 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:56.509402  3765 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.537523  3514 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:56.514325  3508 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.578991  3099 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:15:56.563838  3404 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.579993  3391 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.582441 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.526856  3569 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:15:56.598320 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.598327 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:56.572554  3775 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:15:56.589149  3754 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.646s	user 0.012s	sys 0.004s
I20260709 13:15:56.607507 30706 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:56.615121 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:56.622118 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:56.622648 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.628578 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:56.635129 30552 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:56.634214 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.649565 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:56.650466 31394 raft_consensus.cc:2368] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19.
I20260709 13:15:56.651157 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:56.658993 30559 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:15:56.603767  3607 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.660403 30631 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:15:56.661208 30698 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:56.666445 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:56.652484  3808 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.177s	user 0.002s	sys 0.000s
I20260709 13:15:56.609723  3591 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:56.673035 30703 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:56.670469 30705 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:56.688421 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 335312us
W20260709 13:15:56.670135  3806 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:15:56.672780  3404 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:56.690114 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:56.708755  3374 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:56.716579 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:56.673413  3607 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.722855  3569 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:56.693063  3802 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:15:56.735368  3600 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.609s	user 0.014s	sys 0.000s
I20260709 13:15:56.738564  3736 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.750555  3792 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:56.751274  3792 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:56.752287  3792 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.753368  3792 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:56.747864  3736 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:56.738160  3569 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:56.754510  3514 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.768074  3514 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:56.755934  3767 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.000s
I20260709 13:15:56.756261  3607 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:56.761888  3769 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:56.773087  3769 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.763984  3748 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:56.774078  3748 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:56.775087  3748 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.775377  3691 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.776717  3748 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:56.777208  3181 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:56.777757  3181 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:56.778671  3181 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:56.780133  3181 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:56.782105  3775 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:56.764278  3717 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.004s
I20260709 13:15:56.764566  3569 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:56.784858  3775 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.775666  3508 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:56.764914  3765 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:56.756042  3791 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.711s	user 0.007s	sys 0.000s
I20260709 13:15:56.787925  3691 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:56.753387  3619 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:56.792322  3619 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:56.764689  3253 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:56.793483  3253 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:56.793357  3569 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:56.795804  3569 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:56.893810  3556 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:56.895375  3556 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:56.840286 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.842495 30551 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:56.844324 30552 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
W20260709 13:15:56.844738 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:56.847249 30628 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:15:56.847883 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:56.848927 30629 raft_consensus.cc:2410] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:15:56.851519 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.860208 30633 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:15:56.862586 30554 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:56.735155 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:56.911386 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:56.913319 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 178123us
I20260709 13:15:56.866526 30555 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:56.867856 30379 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:15:56.836915 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:56.915069  3619 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:56.917981 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1641) took 1153 ms. Trace:
I20260709 13:15:56.916702 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.920194 30628 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:56.918740 31361 rpcz_store.cc:276] 0709 13:15:55.764119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.764243 (+   124us) service_pool.cc:224] Handling call
0709 13:15:56.917320 (+1153077us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400684054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:56.917638 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621210880}
I20260709 13:15:56.873744 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.877087 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.879792 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.883019 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.909677 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1816) took 1120 ms. Trace:
I20260709 13:15:56.909323  3774 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:56.922148 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:56.924376 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 470888us
I20260709 13:15:56.934639 30632 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:56.937120 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.940850 30624 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:56.931617 30631 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:15:56.925487 31396 rpcz_store.cc:276] 0709 13:15:55.788814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.792357 (+  3543us) service_pool.cc:224] Handling call
0709 13:15:56.901820 (+1109463us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54004f8654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:56.909358 (+  7538us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1488743296}
I20260709 13:15:56.934643 30628 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:56.956146 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 108952us
I20260709 13:15:56.959481  3649 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:15:56.961334 30550 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:56.961582 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:57.019547 30555 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:56.963642 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.025022 30631 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:56.964833 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1794) took 1313 ms. Trace:
I20260709 13:15:56.966187  3743 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:57.036480  3743 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:57.033659 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:56.971185 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.976886 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1824) took 1167 ms. Trace:
I20260709 13:15:57.043128 31389 rpcz_store.cc:276] 0709 13:15:55.809314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.079916 (+270602us) service_pool.cc:224] Handling call
0709 13:15:56.080310 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:56.520337 (+440027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.976061 (+455724us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54008b29d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:56.976560 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":963877632}
I20260709 13:15:57.046289 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:57.044387 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 386316us
I20260709 13:15:56.979739 31394 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:15:56.979460 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:57.050957  3747 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:15:57.051554 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 193213us
I20260709 13:15:57.053187 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:56.981565 30626 raft_consensus.cc:2368] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:57.054513  3496 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:15:57.055630 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.038254  3743 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:57.057832  3496 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:15:56.983350 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59184.9us
I20260709 13:15:56.983351  3725 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:15:56.990839 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:56.992965 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.062085 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:57.062402 31396 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:57.063989 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.065784 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:56.996219 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.067401 30633 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:57.067791  3747 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:57.069043 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:56.993716 30379 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:57.004665  3556 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.071520 30629 raft_consensus.cc:2376] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14. Re-sending same reply.
I20260709 13:15:57.074750 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:57.075795 31389 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.073819 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.084079 31396 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:15:57.085906 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:57.007705 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.007345 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.012503 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:57.031540 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.025800 30556 rpcz_store.cc:276] 0709 13:15:55.651659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:55.651946 (+   287us) service_pool.cc:224] Handling call
0709 13:15:55.652350 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:55.759635 (+107285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:55.759675 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:55.760243 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:15:55.884321 (+124078us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:15:55.884368 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:56.947580 (+1063212us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:15:56.947729 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:56.947894 (+   165us) raft_consensus.cc:1685] finished
0709 13:15:56.947923 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:56.964531 (+ 16608us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13594,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":24200,"spinlock_wait_cycles":245624704,"thread_start_us":122520,"threads_started":1,"wal-append.queue_time_us":184347}
I20260709 13:15:57.058519  3496 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:57.088573 30631 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:15:57.086660 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.078169 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.006018 30632 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:57.072748 30596 leader_election.cc:400] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19.
I20260709 13:15:57.097764 30596 leader_election.cc:403] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:15:57.060029  3743 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:57.117197 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 108498us
I20260709 13:15:57.119252 30551 raft_consensus.cc:2410] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:57.088716 30379 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.120965 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.123354  3099 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:57.124084 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.125504 30631 raft_consensus.cc:2410] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:15:57.126139 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1056) took 1008 ms. Trace:
W20260709 13:15:57.126807 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 80506.1us
I20260709 13:15:57.128862 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:57.130575 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:57.131740 31394 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:57.100649  3556 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:15:57.113569 30630 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:15:57.119311 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:57.126543 30631 rpcz_store.cc:276] 0709 13:15:56.117384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.110211 (+992827us) service_pool.cc:224] Handling call
0709 13:15:57.126052 (+ 15841us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.151938 30551 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:57.154079 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.156872 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.122220 30379 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 10. New cstate: current_term: 10 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:57.159121 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:57.160362 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 71) took 29378 ms (client timeout 15000 ms). Trace:
I20260709 13:15:57.160246 30379 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 14 to 17, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 17 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:57.161427 30627 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:15:57.163156  3825 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:57.163494 30629 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:57.163786  3825 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:57.164381  3825 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:57.164809 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:57.166853 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56769.3us
I20260709 13:15:57.168239 30523 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:57.168565 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1737) took 1079 ms. Trace:
I20260709 13:15:57.169194 30630 rpcz_store.cc:276] 0709 13:15:56.089057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.089231 (+   174us) service_pool.cc:224] Handling call
0709 13:15:57.168241 (+1079010us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b54004aabd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:57.168440 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1658032512}
I20260709 13:15:57.163239 31388 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:57.170987  3708 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:15:57.172957 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53901.7us
W20260709 13:15:57.160956 30382 rpcz_store.cc:269] 0709 13:15:27.782121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:27.782339 (+   218us) service_pool.cc:224] Handling call
0709 13:15:57.160035 (+29377696us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:47.303057 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:47.303906 (+   849us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:47.303987 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:15:47.304020 (+    33us) tablet.cc:662] Decoding operations
0709 13:15:47.313009 (+  8989us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:47.313128 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:47.313163 (+    35us) tablet.cc:685] Acquiring locks for 24 operations
0709 13:15:47.384530 (+ 71367us) tablet.cc:701] Row locks acquired
0709 13:15:47.384621 (+    91us) write_op.cc:260] PREPARE: finished
0709 13:15:47.384890 (+   269us) write_op.cc:270] Start()
0709 13:15:47.411089 (+ 26199us) write_op.cc:276] Timestamp: P: 1783602947384807 usec, L: 0
0709 13:15:47.439814 (+ 28725us) op_driver.cc:348] REPLICATION: starting
0709 13:15:47.501203 (+ 61389us) log.cc:844] Serialized 9389 byte log entry
0709 13:15:47.835782 (+334579us) op_driver.cc:464] REPLICATION: finished
0709 13:15:48.039428 (+203646us) write_op.cc:301] APPLY: starting
0709 13:15:48.039590 (+   162us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:48.382315 (+342725us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:48.382487 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:49.393948 (+1011461us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:49.394394 (+   446us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 13:15:49.394464 (+    70us) write_op.cc:312] APPLY: finished
0709 13:15:49.429815 (+ 35351us) log.cc:844] Serialized 211 byte log entry
0709 13:15:49.828415 (+398600us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:49.830696 (+  2281us) write_op.cc:454] Released schema lock
0709 13:15:49.837087 (+  6391us) write_op.cc:341] FINISH: Updating metrics
0709 13:15:49.837703 (+   616us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:49.837741 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":38117,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":203202,"apply.run_cpu_time_us":83477,"apply.run_wall_time_us":1798283,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104687,"num_ops":24,"prepare.queue_time_us":301170,"prepare.run_cpu_time_us":41936,"prepare.run_wall_time_us":320634,"raft.queue_time_us":97410,"raft.run_cpu_time_us":64276,"raft.run_wall_time_us":113352,"replication_time_us":395751,"thread_start_us":772232,"threads_started":5,"wal-append.queue_time_us":690813,"wal-append.run_cpu_time_us":44929,"wal-append.run_wall_time_us":2626052}]]}
I20260709 13:15:57.176960  3749 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:57.179234  3748 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:57.166517 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.160182 30628 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:57.158509 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.182765 30555 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:57.180927 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.182210 30631 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:57.185272 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112889us
I20260709 13:15:57.187317 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.185081 30626 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:57.193174 30630 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:57.190800 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1052) took 1074 ms. Trace:
I20260709 13:15:57.194499 30626 rpcz_store.cc:276] 0709 13:15:56.116499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.054968 (+938469us) service_pool.cc:224] Handling call
0709 13:15:57.190723 (+135755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.196933 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.199842 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:57.200654 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64747.3us
I20260709 13:15:57.201900 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.205099 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.205977 30630 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:15:57.203765 30625 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:57.204393 30628 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:57.189090  3496 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:57.214143 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83932.1us
W20260709 13:15:57.215981 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.214197 30522 leader_election.cc:400] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:15:57.220273 30522 leader_election.cc:403] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:15:57.219755 30379 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:57.221366 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:57.222757  3813 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.223821  3684 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:15:57.221824 30379 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:15:57.226220 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.214629 30631 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:57.225762 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.226182 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:57.224449  3754 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:57.231349 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.230013 30627 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:57.233949 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1263) took 1080 ms. Trace:
I20260709 13:15:57.234382 30627 rpcz_store.cc:276] 0709 13:15:56.153229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.200391 (+1047162us) service_pool.cc:224] Handling call
0709 13:15:57.233858 (+ 33467us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.237995  3556 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.238158 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56500.1us
I20260709 13:15:57.238795 30630 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:15:57.238858  3556 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.241607  3099 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.241931 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:57.242394  3099 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:57.242187  3556 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.238519 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.246733 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:57.247793 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 194950us
I20260709 13:15:57.252259 30631 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.248206 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.253822 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:57.255805 30627 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:15:57.249997 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:57.258338 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1042) took 1144 ms. Trace:
I20260709 13:15:57.235242  3754 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:57.260160 30629 rpcz_store.cc:276] 0709 13:15:56.114136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.211611 (+1097475us) service_pool.cc:224] Handling call
0709 13:15:57.258241 (+ 46630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.267076  3754 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:57.270576  3099 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:57.304535 30627 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
W20260709 13:15:57.314741 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102666us
I20260709 13:15:57.283195 30379 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 8, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 8 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:15:57.323143 30625 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:57.315106 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1255) took 1187 ms. Trace:
I20260709 13:15:57.325047 30627 rpcz_store.cc:276] 0709 13:15:56.118103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.260175 (+1142072us) service_pool.cc:224] Handling call
0709 13:15:57.291773 (+ 31598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.305292 (+ 13519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.305499 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.332199 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1043) took 1209 ms. Trace:
I20260709 13:15:57.333756 30625 rpcz_store.cc:276] 0709 13:15:56.114345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.265317 (+1150972us) service_pool.cc:224] Handling call
0709 13:15:57.293432 (+ 28115us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.323805 (+ 30373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.324112 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.335047  3769 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:57.322244 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:57.316565  3556 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.369946 30625 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:15:57.374287 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 128255us
I20260709 13:15:57.356421  3496 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:57.389119 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 96883.5us
I20260709 13:15:57.403060  3801 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:57.404155 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:57.375803 30379 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:57.405169 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:57.406036 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 169966us
W20260709 13:15:57.409149 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 167422us
W20260709 13:15:57.414975 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 157117us
I20260709 13:15:57.411216 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1260) took 1284 ms. Trace:
I20260709 13:15:57.422787 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1035) took 1310 ms. Trace:
I20260709 13:15:57.415823 30629 rpcz_store.cc:276] 0709 13:15:56.119581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.275000 (+1155419us) service_pool.cc:224] Handling call
0709 13:15:57.281379 (+  6379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.388733 (+107354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:57.388790 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:57.395803 (+  7013us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:15:57.403970 (+  8167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.404071 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.404251 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103525,"raft.queue_time_us":104985,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":3926}
I20260709 13:15:57.423574 30630 rpcz_store.cc:276] 0709 13:15:56.112588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.239965 (+1127377us) service_pool.cc:224] Handling call
0709 13:15:57.240485 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:57.240903 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:15:57.262981 (+ 22078us) log.cc:844] Serialized 14 byte log entry
0709 13:15:57.411821 (+148840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:15:57.411895 (+    74us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:57.421830 (+  9935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.421937 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.422706 (+   769us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14415,"prepare.queue_time_us":25726,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":70,"thread_start_us":132300,"threads_started":1}
I20260709 13:15:57.429235 30625 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
W20260709 13:15:57.432258 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186043us
I20260709 13:15:57.429757 30625 pending_rounds.cc:85] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 8
W20260709 13:15:57.424188  3828 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 13:15:57.442639  3404 raft_consensus.cc:2764] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:15:57.438189 30625 pending_rounds.cc:107] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 13:15:57.448936 30625 raft_consensus.cc:2889] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:57.424660  3801 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:57.443048 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.408483  3747 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:15:57.409794  3556 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.429136  3496 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:57.451462 30699 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:57.455528 30379 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:15:57.464973 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.451145  3748 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:15:57.466476  3748 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:57.456180  3551 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:57.490638 30633 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:57.466601  3253 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.672s	user 0.017s	sys 0.006s
I20260709 13:15:57.509110 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:15:57.511020 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49255.2us
I20260709 13:15:57.452863  3749 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:15:57.466989  3748 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:57.524560  3748 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.526293  3748 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:57.510136  3551 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.527424 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.528365 30698 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:57.529286  3551 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.505988  3801 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:57.533073  3826 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.313s	user 0.002s	sys 0.001s
I20260709 13:15:57.534198  3404 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:57.535502  3556 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.536855  3404 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:57.537447  3404 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:57.542717  3508 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.676s	user 0.001s	sys 0.024s
I20260709 13:15:57.546012 30552 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:57.546347 30628 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:57.550602 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:57.551083 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107014us
I20260709 13:15:57.551990 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:57.553138  3754 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:57.555024  3099 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:15:57.555650  3099 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:57.556159  3099 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:15:57.564443  3754 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:57.571022  3508 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:15:57.572870  3508 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:15:57.575217 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54633us
I20260709 13:15:57.522120  3747 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.576866  3747 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:57.577248 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1266) took 1394 ms. Trace:
I20260709 13:15:57.577735 30633 rpcz_store.cc:276] 0709 13:15:56.182693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.085555 (+902862us) service_pool.cc:224] Handling call
0709 13:15:57.577056 (+491501us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400577954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:57.577184 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":908658560}
I20260709 13:15:57.578713  3748 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:15:57.580236 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.580256 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:57.581053 30703 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.580986  3769 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:15:57.581792  3769 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:57.582978  3769 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.584434  3769 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:57.589918 30699 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:57.595534 30379 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:15:57.598232 30624 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.003s
I20260709 13:15:57.607277 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1658) took 1415 ms. Trace:
I20260709 13:15:57.607834 30707 rpcz_store.cc:276] 0709 13:15:56.187683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.223452 (+ 35769us) service_pool.cc:224] Handling call
0709 13:15:56.224076 (+   624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:56.258053 (+ 33977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:56.258090 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.269678 (+ 11588us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.733635 (+463957us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b180020ee40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:57.053228 (+319593us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:15:57.053576 (+   348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.087611 (+ 34035us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.532173 (+444562us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:57.602526 (+ 70353us) raft_consensus.cc:1685] finished
0709 13:15:57.602562 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.602804 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33401,"raft.queue_time_us":36254,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":5640,"spinlock_wait_cycles":32115328,"thread_start_us":290291,"threads_started":1,"wal-append.queue_time_us":939539}
I20260709 13:15:57.616493 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.616971 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.617888  3508 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:57.619462  3508 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:15:57.619664 30700 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:15:57.619978  3508 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:57.621090  3508 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:57.538560  3404 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:57.623512 30701 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:15:57.626711 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 164669us
I20260709 13:15:57.628185 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:57.629483  3769 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:57.630018  3769 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:57.617354 30702 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:57.618002 30706 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:57.630381 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:57.599404 30624 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:15:57.635694 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1753) took 1285 ms. Trace:
I20260709 13:15:57.490046  3496 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.636374 30624 rpcz_store.cc:276] 0709 13:15:56.350172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.045827 (+695655us) service_pool.cc:224] Handling call
0709 13:15:57.635609 (+589782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:15:57.641494 30631 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:15:57.652508 31389 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.009s
I20260709 13:15:57.630451  3769 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:57.662521 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.650799 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:57.670418 30707 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.653451 31389 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:15:57.677527 30700 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:57.522346  3253 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:57.683722  3508 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:57.720320  3514 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:57.721310  3514 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:57.721760 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 91524.2us
I20260709 13:15:57.724131 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:57.724556 30523 leader_election.cc:400] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:15:57.752094 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:57.752592  3181 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.972s	user 0.014s	sys 0.014s
I20260709 13:15:57.753341  3748 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:57.754753  3825 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.590s	user 0.014s	sys 0.004s
I20260709 13:15:57.755398  3825 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.757319 30707 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:57.723124 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:57.725025 30558 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:57.738973 30706 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:15:57.743116  3747 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.748778 30702 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:57.750200 30700 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:57.728238  3099 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:57.791051 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 105704us
I20260709 13:15:57.763530 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:15:57.800501  3829 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.316s	user 0.000s	sys 0.001s
W20260709 13:15:57.787838  3831 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:15:57.798198 30671 leader_election.cc:336] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.724s (SENT)
I20260709 13:15:57.703965  3749 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:57.763892 30379 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:15:57.792896  3832 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:15:57.856195 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:57.828298 30554 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:57.761880  3748 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:15:57.790459  3607 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:15:57.789361  3496 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.758594  3825 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:57.866907 30703 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:15:57.913954 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 208223us
I20260709 13:15:57.920537 31389 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:57.883911  3747 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.922048  3747 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:57.922451 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124332us
I20260709 13:15:57.867112  3607 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:15:57.924695 30671 leader_election.cc:336] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.828s (SENT)
I20260709 13:15:57.920524  3743 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:15:57.884310  3754 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:57.925536 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:57.925987  3754 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:57.925829  3743 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:57.924139 30379 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 12, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 12 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:15:57.932317 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 168985us
I20260709 13:15:57.934193  3253 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:57.939827  3099 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:57.942324  3514 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:57.957471 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:57.867672  3708 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:57.950840 30597 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.042s	sys 0.072s
I20260709 13:15:57.858139  3749 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:15:57.965380 30597 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.043s	sys 0.072s
W20260709 13:15:57.965786 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 541904us
I20260709 13:15:57.964557 30698 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:15:57.926574  3754 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:57.926756  3748 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:57.982036 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:57.924743  3747 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.995270  3099 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:57.996239  3099 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:57.998884  3743 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.000012  3837 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:57.925572  3607 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.000718  3837 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:58.001264  3813 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.778s	user 0.018s	sys 0.000s
I20260709 13:15:58.001602  3607 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:57.999471  3099 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:57.968816  3749 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:57.968427  3684 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:15:58.000207 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75600.5us
I20260709 13:15:58.006052  3404 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:58.006448 30671 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.853s (SENT)
I20260709 13:15:57.988519  3496 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:15:58.007206 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1649) took 2008 ms. Trace:
I20260709 13:15:58.001847  3813 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19.
I20260709 13:15:58.007683 31378 rpcz_store.cc:276] 0709 13:15:55.946077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:56.411958 (+465881us) service_pool.cc:224] Handling call
0709 13:15:56.412372 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:56.419825 (+  7453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:56.419858 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:15:56.420410 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 13:15:56.971347 (+550937us) spinlock_profiling.cc:243] Waited 5.07 ms on lock 0x7b18004474c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:15:57.258481 (+287134us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:15:57.268198 (+  9717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:57.286521 (+ 18323us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:57.799259 (+512738us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:15:57.946159 (+146900us) raft_consensus.cc:1685] finished
0709 13:15:57.946197 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:57.954685 (+  8488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2470,"raft.queue_time_us":1037,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":5739,"spinlock_wait_cycles":22747264,"thread_start_us":221769,"threads_started":1,"wal-append.queue_time_us":1018372}
I20260709 13:15:57.999774  3708 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.019582  3834 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.020339  3834 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:58.021764 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.015516 30379 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:57.978524  3181 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.025805 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:58.033676 30379 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 5 to 9, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 9 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:15:58.033869  3774 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.041236  3837 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.020834  3708 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.052132 30631 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:58.048767 30626 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:15:58.057636 30624 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:15:58.057173 30379 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 12 to 13, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 13 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:15:58.061779 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.062922 30632 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
W20260709 13:15:58.056777  3801 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.012s	sys 0.017s
I20260709 13:15:58.038784  3834 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:58.065490 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:58.067827 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.049613  3747 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.060722  3813 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:58.039103  3181 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.059044  3649 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:58.084576 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1038) took 1971 ms. Trace:
I20260709 13:15:58.085215 30626 rpcz_store.cc:276] 0709 13:15:56.113261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.208368 (+1095107us) service_pool.cc:224] Handling call
0709 13:15:58.084284 (+875916us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54005f41d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:58.084484 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1889255680}
I20260709 13:15:58.075686 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:58.078917 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72515.1us
W20260709 13:15:58.073673  3801 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.012s	sys 0.017s
I20260709 13:15:58.048133  3774 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:58.074043 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 425189us
I20260709 13:15:58.068495  3837 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:58.092958 30671 leader_election.cc:336] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.911s (SENT)
I20260709 13:15:58.104485 31389 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:15:58.081491  3834 raft_consensus.cc:2764] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:15:58.031203  3749 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.114841 30671 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:58.091428  3619 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:15:58.125129 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75108.7us
I20260709 13:15:58.131465 30633 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:58.080518  3747 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:58.134411 31388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.015s	sys 0.046s
W20260709 13:15:58.138235 31388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.016s	sys 0.046s
I20260709 13:15:58.124220  3619 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.089061  3813 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:58.142184 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49297.9us
I20260709 13:15:58.144002  3559 raft_consensus.cc:2764] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:15:58.091732  3649 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.198967 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1065) took 1222 ms. Trace:
I20260709 13:15:58.199785 30627 rpcz_store.cc:276] 0709 13:15:56.976194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.379939 (+403745us) service_pool.cc:224] Handling call
0709 13:15:57.398841 (+ 18902us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:57.439801 (+ 40960us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:57.439844 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:57.487997 (+ 48153us) log.cc:844] Serialized 52 byte log entry
0709 13:15:57.915856 (+427859us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:15:57.993955 (+ 78099us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.030428 (+ 36473us) log.cc:844] Serialized 14 byte log entry
0709 13:15:58.036842 (+  6414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:15:58.036974 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:15:58.198562 (+161588us) raft_consensus.cc:1685] finished
0709 13:15:58.198593 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:15:58.198870 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41283,"prepare.queue_time_us":70932,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":6031,"raft.queue_time_us":109069,"raft.run_cpu_time_us":544,"raft.run_wall_time_us":20240,"spinlock_wait_cycles":15637248,"thread_start_us":347045,"threads_started":1,"wal-append.queue_time_us":679238}
I20260709 13:15:58.182448  3619 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:58.123586  3837 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.186200  3801 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:15:58.217891 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:58.226233 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:58.230732 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.028s
W20260709 13:15:58.233126 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.005s	sys 0.028s
I20260709 13:15:58.208083  3743 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:15:58.247550 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 288357us
I20260709 13:15:58.252071 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:58.256451 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.236094  3748 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:15:58.258989 30671 leader_election.cc:336] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.966s (SENT)
W20260709 13:15:58.254276  3550 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.005s	sys 0.028s
W20260709 13:15:58.263916 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 75439.9us
W20260709 13:15:58.257534  3769 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.004s
I20260709 13:15:58.272717 31378 raft_consensus.cc:2410] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
W20260709 13:15:58.268503 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 181000us
I20260709 13:15:58.276482 31389 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:58.279218 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.280207 30631 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:15:58.270825  3550 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.005s	sys 0.030s
I20260709 13:15:58.285079 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.270540  3708 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:58.274480  3849 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.302647  3837 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.304438 30700 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:58.293766 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:58.309194 30554 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:58.310878 30556 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:58.311472 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:58.315534  3708 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.316737  3852 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:58.276178  3769 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.313776 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.320598  3749 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:58.321257 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.326067 31394 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:15:58.324921 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 336742us
I20260709 13:15:58.329779 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.300524  3181 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:58.331231  3853 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:58.331921  3847 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:58.332722  3649 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.333956 30703 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:15:58.334869 30702 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
W20260709 13:15:58.341368 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 86549.5us
I20260709 13:15:58.338532 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.342980 30700 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:58.345142 31388 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:58.340878  3835 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.346637  3835 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.347069  3099 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.347705  3747 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.348551  3850 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:15:58.348747 30671 leader_election.cc:336] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 2.007s (SENT)
I20260709 13:15:58.349082  3835 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:58.314342  3849 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.349371  3847 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:58.350514 30671 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.694s (SENT)
I20260709 13:15:58.351035 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:58.351342  3849 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.334715  3854 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:58.352070  3774 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.352279  3854 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.354545 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.355502 31396 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:58.347657  3099 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.358018  3099 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.358847 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.339171  3853 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:58.360128  3853 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.361477 31378 raft_consensus.cc:2410] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:15:58.361996  3853 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:58.338883  3748 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.363422 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.364167 31388 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:15:58.367285 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:58.367913  3813 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.387669  3514 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:15:58.368589 30557 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:15:58.391834 30557 pending_rounds.cc:85] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:15:58.392287 30557 pending_rounds.cc:107] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:15:58.392649 30557 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:58.369105  3749 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:15:58.328684  3801 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:15:58.318444  3708 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.372821  3754 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.374307 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.357975  3847 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.375494 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:58.391880 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55968.5us
I20260709 13:15:58.391242 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:58.399528  3838 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:58.409190 31389 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:58.400542  3619 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.401186  3749 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:15:58.401688  3854 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:58.416507 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66040.7us
I20260709 13:15:58.421053 30558 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:58.390388  3514 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:15:58.395377  3850 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:58.403398  3837 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:15:58.419183  3769 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:58.419595  3813 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.420786  3754 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:58.421562 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.420832 30700 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:15:58.416309 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.422590 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.435664 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.438460 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.443174 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:58.419237  3404 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:58.412081  3749 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:15:58.445266 31394 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:58.469131 30631 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:58.439669  3847 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:58.455839 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:58.475459 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84398.8us
I20260709 13:15:58.470901 31396 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:58.479429 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.479532 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.480439 30626 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:58.483549 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 116394us
I20260709 13:15:58.479246 30557 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:15:58.487486 30631 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:15:58.485977 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.491855 30632 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:58.470398  3684 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.499060 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.499403 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.477037  3181 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:15:58.490070 31294 connection.cc:256] RPC call timeout handler was delayed by -1.224428780999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
I20260709 13:15:58.502023 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.485980  3847 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.503530 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.504907 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.504769  3847 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.506827  3569 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:15:58.507606  3099 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:58.507355  3749 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.499665 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:58.499928 30554 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:58.454300  3813 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.510444 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:58.513612 30631 raft_consensus.cc:2376] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16. Re-sending same reply.
I20260709 13:15:58.474298  3404 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.516167 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.519668 30632 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:58.512408  3514 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:58.483259  3838 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:15:58.524645 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:58.525316 30671 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 2.113s (SENT)
I20260709 13:15:58.522840 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:58.527454 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.528545 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.529343 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:58.512315  3708 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:58.535269 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.536501 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.536480  3404 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:58.540587 30558 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:15:58.565352 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.566392 30631 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:15:58.564685 30551 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:58.572983 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:58.625847 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.539458  3835 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:58.539667  3508 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:15:58.511418  3684 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:58.638522 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 122552us
I20260709 13:15:58.638782  3496 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:58.640517 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:58.643250 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73974.6us
I20260709 13:15:58.648795 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.649708 30632 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:15:58.560251  3181 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:58.651789 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.655014 30625 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:58.661626 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:15:58.654268 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 89054.7us
I20260709 13:15:58.663900 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.659518 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.666872 30700 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:58.667207 30703 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:58.668989  3837 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:15:58.669523  3837 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:58.661975  3496 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.671509 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:58.658285 30626 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:15:58.636555  3749 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:15:58.649598  3835 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.677423 30629 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:58.649339  3508 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.516021  3850 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.684507 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:58.654690  3747 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.690033 30705 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:15:58.696627 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:58.700090 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 187262us
I20260709 13:15:58.701732 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.715700 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:58.708948 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:58.690229  3863 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:15:58.676753  3619 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:58.696223 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:58.690369  3857 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:15:58.687778  3835 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:58.689824  3253 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:15:58.690758  3508 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:58.723192 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:58.732894 30624 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:15:58.720851  3769 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.673429  3181 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.742838 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:58.723582  3849 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:58.759995 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58457.4us
I20260709 13:15:58.730888  3099 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.737716  3404 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:15:58.738775  3857 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:58.709705  3747 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:58.772554 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56991.7us
I20260709 13:15:58.722541  3708 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.780166 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:15:58.744233  3253 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.761500  3849 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.670512  3837 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:58.789283  3840 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:15:58.790478  3837 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:58.761140  3619 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:58.781514  3857 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:15:58.789189  3864 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:15:58.787604  3813 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.778379  3754 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:58.799105  3684 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.787463  3849 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:58.799677  3754 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:58.800223  3754 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:58.789170 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:58.790516  3840 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:15:58.802716  3840 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:58.804221  3840 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.805661  3840 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:58.806572  3496 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:58.807482 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:58.808430 30554 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:58.760707  3607 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:58.802604  3649 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:58.811046  3607 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.814064  3875 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:15:58.820148  3875 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.821504  3875 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.776378  3769 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:58.817170  3855 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.170s
W20260709 13:15:58.788328  3860 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:15:58.826148  3099 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:58.851811 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.826826  3708 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:15:58.892428 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 103360us
W20260709 13:15:58.894735 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.897748 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:58.891999 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 195859us
W20260709 13:15:58.903447 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 189702us
I20260709 13:15:58.816547  3870 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:58.902304 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:15:58.909593 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 129532us
I20260709 13:15:58.907192 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:58.904017  3404 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.907792  3684 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:58.911018  3834 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:15:58.938689 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:58.908531  3514 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:58.936789 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:58.935890 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:58.936062 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:58.917724  3496 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:58.947940 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:58.906191  3649 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:15:58.975806 31396 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:58.938237  3834 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.987028  3514 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.951694  3569 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:15:58.989748 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:58.990957  3774 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:58.991245  3607 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.936556  3684 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:58.938561  3870 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:59.019500  3684 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.020030  3870 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.931104  3181 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:58.990996  3813 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:58.927968  3404 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:58.999289 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:15:59.006479  3853 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.644s	user 0.000s	sys 0.017s
I20260709 13:15:58.925421  3253 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:59.010728  3869 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.088s	user 0.002s	sys 0.000s
I20260709 13:15:59.012965 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:59.013341  3888 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:15:59.008927  3774 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:58.947621  3855 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:59.005815  3834 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:59.007063 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:15:59.031894 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 93369.4us
I20260709 13:15:58.993049  3748 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.040319  3496 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:15:59.050933  3253 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.053054  3886 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:59.053224 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.053699  3886 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:15:59.054162  3886 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:15:59.055136  3769 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:59.055487 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.032637  3813 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.043332  3774 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:59.058796  3853 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:15:59.059027 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 69422.2us
I20260709 13:15:59.059427  3853 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.059890 31396 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:59.055648 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.061064 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.061405 30700 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:15:59.061785 30703 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:15:59.034550  3607 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:15:59.065672 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.091439  3889 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:15:59.093945  3889 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.096247  3404 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.099642  3769 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:15:59.100747  3865 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:15:59.104331 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.105492  3849 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.106230  3888 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:15:59.106863  3888 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.107236  3888 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:15:59.110127  3880 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:15:59.112701 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50196.8us
I20260709 13:15:59.058434  3813 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:15:59.043896  3743 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.007s
I20260709 13:15:59.113919 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.130748 30554 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:59.059898  3853 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:59.050729  3874 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:59.060598  3551 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:15:59.060375  3858 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.164860 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.165082 30630 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:15:59.167375 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.049587  3747 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.050886  3496 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.173578 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.098263  3508 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.182884 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.170786 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:15:59.172096 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 149808us
I20260709 13:15:59.085040  3748 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.215869  3748 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.216647  3649 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:59.217260  3649 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.217664  3649 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:59.172825  3801 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:15:59.199008 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 191495us
W20260709 13:15:59.201004 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 140726us
I20260709 13:15:59.224366 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.203663  3838 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:15:59.203758  3878 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:15:59.187346  3858 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.228161 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.228533 30705 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:15:59.229166 30700 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:59.230213 30702 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:15:59.113505  3607 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.242537  3607 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.243450 30632 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:15:59.243817 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.244860 30702 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:15:59.246232 30699 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:15:59.247530 30626 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:15:59.248117 30626 pending_rounds.cc:85] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 12
I20260709 13:15:59.248200 30707 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:15:59.248539 30626 pending_rounds.cc:107] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:15:59.248946 30626 raft_consensus.cc:2889] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:15:59.249624 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.250559 30624 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:15:59.251760 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.252737 30628 raft_consensus.cc:2368] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:15:59.247566 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.253676 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.254025 30630 raft_consensus.cc:2368] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16.
I20260709 13:15:59.255199 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.242383  3754 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.259035 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.259735 30706 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:59.094480  3889 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.176630  3874 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:59.230563  3858 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:15:59.207386  3883 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:15:59.232280  3748 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.235097 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.282766  3748 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.283120 31394 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:59.185232  3496 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:59.236793 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.287042 30631 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:15:59.239239 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.276991 30706 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:59.280329 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:59.291122 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:59.291916 31364 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:15:59.292748 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.292820 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:15:59.274885  3749 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.593s	user 0.016s	sys 0.011s
I20260709 13:15:59.296058 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.298956  3748 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.258318  3888 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.269138  3607 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:59.308799 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 84548us
I20260709 13:15:59.312953 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.314219 30706 raft_consensus.cc:2368] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16.
I20260709 13:15:59.289911  3754 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.315688 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.235884  3801 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.321053 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:59.320482 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:15:59.242556  3774 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.325230 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:15:59.325045 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.327471 30624 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:59.329222 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.336825 30628 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:15:59.343518 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.345935 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
W20260709 13:15:59.352097  3881 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.339s	user 0.007s	sys 0.001s
I20260709 13:15:59.352303 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.316577  3607 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.347473  3850 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:59.355144  3850 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.355593  3850 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:59.356160  3607 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.367168 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.368830 30706 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:15:59.371815 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.375000 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.375908 30627 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:15:59.315429  3837 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:15:59.381223 30631 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:15:59.381539 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 78600.4us
I20260709 13:15:59.379557 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.339458  3749 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:15:59.384476 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.323328  3887 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:15:59.364408  3556 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:15:59.374271  3404 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.362493  3774 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.351940  3801 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:15:59.396221 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75248us
I20260709 13:15:59.392094 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:15:59.394341 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 91529.5us
I20260709 13:15:59.397307 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:15:59.433782 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.440343 30699 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:15:59.423938  3607 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.442926 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.443184  3607 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.445026  3856 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:15:59.445999  3607 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.418018  3813 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.447633  3813 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:59.449841  3895 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:15:59.450140  3813 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.433198  3748 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.452821  3748 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.439337  3569 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:15:59.456254 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:59.457608  3872 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:15:59.458009 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.460813 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.418066  3849 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.461577 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.463217 30702 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:15:59.463722 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.461948 30625 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:15:59.464493 30555 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:15:59.441831 30706 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:15:59.426684  3889 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.475886 31383 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:15:59.469026 30597 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.431761  3550 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.431427  3556 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:59.480644 30707 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:15:59.395071  3834 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.437163  3838 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:15:59.491215 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.418706  3404 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.489262 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.493002  3871 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:15:59.509753 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.510315  3813 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:15:59.510878  3813 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.510939  3404 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.511334  3813 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:59.511597 30559 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:15:59.511889  3889 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:59.512478  3889 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:59.513360 30559 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:15:59.513511  3889 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.493992  3748 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.494225  3556 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.496807 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:59.497663 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 100447us
I20260709 13:15:59.499032 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:59.499850 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:59.500123  3840 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.006s
I20260709 13:15:59.501298 30553 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:15:59.501461 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.503321 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:59.504688  3649 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:15:59.508116  3835 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.753s	user 0.009s	sys 0.004s
I20260709 13:15:59.565711 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.554171 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:59.490737 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.558168 30632 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:15:59.480126  3508 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.437959  3886 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.571296 30556 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:15:59.572450 30559 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:15:59.575214 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73910.2us
I20260709 13:15:59.578209 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:59.391244  3874 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.581269 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:59.590445 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 99820us
I20260709 13:15:59.573318  3856 raft_consensus.cc:2764] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:15:59.491554  3550 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.615895 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:15:59.617595 31389 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:59.607017 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:15:59.621052 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:15:59.619480 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.653453 30630 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:15:59.659754 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:15:59.598536  3857 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.802s	user 0.009s	sys 0.007s
I20260709 13:15:59.662963 30558 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:15:59.664587 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:15:59.577395  3840 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.608551  3837 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:15:59.605142  3834 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.569049  3835 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.587949  3871 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:15:59.716069 30698 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:15:59.580278  3870 raft_consensus.cc:2764] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:15:59.716524 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.608273  3550 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.605506  3888 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.648802  3619 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:15:59.753255  3849 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.727999  3559 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:15:59.759317  3905 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:15:59.763046 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 156127us
I20260709 13:15:59.764498 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.671312  3607 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:15:59.774012 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 158287us
I20260709 13:15:59.716656  3514 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:15:59.773465 30551 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:15:59.671562  3747 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.782315 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:59.783766 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:59.789702 30553 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:15:59.762315  3872 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:15:59.787971 30633 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:15:59.747902  3871 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:15:59.753221  3837 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:59.799113 30556 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.741412  3857 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.816501 30558 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:15:59.785302 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114763us
I20260709 13:15:59.801291 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.727543  3813 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.811125 30698 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:15:59.809410  3853 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:15:59.786959  3559 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:59.835813 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.832196 30553 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:15:59.798362  3619 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:15:59.798055  3834 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.797605  3774 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:15:59.791708  3849 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:15:59.831393 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:15:59.840549 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:15:59.810659  3871 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:15:59.859377  3882 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:59.860497  3882 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.843367 30552 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:15:59.740582  3840 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.835440 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.837651  3559 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:15:59.831077  3569 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.871098 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.798045  3607 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.842401  3853 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.804566  3747 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.897742 30554 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:15:59.898784 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:15:59.903043 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71720.2us
I20260709 13:15:59.874718  3889 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:15:59.884886  3877 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:15:59.911043 30557 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:15:59.911820 31364 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.001s
W20260709 13:15:59.912652 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 109715us
I20260709 13:15:59.915849 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.864974  3834 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.869201  3774 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.918416 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:15:59.912561 31364 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:15:59.906837  3749 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.007s
I20260709 13:15:59.926101 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.879873  3514 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:15:59.907261  3888 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.914537  3889 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:15:59.928376  3888 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:15:59.929107  3909 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 13:15:59.931236  3913 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 13:15:59.931593  3908 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.150s	user 0.001s	sys 0.001s
I20260709 13:15:59.936156 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.937016 30628 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:15:59.938179 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:15:59.929172  3889 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:15:59.943156 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:15:59.944080 30631 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:15:59.945183 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2373) took 1424 ms. Trace:
I20260709 13:15:59.945612 30555 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: []
I20260709 13:15:59.946887 30556 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:59.947254  3834 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.947678  3857 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:15:59.951052  3853 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.951418 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:15:59.917963  3649 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.952273 30553 raft_consensus.cc:2376] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18. Re-sending same reply.
I20260709 13:15:59.953821  3882 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.951131  3559 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.955444  3813 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.957017 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:15:59.957322  3747 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.957582  3559 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.903404  3847 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:15:59.954237  3849 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:15:59.945626 30552 rpcz_store.cc:276] 0709 13:15:58.520801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.527082 (+  6281us) service_pool.cc:224] Handling call
0709 13:15:59.944898 (+1417816us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003a5f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:15:59.945070 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3022175488}
I20260709 13:15:59.966321 30625 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:15:59.960986 30627 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.584s	user 0.015s	sys 0.000s
I20260709 13:15:59.965847  3872 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:15:59.969202 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55657.7us
I20260709 13:15:59.970216  3774 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.881167  3569 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:15:59.971131  3888 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.975731  3840 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:15:59.977782  3847 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.959455  3619 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:15:59.915692  3837 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:15:59.979837 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75157.6us
I20260709 13:15:59.983762  3607 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:15:59.986244 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.988075 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.988379 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:15:59.988443 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:00.047081 30698 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:15:59.989189  3556 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.044178 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.042140 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.046254 30707 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:00.044806 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:15:59.990077 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.049723 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.050396 30707 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:15:59.992988 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:15:59.994227 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:15:59.990203 30557 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:16:00.053037 30557 pending_rounds.cc:85] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 10
I20260709 13:15:59.995167 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:15:59.995426 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.046561 30702 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:15:59.995860  3877 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:15:59.998329 30627 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:00.051915 30559 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:15:59.950457  3496 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.000s
I20260709 13:16:00.056799 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.057541 30551 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:00.028301 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:00.030251 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.058926 30553 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:00.031047 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:00.059463 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.035768 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.060097 30558 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:00.037957 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 159377us
I20260709 13:15:59.967686  3749 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:00.041275 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:00.041790 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.046250 30706 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:15:59.989934 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.070231 30699 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:00.053354 30557 pending_rounds.cc:107] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:16:00.071368 30557 raft_consensus.cc:2889] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:00.048568 30704 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:00.026033 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.061719 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.056226 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:00.078400 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53782.8us
W20260709 13:16:00.080518 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 86395.2us
I20260709 13:16:00.084772 30701 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:00.080710 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.088457 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:00.094941 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 60342.9us
I20260709 13:16:00.032424  3882 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:16:00.033565  3870 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:16:00.098496 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.096868 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.154899 30559 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:16:00.104316 30551 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
W20260709 13:16:00.155407 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58593us
I20260709 13:16:00.154899 30698 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:00.156435 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.165999 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:00.089499  3858 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:00.165813 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 127168us
W20260709 13:16:00.099853  3850 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.744s	user 0.004s	sys 0.010s
I20260709 13:16:00.169759 31364 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:00.090147  3840 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:00.094717  3607 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:00.042502  3922 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.121s
I20260709 13:16:00.056401  3872 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:16:00.176937 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.068059  3496 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.055658  3556 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:00.182548 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:00.100498  3774 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:00.173560  3888 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:00.068871  3877 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:00.185570 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.193630  3921 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:00.196601  3496 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.186324 30558 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:00.180034  3649 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:00.176353  3858 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.178679  3849 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:16:00.242062 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56629.7us
I20260709 13:16:00.249354 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:00.250161 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 152635us
I20260709 13:16:00.184767  3607 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.201006  3813 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:00.179013  3857 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:00.250057 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.187304  3922 consensus_queue.cc:1243] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:00.253858 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:00.254033 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 168083us
I20260709 13:16:00.242087  3888 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:00.182729  3850 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:00.254858 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.255059 30707 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
W20260709 13:16:00.251055 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.879s (SENT) [suppressed 59 similar messages]
W20260709 13:16:00.256032 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 193977us
I20260709 13:16:00.244810  3834 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:00.207629  3882 raft_consensus.cc:2764] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:16:00.247105  3921 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.271765 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.269671 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.248826  3858 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.253223  3607 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:00.299798 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117322us
I20260709 13:16:00.182283  3514 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.256916  3889 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:00.256129  3813 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.270555  3888 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:00.245050  3774 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.317067  3774 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:00.317934  3774 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:00.249194  3649 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.272886  3834 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.272352  3840 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:00.275878  3404 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:00.259452  3932 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:00.280797  3921 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.320569  3813 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.249446  3930 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 13:16:00.356783 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 288492us
I20260709 13:16:00.318426  3774 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:00.310873  3889 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:00.365624 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72740.1us
I20260709 13:16:00.321075  3834 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:00.381697 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.435977  3850 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.436285 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:00.444638 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.445955  3937 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:00.446902  3937 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:00.448009  3937 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.363561  3769 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:00.451660  3927 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:16:00.451989  3769 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.380353  3569 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.386168  3847 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:00.373927  3932 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:00.364305  3404 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.439350 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1287) took 1169 ms. Trace:
I20260709 13:16:00.457434 30630 rpcz_store.cc:276] 0709 13:15:59.269914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.270082 (+   168us) service_pool.cc:224] Handling call
0709 13:16:00.439087 (+1169005us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54004abad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:00.439262 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598265088}
I20260709 13:16:00.409540  3929 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:16:00.439261 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 167591us
W20260709 13:16:00.462234 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 192719us
I20260709 13:16:00.364498  3840 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:00.372193  3508 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:00.452770  3888 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.395089  3889 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:00.479727  3921 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.488215  3921 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.490029  3921 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.491328  3921 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.494837 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 997) took 1015 ms. Trace:
W20260709 13:16:00.500957  3915 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.601s	user 0.003s	sys 0.000s
I20260709 13:16:00.504806  3889 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.495412 30555 rpcz_store.cc:276] 0709 13:15:59.479805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.480466 (+   661us) service_pool.cc:224] Handling call
0709 13:15:59.945383 (+464917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:15:59.946111 (+   728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.494543 (+548432us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b54003a5f50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:00.494764 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068027776}
I20260709 13:16:00.507367 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.507582 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.508862 31389 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:16:00.455169  3769 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:00.520454 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 200852us
W20260709 13:16:00.521056 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.510799 30701 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:00.513993 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 219560us
I20260709 13:16:00.461319  3404 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:00.405762  3774 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.461597  3850 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:00.489030  3886 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:00.525326 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:00.458696  3932 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:00.461539  3849 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:00.461726  3929 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:00.545320 30698 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:00.547333 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.547260 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.548344  3942 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:00.553028  3928 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.295s	user 0.000s	sys 0.002s
I20260709 13:16:00.553583  3889 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:00.554410  3924 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.294s	user 0.001s	sys 0.001s
I20260709 13:16:00.557328 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:00.557191  3847 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:00.559475 30706 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:00.579931 30704 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:00.545154 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:00.565407 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 190000us
I20260709 13:16:00.566869 30701 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:00.528276  3508 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:00.583668 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:00.586686 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.562245  3886 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:16:00.610297  3877 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.592753 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "54da309ba151423ab2739966b9caebd0"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:00.579164  3942 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.599246  3556 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.589952  3649 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:00.616297  3849 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.623283 30523 leader_election.cc:400] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:16:00.645335 30523 leader_election.cc:403] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:00.647789 30523 leader_election.cc:400] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16.
I20260709 13:16:00.648198 30523 leader_election.cc:403] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:00.648346  3942 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.649398  3857 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:16:00.650291  3857 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16.
I20260709 13:16:00.595002  3930 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:00.651656  3930 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:00.652700  3930 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.630048 30698 raft_consensus.cc:606] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.635035  3921 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:00.639194 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 81961.4us
I20260709 13:16:00.656297 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:00.656842  3871 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.802s	user 0.014s	sys 0.001s
I20260709 13:16:00.657469  3871 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.659101  3922 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:00.659858  3922 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:00.661019  3922 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.662902  3870 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:00.591142  3840 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.664768  3888 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.592324  3941 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:16:00.641214  3508 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.600694  3948 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:00.666918  3948 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.592584  3769 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:16:00.671743  3945 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:00.655247 30698 raft_consensus.cc:614] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:00.672472  3945 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.673728  3835 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:00.676206 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.590013  3850 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:16:00.687043 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.683697 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.689177 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.689455 30553 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:00.684774 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.690133 30551 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:00.690311 30555 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:00.690405 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.695691 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.696421 30556 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:00.696921 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.697964 30553 raft_consensus.cc:2410] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:16:00.692854 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.698810 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.699731 30555 raft_consensus.cc:2410] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:16:00.700902 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.702479 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.704916 30558 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:00.709729  3940 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:00.695401  3099 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:16:00.711645  3099 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:00.711341  3834 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.713471 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.715189 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.716325 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:00.717969 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.724161 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:00.738416 30557 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:00.739086 30557 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:00.727105  3875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.164s	user 0.031s	sys 0.036s
I20260709 13:16:00.726955  3835 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:00.726078  3942 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:00.728785  3888 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:00.782801 31292 connection.cc:256] RPC call timeout handler was delayed by -1.247365660000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 13:16:00.737030  3850 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:00.726715  3099 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:16:00.822466 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.740146 30557 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.729701  3769 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:16:00.787214 31292 connection.cc:256] RPC call timeout handler was delayed by -1.1834656599999107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1607s for a response
I20260709 13:16:00.738891  3847 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:00.749886  3948 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.796916 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.798131 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1853) took 1604 ms. Trace:
I20260709 13:16:00.798993  3945 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.745859  3871 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:00.803268  3404 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.807351  3834 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.809070  3607 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.809427  3813 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:00.814675  3858 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.509s	user 0.013s	sys 0.001s
I20260709 13:16:00.759799  3835 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.868485  3404 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.868913  3835 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:00.869378  3813 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:00.827927  3099 consensus_queue.cc:1243] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:00.836445 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.785137  3888 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:00.882979 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.884675 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.885106 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.886008 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.834110 30554 rpcz_store.cc:276] 0709 13:15:59.193814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.195096 (+  1282us) service_pool.cc:224] Handling call
0709 13:15:59.195574 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:15:59.898362 (+702788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.797802 (+899440us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54001aa254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:00.798057 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1604933888}
I20260709 13:16:00.876144 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.888037 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.889577 30704 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:16:00.890789 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.891242 31367 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:00.891521 30554 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:00.892213 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.892560 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.892956 30704 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:00.893260 30554 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:00.838356 30557 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:00.759262  3875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.032s	sys 0.037s
I20260709 13:16:00.846335 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.847162 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.908880 31361 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:00.855970 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:00.911595 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55763.6us
W20260709 13:16:00.849313 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64845.3us
I20260709 13:16:00.850409 31378 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:00.815665 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2106) took 1202 ms. Trace:
I20260709 13:16:00.874151 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:00.820672  3942 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:00.868580  3858 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:00.913563 30703 rpcz_store.cc:276] 0709 13:15:59.613397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.613536 (+   139us) service_pool.cc:224] Handling call
0709 13:15:59.613900 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.491850 (+877950us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:00.492184 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.629664 (+137480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:00.629785 (+   121us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:00.629891 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:00.815319 (+185428us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b5400343854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:00.815405 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:00.815583 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6572,"prepare.queue_time_us":174,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":6761,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":111,"spinlock_wait_cycles":2018336384,"thread_start_us":130298,"threads_started":1,"wal-append.queue_time_us":240898}
I20260709 13:16:00.916462  3858 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:00.916697  3942 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:00.896261 30704 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:00.837167  3850 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:00.898020 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.917618 30551 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:00.917845 30630 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:00.895637 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:00.819540  3857 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:16:00.918236  3850 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:00.900990 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.902143 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.920111 31383 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:00.897969 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.903373 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.921741 30625 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:00.841785 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:00.919409 30633 raft_consensus.cc:2376] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14. Re-sending same reply.
I20260709 13:16:00.923098 30556 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:00.905035 30631 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:16:00.886365 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.912668 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:00.916519 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.914242 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:00.899255 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.899497  3754 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:00.926927 30627 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:00.875355  3949 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:00.928210 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.938300 30559 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:00.938512 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64522.3us
I20260709 13:16:00.939142 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.928958 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.929222 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:00.930855 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.932408 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.943025 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.943236 30552 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.943864 30631 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:00.944886 30557 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 10. (term mismatch)
I20260709 13:16:00.945588 30557 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:16:00.945374 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.946043 30557 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:16:00.946482 30557 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:00.948966 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.949810 30624 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:00.935178 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:00.951097 30554 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.951150 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:00.946491 30633 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:00.957242 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.957669 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.940457 31391 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:00.957300 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:00.959323 30706 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:00.953785 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:00.963707 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:00.908759  3888 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:00.968560 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.968856 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:00.969328 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:00.971807 30631 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 13:16:00.975443 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:00.976516 31378 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:00.978670 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.979715 31367 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:16:00.979549 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:00.980257 30630 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:00.925776  3939 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:00.920522  3607 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:00.984969 30633 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:16:00.985416 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:00.985507 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:00.986433 30626 raft_consensus.cc:2368] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:00.992362 30628 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:16:00.928761  3754 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:00.993332 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:00.993359 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:00.995126 31394 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:00.996193 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:00.998310 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.002365 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.003872 31383 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:16:00.960507  3856 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:01.026121 30703 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:00.965165  3858 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:01.039786 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:01.041245 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:01.064884 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.049111 30703 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:01.044936 30704 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:01.092057 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.095680 31383 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:01.096163 30706 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:01.018066  3962 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:01.098709 30700 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:16:01.105037 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:00.956897  3949 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:01.113842 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.115602 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1679) took 1078 ms. Trace:
I20260709 13:16:01.117393 30706 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:01.119683 30702 rpcz_store.cc:276] 0709 13:16:00.036860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.054564 (+ 17704us) service_pool.cc:224] Handling call
0709 13:16:00.054984 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:00.055234 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:00.153736 (+ 98502us) log.cc:844] Serialized 14 byte log entry
0709 13:16:00.892451 (+738715us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b18003824e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:16:01.115183 (+222732us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:01.115252 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:01.115308 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.115395 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.115536 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93162,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":97,"spinlock_wait_cycles":26957696,"thread_start_us":222400,"threads_started":1}
W20260709 13:16:01.114445  3932 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.556s	user 0.013s	sys 0.003s
I20260709 13:16:01.135800 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:01.047847  3551 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:01.149570 30702 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:01.152855 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.099241  3857 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:01.153368 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.102393  3858 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:01.017906  3931 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:16:01.114884  3942 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:01.221350 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 228057us
I20260709 13:16:01.181360 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:01.182341 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:01.262797 30626 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:01.141623  3939 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:01.205677 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:01.214435 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 221182us
I20260709 13:16:01.215117 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.168972  3932 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:01.154095 30699 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:01.266448 31383 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:01.278000 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.281546 30554 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:01.281656 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:01.296545 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.305676 30702 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
W20260709 13:16:01.310559 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45652us
I20260709 13:16:01.311923 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.319130 31391 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:01.261505  3508 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.322743 31364 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:01.335104 30559 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:01.325093 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.222041  3835 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:01.353094 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.288524  3684 raft_consensus.cc:2764] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 13:16:01.291478  3857 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:01.332808  3956 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.405s	user 0.000s	sys 0.002s
I20260709 13:16:01.320405  3856 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:01.374244 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 62469.1us
I20260709 13:16:01.378664  3937 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:01.294164  3945 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 13:16:01.365950 30624 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:01.371762 30552 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:01.341202  3852 raft_consensus.cc:2764] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:16:01.275835  3942 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.243321  3551 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.361323  3404 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:01.394452  3551 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:01.408507 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.410274 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.374109  3856 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:01.351089  3508 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.402642 31378 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:01.425590 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1677) took 1389 ms. Trace:
I20260709 13:16:01.428752 30707 rpcz_store.cc:276] 0709 13:16:00.030102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.061383 (+ 31281us) service_pool.cc:224] Handling call
0709 13:16:01.418982 (+1357599us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b540037d2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:01.419118 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":417440512}
I20260709 13:16:01.391549  3962 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:01.404354  3958 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
W20260709 13:16:01.439422  3969 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 13:16:01.441617  3963 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 13:16:01.450016  3857 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.460968  3942 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:01.463908  3962 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:01.467424  3966 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:16:01.468938  3932 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:01.427693  3856 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:01.477605 30523 leader_election.cc:400] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16.
I20260709 13:16:01.479847 30631 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:01.481304 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:01.432092 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.436549  3949 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:01.497627 30626 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:01.508663 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.506153 31383 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:16:01.507966 31361 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.588s	user 0.014s	sys 0.000s
I20260709 13:16:01.510951 31361 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:01.509524 31367 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:01.520848 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:01.401185  3858 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.523542 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:01.415529  3939 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.489073  3970 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:01.528972 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.529419 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1004) took 2031 ms. Trace:
I20260709 13:16:01.516273 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.531548 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 206576us
I20260709 13:16:01.533667 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:01.544477 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:01.562000 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.576582 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.577713 30554 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:01.565711 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.579335 31367 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:01.568782 31388 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:01.525647  3961 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:01.574797 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:01.580798 31391 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:01.528033  3404 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:01.573004 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.585436 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.579844 31367 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:01.588907 31383 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:01.535167  3888 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.542s	user 0.015s	sys 0.000s
W20260709 13:16:01.590862 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57322.2us
I20260709 13:16:01.532513  3936 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:01.589020 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:01.594655 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.598986 31361 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:01.595037 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.531636  3977 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:01.601356  3835 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:01.602860 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.566459  3834 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:01.607740 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.570910 30550 rpcz_store.cc:276] 0709 13:15:59.497827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:59.515185 (+ 17358us) service_pool.cc:224] Handling call
0709 13:15:59.515601 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:15:59.516113 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:15:59.516148 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:15:59.516912 (+   764us) log.cc:844] Serialized 52 byte log entry
0709 13:15:59.780997 (+264085us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:00.499826 (+718829us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:01.364902 (+865076us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b1800392860. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf
0709 13:16:01.365086 (+   184us) log.cc:844] Serialized 14 byte log entry
0709 13:16:01.528908 (+163822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:01.529035 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:01.529109 (+    74us) raft_consensus.cc:1685] finished
0709 13:16:01.529127 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:01.529321 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37033,"prepare.queue_time_us":117154,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3906,"raft.queue_time_us":15656,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":219,"spinlock_wait_cycles":20801536,"thread_start_us":509175,"threads_started":3,"wal-append.queue_time_us":326005}
I20260709 13:16:01.605901 31367 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:01.614113 31367 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:01.618697 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:01.621897  3974 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:01.593279  3747 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:01.624948 30554 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:01.603093  3937 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:01.599189  3888 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.626490 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.627574  3978 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:16:01.634490  3850 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:01.599557  3961 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:01.660444  3961 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.605803  3977 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.663007  3977 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:01.667047  3888 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:01.623611  3942 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:16:01.667780 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.668300  3942 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:01.668999  3983 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.625762 31383 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:01.639037 31388 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:01.669468 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.670702 30550 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:01.669665  3983 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.644241 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.651903 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.645478  3835 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:01.658764  3987 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:01.677464 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.642984 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:01.684718 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 111825us
I20260709 13:16:01.653509  3937 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.686604 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.688607 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.695762 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.675212  3747 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:01.678893  3559 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:01.679634  3972 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:01.679286  3973 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:01.701654 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90506us
I20260709 13:16:01.702461 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.690503  3937 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:01.703486 31388 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:16:01.689817  3932 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.712358 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.669445  3942 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.728201 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.728541 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.729398 31383 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:01.718094  3945 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:01.717868  3747 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.714721  3850 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:16:01.753762  3850 consensus_queue.cc:1243] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:01.705430  3972 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.756331  3972 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:01.711556  3973 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.758860  3973 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:01.760157  3929 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.761716  3929 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:01.725355  3983 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:01.725696 30550 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:01.704018  3559 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.727077 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.715073 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:01.735796  3404 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:01.730302 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:01.736558 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.739969  3968 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.534s	user 0.002s	sys 0.000s
I20260709 13:16:01.737491 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:01.773001  3754 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.741441  3551 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.774525  3559 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:01.776526  3551 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:01.701102  3987 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:01.778230 30705 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:01.747541 31383 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:16:01.748955  3967 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.609s	user 0.004s	sys 0.000s
W20260709 13:16:01.779619 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.715488  3932 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:01.762341  3981 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:01.783787  3981 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.782260  3932 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:01.787151  3981 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:01.763264 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:01.788904  3508 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:01.778051  3987 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:01.798442  3985 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 13:16:01.797797 31396 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:01.767818 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:01.768832 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:01.769098 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 54129.6us
I20260709 13:16:01.805382 31361 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 13:16:01.805382 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67966.8us
I20260709 13:16:01.725140 31367 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:16:01.807103 31367 pending_rounds.cc:85] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:16:01.807385 31367 pending_rounds.cc:107] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:16:01.807215 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:01.807713 31367 raft_consensus.cc:2889] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:01.807991 31361 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:01.797025 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:01.811426 31294 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.560s (SENT)
I20260709 13:16:01.774588  3754 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:01.817775 30626 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:16:01.823123 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55456.4us
I20260709 13:16:01.824899 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.795931 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.818949 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.820339 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:01.791597 30555 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:01.826797 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.838420 30557 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:01.847992 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:01.796095  3946 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:01.849254 30553 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:01.848049 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:01.848153 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34250 (request call id 37) took 1266 ms. Trace:
I20260709 13:16:01.879521 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.883052 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.854933 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:01.856776 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.885881 30558 raft_consensus.cc:2368] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:16:01.887631 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.852804 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.879969 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:01.789520  3181 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:01.902964 30552 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:01.891137 30703 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:01.903530 30632 raft_consensus.cc:2393] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:01.770830  3936 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:01.892308 30550 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:16:01.857173 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:01.888379 30553 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
W20260709 13:16:01.852514 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:01.890468 30698 rpcz_store.cc:276] 0709 13:16:00.553330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.584432 (+ 31102us) service_pool.cc:224] Handling call
0709 13:16:01.819206 (+1234774us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5400343854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed7853 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:01.819355 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41879808}
W20260709 13:16:01.898988 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 78498us
I20260709 13:16:01.914529 30627 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:01.914745 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:01.900211 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.855904 30633 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:16:01.902289 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:01.919503 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.890677 30555 raft_consensus.cc:2376] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17. Re-sending same reply.
I20260709 13:16:01.844905  3607 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:01.820591  3769 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:01.849006  3874 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:16:01.912670  3994 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.216s	user 0.000s	sys 0.002s
I20260709 13:16:01.927788  3872 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:01.930920  3404 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:01.935138 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:01.915786  3834 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:16:01.861068  3508 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:16:01.945226 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:01.942543 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.848083  3856 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:01.889509  3946 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:01.906993  3936 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:01.950085 30626 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:01.967939 30630 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:01.854334  3941 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:01.923787  3181 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:01.912891  3932 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:16:01.947482  3838 raft_consensus.cc:2764] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:16:01.961942 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:01.962883 30596 leader_election.cc:336] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.580s (SENT)
I20260709 13:16:01.961776 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:01.985707 30556 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:01.979871 30703 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:01.949600  3882 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:01.950730  3607 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:01.988365 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73737.3us
I20260709 13:16:01.993865 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:01.946408  3769 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:01.970062  3936 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:02.003731 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:01.989209  3970 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:01.953531  3404 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.013332 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.015475 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:02.016126 30556 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:02.000756  3877 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:01.998139  3870 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:16:02.027386 30556 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:16:02.014164  3838 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:02.000519  3874 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:16:02.034091  3404 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:02.039480  3998 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:16:02.047544 30556 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:02.052220 30555 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:02.016625  3882 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 13:16:02.056684 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55917.3us
I20260709 13:16:02.053992  3607 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.058359 30555 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:16:02.069931 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70686.5us
I20260709 13:16:02.087759  3970 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:02.089257 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2398) took 1400 ms. Trace:
I20260709 13:16:02.021859  3937 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.109851 30551 rpcz_store.cc:276] 0709 13:16:00.689273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.713069 (+ 23796us) service_pool.cc:224] Handling call
0709 13:16:02.079930 (+1366861us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54009ca4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:02.089180 (+  9250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":467032960}
W20260709 13:16:02.091413  3997 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.063s	user 0.000s	sys 0.005s
I20260709 13:16:02.095003  3837 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:16:02.092489 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.056355  3181 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.101814 30558 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:16:02.089895  3977 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.122439  3977 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:02.091192  3999 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
W20260709 13:16:02.115361 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.118884 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:02.119462 30553 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:02.130529 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.131402 30703 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:02.131785 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.126509 30557 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [18.11-20.12]
I20260709 13:16:02.135545 30556 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:02.140218  3837 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.143545 30703 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:02.145181  3837 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.135108  3551 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:02.147629 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.148365  3837 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:02.149590 30702 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:02.153129  3550 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:02.153764  3550 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:02.154067 30700 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:16:02.154134  3550 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:02.144356 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.165247 30698 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:02.166152 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.170408  3769 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:02.171638  3856 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:02.173151  3959 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.981s	user 0.004s	sys 0.000s
I20260709 13:16:02.174197  3550 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.177388  3550 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:02.180603  3937 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:02.051926  3838 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.184034  3838 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:02.185914 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.114562  3514 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:16:02.188462 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1087) took 1238 ms. Trace:
I20260709 13:16:02.189018 30624 rpcz_store.cc:276] 0709 13:16:00.950183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.950877 (+   694us) service_pool.cc:224] Handling call
0709 13:16:02.188114 (+1237237us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54005f37d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:02.188358 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951347072}
I20260709 13:16:02.191551 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:02.193197  3937 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:02.194517  3961 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:02.194962  3961 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:02.195966  3961 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.197391  3961 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:02.204471 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.208855 31367 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:16:02.209110  3932 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:02.209374 31367 pending_rounds.cc:85] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 11
I20260709 13:16:02.210297 31367 pending_rounds.cc:107] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:16:02.210537 31367 raft_consensus.cc:2889] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:02.193606  3977 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:02.211374  3977 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.205906  3946 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.211766 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:02.211815  3837 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.142982 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:02.214495 30551 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:02.248029 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.212576  3946 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:02.211833  3977 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:02.209615  3932 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:02.241315  3872 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:02.295305 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 142194us
I20260709 13:16:02.241804  3837 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.302731 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.302302 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.303465 30698 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:02.309684 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.261464  3985 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:16:02.298161  3838 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:02.282287  3550 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:16:02.303043  3882 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.315881  3941 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:02.331338 30698 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
W20260709 13:16:02.342367 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 215426us
I20260709 13:16:02.343412 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.330922  3514 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:02.335227  3877 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:02.332442  3941 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.353706 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.330293  3882 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.355376 30698 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:02.360591  3936 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.363950 30552 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:02.364554 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.365006  3882 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:02.365710  3882 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:02.354770  3514 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.366891 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.362968  3838 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:02.368916  3514 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:02.369608 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.370137  3837 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:02.369599  3838 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.372916  4016 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:02.374007  4016 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:02.366485 31292 leader_election.cc:400] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:02.322901  3932 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.382762 31292 leader_election.cc:403] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:16:02.395720 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 184010us
I20260709 13:16:02.358042  3941 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:02.403414  3837 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:02.403975  3932 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:02.404457  3769 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.405120  3769 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.410681 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.403578  3985 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.410048  3769 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:02.390748  3965 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.128s	user 0.002s	sys 0.005s
W20260709 13:16:02.416586 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73244.6us
I20260709 13:16:02.391975  3882 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:02.390339  3870 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:02.389909  4006 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.425079 30698 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:02.389309  3972 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:02.421667  3985 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.432284  3985 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:02.422107  3514 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:02.424239  3870 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.441445  3870 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:02.442296  3972 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.426196  3877 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:02.445327  3877 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:02.445851  3877 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:02.447540  3877 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.449554  3877 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:02.451606 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 936) took 1584 ms. Trace:
I20260709 13:16:02.452114 30705 rpcz_store.cc:276] 0709 13:16:00.867409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:00.978923 (+111514us) service_pool.cc:224] Handling call
0709 13:16:01.419815 (+440892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:01.778743 (+358928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.450417 (+671674us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b540037d2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:02.451388 (+   971us) spinlock_profiling.cc:243] Waited 441 ms on lock 0x7b540037d2d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:02.451539 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1836871424}
I20260709 13:16:02.435858  4006 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.456079  4006 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:02.457661  4018 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:02.459264 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:02.461558  4018 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:02.461993  4018 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:02.462957  4011 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:02.463526  4011 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:02.463973  4011 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:02.471864 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.470463 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.475044 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.482875  3559 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:02.483343 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:02.483516  3559 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:02.484596  3559 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.484956 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.486192  3559 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:02.487468  3769 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:02.489756  3181 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.492712  3874 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:02.492645  3181 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.493221  3874 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:02.495555  3181 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:02.489243 30552 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:02.500135 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 403300us
I20260709 13:16:02.503258 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:02.503688 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:02.504153 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.504464 31364 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:02.504290 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:02.505039 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:02.506636 30624 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [15.10-17.11]
I20260709 13:16:02.507153 30633 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:02.509151 30632 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:02.510633 30630 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:02.510859 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.511885 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:02.512166 30625 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:02.513350 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:02.514086 30627 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:02.503279 31361 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:02.514447 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:02.516368 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:02.516965 30552 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:02.518539  3888 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:02.518831  3989 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:02.519239 30705 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:02.519450  3989 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:02.519182  3888 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:02.501873 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.520519  3989 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.520860  3888 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.521238 30706 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:16:02.521965 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 379095us
W20260709 13:16:02.522464 31294 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.025s	sys 0.061s
W20260709 13:16:02.522820 31294 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.025s	sys 0.062s
W20260709 13:16:02.523085 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 668138us
I20260709 13:16:02.527102  3991 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:02.528090 30705 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [15.9-20.12]
I20260709 13:16:02.528427  3870 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.530680  3990 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:02.531411  3990 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:02.531894  3990 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:02.535166  3942 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.536369 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.486456 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.538681 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:02.540102 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:02.543154 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.543802 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:02.545704 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.537003 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.549964 31378 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 13:16:02.551836 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 185489us
I20260709 13:16:02.553853 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:02.486227  3936 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:02.567633 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 80229.7us
I20260709 13:16:02.571905 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.573313 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:02.576624 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.583171 30700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:02.583906 30700 raft_consensus.cc:606] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.585736 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:02.516397  3972 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:02.553189  3937 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:02.494213  3874 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.616266 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.574973  3985 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:02.619910 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.636327 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:02.598063 30552 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:02.631105 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.605602 31361 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:02.665959 31361 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:02.606879 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.607280 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.614173 30698 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:02.614923 30706 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:16:02.668447 31388 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:02.617992 30705 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:02.671137 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:02.671838 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.673030 31383 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 8.
I20260709 13:16:02.582332  3870 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:02.618196 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.580333  3404 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:16:02.602471  3877 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:02.463981  3946 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.128s
I20260709 13:16:02.552735  3769 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:16:02.588495  3852 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 13:16:02.588197  4005 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:16:02.528549  3929 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:16:02.704138 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.554287  3942 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.603819  3937 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:02.579697  3834 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:02.574529  3774 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:02.620584 30701 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:02.620630 30703 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:02.619628 31294 leader_election.cc:400] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:16:02.620767 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.627364 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:02.636327 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:02.718441 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98972.3us
I20260709 13:16:02.718557 31378 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:16:02.622777 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.675491 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.721590 31364 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:02.663827 31394 raft_consensus.cc:2376] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17. Re-sending same reply.
I20260709 13:16:02.667171 31361 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.723419 31361 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:02.694037  3874 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:02.602340  3556 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:02.684664 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.615849  3181 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:02.685377 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:02.680430 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 106758us
W20260709 13:16:02.686566 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114794us
I20260709 13:16:02.579999  3987 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:02.690953  3985 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:02.631104  3849 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:02.751451 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.675534  3754 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:02.705235 31383 raft_consensus.cc:2376] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23. Re-sending same reply.
I20260709 13:16:02.702250 30707 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:02.713493  3937 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:02.680469 30706 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:02.722337 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:02.719539 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:02.731382 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 99151.7us
I20260709 13:16:02.672616  3932 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:16:02.671717  3882 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:16:02.734320 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.775722 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:02.702425  3870 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:02.740881 31388 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:02.739720 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.741575 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:02.696336  3550 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:02.679306  3835 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:16:02.749421 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 172798us
I20260709 13:16:02.788239 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:02.759470 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:02.790386 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.761526 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.693280  3977 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:02.773649 31361 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:16:02.579998  4018 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.711993  3747 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:16:02.780849 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.752679  3774 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:02.843168 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 103788us
I20260709 13:16:02.791144 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.791694 31361 pending_rounds.cc:85] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:16:02.845012 31361 pending_rounds.cc:107] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:16:02.845477 31361 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:02.783890 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 64477.2us
I20260709 13:16:02.760917  4005 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:16:02.839937 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 221866us
I20260709 13:16:02.775729  3939 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:02.874521 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:02.865041 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:02.870865 30551 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:02.863817 31378 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:02.867086 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.759879  3181 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:02.880784 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:02.771307  3754 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.908221 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.917701 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.884936  3987 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:02.797539  3550 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:02.892730  3858 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:02.887559  4005 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:02.941633 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 202052us
I20260709 13:16:02.943747  3774 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:02.944321  4041 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:16:02.945827  4028 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:16:02.947036  4024 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
W20260709 13:16:02.949678  4025 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:16:02.949549  4035 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:02.926944  4032 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:02.938401  3870 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:02.946419  3834 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:16:02.956735 31364 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:02.965283 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.928912  3932 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:02.922843  4011 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:02.942369  3987 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:03.011850 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46707.7us
W20260709 13:16:03.015650 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 144663us
I20260709 13:16:02.953182  4038 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:02.999926  3941 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.567s	user 0.006s	sys 0.006s
I20260709 13:16:03.019189 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:02.980340  3858 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:02.999338  3961 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.025149 30630 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 13:16:03.036556 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.029755 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:03.030988 30629 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:03.000959  4035 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.033028 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.044445 30702 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:03.044992 30627 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:03.033749 30633 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:03.012975  3882 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:03.014740  3987 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:03.026692  3932 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:16:03.032323  4005 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.062932  3981 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:03.067449  4036 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 13:16:03.068172 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.127593 30632 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:03.068872  4020 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.071332  3977 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:03.134011 30699 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:03.071448  3747 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:03.072440  3985 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.073846  3929 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:03.074052  4034 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:03.074579 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.076138  4011 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:03.077808  3870 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:03.078447 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.071364  4023 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:16:03.023365  4041 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:03.080006 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.083051 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.084384  3834 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:03.087692 30630 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:03.089643  3945 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:03.094784 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:03.022122  3942 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:03.100409 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.107878 30633 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:03.139525  4029 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:03.184230 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:03.109802 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 152419us
I20260709 13:16:03.111999 30551 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:16:03.112358 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82712.8us
I20260709 13:16:02.999135  3181 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.052719  3769 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:03.124732 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.126554 30706 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:16:03.144012  3929 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.027766  3941 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:03.132175 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:03.140033 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:03.053687  3961 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.215296 30552 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:16:03.215837 30552 pending_rounds.cc:85] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 8
I20260709 13:16:03.216073 30552 pending_rounds.cc:107] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:16:03.156417 30701 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:16:03.156247 30624 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:03.148697 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74227.7us
I20260709 13:16:03.161687 30631 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:03.217594 30552 raft_consensus.cc:2889] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:03.158473  3754 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.159889  3936 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:03.136960  4020 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.223816  4020 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:03.226724  3550 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:03.227262  3929 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.228778  3929 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:03.229478  3941 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.230578  4034 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:03.231390 31391 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:16:03.141052  4035 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.215360 30627 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:03.216742  4050 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:03.232546  4050 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.148200  3838 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:16:03.129441  3981 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.214707  3769 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.233721  3882 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:16:03.160744  4041 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:03.243453  3882 consensus_queue.cc:1243] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:03.220978  3949 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.244135  3882 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:03.244621  3882 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.244474  3949 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.217610  4023 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:16:03.246870  4023 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:03.233033  3936 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:03.247357  4023 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.247658  3936 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:03.248340  4023 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.221012 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:03.238202  3932 consensus_queue.cc:1175] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 19 status { last_received { term: 16 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:03.127080  3858 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.251739  3973 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:03.243597  4041 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:03.255193  3838 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:03.255772  3838 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:03.216988  4044 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:03.235674  4050 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.251065 30706 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:03.215363 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:03.264253 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47350.7us
W20260709 13:16:03.266745 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.269111 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53853.1us
W20260709 13:16:03.269659 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.270054 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.274206 31389 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:03.278944 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.282989  3961 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.238793  4034 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.256865  3838 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.285202 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.286474  3838 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:03.287794  3551 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.288475  3551 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:03.275094  4018 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:03.279933  3939 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:03.291888  3939 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.294641  3949 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:03.295990  3769 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.264420  3852 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.296623  3981 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:03.297055 30627 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:03.297238  3852 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.282994 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:03.303766 30551 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:03.308394 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.308426 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:03.308844 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:03.310699 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.309768 31361 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:03.266881  4023 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:03.317313 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:03.319662 30555 raft_consensus.cc:2410] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:16:03.312312 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.015s	sys 0.015s
W20260709 13:16:03.321586 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.015s	sys 0.015s
W20260709 13:16:03.322157 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 685880us
I20260709 13:16:03.317142 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:03.326875 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:03.326879 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.336123 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:03.298221  4055 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.350399 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:03.285234  4034 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.369066 30551 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:03.361683 31378 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:03.315250  3989 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:03.355566 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.364384 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:03.245584  3882 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:03.366043 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.367369 30629 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:03.319927  4050 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:03.379154 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:03.382440 31396 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:16:03.382134 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.398940 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.402876  3981 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:03.404870  3970 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:03.404987  3981 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:03.405598  3970 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.406056  3970 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:03.407622  4057 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:16:03.409936  4034 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:03.334460  3874 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:16:03.342461  4045 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:03.415110 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:03.415683 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67571.3us
I20260709 13:16:03.421206 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:03.370126  4039 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:03.341439  3941 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:16:03.436684 31383 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:03.393600  4061 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.411368  3888 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.454175 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.411958  4056 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:03.438630  4032 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:03.470010 30556 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:03.384465  4044 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:16:03.376396  3181 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:16:03.473126 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.484515 30631 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:16:03.478359 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.479750 30626 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:03.479060 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.369863  4055 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:03.470996  3990 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.939s	user 0.010s	sys 0.008s
I20260709 13:16:03.488041 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:03.490326 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.532s (SENT)
W20260709 13:16:03.499591 30596 leader_election.cc:336] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.615s (SENT)
I20260709 13:16:03.500167 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.501412 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.501761  4070 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:03.464011  4061 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.503504  4063 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 13:16:03.506101  4061 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:03.508049  4065 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:16:03.511389  3852 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:03.443956  4029 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:16:03.490252  3181 consensus_queue.cc:1243] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:03.483160  4032 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.525355 30626 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:03.527793  3945 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:03.481655 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 139113us
I20260709 13:16:03.530540 31361 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:03.535823 30626 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:03.481858  4006 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:03.562896 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:03.471632  3874 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:16:03.465080  3888 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:03.569643 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.570323 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:03.574664 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:03.615962 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.540107  4032 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:03.618067 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:03.561353  3937 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.752s	user 0.012s	sys 0.002s
I20260709 13:16:03.624430 31383 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:03.629674 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:03.635484  4066 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:16:03.643401  3970 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.647655  3874 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:03.651222  3874 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:03.651629  3874 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.652079  3882 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:03.652531  3874 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:03.653294  3987 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.526s	user 0.003s	sys 0.010s
I20260709 13:16:03.558880  4006 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.654583  4006 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.655557  3929 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:03.656041  4006 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:03.657639 31367 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:16:03.658120 31367 pending_rounds.cc:85] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:03.658463 31367 pending_rounds.cc:107] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:16:03.658761 31367 raft_consensus.cc:2889] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:03.649170  3870 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:03.649946  4055 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:03.663046 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:03.659930 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 278894us
I20260709 13:16:03.660233 30627 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:16:03.665331 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.664601 30630 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:03.554162  3985 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:03.666901 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 163805us
I20260709 13:16:03.645860  3936 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.667732 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.653797  3987 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:03.668921 30596 leader_election.cc:336] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.807s (SENT)
I20260709 13:16:03.669998 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.529075  3990 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:03.675769 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.680930 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:03.688293 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.691180 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.691828 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:03.700531 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:03.701272 30628 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:03.704583 30698 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:03.706830 30698 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:03.709594  4069 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:03.710525  4039 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:03.711534  4039 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:03.675416  4034 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:03.661602  3937 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:03.720330 31383 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:16:03.732466  4072 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:03.733007  4072 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.733896  4072 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:03.737596  4049 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:03.738363  4072 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:03.741611  3973 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:03.742682  3888 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:03.742900  3973 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:03.744112  4049 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:03.744565  4049 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:03.746294 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:03.745536  4049 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:03.752791 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.668489  3985 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:03.756173 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.748154 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:03.700312  3945 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:03.699816  4042 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:03.781823 31396 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:03.703289  4032 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:03.808151 30558 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:03.730300  3870 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:03.729745  4034 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:03.822459 30554 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:03.793512  3887 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:03.826908 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 186485us
I20260709 13:16:03.799429  4042 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:03.833798 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34250 (request call id 38) took 1309 ms. Trace:
I20260709 13:16:03.814015 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.840086 31396 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:16:03.840922 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.841475 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 939) took 1344 ms. Trace:
I20260709 13:16:03.843863 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1685) took 2031 ms. Trace:
I20260709 13:16:03.815176  4073 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:03.836819 30700 rpcz_store.cc:276] 0709 13:16:02.524828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.552522 (+ 27694us) service_pool.cc:224] Handling call
0709 13:16:03.817520 (+1264998us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54005279a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5a520 00007f1806ed7652 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6
0709 13:16:03.833694 (+ 16174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300999936}
I20260709 13:16:03.846969 30707 rpcz_store.cc:276] 0709 13:16:01.812460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:01.812748 (+   288us) service_pool.cc:224] Handling call
0709 13:16:01.813229 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:02.760354 (+947125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.843571 (+1083217us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b54003d8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:03.843792 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043792256}
I20260709 13:16:03.846855 30702 rpcz_store.cc:276] 0709 13:16:02.497457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.516270 (+ 18813us) service_pool.cc:224] Handling call
0709 13:16:02.516607 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.045065 (+528458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:03.841116 (+796051us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b54005f05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:03.841390 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1213340928}
I20260709 13:16:03.819600  3977 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.820641  3870 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:03.858798 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.824409  4034 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:03.861901 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.870518 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:03.877894 31396 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:03.882184 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.883255 30707 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:16:03.884469 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:03.896250 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:03.901059 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:03.837088  4049 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:03.838061  4042 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:03.881649 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:03.907573 30555 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:03.912093 30630 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:03.913271 30626 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:03.856602  3838 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.854521  4061 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.915825 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.916503 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.924948 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:03.924999 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:03.928723 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:03.939744 31294 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:03.924054  3941 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:03.861414  3977 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:03.932163  3858 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:16:04.014324 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.015s (SENT)
W20260709 13:16:03.984236 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 204752us
I20260709 13:16:03.861560  4073 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:03.943363  4051 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:03.931439  3514 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.103s
W20260709 13:16:04.033267  4064 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.686s	user 0.000s	sys 0.003s
I20260709 13:16:03.964396  4047 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:03.927994  3838 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:04.128985 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.932667  4061 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.130321 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.033267  3941 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:04.121510 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:04.123013 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 183425us
I20260709 13:16:04.092164  3991 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.038527  4083 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:04.135473 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.041184  3858 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:04.119303  4050 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.736s	user 0.013s	sys 0.008s
W20260709 13:16:04.134361 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.053804  3987 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:04.143461 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 218590us
I20260709 13:16:04.093948  4020 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.146252 30557 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
W20260709 13:16:04.124765  4018 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.833s	user 0.014s	sys 0.009s
I20260709 13:16:04.154688 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.155975 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:03.963908  4059 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.430s
I20260709 13:16:04.122941  4085 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:03.950495  3870 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:04.094002  4031 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.372s
I20260709 13:16:04.147959  3941 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:04.122654  3837 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:16:04.160085  3937 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:04.103443  4051 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.163394  3858 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:04.184212  4085 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:04.182842  4020 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:04.212924  4027 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.214s	user 0.007s	sys 0.000s
I20260709 13:16:04.187849  4050 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:04.213634  4050 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:04.214278  4050 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:04.215574  4020 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:04.215991  4078 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:16:04.216529  3514 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:04.212814  4085 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:04.217128  3514 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:04.218093  4082 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:04.218361  3514 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:04.219877  4080 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 13:16:04.221086 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 944) took 1698 ms. Trace:
I20260709 13:16:04.221572 30704 rpcz_store.cc:276] 0709 13:16:02.523058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.529689 (+  6631us) service_pool.cc:224] Handling call
0709 13:16:02.530064 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:02.530458 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:02.530495 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:02.535323 (+  4828us) log.cc:844] Serialized 26 byte log entry
0709 13:16:02.667083 (+131760us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:02.667163 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:02.667279 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:03.815946 (+1148667us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:04.220710 (+404764us) raft_consensus.cc:1685] finished
0709 13:16:04.220735 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.221013 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5588,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":15509,"spinlock_wait_cycles":24856320,"thread_start_us":131216,"threads_started":1,"wal-append.queue_time_us":209912}
I20260709 13:16:04.163903  3991 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:04.186128  4016 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.202591  3887 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:04.171535  4083 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:04.270946  4085 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.274042  4085 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.225921  4051 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:04.234553  3882 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.000s
I20260709 13:16:04.236685  3977 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.241945  4016 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.239933  3970 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.253291  4083 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:04.244406  3887 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:04.237089  3932 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:04.238763  3991 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.328981  4073 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:04.332211  4092 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:16:04.285862  4041 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.033s	user 0.005s	sys 0.011s
I20260709 13:16:04.328009  3887 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:04.337852 30698 pending_rounds.cc:85] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 7
I20260709 13:16:04.344940  4061 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.343402  4041 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:04.347119 31389 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:04.347628  3838 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:04.350417  4041 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:04.353216  4023 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.026s	user 0.014s	sys 0.000s
I20260709 13:16:04.325850  4095 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:16:04.338428 30698 pending_rounds.cc:107] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:16:04.361217 30698 raft_consensus.cc:2889] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:04.355101  4050 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.382532 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1045) took 1028 ms. Trace:
I20260709 13:16:04.383345 30555 rpcz_store.cc:276] 0709 13:16:03.354437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.354660 (+   223us) service_pool.cc:224] Handling call
0709 13:16:03.355073 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.909694 (+554621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.382092 (+472398us) spinlock_profiling.cc:243] Waited 525 ms on lock 0x7b54000a0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:04.382424 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208353152}
I20260709 13:16:04.404451  4035 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:04.407330 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1699) took 1130 ms. Trace:
I20260709 13:16:04.439424 31389 rpcz_store.cc:276] 0709 13:16:03.276721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.277354 (+   633us) service_pool.cc:224] Handling call
0709 13:16:03.277710 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:04.347616 (+1069906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.406871 (+ 59255us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400113d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:04.407221 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2458928896}
W20260709 13:16:04.418295 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.420357  3936 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:04.388271 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 266826us
W20260709 13:16:04.437183 30624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.005s	sys 0.014s
W20260709 13:16:04.446014 30624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.006s	sys 0.014s
W20260709 13:16:04.388616  4006 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.732s	user 0.015s	sys 0.000s
W20260709 13:16:04.447821 30559 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.001s	sys 0.018s
W20260709 13:16:04.448289 30559 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.002s	sys 0.018s
W20260709 13:16:04.444857 30699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.012s
W20260709 13:16:04.450631 30699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.013s
I20260709 13:16:04.452031 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:04.455451 30703 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:04.455585 30554 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:16:04.456277 30703 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:04.456682 30703 raft_consensus.cc:614] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:04.438092  4087 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.151s	user 0.005s	sys 0.000s
I20260709 13:16:04.458320 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:04.460886 30554 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:04.446579  4034 consensus_meta.cc:287] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.005s
I20260709 13:16:04.447149  3973 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:04.481062 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 338829us
I20260709 13:16:04.455739  4019 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:04.485817 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 320002us
W20260709 13:16:04.488299 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 395886us
W20260709 13:16:04.493889 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.496454 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:04.511570 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.496843 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:04.500300 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:04.512817 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.500866 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:04.498193 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.490235 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:04.517059 31389 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:04.508280 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:04.512614 30625 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:04.530800 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:04.488672  4020 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
W20260709 13:16:04.626555 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 95868.2us
I20260709 13:16:04.633742 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:04.635785 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:04.640187 31367 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:04.478790  4034 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:04.649214 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 159080us
I20260709 13:16:04.666112 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:04.487075  3973 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.499199  4019 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:04.716518 30700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.020s	sys 0.081s
W20260709 13:16:04.717401 30700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.020s	sys 0.082s
I20260709 13:16:04.647699  4093 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.708688 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:04.715695 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:04.730185 30701 raft_consensus.cc:606] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:04.730962 30701 raft_consensus.cc:614] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:04.706851 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:04.732517 31367 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:16:04.506299  3936 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.140s
I20260709 13:16:04.625240  4061 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:04.624164  3508 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:04.724407 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:04.725772 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:04.741175 30699 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:04.634558  4040 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.623360  4059 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:04.624979  3929 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:04.623888  4103 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:04.780001  3929 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.737638 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:04.647958  4050 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.783893 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:04.784415  4041 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:04.785075  4041 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:04.785524 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:04.640440  4023 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.787048  4023 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.787619  4046 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:04.788249  4046 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:04.738834  4099 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.237s	user 0.003s	sys 0.002s
I20260709 13:16:04.640290  4073 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.791402  4073 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.792176 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2145) took 1076 ms. Trace:
I20260709 13:16:04.793063  3985 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.792694 30707 rpcz_store.cc:276] 0709 13:16:03.715465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.950025 (+234560us) service_pool.cc:224] Handling call
0709 13:16:03.950501 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.951069 (+   568us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.951130 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.032688 (+ 81558us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.315766 (+283078us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:04.317825 (+  2059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.318189 (+   364us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.791919 (+473730us) raft_consensus.cc:1685] finished
0709 13:16:04.791946 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.792107 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1321,"raft.queue_time_us":133,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":55316,"thread_start_us":172980,"threads_started":1,"wal-append.queue_time_us":346209}
I20260709 13:16:04.796334  4083 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.797039  3551 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:04.796917  4083 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.797713  3551 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:04.798076  3551 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:04.799381  4040 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:04.804629  4049 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.888s	user 0.014s	sys 0.003s
I20260709 13:16:04.807065  4093 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.809084 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1812) took 1253 ms. Trace:
I20260709 13:16:04.786141  4041 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.774598  4059 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:04.775894 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1818) took 1003 ms. Trace:
I20260709 13:16:04.776286  3872 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:04.875471 31388 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 13:16:04.715658  4087 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:16:04.876703 31388 pending_rounds.cc:85] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:16:04.889016 31388 pending_rounds.cc:107] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:16:04.889400 31388 raft_consensus.cc:2889] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:04.835628 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1901) took 1576 ms. Trace:
W20260709 13:16:04.869877 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 236348us
I20260709 13:16:04.859727 31383 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:16:04.891741 31383 pending_rounds.cc:85] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:04.892175 31383 pending_rounds.cc:107] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:16:04.892576 31383 raft_consensus.cc:2889] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:04.755262 30550 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:04.790102  4023 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.890103 30558 rpcz_store.cc:276] 0709 13:16:03.259621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.259924 (+   303us) service_pool.cc:224] Handling call
0709 13:16:03.260344 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.811228 (+550884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.835217 (+1023989us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:04.835531 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259255680}
I20260709 13:16:04.876590 30626 rpcz_store.cc:276] 0709 13:16:03.555582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.555854 (+   272us) service_pool.cc:224] Handling call
0709 13:16:03.913053 (+357199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.913685 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.808811 (+895126us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400055ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:04.808988 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":676848896}
W20260709 13:16:04.874716 31391 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1780) took 2265 ms (client timeout 2015 ms). Trace:
I20260709 13:16:04.876536 30624 rpcz_store.cc:276] 0709 13:16:03.772561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.772716 (+   155us) service_pool.cc:224] Handling call
0709 13:16:03.773115 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.822922 (+ 49807us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.822976 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.832725 (+  9749us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.446835 (+614110us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:04.446906 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.768797 (+321891us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b54001fbb54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:04.768869 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.775513 (+  6644us) raft_consensus.cc:1685] finished
0709 13:16:04.775537 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.775817 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6144,"raft.queue_time_us":7705,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":7052,"spinlock_wait_cycles":98967936,"thread_start_us":613687,"threads_started":1,"wal-append.queue_time_us":939477}
I20260709 13:16:04.894073 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.894364 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.896593 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:04.899729 30630 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:04.900871 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.901602 30624 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:04.790313  4046 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.890972 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:04.906190  3888 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:04.898967 31391 rpcz_store.cc:269] 0709 13:16:02.609182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.761141 (+151959us) service_pool.cc:224] Handling call
0709 13:16:04.874321 (+2113180us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400460ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:04.874566 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010981248}
I20260709 13:16:04.907073  4050 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.907888  4050 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.908176 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:04.908900 31391 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:04.909138 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:04.900416 30629 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:04.906031 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:04.917457 30702 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
W20260709 13:16:04.820659  3941 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.002s
W20260709 13:16:04.823621  4072 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.085s	user 0.013s	sys 0.012s
I20260709 13:16:04.913905 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2139) took 1240 ms. Trace:
W20260709 13:16:04.857718  3887 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.003s
I20260709 13:16:04.817401  4093 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.870378  4083 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.935029 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:04.872486  4019 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.928292 30705 rpcz_store.cc:276] 0709 13:16:03.673882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.674037 (+   155us) service_pool.cc:224] Handling call
0709 13:16:03.674381 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.674662 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:03.824743 (+150081us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.913265 (+1088522us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b1800209740. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:16:04.913499 (+   234us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:04.913553 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:04.913609 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.913693 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.913837 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":363278,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"spinlock_wait_cycles":45001600,"thread_start_us":430716,"threads_started":2,"wal-append.queue_time_us":380633}
I20260709 13:16:04.871294  4005 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:04.943677 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:04.871623  3870 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:04.726670  4047 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.208s
I20260709 13:16:04.724905  4063 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:04.838316  3939 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:04.926432 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:04.711457  4020 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.201s
I20260709 13:16:04.813540  4034 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:04.794447  4073 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:04.890367  4042 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.964s	user 0.010s	sys 0.010s
I20260709 13:16:04.864248  3929 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.922533 31383 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:04.887964  3872 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:04.967196  3801 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:16:04.903198  3858 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.675s	user 0.012s	sys 0.002s
I20260709 13:16:04.946587 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 953) took 1373 ms. Trace:
I20260709 13:16:04.809360  3936 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:04.797063  3973 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:04.870939  4040 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:04.809875  3882 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 13:16:04.906862  4103 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:16:05.024740  4103 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:04.993199  3801 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.027041  3801 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:04.920691  3856 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:04.966323 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:05.032094 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 65872.1us
I20260709 13:16:05.033526 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:04.814373  3508 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:04.928344  3769 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:04.972227  4100 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:04.972298  4006 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.061767  4006 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:05.063926  4110 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:16:05.064608  4006 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:04.949182  4093 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.960225  4047 consensus_queue.cc:1243] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:04.963052  4019 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:04.907073  3888 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:04.987632 30698 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.004s	sys 0.011s
I20260709 13:16:04.735734  4016 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:04.962774  4050 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:04.989789 31367 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
W20260709 13:16:04.991273 30559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.007s	sys 0.007s
I20260709 13:16:04.934229  3941 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:04.964344  3939 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:04.971016  3970 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:04.941423  3887 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:05.000553 31383 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:05.135471  4111 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:16:05.003404 30705 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:05.136404 30705 raft_consensus.cc:606] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:05.136991 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1706) took 1287 ms. Trace:
I20260709 13:16:04.863787  3551 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:04.994735 30706 rpcz_store.cc:276] 0709 13:16:03.572891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.573055 (+   164us) service_pool.cc:224] Handling call
0709 13:16:03.573463 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:03.619994 (+ 46531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.620032 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:03.659192 (+ 39160us) log.cc:844] Serialized 52 byte log entry
0709 13:16:03.941796 (+282604us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:04.132213 (+190417us) log.cc:844] Serialized 14 byte log entry
0709 13:16:04.399441 (+267228us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:04.459857 (+ 60416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.459955 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:04.942477 (+482522us) raft_consensus.cc:1685] finished
0709 13:16:04.946279 (+  3802us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:04.946485 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":266795,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":56536,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":18166,"spinlock_wait_cycles":8217856,"thread_start_us":777696,"threads_started":3,"wal-append.queue_time_us":627724}
I20260709 13:16:04.966185  4063 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:05.137534 30700 rpcz_store.cc:276] 0709 13:16:03.849558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.105838 (+256280us) service_pool.cc:224] Handling call
0709 13:16:04.106431 (+   593us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.115605 (+  9174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.115653 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.116414 (+   761us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.718060 (+601646us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:04.718130 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:04.718245 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.136693 (+418448us) raft_consensus.cc:1685] finished
0709 13:16:05.136724 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.136913 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":45752,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":14434,"thread_start_us":555968,"threads_started":1,"wal-append.queue_time_us":933479}
I20260709 13:16:05.021627  4031 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.015169 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:05.139762 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124732us
W20260709 13:16:05.036146 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.032800  3834 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.041437  3508 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.060546  4100 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:05.073621 30553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.005s	sys 0.014s
I20260709 13:16:05.003294  3936 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:05.038339  4035 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.589s	user 0.005s	sys 0.011s
W20260709 13:16:05.088234 30698 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.005s	sys 0.011s
W20260709 13:16:05.125160 30559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.007s	sys 0.007s
I20260709 13:16:05.161897 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1047) took 1223 ms. Trace:
I20260709 13:16:05.081208  4109 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:04.986583  4042 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:05.129113 30552 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.014s	sys 0.006s
I20260709 13:16:05.054857  4103 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:05.141845 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.008363  3858 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:05.153493 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.156721 30633 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:05.158684 30553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.006s	sys 0.014s
I20260709 13:16:05.125033 31367 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 11
I20260709 13:16:05.176883 31367 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:16:05.177280 31367 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:05.147369  3970 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.185499 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:05.160380 31394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.033s	sys 0.057s
W20260709 13:16:05.189802 31394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.034s	sys 0.057s
I20260709 13:16:05.122324  4104 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.122471  4093 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.173993 31364 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:16:05.162292 30559 rpcz_store.cc:276] 0709 13:16:03.938029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.938218 (+   189us) service_pool.cc:224] Handling call
0709 13:16:03.938564 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:03.938872 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:04.448705 (+509833us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.161513 (+712808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.161572 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:05.161617 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.161676 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.161827 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":588147,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6602,"thread_start_us":1221852,"threads_started":2}
I20260709 13:16:05.081080  4019 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.203830 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:05.158011  4016 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:05.194314 30704 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
W20260709 13:16:05.171651 30552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.017s	sys 0.006s
W20260709 13:16:05.163055 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.000s	sys 0.023s
W20260709 13:16:05.196661 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.200675 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:05.175402  4049 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.082554  3872 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.180174  3929 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.187412  3970 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:05.212014 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:05.212522 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.000s	sys 0.023s
I20260709 13:16:05.215648 30707 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:05.158254  3834 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:05.250032 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.216936 30705 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:05.240350 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 731581us
I20260709 13:16:05.160382  3973 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:16:05.159332  4031 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:05.211018  4016 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.196007  4059 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.207880  4019 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:05.204864  4055 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:05.286026 31389 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.002s	sys 0.015s
I20260709 13:16:05.179180  4109 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:05.295416 31361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.010s	sys 0.008s
I20260709 13:16:05.208630  4061 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:05.296259 31361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.011s	sys 0.008s
W20260709 13:16:05.295562 31389 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.003s	sys 0.016s
W20260709 13:16:05.290748 30625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.000s	sys 0.018s
W20260709 13:16:05.297561 30625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.000s	sys 0.019s
I20260709 13:16:05.218088  3887 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.252942  4035 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.278384  3872 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:05.312109 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1779) took 2690 ms. Trace:
I20260709 13:16:05.322103 31388 rpcz_store.cc:276] 0709 13:16:02.608880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:02.792390 (+183510us) service_pool.cc:224] Handling call
0709 13:16:04.875036 (+2082646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:04.897603 (+ 22567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.298788 (+401185us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b5400460ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:05.299047 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4788487040}
I20260709 13:16:05.198076  4104 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:05.248816  4049 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:05.248502  3929 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.293957  4016 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:05.338773 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.839s	user 0.022s	sys 0.030s
I20260709 13:16:05.302639  4085 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:05.218585  3936 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:05.343356 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.022s	sys 0.030s
I20260709 13:16:05.333045 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:05.359483 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 859824us
I20260709 13:16:05.310321  4059 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:16:05.320873  4055 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.363703 31378 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:16:05.378466 31378 pending_rounds.cc:85] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:05.378793 31378 pending_rounds.cc:107] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:16:05.379107 31378 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:05.379101 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46141.2us
I20260709 13:16:05.381065 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:05.370014  4104 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.352249  4085 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:05.372609  4117 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 13:16:05.373672  4115 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:16:05.383798 30632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.042s	sys 0.022s
I20260709 13:16:05.307994  4061 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:05.392721 30632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.043s	sys 0.022s
I20260709 13:16:05.386730  3932 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.396263  3932 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:05.384294  4063 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:05.399346  4063 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:05.400004  4063 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:05.327723  4035 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.398487  3874 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:05.378822 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:05.324033  4077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.001s	sys 0.030s
I20260709 13:16:05.388227  3936 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:05.388263  4102 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:05.386197  3973 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:05.384262 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:05.425004 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 404571us
W20260709 13:16:05.431545 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 388139us
I20260709 13:16:05.407739  3754 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.435144 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:05.438220 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54058.9us
W20260709 13:16:05.386523 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.664s	user 0.018s	sys 0.012s
W20260709 13:16:05.440718 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.018s	sys 0.012s
I20260709 13:16:05.441843 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:05.441171 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 718918us
I20260709 13:16:05.414007  3941 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.411347  3888 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:05.451844 30700 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:05.452247 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:05.453274 30699 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:05.432729 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:05.458468 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:05.462044 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:05.455989 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:05.464847 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:05.468286  4093 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:05.426681  3858 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.422875  4119 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:05.446390  4100 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:05.452639  3754 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:05.512040  4116 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:16:05.446722  4042 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:05.575344 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:05.600335 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.470551  4063 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:05.735256  4031 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.736785  3834 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.841357  4041 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.739699 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1909) took 1601 ms. Trace:
I20260709 13:16:05.758685  3973 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:16:05.759850  4101 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:05.411024  3887 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:16:05.851189  4143 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:05.814499  4138 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:05.852996  4138 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:05.853271  3887 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:05.853541  4138 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:05.450603  4102 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:05.752795 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 294407us
I20260709 13:16:05.755096 30554 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:05.813843  3181 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.857784  3181 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:05.858516 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:05.859356  4126 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 13:16:05.860222 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:05.861742  4122 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:16:05.756340 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:05.863353 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1896) took 1344 ms. Trace:
W20260709 13:16:05.865075  4019 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.002s
I20260709 13:16:05.863914 31389 rpcz_store.cc:276] 0709 13:16:04.519281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.598739 (+ 79458us) service_pool.cc:224] Handling call
0709 13:16:04.599216 (+   477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.643093 (+ 43877us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.643147 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.643857 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.297058 (+653201us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:05.297124 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.297199 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.862973 (+565774us) raft_consensus.cc:1685] finished
0709 13:16:05.863012 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.863268 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43343,"raft.queue_time_us":108067,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":24458,"thread_start_us":559802,"threads_started":1,"wal-append.queue_time_us":829866}
I20260709 13:16:05.865628  4019 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:05.862289 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:05.866677  3754 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.867128  4136 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:05.867641  4136 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:05.836521  4100 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.868727  4136 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:05.869143  4100 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:05.771111 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:05.871762  4100 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.871994 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:05.771423 30700 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:05.874920  3837 raft_consensus.cc:993] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:16:05.875254 30633 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:05.875690  3837 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:05.879961 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:05.880842  4011 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:05.881704 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:05.882042  4020 raft_consensus.cc:1081] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Signalling peer acf852f85f47483d99cbe4020baee627 to start an election
I20260709 13:16:05.882434  4040 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:05.882753  4130 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.883037  4040 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:05.883525  4130 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:05.884253  4040 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:05.885517  4040 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:05.888111 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:05.814079  4119 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.883554 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:05.889979  4119 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:05.890983 30631 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:05.893397  3769 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:05.881712  4011 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:05.854408  3887 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:05.894937  4011 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:05.895454  4011 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:05.895293  3887 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:05.781028 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:05.896690 30703 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:05.769768 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:05.764779 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1709) took 1841 ms. Trace:
I20260709 13:16:05.898202  3887 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.750756  4103 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.758904  3941 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:05.845608  4049 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:05.998951  3941 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.001614  3941 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:05.789821  3801 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.742s	user 0.024s	sys 0.005s
I20260709 13:16:06.002418  3801 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:05.762705  3870 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.816s	user 0.014s	sys 0.011s
W20260709 13:16:05.794581  4112 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.761s	user 0.010s	sys 0.009s
I20260709 13:16:05.763134  3888 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:05.838572 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:05.839279  3858 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.839811  4077 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:06.016878  3858 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.017004  4077 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:05.741093 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1707) took 1891 ms. Trace:
I20260709 13:16:05.842830 30556 rpcz_store.cc:276] 0709 13:16:04.138425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.138706 (+   281us) service_pool.cc:224] Handling call
0709 13:16:04.168922 (+ 30216us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.249976 (+1081054us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.250036 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:05.250100 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:05.250148 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.739310 (+489162us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400291154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:05.739383 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.739611 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485525760}
W20260709 13:16:06.019623  4114 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.610s	user 0.003s	sys 0.001s
I20260709 13:16:05.800247  4109 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.860576  3181 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.895610  3973 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:05.814018  4142 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:05.898454 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1324) took 1766 ms. Trace:
I20260709 13:16:05.898361 30698 rpcz_store.cc:276] 0709 13:16:03.923288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.336880 (+413592us) service_pool.cc:224] Handling call
0709 13:16:04.337471 (+   591us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:04.369469 (+ 31998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.369520 (+    51us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:04.375614 (+  6094us) log.cc:844] Serialized 78 byte log entry
0709 13:16:05.161061 (+785447us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:05.161108 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.161183 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.758596 (+597413us) raft_consensus.cc:1685] finished
0709 13:16:05.760733 (+  2137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.764681 (+  3948us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1288,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":4926,"thread_start_us":716891,"threads_started":1,"wal-append.queue_time_us":787578}
W20260709 13:16:05.889444  3970 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.005s
W20260709 13:16:05.900449  3872 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.570s	user 0.015s	sys 0.002s
I20260709 13:16:05.900663  3837 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:06.064951  3834 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:05.901654 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1790) took 2283 ms. Trace:
W20260709 13:16:05.902062  4135 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:05.842063  4041 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.902658 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1908) took 1769 ms. Trace:
W20260709 13:16:05.904457 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.083987 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:05.905068  4016 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.016s
I20260709 13:16:05.905920 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:16:05.722784 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.005s	sys 0.008s
W20260709 13:16:06.092789 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.005s	sys 0.009s
I20260709 13:16:06.008785  4063 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:05.910146 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1046) took 1977 ms. Trace:
I20260709 13:16:05.910724  3990 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:05.899834 31364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1702) took 2592 ms (client timeout 1554 ms). Trace:
I20260709 13:16:05.912429  4019 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:05.913596 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 65332.3us
I20260709 13:16:06.102574 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:05.886216  4130 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:05.777084  3942 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.097663 30553 rpcz_store.cc:276] 0709 13:16:03.932666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.932862 (+   196us) service_pool.cc:224] Handling call
0709 13:16:03.933252 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.947079 (+ 13827us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:04.381144 (+434065us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.538512 (+1157368us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.890821 (+352309us) spinlock_profiling.cc:243] Waited 6.83 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:05.891370 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.891586 (+   216us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:05.891635 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.891746 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.909763 (+ 18017us) raft_consensus.cc:1685] finished
0709 13:16:05.909891 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.910068 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18527,"prepare.queue_time_us":754844,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":168215,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":18859,"spinlock_wait_cycles":15706240,"thread_start_us":1378812,"threads_started":3,"wal-append.queue_time_us":915844}
I20260709 13:16:05.925879  4085 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.926226 30633 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:05.949707  4061 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.116420 30558 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:06.113710  3990 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:16:06.116888 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:05.952886 30378 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:06.118716 30378 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:06.119041  3942 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.119715  3858 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.120846 30624 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746"
dest_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:46186
I20260709 13:16:06.120606 30378 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 18, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 18 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:06.121837  3942 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:05.953954  3939 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.116461  4130 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:06.123064 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:06.123476  3939 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:06.124596 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:05.957324  4055 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:05.934278  3551 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:06.125232 30633 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:05.959568  3932 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:05.994911  4103 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.997618  4119 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:05.990057  3929 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.004563 30702 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:06.128963  4103 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:06.043259 31294 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.548s (SENT) [suppressed 33 similar messages]
W20260709 13:16:05.985704 30379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 136) took 37070 ms (client timeout 15000 ms). Trace:
I20260709 13:16:06.116439 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:06.064397 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54513.4us
I20260709 13:16:06.131960 30629 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:06.133754 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:05.454762 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:06.135160 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 680473us
I20260709 13:16:06.135293  3929 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:06.135907  3929 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:06.136384  3929 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:06.136143 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:05.918308  4035 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.139034  4035 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:06.065950 31378 rpcz_store.cc:276] 0709 13:16:03.618258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.618593 (+   335us) service_pool.cc:224] Handling call
0709 13:16:03.618962 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.386402 (+1767440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.901377 (+514975us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400683b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:05.901578 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4011511040}
I20260709 13:16:06.139532 31389 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:16:06.140067 31389 pending_rounds.cc:85] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:06.140309 31389 pending_rounds.cc:107] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:16:06.062948 30632 rpcz_store.cc:276] 0709 13:16:04.132600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.132779 (+   179us) service_pool.cc:224] Handling call
0709 13:16:04.133159 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.140503 (+  7344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.140540 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.164465 (+ 23925us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.764829 (+600364us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4800550158. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:16:04.803619 (+ 38790us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b1800147160. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:05.403402 (+599783us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:05.403472 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.403591 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.898211 (+494620us) raft_consensus.cc:1685] finished
0709 13:16:05.898237 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:05.898386 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6859,"raft.queue_time_us":7487,"raft.run_cpu_time_us":3116,"raft.run_wall_time_us":620538,"spinlock_wait_cycles":50723072,"thread_start_us":591241,"threads_started":1,"wal-append.queue_time_us":1665623}
I20260709 13:16:06.141640 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:06.142455 30632 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:06.084138 30550 rpcz_store.cc:276] 0709 13:16:04.137885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.138905 (+  1020us) service_pool.cc:224] Handling call
0709 13:16:04.139365 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:04.893323 (+753958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.902359 (+1009036us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b540028aad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:05.902562 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415784192}
I20260709 13:16:06.085103 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1791) took 2466 ms. Trace:
I20260709 13:16:06.147785  3837 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.091773 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1904) took 2794 ms. Trace:
I20260709 13:16:06.148633  3837 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:06.148859 30552 rpcz_store.cc:276] 0709 13:16:03.281136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.282042 (+   906us) service_pool.cc:224] Handling call
0709 13:16:03.282381 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:03.301671 (+ 19290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:03.301725 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:03.302459 (+   734us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.430431 (+1127972us) spinlock_profiling.cc:243] Waited 4.17 ms on lock 0x7b18002afee0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:05.206690 (+776259us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:05.206795 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.206912 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.018434 (+811522us) spinlock_profiling.cc:243] Waited 5.72 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:06.075294 (+ 56860us) raft_consensus.cc:1685] finished
0709 13:16:06.075317 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.075479 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20797,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":8861,"spinlock_wait_cycles":22742144,"thread_start_us":775802,"threads_started":1,"wal-append.queue_time_us":2055870}
W20260709 13:16:06.093134 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 1.1645e+06us
W20260709 13:16:05.909458  4123 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:16:06.154486 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.159128 31367 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:16:06.159925 31367 pending_rounds.cc:85] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:06.160243 31367 pending_rounds.cc:107] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:16:06.160578 31367 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:06.160725 30378 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 18, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 18 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:06.056787 30555 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:06.159235 31378 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:06.167637 31391 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:06.020396  3888 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:06.021078  4005 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:06.098048 31364 rpcz_store.cc:269] 0709 13:16:03.307483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.307630 (+   147us) service_pool.cc:224] Handling call
0709 13:16:05.899556 (+2591926us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540044f554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:05.899747 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4223879936}
I20260709 13:16:06.177876 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.095111 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:06.063756  3936 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:06.180992 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1331) took 1681 ms. Trace:
I20260709 13:16:06.108208  3181 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:06.112735 30557 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:16:06.066224  4131 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.066915  3973 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:06.123253  4130 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.019026 30704 rpcz_store.cc:276] 0709 13:16:03.850024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.223512 (+373488us) service_pool.cc:224] Handling call
0709 13:16:04.223968 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.221640 (+997672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.740711 (+519071us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b54005f00d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:05.741007 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224312576}
I20260709 13:16:06.140564 31389 raft_consensus.cc:2889] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:06.155575 30627 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:06.156358 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.154006 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:06.156940 30553 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:16:06.085740  3801 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:06.130918 30379 rpcz_store.cc:269] 0709 13:15:28.915145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:28.922079 (+  6934us) service_pool.cc:224] Handling call
0709 13:16:05.985504 (+37063425us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:15:58.501780 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:15:58.502600 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:15:58.503470 (+   870us) write_op.cc:435] Acquired schema lock
0709 13:15:58.503496 (+    26us) tablet.cc:662] Decoding operations
0709 13:15:58.505188 (+  1692us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:15:58.505336 (+   148us) write_op.cc:641] Partition lock acquired for write op
0709 13:15:58.505369 (+    33us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:15:58.507237 (+  1868us) tablet.cc:701] Row locks acquired
0709 13:15:58.507291 (+    54us) write_op.cc:260] PREPARE: finished
0709 13:15:58.507577 (+   286us) write_op.cc:270] Start()
0709 13:15:58.507767 (+   190us) write_op.cc:276] Timestamp: P: 1783602958507466 usec, L: 0
0709 13:15:58.507817 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:15:58.515757 (+  7940us) log.cc:844] Serialized 6309 byte log entry
0709 13:15:59.227167 (+711410us) op_driver.cc:464] REPLICATION: finished
0709 13:15:59.350202 (+123035us) write_op.cc:301] APPLY: starting
0709 13:15:59.350356 (+   154us) tablet.cc:1367] starting BulkCheckPresence
0709 13:15:59.513023 (+162667us) tablet.cc:1370] finished BulkCheckPresence
0709 13:15:59.513167 (+   144us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:15:59.973222 (+460055us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:15:59.973606 (+   384us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:15:59.973673 (+    67us) write_op.cc:312] APPLY: finished
0709 13:15:59.974166 (+   493us) log.cc:844] Serialized 147 byte log entry
0709 13:15:59.974397 (+   231us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:15:59.974590 (+   193us) write_op.cc:454] Released schema lock
0709 13:16:00.034251 (+ 59661us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:00.099148 (+ 64897us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:00.107981 (+  8833us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1027143,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":111730,"apply.run_cpu_time_us":30747,"apply.run_wall_time_us":722090,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71167,"num_ops":16,"prepare.queue_time_us":172582,"prepare.run_cpu_time_us":48362,"prepare.run_wall_time_us":249372,"raft.queue_time_us":174724,"raft.run_cpu_time_us":11109,"raft.run_wall_time_us":94603,"replication_time_us":719161,"thread_start_us":463653,"threads_started":4,"wal-append.queue_time_us":289613,"wal-append.run_cpu_time_us":21500,"wal-append.run_wall_time_us":2191724}]]}
I20260709 13:16:06.092468 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1725) took 2161 ms. Trace:
I20260709 13:16:06.175016 30552 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:06.263024 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.262164 31396 rpcz_store.cc:276] 0709 13:16:03.931546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.940255 (+  8709us) service_pool.cc:224] Handling call
0709 13:16:03.940704 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.011300 (+2070596us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:06.011338 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:06.011397 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:06.011436 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.068041 (+ 56605us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b1800326a40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419
0709 13:16:06.092038 (+ 23997us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54008b29d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:06.092145 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.092380 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":213,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":42772,"spinlock_wait_cycles":4781925376}
W20260709 13:16:06.142335  4124 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.352s	user 0.000s	sys 0.002s
I20260709 13:16:06.180475 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:06.147980 31394 rpcz_store.cc:276] 0709 13:16:03.618583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:03.619166 (+   583us) service_pool.cc:224] Handling call
0709 13:16:04.245925 (+626759us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.246347 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.246375 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.247026 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.499471 (+252445us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:05.195745 (+696274us) log.cc:844] Serialized 14 byte log entry
0709 13:16:05.196159 (+   414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.196274 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:05.402489 (+206215us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:06.020749 (+618260us) raft_consensus.cc:1685] finished
0709 13:16:06.062869 (+ 42120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.084801 (+ 21932us) spinlock_profiling.cc:243] Waited 626 ms on lock 0x7b5400145ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:06.085007 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":910240,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":4173,"raft.queue_time_us":87232,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":32785,"spinlock_wait_cycles":1446131200,"thread_start_us":929224,"threads_started":2,"wal-append.queue_time_us":727113}
I20260709 13:16:06.268217 31364 raft_consensus.cc:2410] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:16:06.076237  3970 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.194759 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.139616  4035 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:06.194773 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.108101 30671 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:06.274026 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 166045us
W20260709 13:16:06.255607 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.220296 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.166479 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.263974 30552 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:06.164628 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:06.279809 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.006s	sys 0.016s
W20260709 13:16:06.280232 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.006s	sys 0.016s
W20260709 13:16:06.280624 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 661087us
I20260709 13:16:06.280678 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.281857 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.278400 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:06.283306 30552 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:06.264202 31383 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:16:06.283722 30672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.011s	sys 0.014s
W20260709 13:16:06.284093 30672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.011s	sys 0.014s
W20260709 13:16:06.284631 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.181478 30625 rpcz_store.cc:276] 0709 13:16:04.499577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.624060 (+124483us) service_pool.cc:224] Handling call
0709 13:16:04.624534 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.624945 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:04.624979 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.639399 (+ 14420us) log.cc:844] Serialized 26 byte log entry
0709 13:16:05.307327 (+667928us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:05.307394 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.307505 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.180730 (+873225us) raft_consensus.cc:1685] finished
0709 13:16:06.180757 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.180925 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":9715,"thread_start_us":543651,"threads_started":1,"wal-append.queue_time_us":1141539}
I20260709 13:16:06.192965 30378 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:06.267004 30632 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:06.269313 31394 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:16:06.098187  4016 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.218776 30550 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:16:06.282992 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:06.287520 31394 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:16:06.309448 31394 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:16:06.307367 30551 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:16:06.128963  4119 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.219436 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.309908 31394 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:06.310496 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:06.276526 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:06.109295  4063 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.327404 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.327895 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.328555 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:06.329195 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2151) took 1126 ms. Trace:
I20260709 13:16:06.329608 30702 rpcz_store.cc:276] 0709 13:16:05.203160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.203510 (+   350us) service_pool.cc:224] Handling call
0709 13:16:06.328962 (+1125452us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54000c7454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:06.329128 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1838947584}
I20260709 13:16:06.331593 30552 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:06.333318 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:06.328747 31394 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:06.337586 30555 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:06.339061 30551 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:06.338869 30378 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:06.340296 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.340667 31364 pending_rounds.cc:85] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:06.341148 31364 pending_rounds.cc:107] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:16:06.341451 31364 raft_consensus.cc:2889] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:06.260789  4130 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:06.259963  4142 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.353047  4142 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.354187  4109 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:06.332406  3801 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:06.355717  3801 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.355775  4142 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.356153  3801 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:06.357087  3870 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.272159  4131 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:06.357765  3870 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.329316  4085 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.358736 30698 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:06.359257  4085 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:06.360054  3870 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.360963 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.360348 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1897) took 1840 ms. Trace:
I20260709 13:16:06.362710 31361 rpcz_store.cc:276] 0709 13:16:04.519656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.520466 (+   810us) service_pool.cc:224] Handling call
0709 13:16:04.520951 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.641924 (+120973us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:04.641975 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:04.716665 (+ 74690us) log.cc:844] Serialized 26 byte log entry
0709 13:16:04.832320 (+115655us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b48000f3fd8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:16:05.296915 (+464595us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:05.296968 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:05.297153 (+   185us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.141434 (+844281us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:06.360080 (+218646us) raft_consensus.cc:1685] finished
0709 13:16:06.360112 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.360277 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":212925,"raft.queue_time_us":176825,"raft.run_cpu_time_us":1446,"raft.run_wall_time_us":122284,"spinlock_wait_cycles":43706496,"thread_start_us":529753,"threads_started":1,"wal-append.queue_time_us":731624}
I20260709 13:16:06.365631  4016 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.148062  3929 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.192667  4005 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.328673  4061 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:16:06.369179  4061 consensus_queue.cc:1243] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:06.369275 30627 raft_consensus.cc:2410] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:16:06.267867  4138 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:06.374914  3949 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.375869  3949 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.377665  4138 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:06.379413 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.383590  3985 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.384119  4041 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.384330  3985 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:06.385440 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.350978  4055 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:06.386904  3973 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.387532 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.388263 31361 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:06.388636  3973 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:06.351990 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.389351 30550 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:06.390385 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.391338  4077 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.393708  3970 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.391498 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.395707 30550 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:06.398213 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.349033 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.405957 30551 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:06.407274 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.343982 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.350917  3941 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:06.259749  3936 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.266321  3181 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.307922  3887 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:06.352265 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.351928  4035 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.399560 30559 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:06.402172 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.404706 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:06.331480  3874 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.917s	user 0.019s	sys 0.009s
I20260709 13:16:06.436203 31391 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:06.408802 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.440676 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.418107 30633 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
W20260709 13:16:06.390666 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.423659 30378 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:16:06.453197 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.425894 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.428541 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.430722 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.386358  3888 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:06.379522  4148 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
W20260709 13:16:06.475060 30628 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.032s	sys 0.071s
W20260709 13:16:06.475498 30628 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.032s	sys 0.071s
I20260709 13:16:06.475899 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:06.475356 30378 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:16:06.477478 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71713.4us
I20260709 13:16:06.478345 31389 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:06.436401 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.437037 31383 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:06.358784  4040 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:06.460747 31367 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:16:06.454950 30550 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:06.457901  3949 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:06.446964  4104 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.469035  3936 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.462299 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:06.489639 30705 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.006s	sys 0.011s
I20260709 13:16:06.472756  3929 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.423133  3941 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.471010  3985 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:06.505410 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.009s	sys 0.031s
I20260709 13:16:06.507980 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:06.505853 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.009s	sys 0.031s
I20260709 13:16:06.506479 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.511188 30378 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:06.490104 30705 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.006s	sys 0.011s
I20260709 13:16:06.490432 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.484100 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:06.493453 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.529124 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:06.495023 31367 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:06.530385 31367 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:16:06.530809 31367 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:06.436736  3887 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:06.498242 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:06.497154 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:06.479111  4093 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:06.532421 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.015s	sys 0.050s
I20260709 13:16:06.537559 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.543871 30550 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:06.544898 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.015s	sys 0.050s
I20260709 13:16:06.548177 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:06.497681  4106 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.000s	sys 0.016s
W20260709 13:16:06.503208 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.019s	sys 0.031s
I20260709 13:16:06.540899 30557 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:06.578749 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.019s	sys 0.032s
I20260709 13:16:06.549817 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:06.554975 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71272.7us
W20260709 13:16:06.505046  4050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.018s	sys 0.021s
I20260709 13:16:06.573801 30378 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:06.483075  4142 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.503022  3941 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:06.592597 30378 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:06.580323 31391 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:06.578886  4106 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.000s	sys 0.018s
W20260709 13:16:06.509502  3508 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.023s	sys 0.016s
I20260709 13:16:06.580387 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:06.581416 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:06.587558 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.603416 30702 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:06.503109  4039 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:06.587113  4139 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:06.613240 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:06.580128 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83086.3us
I20260709 13:16:06.614420 30671 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:06.541332  3887 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:06.583292 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:06.623749 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:06.598701 30378 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:06.625871 30378 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:06.626866  4150 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:06.627591  4150 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:06.628214  4119 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:06.628609  4150 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.629326 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:06.585433  4050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.817s	user 0.019s	sys 0.021s
I20260709 13:16:06.631150 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:06.631799 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:06.632514 30555 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:06.634572  4042 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.637219  4042 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.581713  3936 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:06.603927 30554 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.013s	sys 0.033s
I20260709 13:16:06.638984  3936 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:06.639214 30554 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.013s	sys 0.033s
W20260709 13:16:06.639210 30626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.015s	sys 0.008s
I20260709 13:16:06.635700  4138 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:06.640270  4138 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:06.640710  4138 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:06.642102  4042 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:06.501158  4104 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.644397  4104 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.645058 31383 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:06.639603  3936 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:06.627687 30378 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 13, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 13 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:06.649755 30378 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:16:06.599949 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.010s	sys 0.018s
W20260709 13:16:06.599189  4122 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 13:16:06.601140  3508 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.024s	sys 0.017s
I20260709 13:16:06.596086  3939 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:06.584939  3990 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:16:06.576524  4093 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.595552  4063 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.621358  3949 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:06.585125  3834 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.080s
W20260709 13:16:06.653132 30626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.016s	sys 0.008s
I20260709 13:16:06.705129 30555 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
W20260709 13:16:06.587145  4047 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.000s	sys 0.023s
W20260709 13:16:06.663050 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.010s	sys 0.019s
W20260709 13:16:06.663383 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.676831 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1917) took 1216 ms. Trace:
I20260709 13:16:06.757830 30559 rpcz_store.cc:276] 0709 13:16:05.460674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.461686 (+  1012us) service_pool.cc:224] Handling call
0709 13:16:06.676470 (+1214784us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b5400110dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:06.676729 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153362688}
I20260709 13:16:06.652050 30378 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:16:06.759111 30630 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.014s	sys 0.009s
I20260709 13:16:06.760363 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:06.759604 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:06.596858  4142 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.762352 30559 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:06.636196 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:06.759646 30630 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.014s	sys 0.010s
I20260709 13:16:06.765066 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.766155 30706 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:06.767315 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.769169 30703 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 13:16:06.770792 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 908624us
W20260709 13:16:06.772989 30599 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.019s	sys 0.004s
W20260709 13:16:06.773648 30599 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.019s	sys 0.004s
W20260709 13:16:06.774066 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 620975us
I20260709 13:16:06.774694 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.777102 30378 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 14, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:06.779340 30631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.004s	sys 0.019s
W20260709 13:16:06.780553 30631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.004s	sys 0.019s
I20260709 13:16:06.783555 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:06.792938 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:06.754716  4047 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.001s	sys 0.023s
I20260709 13:16:06.622326  4039 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:06.674643  3551 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.002s
I20260709 13:16:06.755728  3834 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:06.797986 30378 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 13. New cstate: current_term: 13 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:06.810747 30378 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 17, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 17 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:06.671514  4137 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.017s	sys 0.008s
I20260709 13:16:06.749895  3939 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:06.795989 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:06.800704 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.666543  4034 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:06.650547  4138 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.807940 30558 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:06.802474 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.816605 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.748263  4042 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:06.819259 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.817569 30378 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 16 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:06.820518 30703 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:16:06.777288  4063 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:06.666765  4019 consensus_queue.cc:1059] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.148s
I20260709 13:16:06.755219  3508 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:16:06.824474 30707 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:16:06.835919 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:06.867041 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.867390 30699 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:16:06.796329  4032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.002s	sys 0.023s
W20260709 13:16:06.814525  4137 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.017s	sys 0.009s
W20260709 13:16:06.876776 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:06.823690  4034 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:06.870127  4139 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:06.794147  3990 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:16:06.885486 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.888954 30378 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:06.810497  4131 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.908548 30378 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 2 to 11, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:06.911854 30378 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:06.894605 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:06.914587 30631 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:06.844504  3551 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.919672  3551 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.922153  3551 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.862442  4063 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:06.922928  4063 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:06.914052 31391 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:06.853107  4005 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:06.899021 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:06.927485 30627 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
I20260709 13:16:06.928035  4142 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.929028 30700 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:06.929291  4154 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:06.929713  4154 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:06.890756  4034 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:06.899966  3837 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.750s	user 0.017s	sys 0.000s
I20260709 13:16:06.899809  4137 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:06.900554  3970 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:06.902067 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1319) took 2783 ms. Trace:
I20260709 13:16:06.903038  3181 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.904671  4149 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:06.905611  3939 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:06.907052  4011 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.011s	user 0.011s	sys 0.006s
I20260709 13:16:06.863569  4042 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:06.947394 31388 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.018s	sys 0.017s
W20260709 13:16:06.948086 31388 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.808s	user 0.018s	sys 0.017s
W20260709 13:16:06.952329 31294 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.001s	sys 0.030s
W20260709 13:16:06.952855 31294 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.001s	sys 0.030s
I20260709 13:16:06.934391 30628 rpcz_store.cc:276] 0709 13:16:04.118559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:04.127072 (+  8513us) service_pool.cc:224] Handling call
0709 13:16:04.127572 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:04.131841 (+  4269us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:04.840401 (+708560us) spinlock_profiling.cc:243] Waited 7.63 ms on lock 0x7b18002a75a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:16:05.159314 (+318913us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.483205 (+1323891us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.488188 (+  4983us) log.cc:844] Serialized 14 byte log entry
0709 13:16:06.488427 (+   239us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.495784 (+  7357us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.502675 (+  6891us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:06.502725 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.514419 (+ 11694us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:06.872910 (+358491us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b18002a75a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:06.901758 (+ 28848us) raft_consensus.cc:1685] finished
0709 13:16:06.901790 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:06.901955 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153125,"prepare.queue_time_us":1684761,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":142,"raft.queue_time_us":313782,"raft.run_cpu_time_us":510,"raft.run_wall_time_us":36071,"spinlock_wait_cycles":37451776,"thread_start_us":1461678,"threads_started":3,"wal-append.queue_time_us":670852}
W20260709 13:16:06.953240 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 909998us
W20260709 13:16:06.891104  4032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.003s	sys 0.024s
I20260709 13:16:06.662297  3936 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:06.945294 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53261.5us
W20260709 13:16:06.967283  4130 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.007s
I20260709 13:16:06.977025  3834 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:06.977355  4130 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:06.977767  3834 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:06.978778  3973 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.590s	user 0.016s	sys 0.000s
I20260709 13:16:06.979547  3973 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:06.980193  3834 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:06.980104  3973 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.980460  4032 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:06.982348 30701 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:06.934242  3970 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:06.996093 30556 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.092s	sys 0.143s
W20260709 13:16:06.997416 30556 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.092s	sys 0.143s
I20260709 13:16:06.973152 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.997639 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:07.015908 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.933722  3837 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:06.941596 30631 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:06.934983  4149 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:06.980202  4130 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:07.048595  4149 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:06.946276  4011 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:07.050072 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:07.051555 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:06.934041  4137 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:07.052574 30698 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 13:16:06.998337  3801 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.006s
I20260709 13:16:06.996001  3970 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:07.053576  3801 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:07.056042  3801 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.930584  4154 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:07.058115  4034 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:06.957954 30378 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 16, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 16 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:07.051688  4011 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:07.059109 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.060405  4042 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:06.999590 30707 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:07.064604 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:07.076601 31364 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.016s	sys 0.016s
W20260709 13:16:07.077451 31364 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.016s	sys 0.016s
I20260709 13:16:07.078238 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.079260 31391 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:07.080451 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:07.081956 31391 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:07.086112 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:07.072999  4035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.004s	sys 0.019s
W20260709 13:16:07.092960 31361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.013s	sys 0.027s
W20260709 13:16:07.093531 31361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.014s	sys 0.027s
I20260709 13:16:07.095160 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:07.096289 31394 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:07.098258 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.029244  3973 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:07.073284 31378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.000s	sys 0.031s
W20260709 13:16:07.221091 31378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.000s	sys 0.031s
W20260709 13:16:07.014828  4103 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.026s
I20260709 13:16:07.099645 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:07.080477  4040 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:07.229707 30557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.001s	sys 0.014s
W20260709 13:16:07.100605  4035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.005s	sys 0.020s
W20260709 13:16:07.111585 30704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.011s	sys 0.017s
W20260709 13:16:07.241966 30704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.011s	sys 0.018s
W20260709 13:16:07.140837 30552 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.024s	sys 0.000s
W20260709 13:16:07.243551 30552 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.025s	sys 0.000s
W20260709 13:16:07.243574 30523 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.033s	sys 0.061s
W20260709 13:16:07.144292 31396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.013s	sys 0.016s
W20260709 13:16:07.145745  4164 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:16:07.138522  4138 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:07.160624 30705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.007s	sys 0.016s
W20260709 13:16:07.264860 30705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.008s	sys 0.016s
I20260709 13:16:07.265391 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.265889 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2154) took 1839 ms. Trace:
W20260709 13:16:07.176781 30551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.017s
W20260709 13:16:07.267084 30551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.011s	sys 0.018s
I20260709 13:16:07.266849 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:07.184255 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.033s	sys 0.029s
W20260709 13:16:07.188817 30554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.017s	sys 0.009s
W20260709 13:16:07.277491 30554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.017s	sys 0.009s
I20260709 13:16:07.278502 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2428) took 1225 ms. Trace:
I20260709 13:16:07.279000 30554 rpcz_store.cc:276] 0709 13:16:06.053300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.053464 (+   164us) service_pool.cc:224] Handling call
0709 13:16:06.053930 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.054379 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:06.639848 (+585469us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.278043 (+638195us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.278126 (+    83us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:07.278184 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.278272 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.278432 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1037149,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":1907,"thread_start_us":1222681,"threads_started":2}
I20260709 13:16:07.108757 30378 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 17, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 17 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:07.280616 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:07.224372 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 124826us
W20260709 13:16:07.192083  4106 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.000s	sys 0.007s
I20260709 13:16:07.282569 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:07.229277 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.006s	sys 0.010s
W20260709 13:16:07.230204 30557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.001s	sys 0.014s
I20260709 13:16:07.283747  3874 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:07.283802 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.006s	sys 0.010s
I20260709 13:16:07.284606  3874 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:07.287057 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1912) took 1004 ms. Trace:
I20260709 13:16:07.287669 31378 rpcz_store.cc:276] 0709 13:16:06.282583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.282791 (+   208us) service_pool.cc:224] Handling call
0709 13:16:06.283159 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.309138 (+ 25979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.309177 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.333021 (+ 23844us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.221679 (+888658us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:07.221740 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.221852 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.286730 (+ 64878us) raft_consensus.cc:1685] finished
0709 13:16:07.286766 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.286972 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25500,"raft.queue_time_us":44053,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":18735,"thread_start_us":842073,"threads_started":1,"wal-append.queue_time_us":904519}
I20260709 13:16:07.289297  3929 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:07.289819  3929 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.290191  3929 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:07.244581 30523 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.033s	sys 0.061s
I20260709 13:16:07.291500 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:07.291864 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 630957us
I20260709 13:16:07.292465 30628 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:07.292107 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:07.244995 31396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.016s
W20260709 13:16:07.255877 30524 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.053s
I20260709 13:16:07.294652  4005 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:07.294924 30524 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.000s	sys 0.053s
I20260709 13:16:07.295107  4005 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:07.295595  4005 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:07.290295 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.297672 30559 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:07.299485 30559 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:07.300298 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.300884 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:07.301090 30555 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:16:07.301729 30559 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:07.302071 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.302728 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.302942 30555 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:07.260696 31391 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
W20260709 13:16:07.225112  4103 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.000s	sys 0.027s
I20260709 13:16:07.306984 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1793) took 1168 ms. Trace:
I20260709 13:16:07.291410 30378 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 15, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:16:07.310757 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.036s	sys 0.031s
I20260709 13:16:07.307780 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:07.311993 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 695529us
I20260709 13:16:07.313277 30707 raft_consensus.cc:2368] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:07.313887 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.314251 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:07.314530 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.316159 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.320103 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:07.304454 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:07.325378 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.328466 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.307551 31388 rpcz_store.cc:276] 0709 13:16:06.138456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.138650 (+   194us) service_pool.cc:224] Handling call
0709 13:16:06.139140 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.139566 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.139600 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.140188 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.949112 (+808924us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:06.999248 (+ 50136us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.075597 (+ 76349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.075746 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.306692 (+230946us) raft_consensus.cc:1685] finished
0709 13:16:07.306724 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.306907 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70830,"prepare.queue_time_us":14843,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":92901,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":9534,"spinlock_wait_cycles":24960,"thread_start_us":808371,"threads_started":1,"wal-append.queue_time_us":1080991}
I20260709 13:16:07.310982 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:07.363059 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52165.5us
I20260709 13:16:07.329394 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:07.366483 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:07.375595 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:07.387089 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.675807 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2440) took 1316 ms. Trace:
I20260709 13:16:07.677491 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:07.678864 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1921) took 1054 ms. Trace:
W20260709 13:16:07.669785 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.008s	sys 0.012s
I20260709 13:16:07.679772 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:07.681120 30559 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 11.
W20260709 13:16:07.451366 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.015s	sys 0.020s
W20260709 13:16:07.683487 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.015s	sys 0.020s
I20260709 13:16:07.679404 30557 rpcz_store.cc:276] 0709 13:16:06.624796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.624994 (+   198us) service_pool.cc:224] Handling call
0709 13:16:06.625394 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.625833 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.625872 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.626485 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.284123 (+657638us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:07.284186 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.284314 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.678241 (+393927us) raft_consensus.cc:1685] finished
0709 13:16:07.678284 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.678626 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4443,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":657127,"threads_started":1,"wal-append.queue_time_us":779041}
W20260709 13:16:07.680387 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.917s	user 0.009s	sys 0.012s
I20260709 13:16:07.387605 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:07.685667 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 1.04951e+06us
I20260709 13:16:07.390797 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:07.389230 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:07.687460 30628 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:07.679991 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:07.452514 30630 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.017s	sys 0.013s
W20260709 13:16:07.459442 30633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.030s	sys 0.053s
W20260709 13:16:07.690492 30630 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.017s	sys 0.013s
I20260709 13:16:07.690759 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:07.691136 30633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.030s	sys 0.053s
W20260709 13:16:07.383801 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54336.9us
I20260709 13:16:07.454627 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1106) took 1583 ms. Trace:
I20260709 13:16:07.693100 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1828) took 1579 ms. Trace:
I20260709 13:16:07.693392 30624 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:07.694016 30624 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:07.694209 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:07.692585 30626 rpcz_store.cc:276] 0709 13:16:05.871540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.878740 (+  7200us) service_pool.cc:224] Handling call
0709 13:16:05.879231 (+   491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:05.914398 (+ 35167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:05.914436 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:05.915215 (+   779us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.752746 (+837531us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:06.752803 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:06.752890 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.454309 (+701419us) raft_consensus.cc:1685] finished
0709 13:16:07.454337 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.454520 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34636,"raft.queue_time_us":35257,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":837059,"threads_started":1,"wal-append.queue_time_us":1059916}
W20260709 13:16:07.431103 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.012s	sys 0.020s
W20260709 13:16:07.699361 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.013s	sys 0.021s
I20260709 13:16:07.693688 30630 rpcz_store.cc:276] 0709 13:16:06.114008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.114227 (+   219us) service_pool.cc:224] Handling call
0709 13:16:06.114932 (+   705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.115205 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:06.115594 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.691419 (+1575825us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.692073 (+   654us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.692203 (+   130us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:07.692427 (+   224us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.692621 (+   194us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.692927 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68233,"prepare.queue_time_us":92402,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":89,"raft.queue_time_us":89038,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":14703,"thread_start_us":1557768,"threads_started":2,"wal-append.queue_time_us":1024327}
I20260709 13:16:07.690272 30555 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:07.703303 30559 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:07.381441 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.385968 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:07.398468 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.379688 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.434177 31394 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:07.426470  3801 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:07.278229  3837 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:07.328918  3888 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
W20260709 13:16:07.536540  3939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.013s	sys 0.024s
W20260709 13:16:07.475553  3941 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.874s	user 0.012s	sys 0.009s
I20260709 13:16:07.664986 30378 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:07.423629  4019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.007s	sys 0.025s
I20260709 13:16:07.407979  4031 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:07.801582  3801 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:07.809906  3801 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:07.811177 30702 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:07.813841 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.228647  4034 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:07.330600  4035 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:07.815191 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:07.264555  4040 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:07.817380 30627 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:16:07.801783 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 454519us
I20260709 13:16:07.819854  4040 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:07.278788  4042 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:07.820490 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:07.821621  4042 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:07.240985  4050 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:07.430158  4063 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.507s	user 0.012s	sys 0.004s
I20260709 13:16:07.803054 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.361310  4130 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:07.684278 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 742203us
I20260709 13:16:07.824908  4063 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:07.360797  4104 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:07.827566 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2161) took 1389 ms. Trace:
I20260709 13:16:07.827934  4104 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:07.378988  4142 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 13:16:07.427481  4149 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:07.828438 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.829437  4149 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:07.828249 30704 rpcz_store.cc:276] 0709 13:16:06.438086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.438313 (+   227us) service_pool.cc:224] Handling call
0709 13:16:06.438783 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.439102 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:07.276138 (+837036us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.775189 (+499051us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.782814 (+  7625us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:07.826779 (+ 43965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.826937 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.827251 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":452478,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":75765,"raft.run_cpu_time_us":379,"raft.run_wall_time_us":381,"thread_start_us":1183983,"threads_started":3,"wal-append.queue_time_us":332575}
I20260709 13:16:07.370561  4151 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:16:07.849730  4063 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:07.587083  4172 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 13:16:07.569120  4171 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.385s	user 0.002s	sys 0.000s
W20260709 13:16:07.584729  4175 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:16:07.823771  4050 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:07.855204  4050 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:07.585886  4177 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
W20260709 13:16:07.590579  4181 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:16:07.618798  4203 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:07.858914 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.860277 31391 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:07.615304  4207 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:07.803450  3939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.017s	sys 0.025s
I20260709 13:16:07.768903  4189 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:07.769196  4150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.116s	user 0.013s	sys 0.005s
W20260709 13:16:07.867313  4150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.013s	sys 0.005s
I20260709 13:16:07.771960 30707 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:07.868851 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.869939 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:07.772524  4192 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:07.772843 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 468260us
W20260709 13:16:07.766680 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68508.3us
I20260709 13:16:07.676687 30551 rpcz_store.cc:276] 0709 13:16:06.359584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.442130 (+ 82546us) service_pool.cc:224] Handling call
0709 13:16:06.446408 (+  4278us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:06.447012 (+   604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.447061 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:06.489752 (+ 42691us) log.cc:844] Serialized 52 byte log entry
0709 13:16:07.267489 (+777737us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:07.267535 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.267603 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.674888 (+407285us) raft_consensus.cc:1685] finished
0709 13:16:07.674972 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.675504 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2468,"raft.queue_time_us":11615,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":683008,"threads_started":1,"wal-append.queue_time_us":931053}
W20260709 13:16:07.780848 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 91503.9us
I20260709 13:16:07.786604 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1801) took 1615 ms. Trace:
I20260709 13:16:07.787761 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1916) took 1387 ms. Trace:
W20260709 13:16:07.788358 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:07.788697 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:07.791221  4047 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:07.798333  4210 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:07.768838  4195 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 13:16:07.281610  4166 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.141s	user 0.008s	sys 0.004s
I20260709 13:16:07.835454 31394 raft_consensus.cc:2410] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 13:16:07.800770 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 425153us
I20260709 13:16:07.828501  4104 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:07.846597 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1806) took 1673 ms. Trace:
I20260709 13:16:07.846279  4149 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:07.768253 30552 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.005s	sys 0.000s
I20260709 13:16:07.803174 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1923) took 1316 ms. Trace:
I20260709 13:16:07.861671  4063 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
W20260709 13:16:07.866313 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46180.2us
I20260709 13:16:07.804842  4156 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:07.879467 31388 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:07.880932 30628 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:07.867344  4189 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:07.861709  4207 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.923266  4142 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:07.923725  4207 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:07.924336  4142 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:07.807490  4038 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:07.901592 30624 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:07.905928 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1803) took 1734 ms. Trace:
I20260709 13:16:07.897454 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:07.804459  3941 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:07.916774 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.803464  4183 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 13:16:07.943200 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:07.945192  4035 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:07.945705  4035 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:07.946210  4035 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:07.943037  4040 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:07.947276  4040 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.947645  4040 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:07.949360  4031 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:07.922664  3888 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:16:07.915966 31389 rpcz_store.cc:276] 0709 13:16:06.172574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.219966 (+ 47392us) service_pool.cc:224] Handling call
0709 13:16:07.846043 (+1626077us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540002e754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:07.846329 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462267264}
I20260709 13:16:07.952428  4038 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:07.926785 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:07.921108 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 183045us
I20260709 13:16:07.872228 31396 rpcz_store.cc:276] 0709 13:16:06.170965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.264730 (+ 93765us) service_pool.cc:224] Handling call
0709 13:16:06.265064 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.268089 (+  3025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.268123 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.269026 (+   903us) log.cc:844] Serialized 26 byte log entry
0709 13:16:06.492549 (+223523us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b18000222a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:07.294952 (+802403us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:07.295008 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.295083 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.751030 (+455947us) raft_consensus.cc:1685] finished
0709 13:16:07.754086 (+  3056us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.786241 (+ 32155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4042,"raft.queue_time_us":2649,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":36013,"spinlock_wait_cycles":7891840,"thread_start_us":802166,"threads_started":1,"wal-append.queue_time_us":1306323}
I20260709 13:16:07.872411 31361 rpcz_store.cc:276] 0709 13:16:06.400016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.402028 (+  2012us) service_pool.cc:224] Handling call
0709 13:16:06.402502 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.422214 (+ 19712us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.422282 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.429504 (+  7222us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.093991 (+664487us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:07.094056 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.094143 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.753912 (+659769us) raft_consensus.cc:1685] finished
0709 13:16:07.757509 (+  3597us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.787412 (+ 29903us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19256,"raft.queue_time_us":19198,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":132,"thread_start_us":664030,"threads_started":1,"wal-append.queue_time_us":1179849}
I20260709 13:16:07.934355  3834 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:07.938362  4047 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:07.938959  4203 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:07.941911  4189 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:07.943037  3887 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.265s	user 0.014s	sys 0.011s
I20260709 13:16:07.954236 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:07.848019  4130 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.080s
I20260709 13:16:07.918342  4104 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:07.955760 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.954530  4085 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:07.973290  3887 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:07.930409 31364 rpcz_store.cc:276] 0709 13:16:06.171627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.270576 (+ 98949us) service_pool.cc:224] Handling call
0709 13:16:06.340372 (+ 69796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:06.342409 (+  2037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.342461 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:06.396393 (+ 53932us) log.cc:844] Serialized 52 byte log entry
0709 13:16:07.078002 (+681609us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:07.078073 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.078178 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.871070 (+792892us) raft_consensus.cc:1685] finished
0709 13:16:07.883212 (+ 12142us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.905652 (+ 22440us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":132,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":681041,"threads_started":1,"wal-append.queue_time_us":982851}
I20260709 13:16:07.975502 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:07.975792 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.977238  4093 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:07.977541 30702 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:16:07.979099 31389 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:07.979512 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:07.982283 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82002.3us
I20260709 13:16:07.984771 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:07.984889 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:07.987836  4210 consensus_queue.cc:1059] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:16:07.988847  3941 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:07.989310  4210 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:07.989843  4210 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:07.990280  4210 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:07.976682 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:07.983542 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.991088  4142 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:07.991112  4189 raft_consensus.cc:3055] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:07.992111  4189 raft_consensus.cc:740] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:07.993511  4189 consensus_queue.cc:260] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:07.994004  4203 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:07.976397 31364 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:07.997017  4203 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:07.997190 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:07.998173 30702 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:07.999511  4189 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:07.999254 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.012169  4136 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:07.956913  3801 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.002460 30555 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
W20260709 13:16:08.015388 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 187212us
I20260709 13:16:08.015806  4156 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:08.016800  4144 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:08.017261  4039 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.005123  4104 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.020871  4220 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.022132  4159 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.023298 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:08.023978  4195 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.036401  4042 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.037945 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 971) took 1152 ms. Trace:
I20260709 13:16:08.039824 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:07.918521 31367 rpcz_store.cc:276] 0709 13:16:06.487116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.487287 (+   171us) service_pool.cc:224] Handling call
0709 13:16:06.494697 (+  7410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:07.042270 (+547573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.042320 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:07.043708 (+  1388us) log.cc:844] Serialized 52 byte log entry
0709 13:16:07.477729 (+434021us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:07.477799 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.477922 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:07.754005 (+276083us) raft_consensus.cc:1685] finished
0709 13:16:07.757791 (+  3786us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.802875 (+ 45084us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":510390,"raft.queue_time_us":525781,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":137,"thread_start_us":419751,"threads_started":1,"wal-append.queue_time_us":558595}
I20260709 13:16:08.009898 31391 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:08.007246 30706 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:08.009310 31394 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:16:07.891523  4192 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.082871  4195 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:08.037945 31383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1925) took 1531 ms (client timeout 2032 ms). Trace:
W20260709 13:16:08.049760 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106782us
I20260709 13:16:08.048247  3874 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.052156  3887 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.053815 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:08.057334 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.059008  4047 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.098009  3551 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:07.266319 30705 rpcz_store.cc:276] 0709 13:16:05.426709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.426918 (+   209us) service_pool.cc:224] Handling call
0709 13:16:05.457582 (+ 30664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:05.457858 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:06.544615 (+1086757us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.265429 (+720814us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.265505 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:07.265570 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:07.265648 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:07.265822 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12929,"prepare.queue_time_us":1305131,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":64,"raft.queue_time_us":115,"thread_start_us":1413863,"threads_started":2,"wal-append.queue_time_us":670456}
I20260709 13:16:07.956653  3834 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:08.108145  3834 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:08.109125 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1740) took 1040 ms. Trace:
I20260709 13:16:08.109400  4104 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.065078 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 1922) took 1437 ms. Trace:
I20260709 13:16:08.110953 30558 rpcz_store.cc:276] 0709 13:16:06.627513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.628983 (+  1470us) service_pool.cc:224] Handling call
0709 13:16:08.064577 (+1435594us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b540052b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.064784 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":408844160}
I20260709 13:16:08.077917 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:08.113221 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59747.2us
I20260709 13:16:08.113472 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 1742) took 1044 ms. Trace:
I20260709 13:16:08.114287 31388 rpcz_store.cc:276] 0709 13:16:07.068992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.787801 (+718809us) service_pool.cc:224] Handling call
0709 13:16:08.113367 (+325566us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:08.115638  3970 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.052s	user 0.014s	sys 0.009s
I20260709 13:16:08.078457 31364 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:08.088369 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.080493 30700 rpcz_store.cc:276] 0709 13:16:06.885223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.885360 (+   137us) service_pool.cc:224] Handling call
0709 13:16:06.885700 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.929466 (+ 43766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.037293 (+1107827us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b540008cad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.037713 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98601728}
I20260709 13:16:08.102536 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1734) took 1036 ms. Trace:
I20260709 13:16:08.085580 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1732) took 1019 ms. Trace:
I20260709 13:16:08.093564  4159 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.099273  4220 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.101517  4144 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:08.091830 31383 rpcz_store.cc:269] 0709 13:16:06.506504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.507653 (+  1149us) service_pool.cc:224] Handling call
0709 13:16:08.037489 (+1529836us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400145fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.037713 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313000192}
I20260709 13:16:08.119969 31292 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:08.138475 31292 leader_election.cc:403] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:16:08.121392 31394 rpcz_store.cc:276] 0709 13:16:07.066434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.956754 (+890320us) service_pool.cc:224] Handling call
0709 13:16:08.008909 (+ 52155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.091390 (+ 82481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.102451 (+ 11061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.121592 31391 rpcz_store.cc:276] 0709 13:16:07.065787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.882985 (+817198us) service_pool.cc:224] Handling call
0709 13:16:07.931002 (+ 48017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.085096 (+154094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.085451 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.116241  3949 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:16:08.109653 31378 rpcz_store.cc:276] 0709 13:16:07.068436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.075245 (+1006809us) service_pool.cc:224] Handling call
0709 13:16:08.108314 (+ 33069us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.108573 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.108600 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:08.108672 (+    72us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:08.108729 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.108791 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.108907 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.140547 31367 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:08.119978  3551 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:08.125527  4192 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.159309 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.116336  3970 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.168666 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.148545  3949 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.167335 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2426) took 2074 ms. Trace:
W20260709 13:16:08.172492 30699 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1715) took 2390 ms (client timeout 2036 ms). Trace:
W20260709 13:16:08.173056 30699 rpcz_store.cc:269] 0709 13:16:05.785331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:05.871657 (+ 86326us) service_pool.cc:224] Handling call
0709 13:16:08.167763 (+2296106us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b540032a854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.172218 (+  4455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2168916864}
I20260709 13:16:08.174775  4042 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:08.175275 30699 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:08.174935  4144 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.176137 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 975) took 1131 ms. Trace:
I20260709 13:16:08.176566 30699 rpcz_store.cc:276] 0709 13:16:07.045149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.174381 (+1129232us) service_pool.cc:224] Handling call
0709 13:16:08.174939 (+   558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.175818 (+   879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.176061 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.177024  4192 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.177831  4192 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.179069 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2172) took 1116 ms. Trace:
I20260709 13:16:08.179603 30699 rpcz_store.cc:276] 0709 13:16:07.062840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.177419 (+1114579us) service_pool.cc:224] Handling call
0709 13:16:08.177969 (+   550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.178299 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.178327 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:08.178374 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:08.178408 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.178473 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.178599 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.180377  4192 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.187983 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:08.189469  3801 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.190960  3874 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:08.191511  3874 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.191927  3874 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:08.194893  4160 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:08.196246  3551 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.158623  4104 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
W20260709 13:16:08.199193 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 142056us
I20260709 13:16:08.199992 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.200875 30705 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:16:08.206053  4005 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.910s	user 0.009s	sys 0.013s
I20260709 13:16:08.206848  4038 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.180081  4220 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:08.216410 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.198346  3551 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:08.227241 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.219740 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.227587 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:08.229590 30705 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:16:08.235224 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.216415  3949 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:08.249934 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:08.259902 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:08.270150 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 142518us
I20260709 13:16:08.171622 30556 rpcz_store.cc:276] 0709 13:16:06.052314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.052520 (+   206us) service_pool.cc:224] Handling call
0709 13:16:06.085601 (+ 33081us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:06.102294 (+ 16693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:06.102349 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:06.111339 (+  8990us) log.cc:844] Serialized 26 byte log entry
0709 13:16:07.004568 (+893229us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:07.004644 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.069736 (+1065092us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.069828 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.069886 (+    58us) raft_consensus.cc:1685] finished
0709 13:16:08.069903 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.126842 (+ 56939us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4550,"raft.queue_time_us":10079,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":40037,"spinlock_wait_cycles":26589056,"thread_start_us":697424,"threads_started":1,"wal-append.queue_time_us":1095434}
W20260709 13:16:08.290268  4119 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.602s	user 0.028s	sys 0.044s
W20260709 13:16:08.291327  4119 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.662s	user 0.028s	sys 0.045s
W20260709 13:16:08.291675 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 166497us
I20260709 13:16:08.292845 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:08.293407  3872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.097s	user 0.017s	sys 0.018s
W20260709 13:16:08.293977  3872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.152s	user 0.017s	sys 0.018s
I20260709 13:16:08.295423  3936 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.296473  4047 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:08.298753  3929 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.878s	user 0.007s	sys 0.019s
I20260709 13:16:08.299736  4195 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.299594  3929 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.302273 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2165) took 1757 ms. Trace:
I20260709 13:16:08.302910  3941 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:08.304222  4137 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.303891  3941 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.290091 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 969) took 1471 ms. Trace:
I20260709 13:16:08.305868 30701 rpcz_store.cc:276] 0709 13:16:06.818544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.818918 (+   374us) service_pool.cc:224] Handling call
0709 13:16:08.262027 (+1443109us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54005a2dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.289810 (+ 27783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311195520}
I20260709 13:16:08.290457  3181 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:08.307047 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.307240  3181 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.302767 30698 rpcz_store.cc:276] 0709 13:16:06.545012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.545169 (+   157us) service_pool.cc:224] Handling call
0709 13:16:06.545561 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.053094 (+507533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.301660 (+1248566us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b5400050854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.302061 (+   401us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165327104}
I20260709 13:16:08.309964 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.311827 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:08.313078  4047 raft_consensus.cc:2764] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:16:08.314117  4047 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:08.314560  4047 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:08.314987  4047 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:08.316787  4170 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.122s	user 0.005s	sys 0.018s
I20260709 13:16:08.319111  3887 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:08.320142  4159 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.321007 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2170) took 1324 ms. Trace:
I20260709 13:16:08.321485 30706 rpcz_store.cc:276] 0709 13:16:06.996615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.997290 (+   675us) service_pool.cc:224] Handling call
0709 13:16:08.320707 (+1323417us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54009cca54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.320914 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316243840}
I20260709 13:16:08.270519  4104 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.323179  4104 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.322238 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.325897  4195 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:08.327288 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1735) took 1260 ms. Trace:
I20260709 13:16:08.327976 31396 rpcz_store.cc:276] 0709 13:16:07.066893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.972525 (+905632us) service_pool.cc:224] Handling call
0709 13:16:07.973072 (+   547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.973740 (+   668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.973771 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:07.974543 (+   772us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.140720 (+166177us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:08.140767 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.140839 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.326996 (+186157us) raft_consensus.cc:1685] finished
0709 13:16:08.327018 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.327187 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1705,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":216,"thread_start_us":153046,"threads_started":1,"wal-append.queue_time_us":349910}
I20260709 13:16:08.330739  4160 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.331580  4160 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.333120 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.333415 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1053) took 2104 ms. Trace:
I20260709 13:16:08.334858  4149 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:08.335374  4149 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:08.335800  4149 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:08.338284  4195 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:08.339905  3942 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:08.340404  3942 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:08.341655  3942 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.342787  3887 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:08.343899  3942 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:08.344275  3887 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.345996 30624 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.331876 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.311987 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 970) took 1460 ms. Trace:
I20260709 13:16:08.325754 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:08.334491 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 92877us
I20260709 13:16:08.310458 31383 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:08.353418 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82330.6us
I20260709 13:16:08.261987  4038 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:08.228117  4005 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.333901 30552 rpcz_store.cc:276] 0709 13:16:06.229268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.350768 (+121500us) service_pool.cc:224] Handling call
0709 13:16:06.351173 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.458879 (+107706us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:07.244187 (+785308us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.979911 (+735724us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.980054 (+   143us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:07.980247 (+   193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.333059 (+352812us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.333121 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.333341 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34180,"prepare.queue_time_us":697890,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":83,"raft.queue_time_us":45578,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":146,"spinlock_wait_cycles":247044224,"thread_start_us":1451422,"threads_started":2,"wal-append.queue_time_us":550371}
I20260709 13:16:08.368302 30702 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:16:08.347101 30703 rpcz_store.cc:276] 0709 13:16:06.851409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.851645 (+   236us) service_pool.cc:224] Handling call
0709 13:16:06.852083 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:06.984630 (+132547us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:06.995038 (+ 10408us) log.cc:844] Serialized 14 byte log entry
0709 13:16:07.451203 (+456165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:07.451291 (+    88us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:07.451365 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.311439 (+860074us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400435654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.311524 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.311787 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13556,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":2435,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":13556,"spinlock_wait_cycles":303957248,"thread_start_us":433056,"threads_started":1,"wal-append.queue_time_us":585453}
I20260709 13:16:08.367981 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.371527 30705 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:08.368996 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:08.369897 30704 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:08.382953 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:08.383285 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45810.7us
I20260709 13:16:08.383749 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1728) took 1091 ms. Trace:
I20260709 13:16:08.393895 30704 rpcz_store.cc:276] 0709 13:16:07.288751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.237942 (+949191us) service_pool.cc:224] Handling call
0709 13:16:08.358477 (+120535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.380217 (+ 21740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.380457 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:08.383826 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 976) took 1284 ms. Trace:
I20260709 13:16:08.399889 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:46186 (request call id 1832) took 2261 ms. Trace:
I20260709 13:16:08.400456 30624 rpcz_store.cc:276] 0709 13:16:06.120504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.120665 (+   161us) service_pool.cc:224] Handling call
0709 13:16:08.344485 (+2223820us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54004abfd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ed4a3d 00007f180bea4cd0 00007f1805429254 00007f18054291e2 00007f1805429111 00007f180542908c 00007f1805428342 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.376996 (+ 32511us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3614302592}
I20260709 13:16:08.400079 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:08.396575 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 373527us
I20260709 13:16:08.404928 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:08.410470 31361 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:08.405530 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.404512 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.422117 31394 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:16:08.423632 31396 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:08.377621  4035 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.409404  3929 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.379683  4040 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.439303 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.440312 31396 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:08.440572  4039 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:08.442953  4130 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:08.443580  4130 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:08.443936  4224 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:08.445043  4130 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.448783 31378 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:08.451203 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1736) took 1384 ms. Trace:
I20260709 13:16:08.448935 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.427486  3941 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.461216 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.463624 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.458387  3990 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:08.453101 31389 rpcz_store.cc:276] 0709 13:16:07.067196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.980369 (+913173us) service_pool.cc:224] Handling call
0709 13:16:07.980890 (+   521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:07.981523 (+   633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:07.981570 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:07.982382 (+   812us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.136623 (+154241us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:08.136697 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.136816 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:08.446358 (+309542us) raft_consensus.cc:1685] finished
0709 13:16:08.446391 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.451129 (+  4738us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5460,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":130791,"threads_started":1,"wal-append.queue_time_us":277949}
I20260709 13:16:08.477829 30554 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:08.458232  4011 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:08.481551 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.481463 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.481813 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.483222 30704 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:08.486240 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.463780  4159 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:08.495324 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252392us
I20260709 13:16:08.495864  4154 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:08.496508  4040 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:08.497332 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.498874  4104 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.484201  3990 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.504581  3990 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.458758  4042 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:16:08.506112  4042 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:08.506649  4042 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:08.507710  4042 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.509168  4042 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:08.509302  3834 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.484779  4137 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:08.511267  4137 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:08.511729  4137 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:08.515103 31378 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:08.522467 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.522800 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.523182  3834 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.515524 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.482430  4011 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.523931 31378 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:08.524710  4011 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:08.526922  4104 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:08.527436  4104 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:08.527807 31294 leader_election.cc:400] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:08.527868  4104 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:08.529121  4160 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.531116 31383 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:08.532836 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.533103  3181 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.535521 31391 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:08.397934 30699 rpcz_store.cc:276] 0709 13:16:07.093716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.181020 (+1087304us) service_pool.cc:224] Handling call
0709 13:16:08.181443 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.181879 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:08.182300 (+   421us) log.cc:844] Serialized 14 byte log entry
0709 13:16:08.377151 (+194851us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:08.377231 (+    80us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:08.377282 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.377356 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:08.377511 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14964,"prepare.queue_time_us":97737,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":83,"raft.queue_time_us":27703,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":16467,"thread_start_us":167960,"threads_started":1}
I20260709 13:16:08.536921 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.537618 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:08.537783  3929 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:08.539063 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.539961 31361 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:08.540164 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112027us
I20260709 13:16:08.541177 31388 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:08.523815 30702 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:08.547254 31389 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:16:08.537880 31391 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:08.491397 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:08.550555 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:08.548784 31367 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:08.553584 30705 raft_consensus.cc:2393] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:08.549968 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.557154 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:08.558678 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:08.560676 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 122780us
I20260709 13:16:08.560878 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.568908 30624 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:08.576447  4005 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.582803  4197 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:08.583539  4197 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:08.584044  4197 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:08.584718  4136 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.541s	user 0.012s	sys 0.002s
I20260709 13:16:08.588124 31391 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:16:08.591562 31391 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:16:08.591634  4050 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.007s
I20260709 13:16:08.598238 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.593238 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1741) took 1524 ms. Trace:
I20260709 13:16:08.593977 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:08.603865 31388 rpcz_store.cc:276] 0709 13:16:07.068701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.115654 (+1046953us) service_pool.cc:224] Handling call
0709 13:16:08.540971 (+425317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.541646 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.592994 (+ 51348us) spinlock_profiling.cc:243] Waited 425 ms on lock 0x7b540057f150. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.593170 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976539904}
I20260709 13:16:08.596033 31389 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:16:08.597114  4047 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.560828  4156 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:08.599074 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:08.566522  3834 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:08.599948  4130 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:08.630270 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 68498.4us
I20260709 13:16:08.600626 31367 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:08.592541 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:08.557571  4159 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:08.591248 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 128032us
I20260709 13:16:08.609449 31383 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
W20260709 13:16:08.615984  4207 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.692s	user 0.014s	sys 0.003s
I20260709 13:16:08.559615  4144 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 13:16:08.645722 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:08.648386 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:08.627151  4189 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.615s	user 0.011s	sys 0.004s
W20260709 13:16:08.654517 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 171809us
I20260709 13:16:08.657650 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.622478  3834 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.631999 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:08.655709 31396 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:08.653841  4207 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.597182  4011 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.664937  4207 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.665812 30628 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:16:08.667373 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 115110us
I20260709 13:16:08.671177  4136 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:08.616071  4050 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.680902  4136 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:08.682358  3970 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.682229  4136 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.683796  3949 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.684264  4219 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:08.685458 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1928) took 1391 ms. Trace:
I20260709 13:16:08.685727 30707 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:08.685894 30554 rpcz_store.cc:276] 0709 13:16:07.293791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:07.293963 (+   172us) service_pool.cc:224] Handling call
0709 13:16:07.294285 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.499950 (+1205665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:08.685118 (+185168us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b54000bf9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:08.685398 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162404608}
I20260709 13:16:08.687050 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1724) took 1399 ms. Trace:
I20260709 13:16:08.687227 30698 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:08.687443 30707 rpcz_store.cc:276] 0709 13:16:07.287644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.331411 (+1043767us) service_pool.cc:224] Handling call
0709 13:16:08.686820 (+355409us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.686992 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778467200}
W20260709 13:16:08.688235  4232 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.186s	user 0.000s	sys 0.002s
W20260709 13:16:08.691093 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81668us
I20260709 13:16:08.690673  3874 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.667193  3834 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:08.694801 30553 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
W20260709 13:16:08.696491 30553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1049) took 2468 ms (client timeout 1548 ms). Trace:
W20260709 13:16:08.696913 30553 rpcz_store.cc:269] 0709 13:16:06.228026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:06.282664 (+ 54638us) service_pool.cc:224] Handling call
0709 13:16:08.696214 (+2413550us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:08.696361 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5235320064}
I20260709 13:16:08.697695  4149 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.700102  4047 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.666522  4011 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:08.701087  4231 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.213s	user 0.000s	sys 0.002s
I20260709 13:16:08.702488  4035 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:08.706743 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79942.1us
I20260709 13:16:08.707933  4220 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:08.708015 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:08.708602  4220 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:08.709177  4050 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:08.709996 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.709653  4220 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.710976  4222 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:08.711515  4222 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.711894  4222 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:08.714568 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:08.716338  4016 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:08.716785  4016 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:08.716905 31391 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:08.717150  4016 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:08.718544 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.718870  4159 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.719307  4143 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.719867  4143 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.722827 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:08.678041  4039 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:08.724195 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 48169.4us
I20260709 13:16:08.724452  4039 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:08.725031  4039 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:08.726243  3181 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:08.726563  4032 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:08.726904  3181 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:08.727072 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.727449  3181 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:08.731458 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69800.8us
I20260709 13:16:08.735096 30701 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:08.741452 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.701110  4047 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:16:08.747215 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.747037  4189 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:08.748588  4189 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:08.749007  4189 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:08.750355 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.753185  4149 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.753789 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:08.755337  4160 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:08.755685  4138 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:08.756362  4138 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:08.757392  4138 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:08.759717  3551 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.760006 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.761586 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.762331 30700 raft_consensus.cc:2376] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16. Re-sending same reply.
I20260709 13:16:08.763232 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.763643 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.763782 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:08.764554 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:08.764912 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.766122 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.763154 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.760491  3551 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.767138  3888 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:08.767632  3888 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:08.768054 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.768220  4050 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:08.768872 30707 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:08.768708  3888 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.769536  3551 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:08.694281  3874 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:08.770918  4137 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.762423 31367 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:08.766206 31378 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:08.674149  3929 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:16:08.786100 30705 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:08.786846 30705 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:08.771638  4227 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 13:16:08.789434 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:08.788108 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:08.733160  4195 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.794775 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.790066 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:08.805739 30628 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:08.777375 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:08.771718  4189 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:08.818329 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:08.814003  4154 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:08.843701 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:08.843664 31383 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:08.836071 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:08.847697 30702 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:08.876313 30703 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:16:08.877802 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.879460 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.880110 30702 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:08.881628  3874 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.853587 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:08.855243 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67264.5us
I20260709 13:16:08.882472  3874 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:08.836954  4210 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.821s	user 0.015s	sys 0.002s
I20260709 13:16:08.883912  4210 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.886209  4210 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:08.858848 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 164270us
I20260709 13:16:08.858953 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:08.887290  4143 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:08.862847  4011 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:08.839607  4119 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:08.867221 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:08.868046  4149 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:08.871389 30700 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:08.872725  4219 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:08.829849  3936 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:08.885972  3874 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:08.856302 30554 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:08.812042  4195 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.894326 30705 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.941879 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:08.943868 30699 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:08.882159  4222 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:08.946691 30627 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
I20260709 13:16:08.954775 30555 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:08.970526 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:08.975389 30700 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:08.932991  4011 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:08.980193 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:08.989058 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:08.904183  4143 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:16:08.905149  3970 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 13:16:09.002099 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:09.006263 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70014.2us
I20260709 13:16:09.006953 30554 raft_consensus.cc:2368] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15.
I20260709 13:16:08.903875  4016 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:08.928974  4154 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:09.000722 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 429831us
I20260709 13:16:09.026820 30631 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:09.027076 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:08.990203  4189 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:09.009217  3887 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.008s	sys 0.019s
W20260709 13:16:09.029915 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 330192us
I20260709 13:16:09.035527 30673 leader_election.cc:400] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15.
I20260709 13:16:09.035315 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:09.035944 30673 leader_election.cc:403] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:16:09.037801 30703 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:16:09.025727  4159 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:09.015401  4137 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:09.044607 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154314us
I20260709 13:16:09.057229 30627 raft_consensus.cc:2410] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:09.057334 30630 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 15. Current term is 16. Ops: []
W20260709 13:16:09.062738 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.028322  4210 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 13:16:09.050980  4218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.009s	sys 0.017s
I20260709 13:16:09.076270 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.048836  4159 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:09.079552 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.065459  3887 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:09.094023 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:09.099236 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:09.082942  4218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.009s	sys 0.017s
W20260709 13:16:09.091671  3990 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.010s	sys 0.012s
I20260709 13:16:09.118425 30626 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:16:09.052963  4220 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:16:09.116963 30633 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:09.118321  4149 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:09.121387 30630 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:09.122023  4152 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:16:09.096287  4159 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:09.064051  3834 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:09.119151  3990 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.010s	sys 0.013s
W20260709 13:16:09.126197  4104 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.597s	user 0.009s	sys 0.006s
I20260709 13:16:09.113026  3887 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:09.142915 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58548.5us
I20260709 13:16:09.147262 30632 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:09.151860  3949 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:09.142786  4160 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:09.159174  4160 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:09.159878  4011 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:09.160434  4160 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.161397  4154 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:09.144030 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.164680 30631 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:09.165803 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.167402  4222 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:09.172178  4195 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:09.174093  4156 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:09.192979  4156 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:09.194056  4156 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.194901  4143 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.195617  4119 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:09.196095 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.197758 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.199430 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.202992 30625 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:09.178249 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.176383  4207 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.159236 30629 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:09.156488  3834 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:09.145707  4218 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:16:09.187954  4244 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:09.213001 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.191290 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.191803  4159 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.192291 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.195775  4143 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:09.197381  3887 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.099509  4137 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:16:09.175463  4243 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:09.200438  3888 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:09.195643  4156 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:09.191376 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45050.7us
I20260709 13:16:09.228937 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:09.177920  4016 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:09.184099 30633 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:09.214579 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.221760 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.216075  3551 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:09.181823  4104 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:09.222496 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 162460us
I20260709 13:16:09.196192  4119 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:09.215770  4159 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:09.238421 30629 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:09.240310 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.256177 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.241430 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:09.258504 31388 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
W20260709 13:16:09.243275  4042 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.734s	user 0.017s	sys 0.000s
I20260709 13:16:09.247248 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:09.248090  3181 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.520s	user 0.013s	sys 0.002s
I20260709 13:16:09.248936 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:09.267550 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:09.216349  4207 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.236229  3936 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:09.268623 31396 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:09.267750 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45560.7us
I20260709 13:16:09.249943  4143 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:09.254492 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.294929 30633 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:09.291662 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:09.299594 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:09.303171 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.304644 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:09.233651  3551 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:09.226781  4039 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:09.317375 30553 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:09.308372 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.308169 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.238400  4136 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:09.315297 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.318427 30701 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:09.325169 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:09.326018 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.320603 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.332472 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.328223 30629 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:09.254944  4203 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:16:09.334764 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:09.292770  4104 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:09.345288 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.313068  4219 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:09.325505 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.313535  3874 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:09.294382  3936 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:09.342517 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.346113 30629 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:09.255142  3929 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:16:09.336118 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.347919 30624 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:09.349463 30706 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:16:09.339347 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.358078 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.354545 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.358834 30553 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:09.329447  4130 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.009s
I20260709 13:16:09.367419 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.295979  3181 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.368391 30629 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:09.372578 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.372697 30628 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:16:09.372656 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:09.267475  4222 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 13:16:09.379384 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:09.379978 31394 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:16:09.377696 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:09.379469 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:09.396605 30702 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:09.400593 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.401472 30552 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:09.414724 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:09.416044 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.417564 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.418340 30633 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:09.419536  4016 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:09.420460  4249 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 13:16:09.420885  4218 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:09.421481  4218 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:09.421736 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.422591 30558 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:09.422443  4218 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.422924 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.424425 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.425712 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.426570  4254 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:16:09.427551  4197 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.843s	user 0.014s	sys 0.001s
I20260709 13:16:09.428033  4197 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.428344  4254 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:09.429692 30698 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:09.430267  4197 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:09.430792  4256 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:09.384236  3551 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.426571  4242 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:16:09.432415 30706 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:16:09.432600 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:09.433712 30625 raft_consensus.cc:1240] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:09.430459 30553 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:09.439865 30552 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:09.340263  3834 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:16:09.525410 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:09.536938 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.540436 30552 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:09.541671 30633 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
W20260709 13:16:09.538748 30673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.057s	sys 0.072s
W20260709 13:16:09.547164 30673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.057s	sys 0.072s
W20260709 13:16:09.547641 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 512292us
I20260709 13:16:09.523581  4256 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:09.551669  3834 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:09.587037  3834 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:09.587774  4256 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.588315  3834 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:09.589617  4256 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:09.552354 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.590823 30552 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:09.384842  4119 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.592875  3551 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:09.553409 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 174476us
I20260709 13:16:09.554072  4031 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:09.597128 30701 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:09.597434  4031 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:09.598555  4031 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.599347  4242 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:16:09.600597 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.562835  3936 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.564095  4039 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:09.602515  3936 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:09.384581  4195 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 13:16:09.603433  4195 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:09.603847  4195 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:09.604846  4195 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:09.606281  4195 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:09.565116 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.590044  3834 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:09.612568 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.613353 30553 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:09.613332 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 180840us
W20260709 13:16:09.550877  4189 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.005s
I20260709 13:16:09.574589  4207 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:09.577788  3181 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:09.622155 30707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.041s	sys 0.110s
W20260709 13:16:09.637601 30707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.044s	sys 0.112s
I20260709 13:16:09.598531 30629 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:09.635211 30552 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 14.
I20260709 13:16:09.639168 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.548705  4197 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:09.677515 30553 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:09.657433  4197 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:09.686918 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:09.691128 30556 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:09.692305  4197 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.698428 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.698698 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.702237 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:09.703557 30701 raft_consensus.cc:2368] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
W20260709 13:16:09.692324  4149 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
I20260709 13:16:09.721560 30629 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:09.758948 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.759634 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:09.722309 30672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.035s	sys 0.060s
W20260709 13:16:09.761296 30672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.035s	sys 0.060s
W20260709 13:16:09.762120 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.762214 30699 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:09.763656 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:09.710543  4259 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:09.765136 30625 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:09.777051 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.702793  4203 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:09.820199 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.705387  4207 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:09.823452 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:09.839260 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.842993 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:09.712185  4042 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:09.770288 30626 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.002s	sys 0.015s
W20260709 13:16:09.847795 30626 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.004s	sys 0.017s
I20260709 13:16:09.800472  4149 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15.
I20260709 13:16:09.853729 30704 raft_consensus.cc:2368] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:09.855618 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.850111 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:09.860934 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:09.817384  3936 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:09.853289  4207 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.871277 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.819417  4143 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:09.892369 31291 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.013s
W20260709 13:16:09.893464 31291 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.005s	sys 0.013s
W20260709 13:16:09.894243 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.868218  4242 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:09.876796 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.861830  3888 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:09.897939 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:09.872666 31364 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.016s	sys 0.029s
W20260709 13:16:09.975292 31364 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.017s	sys 0.029s
I20260709 13:16:09.878768  3936 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.858397  4203 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.884743  4138 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:09.936427 31391 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:09.859766  4042 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:09.873787  4189 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:09.932441 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:09.955163 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.004403 30628 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:09.892218  4237 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.777s	user 0.001s	sys 0.004s
I20260709 13:16:09.901582 30629 raft_consensus.cc:2376] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17. Re-sending same reply.
I20260709 13:16:10.034394 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.035274 30629 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:09.975317 30625 raft_consensus.cc:2376] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16. Re-sending same reply.
I20260709 13:16:09.978698 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1926) took 2035 ms. Trace:
W20260709 13:16:09.963982  4218 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.060s	sys 0.092s
I20260709 13:16:09.895848  4143 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:09.982547 30706 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:16:09.973683  3874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.005s	sys 0.013s
W20260709 13:16:09.973639  3929 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.009s	sys 0.009s
I20260709 13:16:09.971551  4039 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:09.983754  4138 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:10.040759 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1738) took 1279 ms. Trace:
I20260709 13:16:10.045907 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.042893 30554 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:10.039803 31396 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:10.036764 31364 rpcz_store.cc:276] 0709 13:16:07.943311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.141464 (+198153us) service_pool.cc:224] Handling call
0709 13:16:08.215975 (+ 74511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.227033 (+ 11058us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:08.450284 (+223251us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.206879 (+756595us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b18002efa20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:16:09.978002 (+771123us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.978243 (+   241us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:09.978306 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:09.978357 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.978449 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:09.978629 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28126,"prepare.queue_time_us":626609,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":118,"raft.queue_time_us":13333,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":13984,"spinlock_wait_cycles":33087232,"thread_start_us":1332140,"threads_started":3,"wal-append.queue_time_us":670658}
I20260709 13:16:10.043339  4039 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:10.049587  4039 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:10.050956 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.051299 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70190.8us
W20260709 13:16:10.052004 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:09.983611  4189 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:10.058799 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 293963us
I20260709 13:16:10.055771 30629 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:10.052965 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:10.060045 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:09.976686  4242 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:10.039280  4218 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.060s	sys 0.093s
W20260709 13:16:10.045715  3929 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.010s	sys 0.009s
I20260709 13:16:10.064177 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:10.043056  3874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.006s	sys 0.013s
I20260709 13:16:10.069389 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1110) took 1343 ms. Trace:
I20260709 13:16:10.075815 30624 rpcz_store.cc:276] 0709 13:16:08.726077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.740047 (+ 13970us) service_pool.cc:224] Handling call
0709 13:16:10.069056 (+1329009us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b54004a97d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.069271 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267448704}
I20260709 13:16:10.071182 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1857) took 1014 ms. Trace:
I20260709 13:16:10.073210 30625 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:10.025045  4154 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:16:10.067317  4039 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.093650 30631 rpcz_store.cc:276] 0709 13:16:09.056800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.056973 (+   173us) service_pool.cc:224] Handling call
0709 13:16:10.070938 (+1013965us) spinlock_profiling.cc:243] Waited 74.9 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.071114 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172322560}
I20260709 13:16:10.075564 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:10.076363 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.084301 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:10.029400  3888 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:10.046381 30707 rpcz_store.cc:276] 0709 13:16:08.761161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.769856 (+  8695us) service_pool.cc:224] Handling call
0709 13:16:08.770208 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:08.770814 (+   606us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:08.771711 (+   897us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.972284 (+1200573us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.012011 (+ 39727us) log.cc:844] Serialized 14 byte log entry
0709 13:16:10.012432 (+   421us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:10.012519 (+    87us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:10.040492 (+ 27973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.040569 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.040699 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5096,"prepare.queue_time_us":296,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":54,"raft.queue_time_us":17165,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":16310,"spinlock_wait_cycles":26715264,"thread_start_us":1198457,"threads_started":2,"wal-append.queue_time_us":1087456}
I20260709 13:16:10.077198 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.095149 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:10.095487  4263 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:16:10.095824 30559 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:10.096994 30557 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:10.120937 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.097703 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:10.098892  4260 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
W20260709 13:16:10.099817  4264 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:16:10.100852  4262 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 13:16:10.102082  4103 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:16:10.102644  4258 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.102943  3936 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:10.103830 30699 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:16:10.075564  3874 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:10.075268 30632 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:10.054188  4138 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:10.131245 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47041.4us
I20260709 13:16:10.115293  4143 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:10.132899 31391 raft_consensus.cc:2368] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:10.133172  4268 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:10.133535  4237 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:10.134152  4203 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:10.112588  4050 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:10.138111  4242 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:10.191243 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.141988  3888 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:10.142503 30550 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.143999 30553 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:10.063716  4031 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.113729  4207 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:10.125566 30629 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:16:10.112305 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.113685 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.130828 30552 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:10.201236 30624 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:10.124352 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.153012 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:10.120687  3834 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.003s
W20260709 13:16:10.112843  4136 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.002s
I20260709 13:16:10.199856 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.157624 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:10.159160 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64529us
W20260709 13:16:10.106539  4219 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.743s	user 0.010s	sys 0.000s
W20260709 13:16:10.161785 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.205587 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.162524 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.164917 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:10.206208 30631 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:10.121345  4250 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:16:10.117072 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.172729 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.171547 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:10.211102 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.177804  4011 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:10.179555 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.181217  4130 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:10.185746  4271 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:10.139091  4189 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:10.215297 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43879.4us
I20260709 13:16:10.196470 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.215708 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.216137 30559 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.216212  4271 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:10.216481 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:10.216923  4271 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:10.217082 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.161324  4103 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:10.218997  4275 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:10.219357  4103 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.202682 30701 raft_consensus.cc:2393] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:10.104638 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.221261 30704 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:10.207175 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.222414 30629 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:10.223495 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.224812 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:10.203333  3551 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.007s
I20260709 13:16:10.213076 30557 raft_consensus.cc:2393] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:10.218344  4269 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:10.207785 30625 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:10.226131 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.231375 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.197911  4268 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:10.241675 30559 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:16:10.250852 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.229036 30552 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:10.240201 30553 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:10.245324 30555 raft_consensus.cc:2368] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:16:10.281503 30629 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:10.286144 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:10.174021  4258 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.281883  4269 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:10.310508  4267 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
I20260709 13:16:10.310660 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.311998 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.313223 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.314278 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.315735  3181 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.334642  3181 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.335932 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2182) took 1632 ms. Trace:
I20260709 13:16:10.336453 30698 rpcz_store.cc:276] 0709 13:16:08.703911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.704534 (+   623us) service_pool.cc:224] Handling call
0709 13:16:08.704935 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:09.430092 (+725157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.335664 (+905572us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b540032b254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:10.335868 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666308352}
I20260709 13:16:10.337131  3181 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:10.316392  4266 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:10.337407 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1129) took 1156 ms. Trace:
W20260709 13:16:10.337944 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50860.9us
I20260709 13:16:10.338201 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1127) took 1158 ms. Trace:
I20260709 13:16:10.339604 30522 leader_election.cc:400] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:10.340095  4258 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:10.341203  4137 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:16:10.345104  4156 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.111s	user 0.013s	sys 0.000s
I20260709 13:16:10.346149 30630 raft_consensus.cc:2410] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:16:10.347543  4268 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:10.351936  4016 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.352358  4218 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.352555  4016 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.352908  4218 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.204193  4031 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.354952  4218 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:10.308702  4235 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.201s	user 0.005s	sys 0.000s
I20260709 13:16:10.355530  4218 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:10.356235 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:10.309094 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.357374 30633 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 13:16:10.357754 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:10.358355 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.320577 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:10.359041 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1815) took 1894 ms. Trace:
I20260709 13:16:10.359757 30599 leader_election.cc:400] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:16:10.360126 30599 leader_election.cc:403] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 13:16:10.359069 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 213662us
I20260709 13:16:10.203581  4203 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
W20260709 13:16:10.327418 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.209615  4136 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:10.335786 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.338725 30626 rpcz_store.cc:276] 0709 13:16:09.179861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.214209 (+ 34348us) service_pool.cc:224] Handling call
0709 13:16:09.214589 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:09.215153 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:09.215185 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:09.215870 (+   685us) log.cc:844] Serialized 26 byte log entry
0709 13:16:09.848336 (+632466us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:09.848393 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.848725 (+   332us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.337965 (+489240us) raft_consensus.cc:1685] finished
0709 13:16:10.337989 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.338138 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1390,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":8098,"thread_start_us":631970,"threads_started":1,"wal-append.queue_time_us":806113}
I20260709 13:16:10.218008  4271 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:10.327328 30550 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:10.355731  4195 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.001s
I20260709 13:16:10.339993 30522 leader_election.cc:403] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:16:10.210088  4250 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:10.367887 30631 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:10.395721 30705 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:10.414422 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.359478 31361 rpcz_store.cc:276] 0709 13:16:08.464889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.567227 (+102338us) service_pool.cc:224] Handling call
0709 13:16:08.575519 (+  8292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:08.576462 (+   943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:08.576495 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:08.577131 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:16:08.946666 (+369535us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:09.002545 (+ 55879us) log.cc:844] Serialized 14 byte log entry
0709 13:16:09.003108 (+   563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:09.003231 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:10.198435 (+1195204us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:10.358710 (+160275us) raft_consensus.cc:1685] finished
0709 13:16:10.358825 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:10.358970 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":246962,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":319,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":10245,"spinlock_wait_cycles":19345664,"thread_start_us":363179,"threads_started":1,"wal-append.queue_time_us":463647}
I20260709 13:16:10.208927  4219 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:10.213061  4011 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:10.232699  4130 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:10.386478 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:10.395697 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.396605 30624 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:10.337882 30632 rpcz_store.cc:276] 0709 13:16:09.208511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.215280 (+  6769us) service_pool.cc:224] Handling call
0709 13:16:10.071472 (+856192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.174833 (+103361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.337204 (+162371us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b5400055ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.337342 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968120704}
W20260709 13:16:10.412853  4260 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:16:10.415184 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 197388us
I20260709 13:16:10.418800 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.431531 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.431849 30550 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:10.419649 30558 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.10-20.10]
W20260709 13:16:10.373123  4256 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.783s	user 0.008s	sys 0.008s
I20260709 13:16:10.415534 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1114) took 1425 ms. Trace:
I20260709 13:16:10.327280  3874 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:10.428892 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89455.5us
I20260709 13:16:10.429350  3181 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:10.430289 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.427937 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:10.416553  4251 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:16:10.412384  3990 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:10.433166 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.434429  4042 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:10.463197  3990 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:10.464383  3990 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.434244 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.465972 30624 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:10.467206 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.468051 30624 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:10.469859 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:10.471251 31364 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:10.472044  4119 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:10.473095 30557 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:16:10.473675 30556 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:10.469816 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.475947 30624 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:10.439301 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.477293 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.477504 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.440043 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:10.440225  4130 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:10.415381  4156 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15.
I20260709 13:16:10.447031 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.436617 30627 rpcz_store.cc:276] 0709 13:16:08.990730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.990887 (+   157us) service_pool.cc:224] Handling call
0709 13:16:10.400313 (+1409426us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5400122354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.415425 (+ 15112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50113792}
I20260709 13:16:10.484616 30557 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:10.449487 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:10.436995  4149 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:16:10.451494 30552 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:10.493225  4149 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:10.356241  4218 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.501819  4218 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:10.435907 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76326.8us
I20260709 13:16:10.452539  4250 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.505323  4250 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:10.451766 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.439350 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.511168 30625 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:10.511338 30626 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:16:10.453034 30555 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:10.455633 30628 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:10.453083  4016 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:10.456635  4104 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:10.456257  4269 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:10.515394  4104 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:10.516925  4104 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:10.457481 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.519603 31383 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:10.431635  4220 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.460861 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.460906 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:10.415738  3887 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:16:10.520943  4220 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.464856 31378 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:10.413126  4213 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:16:10.448093 31396 pending_rounds.cc:85] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:16:10.523253 31396 pending_rounds.cc:107] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:16:10.475096 30556 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:10.521184 31389 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:10.448135 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:10.523649 31396 raft_consensus.cc:2889] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:10.432783 30631 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:16:10.497721  4149 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.524259 30559 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:10.660007  4220 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:10.625231 30700 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:16:10.675236 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.527697 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:10.581161 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 120387us
W20260709 13:16:10.668237 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 382211us
I20260709 13:16:10.678793 30705 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:16:10.679646 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 260742us
W20260709 13:16:10.679803 30705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2177) took 2115 ms (client timeout 1663 ms). Trace:
W20260709 13:16:10.680414 30705 rpcz_store.cc:269] 0709 13:16:08.564742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:08.564902 (+   160us) service_pool.cc:224] Handling call
0709 13:16:08.779930 (+215028us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b54005279a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806ee0e18 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:16:10.363164 (+1583234us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b580057a1d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806ee127c 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c
0709 13:16:10.679717 (+316553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653509760}
W20260709 13:16:10.684196 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.426779  4275 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:10.695832 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:10.695863 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.697817 31361 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:10.526952  4256 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:10.661221  4130 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:10.707716 30557 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [11.7-15.9]
I20260709 13:16:10.683036  4136 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:10.708492  4136 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.710019 30632 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:10.710881 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1347) took 1107 ms. Trace:
I20260709 13:16:10.711192 30632 rpcz_store.cc:276] 0709 13:16:09.603405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.709458 (+1106053us) service_pool.cc:224] Handling call
0709 13:16:10.709824 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.710514 (+   690us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:10.710819 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:10.711738 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.712649 30632 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:10.608224  4213 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:10.714775  4213 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:10.716040  4213 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:10.701574 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.721176 30624 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:16:10.721616 31383 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:16:10.722754 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.673954  4219 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.729007 30550 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:16:10.617151  4138 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:10.728348  4119 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:10.771881 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.786933 30632 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:10.787381 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1356) took 1175 ms. Trace:
I20260709 13:16:10.791386 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:10.793609 31361 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:16:10.795251 30551 raft_consensus.cc:2368] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23.
I20260709 13:16:10.796489  4138 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:10.797631 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.788112 30628 rpcz_store.cc:276] 0709 13:16:09.612349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.037256 (+424907us) service_pool.cc:224] Handling call
0709 13:16:10.787112 (+749856us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54002012d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.787284 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907656064}
I20260709 13:16:10.777096  4137 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:10.800238  4137 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:10.801244  4137 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.767988  4136 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:10.802819 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1130) took 1569 ms. Trace:
I20260709 13:16:10.803277 30630 rpcz_store.cc:276] 0709 13:16:09.233565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.238230 (+  4665us) service_pool.cc:224] Handling call
0709 13:16:10.786296 (+1548066us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54004a92d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:10.802731 (+ 16435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501717376}
W20260709 13:16:10.804641 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.803458 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:10.812063 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 135017us
I20260709 13:16:10.813687 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:10.770570  4218 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:10.816383 30523 leader_election.cc:400] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:10.771270  4256 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:10.881482 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:10.764061  4219 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:10.882074 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 65884.8us
I20260709 13:16:10.882678 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:10.884253 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80859us
I20260709 13:16:10.888933 30632 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:10.889549 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:10.890728 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:10.896633 30523 leader_election.cc:400] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14.
I20260709 13:16:10.898310 30624 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:10.782317  4119 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:10.903432 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.877902  4271 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:10.912985 30628 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:10.879877  4286 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:16:10.915194 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.917053 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.918267 30670 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:10.920511 31367 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:10.930528 30632 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:10.898800  4138 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:10.895756  4255 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:16:10.933902 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:10.939289 30701 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:10.926311 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:10.940943 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.928880 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:10.950588 30705 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:10.963497 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:10.968366 30628 raft_consensus.cc:2410] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:16:10.969451 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.969846 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.969866 30702 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:10.970928 30707 raft_consensus.cc:2410] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:16:10.977938 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.970853 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:10.982000 30698 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:10.979130 30704 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:10.924867  4149 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:10.884058  4218 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:10.930061  3887 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:10.932050  3874 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:10.956099 30523 connection.cc:256] RPC call timeout handler was delayed by -1.1661870510001791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 13:16:10.998132 30628 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:10.921095  4271 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:11.011597 30523 connection.cc:256] RPC call timeout handler was delayed by -1.1616870510001718s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 13:16:10.988929  4285 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:16:11.012312 30523 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.552s (SENT)
I20260709 13:16:10.893460  4275 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.015285 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.019217  3887 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.023175  4285 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:11.023617  4213 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:11.047580  4288 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:16:11.047940  4254 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.055184  4254 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.010560  3874 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.057723  4254 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.058858  3874 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:11.060369 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.061452 30628 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:11.064540 30632 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 13:16:11.065236 30632 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 12
I20260709 13:16:11.065582 30632 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 13:16:11.065899 30632 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:10.995761  4256 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:11.069590 30624 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:16:11.049868  4213 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:11.071692  4271 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:11.073056  4289 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:11.071753  4213 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:11.075234  4119 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.028291  3929 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:11.085930 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:11.086410 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.096467 30629 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:11.045019  4286 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:11.083052 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:11.090986 30523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.804s (SENT) [suppressed 41 similar messages]
W20260709 13:16:11.108647 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 182434us
I20260709 13:16:11.114715 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1353) took 1509 ms. Trace:
I20260709 13:16:11.117857 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:11.092626  4269 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:11.091336  4103 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:16:11.118574 30629 raft_consensus.cc:2376] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17. Re-sending same reply.
I20260709 13:16:11.123361 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.125129 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.126384 30629 raft_consensus.cc:2410] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:16:11.118512 30631 rpcz_store.cc:276] 0709 13:16:09.605703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.713238 (+1107535us) service_pool.cc:224] Handling call
0709 13:16:10.713549 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.713844 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:10.910313 (+196469us) log.cc:844] Serialized 14 byte log entry
0709 13:16:11.114287 (+203974us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.114356 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:11.114411 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.114487 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.114648 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3268,"prepare.queue_time_us":299641,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":10300,"thread_start_us":330066,"threads_started":2}
I20260709 13:16:11.111967  4136 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.109556  4119 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:11.114253  4156 raft_consensus.cc:2764] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:16:11.135121 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.135669  4136 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.136260 30557 raft_consensus.cc:2410] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:16:11.137378 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.138046  4136 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:11.135249  4292 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:16:11.139423  4218 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:11.135397  4119 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.140117  4119 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:11.050550  3887 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.138522 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.144110 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:11.135308 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.126786  4039 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.170567  4039 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.107604  3929 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.087294  4285 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.110128  4254 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:16:11.177094 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:11.177116 30550 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:11.177289 30629 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:16:11.143226  4250 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.637s	user 0.016s	sys 0.000s
W20260709 13:16:11.128917 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 200141us
I20260709 13:16:11.194334 30557 raft_consensus.cc:2410] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:16:11.194415 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.202162 30631 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:11.143230  4219 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.173170  4039 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.206259 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:11.207875 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 116953us
I20260709 13:16:11.208977 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.139539  3990 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
W20260709 13:16:11.210655 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 98665.6us
W20260709 13:16:11.211656 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.211952 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0437870509999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:16:11.212459 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0122870510001576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 13:16:11.202330  4257 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.215242 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 97540.2us
W20260709 13:16:11.217391 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.219704 30550 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:16:11.226073 30629 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:11.227355 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.228310 30629 raft_consensus.cc:2410] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:16:11.230657 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1001) took 1040 ms. Trace:
I20260709 13:16:11.231176 30700 rpcz_store.cc:276] 0709 13:16:10.190328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.190528 (+   200us) service_pool.cc:224] Handling call
0709 13:16:11.230383 (+1039855us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54003d8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:11.230520 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966165888}
I20260709 13:16:11.232654 30700 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:11.210992  4042 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:11.239744 30706 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1746) took 1541 ms (client timeout 2002 ms). Trace:
I20260709 13:16:11.238284 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:11.240449 30706 rpcz_store.cc:269] 0709 13:16:09.697881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.698056 (+   175us) service_pool.cc:224] Handling call
0709 13:16:11.239470 (+1541414us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b5400050d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:11.239654 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425397632}
W20260709 13:16:11.239861 30554 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1080) took 1996 ms (client timeout 1808 ms). Trace:
W20260709 13:16:11.242529 30554 rpcz_store.cc:269] 0709 13:16:09.243048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.272746 (+ 29698us) service_pool.cc:224] Handling call
0709 13:16:11.239677 (+1966931us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b540019b754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:11.239788 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":767188864}
I20260709 13:16:11.245009 30631 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 12: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:11.245330 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1088) took 1724 ms. Trace:
I20260709 13:16:11.242578 30629 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:11.234404 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.247044 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:11.247012 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.248355 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1954) took 1103 ms. Trace:
I20260709 13:16:11.250558 30705 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:11.209046  4220 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:16:11.250782 30559 rpcz_store.cc:276] 0709 13:16:10.144810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.247743 (+1102933us) service_pool.cc:224] Handling call
0709 13:16:11.247978 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.248137 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:11.248154 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.248187 (+    33us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:11.248208 (+    21us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.248241 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.248310 (+    69us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:11.257383 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49050.9us
I20260709 13:16:11.245882 30558 rpcz_store.cc:276] 0709 13:16:09.521537 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.521841 (+   304us) service_pool.cc:224] Handling call
0709 13:16:09.544180 (+ 22339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:10.432778 (+888598us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.238059 (+805281us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b540028a5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:11.245241 (+  7182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2012668160}
I20260709 13:16:11.261179 30631 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 13:16:11.261670 30631 pending_rounds.cc:85] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 8
I20260709 13:16:11.261919 30631 pending_rounds.cc:107] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 13:16:11.262159 30631 raft_consensus.cc:2889] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:11.243923  4042 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.256168 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.264829 30700 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:11.265894 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.220466  4149 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
W20260709 13:16:11.254350  4295 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 13:16:11.267772 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.266383  4042 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:11.252895  4213 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.255506  4130 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.244581  4219 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:16:11.221653  4257 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.265782  4308 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.271158 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.275254  4256 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:16:11.268946 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:11.278599 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.317360  4308 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.318292 30558 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:11.318384 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49557.3us
I20260709 13:16:11.319722 30558 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:11.320832  4309 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:11.321249 30550 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:11.320871 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.324168 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.324973 31396 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:11.294528 30705 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:16:11.294037 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.316247 30552 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:11.295320 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.296619 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.327603 30700 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:11.263329  4039 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:11.258037  4156 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.305068  3990 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.316200 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.333395 31364 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:11.332266  4103 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:11.340036 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.340152 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.349079 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:11.341004 31389 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:11.353578 31361 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:16:11.355938 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.362358 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.363318 31396 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:11.363371  4310 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:11.365840 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.365764 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:11.346068  4219 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:11.308868  4130 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:11.369371 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 94758.6us
W20260709 13:16:11.370862 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.354990  4103 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:11.372162  4137 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:11.359249  4149 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:11.338199  4156 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.336661  4039 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.372495 30597 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:11.369776  4219 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:11.376575  3887 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:11.380509  4130 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.338706  3990 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.391546  4031 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:11.392796  4156 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.370457  4310 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.399988  4137 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:11.401461  4310 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:11.398761  4119 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:11.374987  4149 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.366745  4309 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:16:11.373914  4159 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:11.384913  4300 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:16:11.417411 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:11.419939 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.425824 30703 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:11.426895 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50479.8us
I20260709 13:16:11.402951  4149 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:11.402134  4042 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.418109  4159 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:11.428380 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:11.417408  4275 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.423913  4156 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.459358  4039 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.455263  4213 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.459359  4156 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:11.443571 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.478904 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.441251 30556 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:11.443452  4219 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:11.421902  4316 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:16:11.462067  4308 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.441978  4159 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:11.459466  4275 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.461083  4119 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:11.448902  4042 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.466038  4156 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.431871  4256 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:16:11.423422  3887 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:11.437813 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65060.8us
I20260709 13:16:11.467854 30627 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:11.439005 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.499151 30557 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:11.480281 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2464) took 1301 ms. Trace:
I20260709 13:16:11.499501 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.499939 30556 rpcz_store.cc:276] 0709 13:16:10.178717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.430176 (+1251459us) service_pool.cc:224] Handling call
0709 13:16:11.440979 (+ 10803us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.479807 (+ 38828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.480190 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:11.498383 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1351) took 1893 ms. Trace:
I20260709 13:16:11.490406  4130 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:11.501767 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:11.491466 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2455) took 1373 ms. Trace:
I20260709 13:16:11.502135  4130 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.502584 30551 rpcz_store.cc:276] 0709 13:16:10.118219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.216790 (+ 98571us) service_pool.cc:224] Handling call
0709 13:16:11.491198 (+1274408us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:11.491395 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1327620480}
I20260709 13:16:11.489894  4011 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.491482 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.491791 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1352) took 1886 ms. Trace:
I20260709 13:16:11.445042  3990 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.492086  4317 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.489943  4316 consensus_queue.cc:1243] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:11.502739  4130 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:11.493749 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:11.531086 31383 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:11.494025  4275 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:11.497145 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.535846 30626 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:11.536748 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1139) took 1360 ms. Trace:
I20260709 13:16:11.537232 30626 rpcz_store.cc:276] 0709 13:16:10.176311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.496741 (+1320430us) service_pool.cc:224] Handling call
0709 13:16:11.536641 (+ 39900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:11.538516 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:11.539494 31394 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:11.506595 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:11.508430 30706 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:11.501605 30627 rpcz_store.cc:276] 0709 13:16:09.605012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.423744 (+1818732us) service_pool.cc:224] Handling call
0709 13:16:11.442594 (+ 18850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.497974 (+ 55380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.498305 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:11.517553 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:11.516491 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 88197.9us
I20260709 13:16:11.517534 30625 rpcz_store.cc:276] 0709 13:16:09.605355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.398769 (+1793414us) service_pool.cc:224] Handling call
0709 13:16:11.399408 (+   639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.399694 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:11.400078 (+   384us) log.cc:844] Serialized 14 byte log entry
0709 13:16:11.491239 (+ 91161us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.491324 (+    85us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:11.491374 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.491447 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.491730 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2285,"prepare.queue_time_us":2318,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":5649,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":137,"thread_start_us":88274,"threads_started":1,"wal-append.queue_time_us":114734}
I20260709 13:16:11.513154  3936 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:11.604414 31383 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:11.528134  4210 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:16:11.551506 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 1.35314e+06us
I20260709 13:16:11.563176 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:11.552728 31394 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:11.511530  4104 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:11.523423  4257 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.660704  4257 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:11.661264  4257 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:11.662024 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.663836  4159 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.648744 31389 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:11.607894 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:11.597688 30551 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:11.584342 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.674276 30557 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:11.607118 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.597299 30554 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:11.542994 31378 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:11.676100 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:11.650069 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.551741  3929 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:11.568470 31361 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:16:11.563606 31388 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 11. Current term is 12. Ops: []
I20260709 13:16:11.584012 30556 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:11.560577  4189 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:11.541517  4011 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:11.578998  4220 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
W20260709 13:16:11.655402 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 137958us
I20260709 13:16:11.659657  4210 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:11.596555  4316 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.580667 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.571951 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:11.701167 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 129281us
W20260709 13:16:11.681717  4322 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
I20260709 13:16:11.666528  4159 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:11.681483  4130 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.677019 30559 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:11.711469 30698 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:16:11.689375 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.690442 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:11.657804 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:11.694164 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 86413.3us
I20260709 13:16:11.695468 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:11.555215 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1952) took 1659 ms. Trace:
I20260709 13:16:11.716812 31367 rpcz_store.cc:276] 0709 13:16:09.895803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:09.895970 (+   167us) service_pool.cc:224] Handling call
0709 13:16:09.935693 (+ 39723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:10.923697 (+988004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.554796 (+631099us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b5400523854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:11.555119 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264189824}
I20260709 13:16:11.573843  4308 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:16:11.699282 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.682803  4291 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.684s	user 0.001s	sys 0.004s
I20260709 13:16:11.720484 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:11.709583 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:11.725178 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 67462.2us
I20260709 13:16:11.662277  4257 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:11.696141  4319 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:11.729609 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.699225  4210 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:11.539582  4317 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.733712 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.727630 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.710824  4119 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:11.736943 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.758649 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.724539  3929 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:11.750399 30698 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:16:11.768390 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79117.6us
I20260709 13:16:11.737852 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:11.762476 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.722666  4308 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:11.774693 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:11.775833 30701 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:11.764346 30559 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:11.705480  4042 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 13:16:11.751099 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55729.8us
I20260709 13:16:11.784276 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.786206 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:11.770642 31367 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:11.787286 30554 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:16:11.788895 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.787220 30624 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:11.749603  4152 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:11.783013 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:11.791384 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71068.7us
I20260709 13:16:11.792115 31378 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:11.740546  4159 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:11.795471 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:11.754869  4119 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:11.739996  4302 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:11.705629  4316 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.758193  4197 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.735031  4325 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:11.757855  3990 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:11.803611 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:11.806258 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:11.806754 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.783826  4269 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.653s	user 0.015s	sys 0.000s
W20260709 13:16:11.814112 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.816162 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:11.816954 30626 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:11.783818  4319 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.824142 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 133212us
I20260709 13:16:11.828330 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:11.859890 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.865316 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2465) took 1655 ms. Trace:
I20260709 13:16:11.808939  4119 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:11.816339  4160 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:16:11.865882  4310 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.011s
I20260709 13:16:11.866150 30555 rpcz_store.cc:276] 0709 13:16:10.179050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.401822 (+1222772us) service_pool.cc:224] Handling call
0709 13:16:11.402236 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:11.402541 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:11.682479 (+279938us) log.cc:844] Serialized 14 byte log entry
0709 13:16:11.834063 (+151584us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:11.834153 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:11.834236 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:11.834338 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:11.834543 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26909,"prepare.queue_time_us":26679,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":29140,"thread_start_us":395477,"threads_started":2}
I20260709 13:16:11.836764  4319 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:11.830422  4137 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:11.824395  4152 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:11.816630  4197 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:11.868050  4250 raft_consensus.cc:2764] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:16:11.810379  4159 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:11.896893 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 126829us
I20260709 13:16:11.903508 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.900658 30625 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:11.928664 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 146268us
I20260709 13:16:11.933012 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:11.935153 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:11.895185 31367 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:11.819671  3970 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:16:11.940843 30555 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:11.896345  4275 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:11.947942 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 155740us
I20260709 13:16:11.948098 30706 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:11.948939 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:11.927578  4031 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:11.966871  4031 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:11.927215  4269 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:11.817461  4302 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.927867  4016 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:11.958823 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:11.960317  4219 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:11.970429 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:11.953497 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.951421 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:11.976835 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.000640 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:12.007176 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 109014us
I20260709 13:16:11.956305  4275 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.009681 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.011458 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.012785 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:12.014000 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 81090.6us
I20260709 13:16:11.995760  4213 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.019553 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.019872  4213 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.021399 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.023010  4269 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:12.023794  4269 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:12.024317 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.025113 30554 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:12.025426 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:11.904824  4308 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.026873  4269 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:12.027961  4213 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.012351  4275 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:11.945693  4149 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.003523  3990 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:12.031808  4159 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:11.943522  4152 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:12.051892 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51361.2us
W20260709 13:16:12.052567 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49226.4us
I20260709 13:16:12.060317 30706 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:12.003523  4016 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.031903  4130 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.069022 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:12.072708  4016 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:12.074260  4331 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:16:12.075083  3990 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:12.078909  4160 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:12.079301  4197 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.081653  4337 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.083151  4316 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.047421  4338 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:12.083523  4302 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.044873  4334 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:12.086153  4136 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.087947  4336 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:16:12.064855  4317 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.088688 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.089514 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.038787  4143 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:12.041852  4275 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:12.098091 30626 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:11.993749  4031 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.048137  3970 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:16:12.101468 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.064447  4272 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:12.111611 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51523.7us
I20260709 13:16:12.117758 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:12.083582  4149 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.099471  4130 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.123947 31391 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:16:12.125592 30552 raft_consensus.cc:2410] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:16:12.094235  4042 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 13:16:12.190251 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 164852us
I20260709 13:16:12.118728  3990 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:16:12.157461 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:12.182417 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 113519us
W20260709 13:16:12.184816 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67169.8us
I20260709 13:16:12.185670 31391 pending_rounds.cc:85] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:12.195641 31391 pending_rounds.cc:107] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:16:12.194044 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.196195 31391 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:12.115465  4338 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.144618  4272 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:12.147009  4143 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:12.245812 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:12.241475 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 43570.8us
W20260709 13:16:12.198271 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 131336us
W20260709 13:16:12.221735 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.190853  4011 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:12.312002 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 117216us
I20260709 13:16:12.143676  4257 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:12.135170  4136 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.308478 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.238497  4338 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:12.134290  4016 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:16:12.118203  4337 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.238956  4272 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:12.192806  4130 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.316675 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.332455 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.337250  4130 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:12.200544 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.318995 31394 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:12.321695  3929 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:12.322026  4143 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.308048  3990 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:12.332861 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:12.355343  4356 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:12.355580  4119 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.356235  4356 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.356808  4356 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:12.358070 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 164223us
I20260709 13:16:12.354075 30599 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.359283 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:12.348224 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.359270 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.375901 30599 leader_election.cc:400] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23.
I20260709 13:16:12.376649 30599 leader_election.cc:403] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:12.383620 31394 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:12.383496 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:12.357300  4352 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:12.390352 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 189939us
I20260709 13:16:12.392942 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:12.393424 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:12.394253 30699 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:12.394492 30702 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: [15.9-21.12]
I20260709 13:16:12.394979 30700 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:12.330811  4317 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:16:12.357375  4130 raft_consensus.cc:2764] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 13:16:12.385846  3936 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.745s	user 0.014s	sys 0.000s
I20260709 13:16:12.347668  4316 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:16:12.368671  4336 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:12.438186 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79042.7us
I20260709 13:16:12.441568 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.444654 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:12.369297  4354 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:12.368657  3970 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.388849  4275 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:16:12.437458  4210 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.674s	user 0.014s	sys 0.000s
I20260709 13:16:12.397995  4016 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:12.445930 30704 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:12.403389  4309 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:12.440205  3936 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:12.368407  4197 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
W20260709 13:16:12.447171 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71538.4us
I20260709 13:16:12.426834  4220 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:12.450578  3929 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:12.362457  3990 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.456564  4354 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.368175  4042 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 13:16:12.464860 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 78554.3us
W20260709 13:16:12.469614 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71037us
I20260709 13:16:12.476897 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.368589  4149 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:16:12.468732 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.403774  4346 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 13:16:12.425120  4355 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.105s
W20260709 13:16:12.468587 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73103.1us
I20260709 13:16:12.435015  4299 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:16:12.444638  4159 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:16:12.475414 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 104103us
I20260709 13:16:12.398638  4269 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:16:12.482236 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:12.469806  4309 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.464365  4210 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.491838  4364 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23.
W20260709 13:16:12.493186  4350 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
W20260709 13:16:12.494261  4353 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:16:12.405470  4352 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.525970  4339 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:12.526137  4042 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:12.499696  4016 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:16:12.469743  4220 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.461196  3970 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.479626  4354 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:12.486888 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.510332 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:12.470281  4257 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:16:12.512351 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.498989  4355 consensus_queue.cc:1243] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:12.496989  4275 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:16:12.515571  4344 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:16:12.520164 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.559571 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:12.560423 31364 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:12.505887  4349 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:12.525586  4360 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:12.526333 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.505049 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.529891  4220 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:12.531953 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55147us
I20260709 13:16:12.605070 31378 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:12.532696 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:12.605769 31388 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:12.533373  4136 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:12.544348  4364 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:12.608664 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 140059us
I20260709 13:16:12.533941  4119 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.612553 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.613437 30552 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:12.613509  4361 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.613786  4210 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:12.614152  4361 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.534039  3970 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.518500  4309 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:12.631433 31364 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:12.536510 30550 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:12.536929 31383 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
W20260709 13:16:12.539593 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52945.2us
I20260709 13:16:12.603354  4337 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.527935  4352 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:12.606150 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.505537  3929 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:16:12.556433  4197 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:12.555624  4336 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:12.617729 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.617964 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.638195 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.638844 30552 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:12.619751 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.565945  4138 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:12.610106  4350 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:16:12.633525 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.546648 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.605952  4360 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:12.616817  4189 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.921s	user 0.010s	sys 0.006s
I20260709 13:16:12.610785  4364 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:12.644464  4342 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
I20260709 13:16:12.644961  3990 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:12.645591  4359 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:16:12.657574 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 111090us
I20260709 13:16:12.662066 31388 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:12.668046 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:12.672695 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.631490  4361 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.675458 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:12.676370 30626 raft_consensus.cc:2368] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15.
I20260709 13:16:12.681347 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.683634 31396 pending_rounds.cc:85] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 7
I20260709 13:16:12.688894 31396 pending_rounds.cc:107] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:16:12.689385 31396 raft_consensus.cc:2889] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:12.660305  4317 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.663812  4299 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:12.690439 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.661585  4364 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:12.669373  4103 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.697559 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.698529 30632 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:12.680897  4352 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:12.677541  4302 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.687657 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:12.670696  4138 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:12.682716  4210 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:12.700749  4016 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:12.699409  4366 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:16:12.721524  4357 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
W20260709 13:16:12.725935 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117207us
I20260709 13:16:12.722847 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.695734  4189 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:16:12.729565 30629 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:12.730688 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:12.731108 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:12.732205 30557 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 14: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:12.734074 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.734328 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.734417 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:12.736479 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.733908 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.737950 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:12.731468 30556 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22. Re-sending same reply.
I20260709 13:16:12.718526  4365 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.738603 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.700585  4103 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.698788  4317 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.707762  3949 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.738857 30624 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 16: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:12.740684 30631 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:12.736567 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.749354 30551 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:12.747259  4365 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.755406 30554 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:12.708443  4302 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.728507  3990 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.690083  4360 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.728709  4016 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:12.761690 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.772094 30553 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:12.761433  3949 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.737177  4210 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:12.754206 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:12.754195  4365 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:12.765355 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.784714 31367 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:12.768324  4156 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:12.761143 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.748770  4362 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:12.775038 30627 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:16:12.768491  4302 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:12.794241 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50791.6us
I20260709 13:16:12.795660 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.783288  4210 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.798377  4210 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:12.799778  3949 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.793818  3929 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:12.803406  3929 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:12.803073  4016 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.804428  3929 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.805210  4308 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:12.805395  4349 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:12.724466  4336 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.805095 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.807461 31367 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:12.770237  4138 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:12.797567 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:12.817014 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56000.9us
I20260709 13:16:12.833425 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.790434  4367 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.836526 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.793072  4156 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.811023  4360 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:12.773944  3990 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.796777  4366 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:12.848835 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.849653 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:12.824676  4138 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.851408  4366 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.824265  3942 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:12.853111  4138 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:12.856473 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:12.847576  4156 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:12.850440 30628 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22. Re-sending same reply.
I20260709 13:16:12.830698  4349 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:12.862901  4349 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.863348  4349 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:12.870961 31292 connection.cc:256] RPC call timeout handler was delayed by -2.5712228579998282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
I20260709 13:16:12.832824  4362 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:12.845512  4361 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:12.840890 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:12.841179  4374 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:16:12.837767 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:12.884881 30550 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:16:12.847293 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:12.875178 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:12.875326 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:12.887514 31292 connection.cc:256] RPC call timeout handler was delayed by -2.1490715800000544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 13:16:12.883237  4250 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:16:12.887956 31292 connection.cc:256] RPC call timeout handler was delayed by -1.9096715799998947s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1789s for a response
I20260709 13:16:12.888832 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.889827 30628 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
W20260709 13:16:12.888283 31292 connection.cc:256] RPC call timeout handler was delayed by -1.6567715799999405s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.207s for a response
I20260709 13:16:12.891644 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:12.891706 31292 connection.cc:256] RPC call timeout handler was delayed by -1.5271715799999583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 13:16:12.892287 30628 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:12.892364 31292 connection.cc:256] RPC call timeout handler was delayed by -1.1645683229999122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
W20260709 13:16:12.892906 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0004265259999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
I20260709 13:16:12.837087  4189 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:16:12.898161 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.899031 30624 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:12.900074 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.853173  3942 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.900768  3942 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:12.900327  4354 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.868527  4367 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.895563 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:12.895699  4362 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:12.900141  4319 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:12.911135 30624 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:12.910243 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.912408 30630 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:12.924888  4119 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:12.924990 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.925618  4119 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:12.908658  4365 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.926095 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:12.927989 30624 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:12.929126 30624 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:12.929808 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:12.839748 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.926641  4119 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:12.917393  4355 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:12.934983  4382 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:12.935560  4382 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.935598  4213 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:12.935957  4382 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:12.936138  4213 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:12.936643  4213 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:12.885921  4219 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.882s	user 0.010s	sys 0.007s
I20260709 13:16:12.938992 30632 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:12.969611 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:12.918126  4356 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.000s
I20260709 13:16:12.926369  4365 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:12.975711 30633 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:12.894520  4250 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:12.908303  4317 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:12.941200  4272 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.592s	user 0.014s	sys 0.000s
I20260709 13:16:12.935040  4355 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:12.982975 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 143353us
I20260709 13:16:12.984580 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:12.981819 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:12.967379  4143 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:12.987200  4275 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:13.013969 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:12.968825  4156 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:12.976629  4210 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.019687 30628 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:13.021152 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:13.021425  4348 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:13.021610  4269 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:16:12.980556  4356 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.014503  4354 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.013441  3929 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:13.026692 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:13.043180  4349 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.044697  4309 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.031404 30626 raft_consensus.cc:2368] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:16:13.031399  4143 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:12.990677  4272 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:13.022228  4365 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.054939  4363 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:13.018211  4189 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:13.050415  4355 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.056445  3990 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:13.074738 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48127.8us
I20260709 13:16:13.077654 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1881) took 1320 ms. Trace:
W20260709 13:16:13.080102 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56160us
I20260709 13:16:13.090175 30625 rpcz_store.cc:276] 0709 13:16:11.756943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.757065 (+   122us) service_pool.cc:224] Handling call
0709 13:16:13.077309 (+1320244us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:13.077563 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265490688}
W20260709 13:16:13.077167  4338 consensus_meta.cc:287] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.005s
I20260709 13:16:13.068372  4275 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:13.081952  4361 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:13.107508 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1763) took 2408 ms. Trace:
I20260709 13:16:13.111668 31391 rpcz_store.cc:276] 0709 13:16:10.693636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:10.693920 (+   284us) service_pool.cc:224] Handling call
0709 13:16:10.752103 (+ 58183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:12.196675 (+1444572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.101336 (+904661us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54000032d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:13.101566 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3153453824}
I20260709 13:16:13.084131  4189 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.121783  4308 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.123904  4349 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.123708  4308 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.100114  4355 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:13.100834  4275 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.083001  3929 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:13.032387  4250 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:13.081518  4143 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.127493  4189 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:13.131313  4361 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:13.134473  4275 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.101668  4338 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.111174  4220 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.137465  4138 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 13:16:13.116086  3942 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:13.159024  4197 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.516s	user 0.015s	sys 0.000s
I20260709 13:16:13.119067  4372 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:16:13.177170 30630 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:13.168185  4367 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.173746  4197 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:13.182869  4156 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.196367  4210 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.196630  4309 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.199291  4272 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:13.205468  4390 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 13:16:13.215317  4389 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:16:13.204238  4361 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.204732  4356 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.220599  4338 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:13.209980  4387 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.129s	user 0.002s	sys 0.000s
I20260709 13:16:13.237366  4031 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:13.239326 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.240602  4143 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.241031  4316 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.241645  4316 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:13.244279  4351 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.000s	user 0.009s	sys 0.000s
I20260709 13:16:13.246485 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1978) took 1291 ms. Trace:
I20260709 13:16:13.246945 30555 rpcz_store.cc:276] 0709 13:16:11.954880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:11.955081 (+   201us) service_pool.cc:224] Handling call
0709 13:16:11.955613 (+   532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:12.004972 (+ 49359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:12.005024 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:12.016440 (+ 11416us) log.cc:844] Serialized 52 byte log entry
0709 13:16:12.062748 (+ 46308us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:12.064471 (+  1723us) log.cc:844] Serialized 14 byte log entry
0709 13:16:12.181964 (+117493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:12.182055 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:13.243619 (+1061564us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:13.246094 (+  2475us) raft_consensus.cc:1685] finished
0709 13:16:13.246209 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:13.246414 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79745,"prepare.queue_time_us":235,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":181480,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":49209,"spinlock_wait_cycles":140757376,"thread_start_us":97918,"threads_started":2,"wal-append.queue_time_us":108086}
W20260709 13:16:13.251767 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:13.253053 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:13.257879  4352 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.553s	user 0.014s	sys 0.000s
I20260709 13:16:13.257978  4104 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:13.255632  4356 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.258841  4220 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:13.260344  3942 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.261030 30698 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:13.263516 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.267638 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.270027 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.275537 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:13.280335 30702 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:13.284727 30671 leader_election.cc:400] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15.
I20260709 13:16:13.305614 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.305728 30704 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:13.306929 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:13.307083 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.307111 30701 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:16:13.308634 30703 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:13.309455 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.310493 30702 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:16:13.311960 30699 raft_consensus.cc:3060] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:13.312885 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:13.273670  4104 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:13.279999  4215 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.321074 30700 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:13.282768 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:13.316767 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 335088us
I20260709 13:16:13.279999  4352 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:13.305361  4348 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:13.325344 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:13.327045  4104 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.328352  4104 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.331647  4316 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:13.332928  4364 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.628s	user 0.014s	sys 0.000s
I20260709 13:16:13.333463  4364 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.335711  4382 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.323153  4220 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:13.339454  3942 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:13.339327  4215 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.342197  4215 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.340133  3942 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.338359  4382 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.343122  3942 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:13.370467 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.315534  4302 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:13.287752  4356 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.375521 31388 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:13.374476 30699 raft_consensus.cc:2468] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:13.379302 30625 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:16:13.400219 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:13.400228 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:13.401345 30629 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:13.284829 30555 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:16:13.404160 30555 raft_consensus.cc:606] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
W20260709 13:16:13.403155 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.677s (SENT)
I20260709 13:16:13.406175 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:13.401211 30625 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:13.407115 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:13.375667  4039 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:13.414245 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 89113.4us
I20260709 13:16:13.206182  3929 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.372887  4316 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:16:13.368494  3990 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 13:16:13.421192 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 322222us
I20260709 13:16:13.423374 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.374181  4302 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.399250  4325 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:16:13.428000  4325 consensus_queue.cc:1243] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:13.428675  4325 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:13.429031  4325 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:13.431811  4275 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:13.399017  4031 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:16:13.435505 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.342732  4348 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.410025  4354 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:13.438966 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.419118  4402 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:13.434312  4275 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:13.442958  4392 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 13:16:13.434966  4302 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:13.454977 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:13.460032  4395 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 13:16:13.457163  4137 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:13.456871  4403 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:13.461328  4137 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:13.461946  4397 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:16:13.462432  4137 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:13.463107  4364 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.463516 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:13.463994  4137 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:13.464188 30699 raft_consensus.cc:606] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:13.464663 30699 raft_consensus.cc:614] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:13.470757  4356 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.471616 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:13.472303 30699 raft_consensus.cc:606] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:13.472630 30699 raft_consensus.cc:614] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:13.473258  4396 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:16:13.476457 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:13.477329 30699 raft_consensus.cc:606] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:13.446694  4272 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:13.477902 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.443492  4360 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:13.479010  4360 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:13.479346  4360 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:13.479563 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:13.480218  4360 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.480873 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:13.481577 30627 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:13.481642  4360 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.483181  4039 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.483886 30553 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:13.484337 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.483903 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.489305 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.494364 30558 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d847606257cc4750be11b056651106a6"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:16:13.495332 30558 raft_consensus.cc:606] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:13.495787 30558 raft_consensus.cc:614] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:13.499939 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.487953 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:13.505343 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50482.3us
I20260709 13:16:13.490244 30552 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:13.506533 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:13.507577 30627 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
W20260709 13:16:13.520792 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 111023us
I20260709 13:16:13.527529 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.535087 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.534097 30597 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:13.456974  4402 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:13.538931  4309 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.544263  4309 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.509341  4039 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.549129  4039 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.550904 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:13.552596 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:13.553388 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.555511 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.558521 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.513797  4403 raft_consensus.cc:2764] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:16:13.519593  4339 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.567611 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.568869 30554 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
W20260709 13:16:13.571305  4213 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.634s	user 0.016s	sys 0.000s
I20260709 13:16:13.552352  4398 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:13.574649  4398 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:13.575080  4398 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.579263 30559 raft_consensus.cc:2410] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:16:13.589063  4325 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.591094  4319 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:13.563334  4197 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:16:13.563475  4219 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.563647  4220 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:13.592849  4011 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:13.593348  4220 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:13.593649  4011 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:13.563057  4362 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:13.561869  4272 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:13.561122  4316 raft_consensus.cc:2804] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:13.595171  4402 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.595773  4130 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:13.596326  4316 raft_consensus.cc:697] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:13.596541  4130 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:13.597517  4316 consensus_queue.cc:237] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.563169  4152 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.571954  4352 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:13.599090  4152 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.599788  4339 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.600345  4250 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:13.601568  4152 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.602232  4339 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.604705  4103 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.564572  4366 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:16:13.592924  4219 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.607353 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:13.608356 30630 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:13.608839  4189 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:13.609531  4219 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.593873  4220 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:13.561149  4309 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.574075  4213 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.591879  4319 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:13.614554 30626 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:16:13.621143 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:13.616560 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.621557 30633 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:13.638552 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.633195 30626 pending_rounds.cc:85] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 11
I20260709 13:16:13.641597 30626 pending_rounds.cc:107] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:16:13.642007 30626 raft_consensus.cc:2889] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:13.675325 31389 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:13.661456 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.657656 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.686196 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:13.687438 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58858.9us
I20260709 13:16:13.689903 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.684089 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:13.692821 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.694850 31367 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
W20260709 13:16:13.695623 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:13.699649 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.595245  4011 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:13.702222 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.702065  4189 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:13.708835 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.634892  4254 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:13.710325  4254 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:13.713490  4254 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.715448 31394 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:16:13.718735  4346 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:13.725699 31378 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:13.727056 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.727257 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.728142 31378 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:13.631577  4325 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:13.728559 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.729212 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.729272 31367 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:13.731096 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.732319 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:13.690971 31361 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:13.728605 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.731112 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.730206 31378 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:13.736569 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.740377 30630 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
W20260709 13:16:13.741613 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:13.743265 31396 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:13.618690  4367 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.597968  4130 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.753489 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.754897  4302 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:13.763470 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 74729.6us
I20260709 13:16:13.755630  4011 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:13.730154 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.677809  4352 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:13.776328 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.777402 30554 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.774959 31389 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:13.621315  4104 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.127s
I20260709 13:16:13.787245 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.788812 31394 raft_consensus.cc:2410] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:16:13.710382 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.791996 30559 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:13.693825  4272 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:13.798842 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.796722 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:13.774003  4103 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:13.773950  4309 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:13.821583 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:13.831002 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66297.6us
I20260709 13:16:13.836609 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.840715 30554 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:13.838879 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.841843 31378 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:16:13.776057  4367 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.795295  4299 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:16:13.836730  4220 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.865670 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.797137  4256 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:16:13.871446  4308 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.886209 30550 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:13.872364  4319 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:13.875152  4410 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:16:13.881276 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.915154 30555 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:13.882505  4302 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:13.888464  4213 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:13.888957 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.923705 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.890579 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.924764 31361 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:13.925894 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.927110 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.890954 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.927691  4130 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:13.928246 31396 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:13.928834  4367 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:13.891429 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.930348 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.872238 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.821609  4103 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:13.931306 31394 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:13.931501 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.932210 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.933099 31394 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:16:13.896131  4220 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:13.936064 31361 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:13.936626  4366 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:13.937043  4366 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:13.907668 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 174043us
I20260709 13:16:13.811846  4272 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.808681  3942 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:13.934778 31394 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:13.896013 30553 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:13.918578 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.929219 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:13.932993  4103 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.846138 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:13.937870 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.948103 31383 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:13.940613  3942 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:13.939865  4272 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:13.969696 31361 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:13.942468 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.943517 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.970757 31388 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:13.944376 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.944923 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.956585 31391 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
W20260709 13:16:13.977517 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 131477us
I20260709 13:16:13.983448 30700 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:13.983660 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:13.990350 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:13.989234 31378 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:16:14.008641 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.036969 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:14.039980 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:14.146008 30631 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:14.147830 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:14.148065 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 175583us
W20260709 13:16:14.150254 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.152105 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 200878us
I20260709 13:16:14.194075 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:14.195214 30631 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:14.203235 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 163333us
I20260709 13:16:14.208472 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:13.951367  4319 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:14.216058 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.210682 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:14.224272 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:14.225118 30630 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:14.247989 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.257794 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:14.266498 30553 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:14.266965 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:14.269492 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:14.232023 30556 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:16:14.259832 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 265600us
I20260709 13:16:14.260141 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:14.277473 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:14.199458  4347 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:14.192463  4256 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:16:14.292302 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139178us
I20260709 13:16:14.151955  4339 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:14.294972 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:14.223121 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:13.970415  4220 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.308s
I20260709 13:16:14.323419 30631 raft_consensus.cc:2468] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:14.302886 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:14.199002  4104 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:14.274013  4308 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.317893 30707 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:14.224231  4189 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.303139 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:14.331255 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 108198us
I20260709 13:16:14.334897 31388 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:14.329422 30553 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:14.310333 30630 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:16:14.413331 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72083.9us
W20260709 13:16:14.413666 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 110893us
I20260709 13:16:14.419656 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:14.419682 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:14.298447  4364 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.140s
I20260709 13:16:14.341813 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.420738 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:14.414510 30671 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:14.554278 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:14.298840  4367 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.313944  4372 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:14.340860  4316 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.291774  4016 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.238s
I20260709 13:16:14.330377  4352 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:14.298120  4119 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:14.573426 30703 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:14.422137  4219 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:14.577527 30630 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:16:14.419695  4362 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:14.422837  4256 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:14.340651  4365 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:14.581523 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 160840us
I20260709 13:16:14.528636  4402 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:14.603755 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:14.452195  4189 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:14.600014  4299 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:14.616667 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.616871  4299 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.617446  4299 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:14.623474  4011 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.002s
I20260709 13:16:14.577176  4367 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:14.626358 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.626797 30702 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:14.627259 30706 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 13:16:14.627934 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.628515 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.628665 30698 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:14.629366 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:14.629760 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:14.630115 30553 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:14.605052  4335 raft_consensus.cc:2804] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:14.631421 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1986) took 1129 ms. Trace:
I20260709 13:16:14.633837 30705 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:16:14.634016 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.630555 30706 raft_consensus.cc:2393] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:14.637985 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:14.631448 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:14.638765 30553 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:14.627938 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.640919 30702 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:14.331282  4339 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:14.421314  4255 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.168s
I20260709 13:16:14.410955  4104 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:14.445513  4308 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:14.642962 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:14.692468 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 463538us
I20260709 13:16:14.643134 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:14.694994 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.695762 30553 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:14.697021  4402 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.697345 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:14.697855  4402 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:14.699685 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:14.700912  4402 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:14.701298 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:14.701956 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.702126 30555 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:14.695021 31396 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:14.649165 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:14.629679 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:14.632017 30558 rpcz_store.cc:276] 0709 13:16:13.502172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.572325 (+ 70153us) service_pool.cc:224] Handling call
0709 13:16:13.572740 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.610885 (+ 38145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.610922 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.621194 (+ 10272us) log.cc:844] Serialized 26 byte log entry
0709 13:16:13.804314 (+183120us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:13.804361 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.630997 (+826636us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:14.631049 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.631113 (+    64us) raft_consensus.cc:1685] finished
0709 13:16:14.631127 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.631358 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4340,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"spinlock_wait_cycles":86508800,"thread_start_us":116578,"threads_started":1,"wal-append.queue_time_us":316049}
I20260709 13:16:14.706866 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:14.706888 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.707845 30698 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:14.708585 30558 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:16:14.661862 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 400394us
W20260709 13:16:14.711133 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 157010us
I20260709 13:16:14.698251 30553 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:14.713052 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:14.571456 31383 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:14.714571 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:14.715168  4352 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:14.715763  4352 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:14.716027 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:14.717598 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:14.653995 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:14.685714 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:14.614287  4382 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:14.687525 31391 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:14.598011  4119 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:14.609795  4219 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:14.707177 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:14.738080 30555 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:14.723580 30700 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:14.582320  4316 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:14.685828 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.740867 30702 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:14.725334 30556 raft_consensus.cc:1275] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:16:14.741699 30556 pending_rounds.cc:85] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 10
I20260709 13:16:14.689502 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:14.741974 30556 pending_rounds.cc:107] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:16:14.742285 30556 raft_consensus.cc:2889] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:14.732517 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:14.742656 30703 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:14.742950 30631 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:14.728215 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 83678.5us
W20260709 13:16:14.733141 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 318751us
I20260709 13:16:14.734411 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:14.745036 30706 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:14.735193 31396 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:14.742316 30550 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:16:14.731053 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 137362us
I20260709 13:16:14.755784 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:14.756481 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 102622us
I20260709 13:16:14.745143 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:14.607837  4365 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:14.757342 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:14.690488  4335 raft_consensus.cc:697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:14.733358  4382 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:14.747191  4398 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.172s	user 0.012s	sys 0.002s
W20260709 13:16:14.772457 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 352869us
I20260709 13:16:14.772928 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:14.746135  3942 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:14.786551 30631 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:14.775596 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:14.747232  4349 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:14.616878  4256 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:14.777704 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66690.3us
I20260709 13:16:14.763242  4016 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:14.778867  3990 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:14.771531  4382 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:14.822377 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1110) took 1796 ms. Trace:
W20260709 13:16:14.816402  4130 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.888s	user 0.011s	sys 0.004s
I20260709 13:16:14.829806 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:14.786520  4420 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:14.779130  4440 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:14.793242  3942 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:14.827838  4016 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:14.960250 30557 rpcz_store.cc:276] 0709 13:16:13.025885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.026029 (+   144us) service_pool.cc:224] Handling call
0709 13:16:13.026433 (+   404us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:16:13.047839 (+ 21406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.047892 (+    53us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:16:13.073956 (+ 26064us) log.cc:844] Serialized 78 byte log entry
0709 13:16:13.195249 (+121293us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:13.279204 (+ 83955us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.281248 (+  2044us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.466847 (+185599us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b1800199d20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:14.684700 (+1217853us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b1800199d20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:16:14.821544 (+136844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.821754 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:14.821866 (+   112us) raft_consensus.cc:1685] finished
0709 13:16:14.821887 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:14.822247 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4098,"prepare.queue_time_us":82230,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":52193,"raft.run_cpu_time_us":411,"raft.run_wall_time_us":9933,"spinlock_wait_cycles":55887360,"thread_start_us":280250,"threads_started":3,"wal-append.queue_time_us":137740,"wal-append.run_cpu_time_us":4165,"wal-append.run_wall_time_us":1246894}
I20260709 13:16:14.982527 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:14.818383  4308 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:14.981740 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 160792us
I20260709 13:16:14.775594  4219 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.211s
I20260709 13:16:14.954978  4349 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:14.796767  4210 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.222s
I20260709 13:16:14.785557  4159 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.234s
I20260709 13:16:14.779191  4436 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.250s
I20260709 13:16:14.780959  4398 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:14.969216  4362 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:15.060623 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:15.056536 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 274117us
I20260709 13:16:14.967438  4429 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:15.073067 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 315792us
W20260709 13:16:15.083739 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 742910us
I20260709 13:16:15.109350 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:15.019320  4420 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:15.038678  3942 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:16:15.132522 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 359679us
W20260709 13:16:15.136754 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.021467  4441 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:15.138315 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:15.050058  4440 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:15.073149  4016 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:15.157511 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48308.1us
I20260709 13:16:15.173301 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:15.054417  4443 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:15.169559 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:15.097482  4308 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.175173 31364 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:15.076725  4439 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:15.186749 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:15.187556 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 412264us
I20260709 13:16:15.095331  4450 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:15.214628 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:15.134574  4420 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:15.226679  4189 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:15.094763  3990 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.108s
I20260709 13:16:15.179437  4272 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:16:15.233394 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.179683  4436 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:15.181586  4443 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:15.253038 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66396.5us
I20260709 13:16:15.067852  4335 consensus_queue.cc:237] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.220428  4441 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:16:15.134323  4429 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.227764  4449 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:15.238517  4104 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:15.251868  4437 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.239238  4272 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:15.267086 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:15.302397 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 238136us
W20260709 13:16:15.303411 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.006s	sys 0.014s
W20260709 13:16:15.308094 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.216171  4308 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:15.226672  4210 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 13:16:15.319149 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.007s	sys 0.014s
I20260709 13:16:15.234822  4309 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:16:15.319897 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:15.323884 30633 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:15.361138 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 599358us
I20260709 13:16:15.376255 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:15.252461  4189 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.255213  4443 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:15.259684  4452 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:15.306056  4450 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:15.385485 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 157536us
W20260709 13:16:15.385618 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 131019us
I20260709 13:16:15.384580 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:15.359503  4250 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.758s	user 0.013s	sys 0.007s
I20260709 13:16:15.315950  4436 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:15.392879 30700 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.003s
I20260709 13:16:15.316736  4159 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:15.394542 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.290557  4119 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:15.308267  4365 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.237905  4349 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.386201  4441 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.310812  4458 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:15.375077  4220 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
W20260709 13:16:15.443516 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47517.1us
I20260709 13:16:15.451359 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:15.406148 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86379.5us
I20260709 13:16:15.410276 31292 leader_election.cc:400] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
I20260709 13:16:15.410236 30700 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:15.454494 31292 leader_election.cc:403] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:15.404704  4189 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.374959  4103 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:15.415396  4159 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.417604 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:15.418526 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:15.461577 30553 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:15.464938 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:15.419057 30633 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:15.411986  4436 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.321198  4449 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:15.437306  4365 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:15.411520  4346 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:16:15.333132  4437 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.469959 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.450951  4458 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.359947  4440 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.464242  4159 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:15.406574  4452 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.471629  4436 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:15.501318 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:15.504634 30633 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:15.509205 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44350.5us
I20260709 13:16:15.510411 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:15.410498  4450 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:15.512846 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 102745us
I20260709 13:16:15.520170  4469 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:16:15.521203  4453 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:16:15.522737  4445 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:15.523507  4431 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:15.525148 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1012) took 1534 ms. Trace:
I20260709 13:16:15.525547 30701 rpcz_store.cc:276] 0709 13:16:13.990568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.990747 (+   179us) service_pool.cc:224] Handling call
0709 13:16:13.991214 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:14.260900 (+269686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.261240 (+   340us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:14.277132 (+ 15892us) log.cc:844] Serialized 52 byte log entry
0709 13:16:14.660051 (+382919us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:14.660135 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.660250 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.524879 (+864629us) raft_consensus.cc:1685] finished
0709 13:16:15.524910 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.525077 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":277381,"raft.queue_time_us":268885,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":16299,"thread_start_us":342578,"threads_started":1,"wal-append.queue_time_us":1074417}
W20260709 13:16:15.527333  4451 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:16:15.528770  4466 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:15.428541  4272 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.441001  4441 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.519131  4429 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.490314  4398 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:15.530357 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1915) took 1541 ms. Trace:
I20260709 13:16:15.489957  4458 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:15.505609  4210 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.555737 30632 rpcz_store.cc:276] 0709 13:16:13.988499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.330345 (+341846us) service_pool.cc:224] Handling call
0709 13:16:14.349221 (+ 18876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:14.710355 (+361134us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.710929 (+   574us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:14.717650 (+  6721us) log.cc:844] Serialized 26 byte log entry
0709 13:16:15.098226 (+380576us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:15.098282 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.098381 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.530072 (+431691us) raft_consensus.cc:1685] finished
0709 13:16:15.530110 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.530284 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68554,"raft.queue_time_us":256114,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":5810,"thread_start_us":663103,"threads_started":2,"wal-append.queue_time_us":519323}
I20260709 13:16:15.567791 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:15.570129 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.572881 30632 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:15.577492 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.530027  4455 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:15.503939  4309 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:16:15.560267  4347 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.260s	user 0.010s	sys 0.006s
W20260709 13:16:15.613387 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.611132 30632 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:15.553786  4462 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:15.555106  4250 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:16:15.597842  4405 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 13:16:15.557049  4449 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:15.685240 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.006s	sys 0.029s
W20260709 13:16:15.689827 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.006s	sys 0.029s
I20260709 13:16:15.595309  4450 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:15.609692  4398 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17.
W20260709 13:16:15.670811  4319 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.398s	user 0.009s	sys 0.007s
I20260709 13:16:15.656745  4434 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:15.701725 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 532255us
I20260709 13:16:15.675729  4347 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:15.707995 30626 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:15.523516  4445 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.675170  4455 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:15.800828 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:15.803083 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:15.673633  4472 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:15.603837  4210 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:15.716321 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:15.768535  4432 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:16:15.768968  4475 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:15.770154  4471 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.258s
I20260709 13:16:15.770905  4467 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:15.771773  4461 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:15.772100 30553 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:15.772533  4382 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.785s	user 0.014s	sys 0.000s
I20260709 13:16:15.773829 30557 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:15.774173  4457 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:15.776377  4438 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.609227  4466 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:16:15.782066  4452 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:15.866735  4452 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.867427  4438 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:15.869181  4429 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.768258 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:15.870028  4429 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:15.787518  4308 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:15.788455  4433 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
I20260709 13:16:15.789804  4456 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:15.680343  4437 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.679068  4250 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:15.703362  4448 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.372s	user 0.002s	sys 0.000s
I20260709 13:16:15.692178  4462 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:15.899408 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:15.790287  4441 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.819754 30631 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:16:15.824390 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.900228  4462 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:15.901517  4462 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:15.901739 30631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1170) took 1348 ms (client timeout 1752 ms). Trace:
I20260709 13:16:15.872653  4382 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:15.902326 30631 rpcz_store.cc:269] 0709 13:16:14.553271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.796131 (+242860us) service_pool.cc:224] Handling call
0709 13:16:15.901499 (+1105368us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54004abad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:15.901666 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887290112}
I20260709 13:16:15.904052 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.905033 30631 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:16:15.905179  4382 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:15.906174 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.906774  4477 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.907683 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.907514  4477 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:15.909999 30704 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:15.910192  4477 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.831415 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:15.835628 31293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.010s	sys 0.002s
W20260709 13:16:15.968433 31293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.011s	sys 0.002s
W20260709 13:16:15.969429 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.847333 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:15.820328  4405 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:16:15.861649 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:15.864231 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:15.858057  4439 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:15.988178 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:15.865924  4481 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.870304 30554 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
W20260709 13:16:15.988224 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 156971us
I20260709 13:16:15.990461 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:15.990767 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:15.786908  4468 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:16:15.826139  4398 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.826488  4103 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:15.838248  4475 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.023495  4475 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:16.025198  4458 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:15.877452 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 342699us
W20260709 13:16:16.028126 30596 leader_election.cc:336] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.966s (SENT)
I20260709 13:16:16.027794  4481 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.028651 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.006263 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:15.847214  4319 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.836499  4449 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.855840  4435 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.867309  4430 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:15.895661 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1898) took 2507 ms. Trace:
I20260709 13:16:15.898826  4463 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.877023  4467 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:15.790481  4299 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.173s	user 0.011s	sys 0.002s
I20260709 13:16:15.877283  4457 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:15.713733  4434 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:15.911362 30631 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:16.053452 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1919) took 1904 ms. Trace:
W20260709 13:16:15.911973 31364 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 1987) took 2013 ms (client timeout 1509 ms). Trace:
I20260709 13:16:15.911283  4455 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:15.914319  4210 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:15.873833  4473 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.107s
W20260709 13:16:15.915325  4444 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.849s	user 0.003s	sys 0.000s
I20260709 13:16:15.826297  4472 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.030423  4481 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:15.884465  4471 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.884517  4469 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:15.867331  4452 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.889897  4456 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:15.895933  4368 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:15.926863 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:15.879410  4466 consensus_queue.cc:1243] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 21.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:15.930670  4470 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:15.931209 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1766) took 1320 ms. Trace:
I20260709 13:16:15.932708  4429 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:15.933936  4476 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:16:15.935756  4479 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.874226  4309 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.911283  4250 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.057384  4299 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:15.945348 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1775) took 1318 ms. Trace:
W20260709 13:16:15.950130  4443 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.541s	user 0.008s	sys 0.004s
W20260709 13:16:15.950670  3942 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.775s	user 0.012s	sys 0.003s
I20260709 13:16:15.950011  3929 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:16.125382  3929 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.128903  3929 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.129657  3929 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:15.950711  4403 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.923844  4486 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:15.953991 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:15.956370 30633 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:16.130960  4403 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:15.971726 30553 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:16.132169  4491 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:16.132372 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.132736 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.133430 30553 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:16.133697  4403 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.133790 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.971801 30559 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:15.959965 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:16.135170 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 175373us
I20260709 13:16:16.135720 30559 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:16.136368 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:15.993325 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2488) took 1734 ms. Trace:
I20260709 13:16:16.137557 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:15.957437 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.022746  4484 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.029882 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:15.949918  4460 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.003s
W20260709 13:16:16.014225  4436 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
W20260709 13:16:16.031616 30596 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.999s (SENT)
I20260709 13:16:16.139684 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.043836 30625 rpcz_store.cc:276] 0709 13:16:13.381130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.409593 (+ 28463us) service_pool.cc:224] Handling call
0709 13:16:13.409947 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.416082 (+  6135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.416120 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.427552 (+ 11432us) log.cc:844] Serialized 26 byte log entry
0709 13:16:15.240941 (+1813389us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b18002d4900. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:15.460431 (+219490us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:15.846690 (+386259us) log.cc:844] Serialized 14 byte log entry
0709 13:16:15.874585 (+ 27895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.874723 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.874773 (+    50us) raft_consensus.cc:1685] finished
0709 13:16:15.888419 (+ 13646us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.888660 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26192,"prepare.queue_time_us":462991,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":5155,"raft.queue_time_us":63168,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":5604,"spinlock_wait_cycles":16169728,"thread_start_us":560033,"threads_started":2,"wal-append.queue_time_us":2095112}
I20260709 13:16:16.140903 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.141396 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:15.989398  4348 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.053992 30631 rpcz_store.cc:276] 0709 13:16:14.149545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.908767 (+1759222us) service_pool.cc:224] Handling call
0709 13:16:15.909157 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.053076 (+143919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.053375 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.143903 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:16.054489 31364 rpcz_store.cc:269] 0709 13:16:13.898220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.923405 (+ 25185us) service_pool.cc:224] Handling call
0709 13:16:15.911752 (+1988347us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400680454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:15.911884 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876177792}
W20260709 13:16:16.040007  4418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.012s	sys 0.010s
I20260709 13:16:16.008692  4103 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.038880  4459 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:16.050271  4435 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.030567  4449 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:15.916730  4453 raft_consensus.cc:993] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: : Instructing follower 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:16:16.149982  4481 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:16.064385 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1910) took 2080 ms. Trace:
W20260709 13:16:16.164640  4418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.546s	user 0.012s	sys 0.010s
I20260709 13:16:16.068264  4458 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.069273  4434 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.070113  4366 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.040004  4398 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.042724  4393 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:16.074429  4420 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.000s
I20260709 13:16:16.007719  4439 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.032856  4478 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:16.053135  4457 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:16.030454  4319 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:16.088090 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 120708us
I20260709 13:16:16.030655  4159 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.096112 30703 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:16.094913 30705 rpcz_store.cc:276] 0709 13:16:14.612249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.612402 (+   153us) service_pool.cc:224] Handling call
0709 13:16:14.612873 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:14.634441 (+ 21568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.930884 (+1296443us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b540032a854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:15.931135 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47306624}
I20260709 13:16:16.095306  4466 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:15.941011  4367 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.106120 30698 rpcz_store.cc:276] 0709 13:16:14.626536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.709039 (+ 82503us) service_pool.cc:224] Handling call
0709 13:16:14.709733 (+   694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.763802 (+ 54069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.763841 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:14.763885 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:14.763922 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:15.945029 (+1181107us) spinlock_profiling.cc:243] Waited 53.8 ms on lock 0x7b5400342e54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:15.945089 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:15.945271 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123764096}
I20260709 13:16:16.109946  4479 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.111909  4456 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.112159  4472 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.112684  4467 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.113588  4452 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.114058  4443 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.115127 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.080878  4471 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:16.116659 30597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 1.700s (SENT) [suppressed 20 similar messages]
I20260709 13:16:16.117944 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.219679 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.220322 31378 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:16.119702 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.122431  4299 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:16.122784  4427 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.453s	user 0.009s	sys 0.000s
W20260709 13:16:16.124603  4016 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.910s	user 0.005s	sys 0.009s
I20260709 13:16:16.054255  4463 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.131052  4486 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:16.032786 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.145898 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:16.139437 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 109645us
I20260709 13:16:16.134367  4403 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:16.148546 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 116961us
I20260709 13:16:16.138669 31396 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:16.242820 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.242821 30559 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:16.148823 30625 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:16.134653 30633 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 12.
I20260709 13:16:16.160600 30553 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:16.137359 30550 rpcz_store.cc:276] 0709 13:16:14.259370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.259756 (+   386us) service_pool.cc:224] Handling call
0709 13:16:15.926058 (+1666302us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b5400112954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:15.993226 (+ 67168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1058542848}
I20260709 13:16:16.175038 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.022796  4488 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:16.083451  4469 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.271493  4471 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.272065  4469 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.180301 30627 rpcz_store.cc:276] 0709 13:16:13.983568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.984120 (+   552us) service_pool.cc:224] Handling call
0709 13:16:14.150060 (+165940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:14.199961 (+ 49901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.200014 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:14.277757 (+ 77743us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.720182 (+442425us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:14.720227 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.720306 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:15.914341 (+1194035us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:16.060778 (+146437us) raft_consensus.cc:1685] finished
0709 13:16:16.060811 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.064281 (+  3470us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49293,"raft.queue_time_us":50070,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":14462,"spinlock_wait_cycles":7743744,"thread_start_us":243407,"threads_started":1,"wal-append.queue_time_us":491068}
I20260709 13:16:16.044772  4450 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:16.135712  4475 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.130255  3929 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.149293  4250 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:16.184319 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.166348  4441 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:16.276368  4479 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.150898  4429 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:16.183256  4481 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.277889  4481 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:16.200567  4420 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.169162  4103 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:16.280401  4420 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.170578  4335 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.277137 30555 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:16.281270  4335 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.139124  4490 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:16.171886  4459 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:16.108196  4309 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.291706  4459 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:16.293601  4449 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.294042  4309 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.295472  4435 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.140019  3942 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.101302  4470 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.298821 31394 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:16.300577  3942 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.207279  4393 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.301405  3942 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:16.149073  4436 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.303083  4463 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.303787  4486 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.300655  4470 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.304538  4439 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.304958  4436 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.283396  4475 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.185766  4455 raft_consensus.cc:1081] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Signalling peer 015ca80eba674552b34856a4f20271f1 to start an election
I20260709 13:16:16.205363 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.306865 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.307053  4319 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.205396 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1867) took 2268 ms. Trace:
I20260709 13:16:16.166860  4348 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.138733  4484 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.215039 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.312266 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:16.218271 31367 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:16.223615 31389 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:16.193549  4489 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:16:16.243511 31364 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:16.247879 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.242731 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:16.316290 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.316754 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 74131.4us
I20260709 13:16:16.259820  4159 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:16.320076 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.262948 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.263024 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:16.264034  4403 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:16.310917  4428 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.519s	user 0.000s	sys 0.017s
I20260709 13:16:16.218062  4434 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:16.305109  4445 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.269194  4494 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:16.282946 30631 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:16.283887 31396 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:16.290571 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.286887  4490 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.302035  3942 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.206000 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.308144 31361 rpcz_store.cc:276] 0709 13:16:13.937386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.970825 (+ 33439us) service_pool.cc:224] Handling call
0709 13:16:14.208557 (+237732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.741336 (+532779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:14.741372 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:14.741421 (+    49us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:14.741452 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.205135 (+1463683us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b54002b07d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:16.205195 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.205336 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1224695040}
I20260709 13:16:16.276403  3929 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.207494  4466 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.312389 31388 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
W20260709 13:16:16.222740 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 106153us
I20260709 13:16:16.311609 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:16.318183 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:16.310724  4495 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
I20260709 13:16:16.266188  4443 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.324043 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.268575 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1912) took 2283 ms. Trace:
I20260709 13:16:16.322961  4435 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:16.347958 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:16.363525 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.386273 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:16.387877 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:16.283612  4016 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:16.346095  4508 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:16.394284  4435 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:16.408860  4435 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:16.277920  4488 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.411442  4439 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:16.407464 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:16.395740 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77668.7us
I20260709 13:16:16.423105 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.394186 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.432255 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.386264 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.437675 31361 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:16.452471 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:16.477324 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54348.2us
I20260709 13:16:16.496721 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.496721 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.512773 30698 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:16.514112 31367 raft_consensus.cc:2376] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17. Re-sending same reply.
I20260709 13:16:16.384650  4319 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:16.376226  4490 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:16.389514 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 78024.2us
I20260709 13:16:16.328331  4489 consensus_queue.cc:1243] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:16:16.393263 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.392542 30624 rpcz_store.cc:276] 0709 13:16:13.985430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.985601 (+   171us) service_pool.cc:224] Handling call
0709 13:16:13.986027 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.986510 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.986550 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:14.151056 (+164506us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.303163 (+152107us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:14.303216 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.303297 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.075049 (+1771752us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:16.268140 (+193091us) raft_consensus.cc:1685] finished
0709 13:16:16.268178 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.268495 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":894,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":58477,"spinlock_wait_cycles":767611776,"thread_start_us":150529,"threads_started":1,"wal-append.queue_time_us":453800}
I20260709 13:16:16.540289 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:16.392267  4475 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:16.533535 30705 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:16.533864 30702 raft_consensus.cc:2368] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:16.479406 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:16.540804 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.564878 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:16.566275 30624 raft_consensus.cc:2376] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17. Re-sending same reply.
W20260709 13:16:16.387046 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96649.7us
I20260709 13:16:16.387599  4494 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:16.543447 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1115) took 2795 ms. Trace:
I20260709 13:16:16.406159  4393 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.545859  4488 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.551102  4309 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.334075  4434 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.577452 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1015) took 1029 ms. Trace:
I20260709 13:16:16.578207 30705 rpcz_store.cc:276] 0709 13:16:15.548266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.517734 (+969468us) service_pool.cc:224] Handling call
0709 13:16:16.518133 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.577136 (+ 59003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.577388 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:16.552336  4425 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.934s	user 0.000s	sys 0.005s
I20260709 13:16:16.558157  4420 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:16.558522 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.559836  4484 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.567834 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:16.582087 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 102833us
I20260709 13:16:16.588636 30624 raft_consensus.cc:2376] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23. Re-sending same reply.
I20260709 13:16:16.588909 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:16.568423 30552 rpcz_store.cc:276] 0709 13:16:13.747946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.886197 (+138251us) service_pool.cc:224] Handling call
0709 13:16:13.886534 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.901267 (+ 14733us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:13.901316 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.946466 (+ 45150us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.533189 (+586723us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:14.533258 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.533362 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.282220 (+1748858us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:16.534471 (+252251us) raft_consensus.cc:1685] finished
0709 13:16:16.534498 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.543336 (+  8838us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":131,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":6975,"spinlock_wait_cycles":403395328,"thread_start_us":387181,"threads_started":1,"wal-append.queue_time_us":784442}
I20260709 13:16:16.481324  4348 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.497007  4335 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.591303 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:16.592095 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1016) took 1042 ms. Trace:
I20260709 13:16:16.596259  4016 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.597095 31361 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:16.598939  4319 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:16.599454  4443 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.600026  3942 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:16.600214 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1112) took 2853 ms. Trace:
I20260709 13:16:16.658787 30551 rpcz_store.cc:276] 0709 13:16:13.747071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.884600 (+137529us) service_pool.cc:224] Handling call
0709 13:16:13.884922 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.885321 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.885354 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.885891 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.332460 (+446569us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:14.332513 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.332594 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.551595 (+2219001us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:16.599853 (+ 48258us) raft_consensus.cc:1685] finished
0709 13:16:16.599879 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.600140 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1636,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":775482752,"thread_start_us":446241,"threads_started":1,"wal-append.queue_time_us":713910}
I20260709 13:16:16.587353  4445 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:16.543315  4500 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 13:16:16.600926 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.520354  4439 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:16.602577  4462 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.700s	user 0.006s	sys 0.015s
I20260709 13:16:16.586786  4494 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:16.602833  4309 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.605790  4434 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.618539 31367 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:16:16.619315  4347 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:16.589754 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.627864 30523 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:16.625818 30702 rpcz_store.cc:276] 0709 13:16:15.550180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.899036 (+348856us) service_pool.cc:224] Handling call
0709 13:16:16.591843 (+692807us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b54003d8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:16.592029 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456785664}
I20260709 13:16:16.720024 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:16.632515 30552 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:16:16.722543 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.562399  4475 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:16.667443 30705 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:16.673609 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.682812 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2005) took 1988 ms. Trace:
I20260709 13:16:16.683607  4435 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:16.678062  4459 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.689997 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.691212 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 1999) took 1998 ms. Trace:
I20260709 13:16:16.692114 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:16.713878  4398 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.735965 30701 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:16.630931  4484 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.738642 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.739225  4484 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.740129 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.741255 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:16.741448  4339 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.743901 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.744304 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.744827 30630 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:16.745666 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.745769 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.746888  4466 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.726863  4475 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:16.710907  4349 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:16.716059 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.724604  4347 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:16.720940 30702 raft_consensus.cc:2376] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18. Re-sending same reply.
W20260709 13:16:16.671133  4509 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.255s	user 0.001s	sys 0.001s
W20260709 13:16:16.728583 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 100815us
I20260709 13:16:16.752197 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.664352  4441 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:16.730810 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.727965 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.732543 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.756783 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.694520 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1908) took 3125 ms. Trace:
I20260709 13:16:16.735877 30556 rpcz_store.cc:276] 0709 13:16:14.692843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.698733 (+  5890us) service_pool.cc:224] Handling call
0709 13:16:14.699125 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:14.742885 (+ 43760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.690919 (+1948034us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:16.691139 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59226368}
I20260709 13:16:16.755909 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.758451 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.758853 30702 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:16:16.758855 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.640437  4319 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:16.729283  4457 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 13:16:16.738338  4454 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:16.735173  4481 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.707803  4103 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.666141  4443 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.749799 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.746556 30630 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:16.783313 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.784108 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.784906 30630 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:16.785543 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.733924  4137 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:16.750943  4347 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:16.750790  4349 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:16.733541 30624 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:16.759972 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:16.731760 30558 rpcz_store.cc:276] 0709 13:16:14.706952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.709442 (+  2490us) service_pool.cc:224] Handling call
0709 13:16:15.953426 (+1243984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.953697 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:16.008070 (+ 54373us) log.cc:844] Serialized 14 byte log entry
0709 13:16:16.199133 (+191063us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:16.199230 (+    97us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:16.199303 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.199409 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:16.682514 (+483105us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400112950. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:16.682726 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":77365,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":63,"raft.queue_time_us":3348,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":5970,"spinlock_wait_cycles":2860143872,"thread_start_us":187883,"threads_started":1,"wal-append.queue_time_us":309656}
I20260709 13:16:16.795161 30706 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:16:16.757643 30626 rpcz_store.cc:276] 0709 13:16:13.569010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.679848 (+110838us) service_pool.cc:224] Handling call
0709 13:16:13.680369 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:15.765338 (+2084969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.694124 (+928786us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54000712d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:16.694424 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4662080768}
I20260709 13:16:16.757683 30551 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:16.755895 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.765214 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.800518 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:16.757000 30552 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:16.769009  4319 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:16.738487  4424 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 2.095s	user 0.003s	sys 0.013s
I20260709 13:16:16.785770 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.742350 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:16.773000  4439 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:16.759464  4403 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:16.775530 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.805922 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:16.776693  4450 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:16.813983 30701 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:16.814658 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:16.809656 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53834us
I20260709 13:16:16.810818 30702 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:16.818588 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:16.819180 30702 raft_consensus.cc:1275] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:16.819990 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.822415  4443 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.763861  4339 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:16:16.804666  4137 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:16.825565 31389 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.512s	user 0.015s	sys 0.000s
I20260709 13:16:16.827286 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:16.827785 30626 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:16.827334 31389 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:16.828706 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1920) took 2548 ms. Trace:
I20260709 13:16:16.829604 30626 rpcz_store.cc:276] 0709 13:16:14.280389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.805859 (+2525470us) service_pool.cc:224] Handling call
0709 13:16:16.827500 (+ 21641us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.828318 (+   818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.828629 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:16.823487 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.833900 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.834843 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:16.836215 30633 raft_consensus.cc:2410] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:16:16.834790 30552 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:16.839445 30552 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:16.773809  4441 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.779881  4457 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.823467  3942 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:16:16.819221  4422 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 2.476s	user 0.003s	sys 0.020s
I20260709 13:16:16.813943  4467 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:16.816628  4463 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:16.842552 30626 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:16:16.854432 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1168) took 2302 ms. Trace:
I20260709 13:16:16.855813 30631 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:16.803627  4454 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:16.856667 30557 raft_consensus.cc:2368] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:16.854983 30626 rpcz_store.cc:276] 0709 13:16:14.552613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.840196 (+2287583us) service_pool.cc:224] Handling call
0709 13:16:16.842202 (+  2006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:16.853998 (+ 11796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.854347 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.843575  4462 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:16.863919 30626 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:16.851814  4339 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:16.866452 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:16.850857  4103 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.886222 30552 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:16.855759  3942 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:16.846275  4490 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:16.864323  4462 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:16.900326 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:16.901830 30698 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:16.895466 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1172) took 2341 ms. Trace:
W20260709 13:16:16.883833  4514 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:16:16.917603 30626 rpcz_store.cc:276] 0709 13:16:14.553878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.859050 (+2305172us) service_pool.cc:224] Handling call
0709 13:16:16.861261 (+  2211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.889027 (+ 27766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.895358 (+  6331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.920763 30706 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:16.845605  4349 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:16.918519  4481 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:16.950855  4459 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:16.961069  4515 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:16:16.968670  4272 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:16.970968  4366 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:16.972354 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1119) took 1456 ms. Trace:
I20260709 13:16:16.976683 30555 rpcz_store.cc:276] 0709 13:16:15.516431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.556736 (+ 40305us) service_pool.cc:224] Handling call
0709 13:16:15.656334 (+ 99598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.281236 (+624902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.971936 (+690700us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54000f05d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:16.972260 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427098880}
I20260709 13:16:16.973937 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1402) took 1570 ms. Trace:
I20260709 13:16:16.978940 30632 rpcz_store.cc:276] 0709 13:16:15.403649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.577067 (+173418us) service_pool.cc:224] Handling call
0709 13:16:16.973676 (+1396609us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b540007d5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:16.973856 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63726976}
I20260709 13:16:16.884567  4137 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:16.975759  4516 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:16:16.984457 30632 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:16.990011 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1393) took 1589 ms. Trace:
I20260709 13:16:16.954005  4309 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:16.990860 30632 rpcz_store.cc:276] 0709 13:16:15.400917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.983596 (+1582679us) service_pool.cc:224] Handling call
0709 13:16:16.984159 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.989576 (+  5417us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.989921 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.995436 30632 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:16.996599 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1394) took 1595 ms. Trace:
W20260709 13:16:17.000557  4421 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 2.341s	user 0.008s	sys 0.008s
I20260709 13:16:17.008368  4316 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.012296 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1397) took 1610 ms. Trace:
I20260709 13:16:17.012887 30633 rpcz_store.cc:276] 0709 13:16:15.402020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.011266 (+1609246us) service_pool.cc:224] Handling call
0709 13:16:17.011692 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.011925 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.011956 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.012019 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:17.012053 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.012114 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.012227 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.015408 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1868) took 1079 ms. Trace:
I20260709 13:16:17.015955 31389 rpcz_store.cc:276] 0709 13:16:15.936267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.953567 (+ 17300us) service_pool.cc:224] Handling call
0709 13:16:17.015175 (+1061608us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b5400717ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.015336 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212136320}
I20260709 13:16:17.018447  4316 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.019191  4316 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:17.019639 30557 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1123) took 1226 ms (client timeout 1629 ms). Trace:
W20260709 13:16:17.020125 30557 rpcz_store.cc:269] 0709 13:16:15.793452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.861265 (+ 67813us) service_pool.cc:224] Handling call
0709 13:16:17.019325 (+1158060us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b54000bf9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.019554 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2035368832}
I20260709 13:16:16.997280 30632 rpcz_store.cc:276] 0709 13:16:15.401178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.994471 (+1593293us) service_pool.cc:224] Handling call
0709 13:16:16.995139 (+   668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.996177 (+  1038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.996516 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.023612 30632 raft_consensus.cc:1240] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:17.027069 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1400) took 1624 ms. Trace:
I20260709 13:16:17.002334 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1909) took 3382 ms. Trace:
I20260709 13:16:17.033524 30632 rpcz_store.cc:276] 0709 13:16:15.403088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.022459 (+1619371us) service_pool.cc:224] Handling call
0709 13:16:17.023325 (+   866us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.026589 (+  3264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.026981 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.031545 30629 rpcz_store.cc:276] 0709 13:16:13.620133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.737726 (+117593us) service_pool.cc:224] Handling call
0709 13:16:13.738066 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:13.738472 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:13.738500 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:13.773075 (+ 34575us) log.cc:844] Serialized 26 byte log entry
0709 13:16:14.199734 (+426659us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:14.199802 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:14.199912 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:16.999817 (+2799905us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:17.001931 (+  2114us) raft_consensus.cc:1685] finished
0709 13:16:17.001958 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.002255 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8630,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":6791,"spinlock_wait_cycles":621918464,"thread_start_us":382330,"threads_started":1,"wal-append.queue_time_us":751731}
I20260709 13:16:17.039974 30632 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:17.044633 30629 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:17.049576 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1401) took 1646 ms. Trace:
I20260709 13:16:17.050175 30629 rpcz_store.cc:276] 0709 13:16:15.403385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.038063 (+1634678us) service_pool.cc:224] Handling call
0709 13:16:17.044212 (+  6149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.048952 (+  4740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.049235 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.045419  4316 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.054746 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1403) took 1639 ms. Trace:
W20260709 13:16:17.055105 30704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1765) took 2444 ms (client timeout 3080 ms). Trace:
W20260709 13:16:17.055313 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 254891us
W20260709 13:16:17.057227 30704 rpcz_store.cc:269] 0709 13:16:14.610294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.610413 (+   119us) service_pool.cc:224] Handling call
0709 13:16:17.054769 (+2444356us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400343354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.055015 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2972062976}
I20260709 13:16:17.056820 30632 rpcz_store.cc:276] 0709 13:16:15.415777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.039222 (+1623445us) service_pool.cc:224] Handling call
0709 13:16:17.039686 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.054388 (+ 14702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.054674 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.063282 30551 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:17.085407 30629 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:17.086561 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1174) took 1410 ms. Trace:
I20260709 13:16:17.087534 30629 rpcz_store.cc:276] 0709 13:16:15.676191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.051417 (+1375226us) service_pool.cc:224] Handling call
0709 13:16:17.085119 (+ 33702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.086166 (+  1047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.086477 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:17.134377 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49640.2us
I20260709 13:16:16.985241  3942 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.066509  4457 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.139910 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.141703 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.142551 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:17.144399 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 309609us
I20260709 13:16:17.145253 30704 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:17.075201  4213 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:17.148229 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.146317 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1894) took 3766 ms. Trace:
I20260709 13:16:17.147706 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:17.154665 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.135529  4441 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.155763 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.158772 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1408) took 1199 ms. Trace:
I20260709 13:16:17.158883 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.145282 30551 raft_consensus.cc:3055] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:17.148691 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.160323 30698 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:17.159833 30629 rpcz_store.cc:276] 0709 13:16:15.959192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.088579 (+1129387us) service_pool.cc:224] Handling call
0709 13:16:17.153162 (+ 64583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.153393 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.153418 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.153468 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:17.153511 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.153604 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.158682 (+  5078us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:16.959836  4462 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:17.161408 30551 raft_consensus.cc:740] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:17.162335 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.163934 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:17.137166  4438 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.164878 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.175787 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.153245 30628 rpcz_store.cc:276] 0709 13:16:13.379871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.380027 (+   156us) service_pool.cc:224] Handling call
0709 13:16:13.380460 (+   433us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:13.431781 (+ 51321us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:16:13.432196 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.528405 (+ 96209us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:13.566801 (+ 38396us) log.cc:844] Serialized 52 byte log entry
0709 13:16:13.616458 (+ 49657us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:13.619233 (+  2775us) log.cc:844] Serialized 14 byte log entry
0709 13:16:13.619594 (+   361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:13.879335 (+259741us) spinlock_profiling.cc:243] Waited 8.53 ms on lock 0x7b18002c08c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:14.759803 (+880468us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b4800550158. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:16:14.967955 (+208152us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b18002c08c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:15.136634 (+168679us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b18002c08c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419
0709 13:16:17.145923 (+2009289us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.145988 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.146053 (+    65us) raft_consensus.cc:1685] finished
0709 13:16:17.146067 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.146250 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37651,"prepare.queue_time_us":5696,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":2203,"raft.queue_time_us":4250,"raft.run_cpu_time_us":12194,"raft.run_wall_time_us":1243424,"spinlock_wait_cycles":180977920,"thread_start_us":95645,"threads_started":1,"wal-append.queue_time_us":250199,"wal-append.run_cpu_time_us":7418,"wal-append.run_wall_time_us":2063061}
I20260709 13:16:17.153959  4346 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:17.180152 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.194370 30628 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:17.157678  3990 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.200928 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.163367  4457 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:17.165874  4463 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:17.183022 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:17.140302  4316 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:16:17.193389 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2013) took 1338 ms. Trace:
I20260709 13:16:17.205152 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.206703 30555 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:17.206812 30550 rpcz_store.cc:276] 0709 13:16:15.854782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.542738 (+687956us) service_pool.cc:224] Handling call
0709 13:16:16.543238 (+   500us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:16.543745 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:16.543779 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:16.544506 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 13:16:16.864271 (+319765us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:16.864335 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.864427 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.193127 (+328700us) raft_consensus.cc:1685] finished
0709 13:16:17.193151 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.193320 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1207,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":95,"thread_start_us":299060,"threads_started":1,"wal-append.queue_time_us":473066}
I20260709 13:16:17.209677 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:17.210018 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.160419  4215 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:16:17.211184 30550 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:17.210871 30557 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:17.198771 30551 consensus_queue.cc:260] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:17.218499 30551 raft_consensus.cc:3060] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:17.222071 30700 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:16:17.193697  4346 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.210359 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.180426  4438 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:17.201042  4490 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.193257  4459 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:17.213318 30698 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:17.196274  4454 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.230942 30551 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:16:17.193879  4441 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:17.232694 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1923) took 1265 ms. Trace:
I20260709 13:16:17.245162 30629 rpcz_store.cc:276] 0709 13:16:15.967250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.203630 (+1236380us) service_pool.cc:224] Handling call
0709 13:16:17.232052 (+ 28422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.232279 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.232308 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.232360 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:17.232418 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.232483 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.232624 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.162533  4213 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.234683 30555 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:17.201853  4339 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:17.235715 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.241466 30551 pending_rounds.cc:85] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:16:17.255730 30551 pending_rounds.cc:107] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:16:17.237151  4346 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:17.256068 30551 raft_consensus.cc:2889] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:17.258911 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.232195  4457 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:17.260613 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:17.260900 30700 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:17.205991  3990 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.242297  4459 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.242069  4137 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:17.241541  4130 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:17.265041 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.247041  4523 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:17.262355  4522 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:17.269716 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.262995  4352 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.267778  4490 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:17.268090  4459 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:17.252904  4441 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.297819  4438 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:17.309603 30700 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:17.269466  4250 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:16.078912 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:17.298592  4522 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:17.331490  4521 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:16:17.271148  4130 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:17.311760  4347 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.506s	user 0.013s	sys 0.000s
I20260709 13:16:17.303486  4481 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:17.296398  4523 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:17.320067  4494 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.591s	user 0.015s	sys 0.001s
I20260709 13:16:17.359100  4189 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.326920  4438 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.363178  4316 consensus_queue.cc:1059] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:17.367641  4526 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:16:17.367779 30699 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1010) took 3377 ms (client timeout 1911 ms). Trace:
W20260709 13:16:17.407286 30699 rpcz_store.cc:269] 0709 13:16:13.989888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:13.990053 (+   165us) service_pool.cc:224] Handling call
0709 13:16:17.367440 (+3377387us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b5400341f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.367673 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5104927488}
I20260709 13:16:17.374120  4459 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.376495 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1926) took 1387 ms. Trace:
I20260709 13:16:17.417445 30625 rpcz_store.cc:276] 0709 13:16:15.989393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.375323 (+1385930us) service_pool.cc:224] Handling call
0709 13:16:17.375776 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.376026 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.376075 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.376139 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:17.376185 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.376258 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.376411 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.418360 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1925) took 1450 ms. Trace:
I20260709 13:16:17.418853 30631 rpcz_store.cc:276] 0709 13:16:15.967843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.395126 (+1427283us) service_pool.cc:224] Handling call
0709 13:16:17.417791 (+ 22665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.418003 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.418033 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.418079 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:17.418111 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.418172 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.418299 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.419432 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1927) took 1411 ms. Trace:
I20260709 13:16:17.419833 30625 rpcz_store.cc:276] 0709 13:16:16.008510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.418551 (+1410041us) service_pool.cc:224] Handling call
0709 13:16:17.418912 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.419123 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.419148 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.419192 (+    44us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:17.419224 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.419279 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.419377 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.378549 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.421748 30627 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:16:17.379658  4520 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:16:17.380666  4527 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:17.388136  4213 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.390874 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1017) took 1424 ms. Trace:
I20260709 13:16:17.429155 30706 rpcz_store.cc:276] 0709 13:16:15.966189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.796706 (+830517us) service_pool.cc:224] Handling call
0709 13:16:16.917989 (+121283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.923498 (+  5509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.390654 (+467156us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001f22d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.390807 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277874560}
I20260709 13:16:17.391479  4197 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.392689 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 1.1297e+06us
W20260709 13:16:17.392990 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.393158  4436 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:17.393332  4484 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.395671 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.394073 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:17.402849  4523 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:17.387156  4529 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:17.399394 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.443245 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.443296 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.444798 30699 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:17.362134 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:17.406941  4463 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.404245 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 210610us
I20260709 13:16:17.333798  4522 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:17.456730 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.458243 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.460233  4130 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.422439 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1180) took 1219 ms. Trace:
I20260709 13:16:17.395115  3990 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.441455 30554 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
I20260709 13:16:17.442391  4213 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:17.444021 30556 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:17.399912 30551 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:16:17.472023 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1021) took 1260 ms. Trace:
I20260709 13:16:17.472510 30701 rpcz_store.cc:276] 0709 13:16:16.211446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.815063 (+603617us) service_pool.cc:224] Handling call
0709 13:16:16.815354 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.869759 (+ 54405us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:16.883736 (+ 13977us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.295071 (+411335us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.295298 (+   227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.295343 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:17.295384 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.471761 (+176377us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b5400184cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.471817 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.471961 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12976,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":2188,"raft.queue_time_us":2503,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":6271,"spinlock_wait_cycles":124447616,"thread_start_us":403187,"threads_started":2,"wal-append.queue_time_us":347721}
W20260709 13:16:17.386405  4475 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.623s	user 0.005s	sys 0.007s
I20260709 13:16:17.446009 30625 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:16:17.442289 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 48316.5us
I20260709 13:16:17.412606  4438 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:17.449790 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.454365 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:17.454432 30704 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:17.484804 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.404361  4494 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:17.406661  4481 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:17.423377 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1779) took 2789 ms. Trace:
I20260709 13:16:17.463486 30627 rpcz_store.cc:276] 0709 13:16:16.202705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.378084 (+1175379us) service_pool.cc:224] Handling call
0709 13:16:17.422358 (+ 44274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:17.463896 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.487552 30703 rpcz_store.cc:276] 0709 13:16:14.627753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:15.171656 (+543903us) service_pool.cc:224] Handling call
0709 13:16:15.210044 (+ 38388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.202776 (+992732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.416398 (+1213622us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b5400630454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.416660 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037094272}
I20260709 13:16:17.464357 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.463486 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.396448  4347 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.466358 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:17.468936 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:17.462482 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.471414 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1931) took 1003 ms. Trace:
W20260709 13:16:17.526769 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.526887 30624 rpcz_store.cc:276] 0709 13:16:16.468666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.589427 (+120761us) service_pool.cc:224] Handling call
0709 13:16:17.471186 (+881759us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54005f41d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.471343 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33295232}
I20260709 13:16:17.528167 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.472245 30551 pending_rounds.cc:85] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 8
I20260709 13:16:17.528307 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.456583  4348 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.479226 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1177) took 1317 ms. Trace:
I20260709 13:16:17.531638 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:17.471113  4319 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.657s	user 0.015s	sys 0.000s
I20260709 13:16:17.481405 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.532316 31389 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
W20260709 13:16:17.486382 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.503408 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.355458  4457 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:17.524149 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69916.2us
I20260709 13:16:17.464179  4316 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:17.528776 30551 pending_rounds.cc:107] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:16:17.539892 30551 raft_consensus.cc:2889] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:17.445772 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 83725.9us
I20260709 13:16:17.528885 30624 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:17.490499 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.530644 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.534061 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.461546  4436 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:17.544857 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 311601us
I20260709 13:16:17.545051 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.532088 30625 rpcz_store.cc:276] 0709 13:16:16.161595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.423571 (+1261976us) service_pool.cc:224] Handling call
0709 13:16:17.445726 (+ 22155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.478810 (+ 33084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.479151 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.538455 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2498) took 1379 ms. Trace:
I20260709 13:16:17.454563  4454 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.546934 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:17.546124 30553 rpcz_store.cc:276] 0709 13:16:16.158861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.562546 (+403685us) service_pool.cc:224] Handling call
0709 13:16:16.575664 (+ 13118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:16.576206 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:16.576255 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:16.654914 (+ 78659us) log.cc:844] Serialized 26 byte log entry
0709 13:16:16.955409 (+300495us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:16.955468 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:16.955573 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.538125 (+582552us) raft_consensus.cc:1685] finished
0709 13:16:17.538160 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.538358 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":727,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":32502,"thread_start_us":228901,"threads_started":1,"wal-append.queue_time_us":577772}
I20260709 13:16:17.548270 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.506327 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:17.539250 30706 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:17.555637 30633 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:17.549824 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.556612 31383 raft_consensus.cc:2376] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18. Re-sending same reply.
W20260709 13:16:17.524523  4445 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.006s
I20260709 13:16:17.549391 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.557753 31394 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:17.549639 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.550457 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:17.524269  4439 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.666s	user 0.015s	sys 0.000s
I20260709 13:16:17.557046 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:17.561257 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55104.9us
I20260709 13:16:17.562042 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.455176  4197 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.558781 31388 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:17.560622 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.559244 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.562201 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.562922 31396 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:17.555521 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.524672  4352 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:17.489030  4213 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:17.525156  4372 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:17.531337  4272 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.531039  4491 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.569736 31367 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:17.574008 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.537328  4493 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:17.538529  4484 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:17.583180 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.580482 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.553004  3929 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.554900  4103 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:17.488499  4475 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:17.572925  4454 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.595150 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.560132  4316 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:17.564515  4439 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:17.468875  4463 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.571826  4445 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:17.610087 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 67257.6us
I20260709 13:16:17.727954 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:17.730681  4489 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:17.731148  4489 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:17.731379  4459 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.540735  4348 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.541097  4319 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.618227  4481 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:17.624169 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 160761us
I20260709 13:16:17.587873  4402 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:17.626319 31396 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:17.755100  4402 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:17.755712  4402 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:17.755837 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.757308 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.587371  4538 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:17.759105  4538 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:17.759389 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:17.760111  4445 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:17.760226  4538 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.560708  4436 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.629379 31394 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:17.762362 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:17.608794 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 69286.5us
I20260709 13:16:17.631493 31292 leader_election.cc:400] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:17.764134 31292 leader_election.cc:403] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:16:17.764829 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 141169us
I20260709 13:16:17.641245 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.645130 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2495) took 1838 ms. Trace:
I20260709 13:16:17.581090  4352 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.646009 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.769670 31383 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:17.761816 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:17.651623 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1399) took 2249 ms. Trace:
I20260709 13:16:17.662710 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 1994) took 1555 ms. Trace:
I20260709 13:16:17.672052  4213 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.770560 31391 rpcz_store.cc:276] 0709 13:16:16.107432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.107625 (+   193us) service_pool.cc:224] Handling call
0709 13:16:16.114463 (+  6838us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.134888 (+ 20425us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:16.207808 (+ 72920us) log.cc:844] Serialized 14 byte log entry
0709 13:16:16.219672 (+ 11864us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.383305 (+1163633us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b180034d380. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:16:17.475116 (+ 91811us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.477542 (+  2426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.477617 (+    75us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:17.477676 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.662299 (+184623us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b5400460ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.662368 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.662597 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1126864,"prepare.queue_time_us":399515,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":6176,"raft.queue_time_us":1177160,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":81567,"spinlock_wait_cycles":91130880,"thread_start_us":167680,"threads_started":3,"wal-append.queue_time_us":7686}
I20260709 13:16:17.779793 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.673810  4349 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:17.674194  4347 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.677114  4197 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.681835 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:17.792613 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.683578 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:17.793881 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.628950  4488 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:17.635847  3929 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.686192 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:17.619378  4272 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.593341  4403 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:16:17.689978  4490 raft_consensus.cc:3055] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:17.598294  4302 consensus_queue.cc:1048] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:16:17.691123  4316 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:17.693886 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.856421 30624 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:17.856577 31396 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
W20260709 13:16:17.798063  4528 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.339s	user 0.004s	sys 0.000s
I20260709 13:16:17.695092 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.612303  4372 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.592387  4189 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:16:17.697427  4435 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.628042  4493 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.704128 30630 raft_consensus.cc:2410] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:16:17.705380  4462 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:17.708953  4454 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.709851 30377 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 10, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 10 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:17.711025  4475 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.886247  4436 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.724189  4494 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.643090  4103 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:17.888386  3929 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.888885  4103 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.731532  4489 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:17.631907  4532 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:17.734428  4348 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.735332 30626 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:16:17.892485  4532 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:17.735229 31389 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:17.616405 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.893007  4532 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:17.734589  4319 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.727954  4463 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.623677 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.895287 31367 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:17.896371 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.897620 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.898213 31367 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:17.742602  4398 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:16:17.743012  4346 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:17.746896 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1924) took 1779 ms. Trace:
W20260709 13:16:17.747577 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.899290 30628 rpcz_store.cc:276] 0709 13:16:15.967580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.207901 (+1240321us) service_pool.cc:224] Handling call
0709 13:16:17.208292 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.220888 (+ 12596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.220925 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.234411 (+ 13486us) log.cc:844] Serialized 26 byte log entry
0709 13:16:17.549964 (+315553us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:17.550016 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.550113 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:17.746538 (+196425us) raft_consensus.cc:1685] finished
0709 13:16:17.746575 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.746820 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12104,"raft.queue_time_us":23158,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":14051,"thread_start_us":294005,"threads_started":1,"wal-append.queue_time_us":408245}
I20260709 13:16:17.900728  4346 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:17.901440 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.766534 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:17.766161 30559 rpcz_store.cc:276] 0709 13:16:15.806747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.136715 (+329968us) service_pool.cc:224] Handling call
0709 13:16:16.137043 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:16.243485 (+106442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.644822 (+1401337us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54001225d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.645061 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":242296192}
I20260709 13:16:17.768024  4104 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18.
I20260709 13:16:17.619366  4491 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:17.907548 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.770464 30633 rpcz_store.cc:276] 0709 13:16:15.402533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.013962 (+1611429us) service_pool.cc:224] Handling call
0709 13:16:17.014358 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.556099 (+541741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.651176 (+ 95077us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400865bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:17.651512 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244314368}
I20260709 13:16:17.771571 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.774844 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.911362 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:17.911813 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 137056us
I20260709 13:16:17.913058  4398 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:17.913480 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1179) took 1751 ms. Trace:
I20260709 13:16:17.855441  4347 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:17.775174 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:17.914157  4398 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:16:17.915050 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 139939us
I20260709 13:16:17.913949 30633 rpcz_store.cc:276] 0709 13:16:16.162331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.912528 (+1750197us) service_pool.cc:224] Handling call
0709 13:16:17.912897 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.913116 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.913146 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:17.913198 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:17.913234 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.913297 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:17.913420 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.917330  4493 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.915174  4398 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:17.794142 30625 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:17.918375 31364 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:17.919723  4398 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:17.632262  4466 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 13:16:17.920670  4398 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.921501  4466 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:17.921902  4466 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:17.921974  4398 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:17.816609 30627 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:17.926930  4372 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.927227 30627 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:16:17.927945 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1933) took 1390 ms. Trace:
I20260709 13:16:17.928239 30627 rpcz_store.cc:276] 0709 13:16:16.537163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.926480 (+1389317us) service_pool.cc:224] Handling call
0709 13:16:17.926873 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.927656 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.927884 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:17.929033 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:17.694429 30698 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:17.791900  4352 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:17.866958 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.779309  4250 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:17.963960 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.964927 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.965915 31396 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:17.782884  4525 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:17.873370 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:17.879200 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 318652us
I20260709 13:16:17.846545  3990 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.834939  4505 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:16:17.901737 30699 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:17.917457 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:17.790925  4393 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:16:17.835667 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 149603us
I20260709 13:16:17.922930  4466 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:17.857784 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:17.751701 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.762961  4441 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:17.935803 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:18.025403 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 337765us
I20260709 13:16:17.939040 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.939795 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.954823 31383 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 22 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:16:17.849388  4490 raft_consensus.cc:740] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:17.873795  4403 consensus_queue.cc:1243] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:16:17.834573  4486 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.121s
W20260709 13:16:17.970016 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 198511us
I20260709 13:16:17.969782 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:17.988361 31391 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:18.055591 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1928) took 2026 ms. Trace:
I20260709 13:16:17.995675 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.996217 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:18.057170 31389 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:18.057829 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:17.997926 30632 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:17.997799 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1934) took 1442 ms. Trace:
W20260709 13:16:18.018278 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.059352 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:18.059051 30630 rpcz_store.cc:276] 0709 13:16:16.555776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.785550 (+229774us) service_pool.cc:224] Handling call
0709 13:16:17.997476 (+1211926us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54007229d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:17.997718 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059630976}
I20260709 13:16:18.061456 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:18.026194 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90998.6us
I20260709 13:16:18.063532 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:18.029403 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:18.064828 30624 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:17.991267  4346 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:18.072969 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1922) took 3310 ms. Trace:
I20260709 13:16:18.007296  4437 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:17.966001  4189 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:17.967105  4372 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:18.043216 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:17.963052  4250 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:18.062328 30630 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
W20260709 13:16:18.045042  4366 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.059s	user 0.014s	sys 0.002s
W20260709 13:16:18.071648 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 320077us
I20260709 13:16:18.071681 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:18.073580 30626 rpcz_store.cc:276] 0709 13:16:14.762466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.924252 (+2161786us) service_pool.cc:224] Handling call
0709 13:16:16.928304 (+  4052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.892852 (+964548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.054391 (+161539us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54004a83d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:18.072873 (+ 18482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855234944}
I20260709 13:16:17.947985 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:18.056459 30631 rpcz_store.cc:276] 0709 13:16:16.028843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.419760 (+1390917us) service_pool.cc:224] Handling call
0709 13:16:17.450709 (+ 30949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.450951 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:17.579579 (+128628us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.055096 (+475517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.055187 (+    91us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:18.055252 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.055339 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.055502 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21793,"prepare.queue_time_us":342366,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":73,"raft.queue_time_us":27778,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":173,"thread_start_us":581191,"threads_started":2}
W20260709 13:16:18.103122 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 155305us
I20260709 13:16:18.104700 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:18.037443  4435 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:18.087133 31364 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:18.090871 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:18.069806 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:17.988446  4525 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:17.950827  4523 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:18.160264 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59846.8us
I20260709 13:16:18.161904 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:18.165529 31389 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:18.071390  4348 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:16:18.172144 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 102476us
I20260709 13:16:18.006928  3990 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.025341  4454 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:16:18.170737 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66276.3us
W20260709 13:16:18.103278  4522 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.646s	user 0.011s	sys 0.002s
I20260709 13:16:18.176455 31383 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:16:18.045395  4441 consensus_queue.cc:1048] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 13:16:18.155787 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:18.183701 30628 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:18.176481 31389 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:16:18.102011  3942 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:18.159039  4435 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.190140 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:18.159837  4525 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:18.193542 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1405) took 2235 ms. Trace:
I20260709 13:16:18.197420 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:18.197080 30632 rpcz_store.cc:276] 0709 13:16:15.958116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.068354 (+1110238us) service_pool.cc:224] Handling call
0709 13:16:17.997679 (+929325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.072426 (+ 74747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.193231 (+120805us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54007229d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:18.193449 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101692672}
I20260709 13:16:18.173823  4436 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:18.172837  4348 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:18.203770 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:18.201381 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:18.177901  4346 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:18.213312 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 141761us
I20260709 13:16:18.104879  4541 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:16:18.224891 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:18.239938 31294 connection.cc:256] RPC call timeout handler was delayed by -1.1900493890000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 13:16:18.209699  4438 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.001s
I20260709 13:16:18.092620  4493 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:18.247157 31364 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:16:18.240883 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:18.197921  4435 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:18.091696  4372 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:18.201956  4470 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:18.098847  4250 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:18.204627  4309 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:18.210093  4532 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:18.029011  4434 consensus_queue.cc:1175] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5a03a8214bba4eb1b0ff00895c565564" responder_term: 19 status { last_received { term: 17 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:18.263520  4213 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:18.210296  4436 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:18.260012  4438 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.284304  4523 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:18.287312  4545 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:16:18.288189  4491 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.289043  4532 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:18.289944 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2006) took 2065 ms. Trace:
I20260709 13:16:18.187886  4522 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.291455  4491 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.290365 31361 rpcz_store.cc:276] 0709 13:16:16.224216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.627650 (+403434us) service_pool.cc:224] Handling call
0709 13:16:16.628040 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.393877 (+765837us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:17.525817 (+131940us) log.cc:844] Serialized 14 byte log entry
0709 13:16:17.771846 (+246029us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.791028 (+ 19182us) log.cc:844] Serialized 26 byte log entry
0709 13:16:17.791256 (+   228us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:17.791297 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.791379 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.289677 (+498298us) raft_consensus.cc:1685] finished
0709 13:16:18.289703 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.289873 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":790476,"prepare.queue_time_us":208959,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":772900,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":11744,"thread_start_us":333521,"threads_started":2,"wal-append.queue_time_us":448263}
I20260709 13:16:18.256160 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:18.292479  4522 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:18.292634 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:18.233969  4366 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:18.272154  4470 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:18.293426  4470 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:18.293964 31361 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6"
dest_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:50024
W20260709 13:16:18.294803  4402 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.005s
I20260709 13:16:18.295495  4402 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.297676  4402 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.251684  4254 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:18.292861 31364 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:18.298868  4505 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:18.299737  4348 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.300640  4436 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:18.301146  4348 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:18.302583  4470 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.304702 31391 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:18.307304 31361 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:16:18.306804  4470 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.260507  4544 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:18.311198  4544 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:18.312271  4544 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.313514 31361 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:18.260758  4493 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:18.314965  4493 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:18.316118 31364 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:18.317672  4436 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.248587  3929 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:16:18.330567 31364 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:18.331221 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.323776 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:18.327478 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:18.101372  4437 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.259076  4372 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:18.333719 31364 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 13:16:18.277277  4272 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:18.201833  4490 consensus_queue.cc:260] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.352072 31394 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 13:16:18.354122 31394 pending_rounds.cc:85] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 13
I20260709 13:16:18.354450 31394 pending_rounds.cc:107] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 13:16:18.354779 31394 raft_consensus.cc:2889] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:18.329137  4352 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:16:18.358256  4349 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.000s
W20260709 13:16:18.385370  4445 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.002s
I20260709 13:16:18.386111  4445 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:18.389040  4436 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:18.390261  4548 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:16:18.396623  4466 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:18.399888  4535 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.527s	user 0.000s	sys 0.002s
I20260709 13:16:18.403381  4438 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:18.408422  4489 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 13:16:18.409075  4489 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:18.412513  4550 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.149s	user 0.002s	sys 0.000s
I20260709 13:16:18.415083  4488 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:18.415647  4488 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:18.415745  4435 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.416613  4488 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.417899  4488 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:18.421655  4525 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.425374  4493 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.427600  4489 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:18.429427  4493 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:18.431423  4525 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:17.989804 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:18.491492 30625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.003s	sys 0.027s
W20260709 13:16:18.492398 30625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.003s	sys 0.027s
I20260709 13:16:18.087415 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:18.496325 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1412) took 2208 ms. Trace:
I20260709 13:16:18.342598 31364 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:18.503952 31364 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:18.504570 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:18.506898 31364 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:18.507725 30629 raft_consensus.cc:2368] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:18.168473 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:18.509262 31364 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
W20260709 13:16:18.511379 30633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.001s	sys 0.020s
W20260709 13:16:18.496960 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 409461us
I20260709 13:16:18.511765 31367 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 24 because replica has last-logged OpId of term: 22 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 11.
W20260709 13:16:18.512012 30633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.001s	sys 0.021s
I20260709 13:16:18.512476 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2246) took 1150 ms. Trace:
I20260709 13:16:18.513490 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1878) took 1022 ms. Trace:
W20260709 13:16:18.514644 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 524789us
W20260709 13:16:18.516710 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 275763us
I20260709 13:16:18.514359 31378 rpcz_store.cc:276] 0709 13:16:17.490554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.491370 (+   816us) service_pool.cc:224] Handling call
0709 13:16:17.491722 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.492125 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.492152 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.531537 (+ 39385us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.042493 (+510956us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:18.095524 (+ 53031us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.096304 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.096435 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.411857 (+315422us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:18.512631 (+100774us) raft_consensus.cc:1685] finished
0709 13:16:18.512804 (+   173us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.513210 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8269,"prepare.queue_time_us":46050,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":3438,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":73579,"spinlock_wait_cycles":21439360,"thread_start_us":454215,"threads_started":1,"wal-append.queue_time_us":695066}
W20260709 13:16:18.506907 30627 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.004s	sys 0.004s
I20260709 13:16:18.522078  4103 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:18.522480 30627 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.004s	sys 0.004s
I20260709 13:16:18.498821 30706 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:18.506304 30632 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:18.513829 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1411) took 2256 ms. Trace:
I20260709 13:16:18.584192 30630 raft_consensus.cc:1275] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:16:18.568619  4138 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.562161 30633 rpcz_store.cc:276] 0709 13:16:16.257021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.916611 (+1659590us) service_pool.cc:224] Handling call
0709 13:16:17.916957 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.917218 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:16:17.917564 (+   346us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.513020 (+595456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.513154 (+   134us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:18.513324 (+   170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.513485 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.513749 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":80120,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"thread_start_us":594655,"threads_started":1}
I20260709 13:16:18.496982 30625 rpcz_store.cc:276] 0709 13:16:16.287396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.918782 (+1631386us) service_pool.cc:224] Handling call
0709 13:16:17.919181 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:17.919619 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.922856 (+  3237us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:17.923406 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.493498 (+570092us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:18.494571 (+  1073us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.495190 (+   619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.495404 (+   214us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.495568 (+   164us) raft_consensus.cc:1685] finished
0709 13:16:18.495608 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.496078 (+   470us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3121,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"thread_start_us":569081,"threads_started":1,"wal-append.queue_time_us":460423}
I20260709 13:16:18.364878  4352 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:18.591351 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1937) took 1973 ms. Trace:
I20260709 13:16:18.523239  4103 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.511374 31361 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.596750 30630 rpcz_store.cc:276] 0709 13:16:16.615629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.510030 (+1894401us) service_pool.cc:224] Handling call
0709 13:16:18.531526 (+ 21496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.584998 (+ 53472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.588963 (+  3965us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.513099 30700 rpcz_store.cc:276] 0709 13:16:17.361923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.368707 (+  6784us) service_pool.cc:224] Handling call
0709 13:16:17.369176 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:17.371075 (+  1899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:17.371116 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:17.371879 (+   763us) log.cc:844] Serialized 52 byte log entry
0709 13:16:17.597928 (+226049us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:17.598001 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:17.598106 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.399170 (+801064us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:18.511584 (+112414us) raft_consensus.cc:1685] finished
0709 13:16:18.511629 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.512114 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":7714048,"thread_start_us":225496,"threads_started":1,"wal-append.queue_time_us":389626}
I20260709 13:16:18.643443  4138 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.671021 31361 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:18.674717 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.675931 30559 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:18.677940  4319 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:18.680721  4445 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.687508 30626 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:18.688722 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1936) took 2103 ms. Trace:
I20260709 13:16:18.689229 30626 rpcz_store.cc:276] 0709 13:16:16.585455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.497968 (+1912513us) service_pool.cc:224] Handling call
0709 13:16:18.594904 (+ 96936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.688263 (+ 93359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.688602 (+   339us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:18.692335 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81512.5us
I20260709 13:16:18.692628  4352 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.693990 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:18.695096 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:18.695561  4138 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.695775  4454 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:16:18.697216 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 95393.8us
I20260709 13:16:18.698241 30551 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:18.714257 30633 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:18.715224  4103 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.715463 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1941) took 1559 ms. Trace:
I20260709 13:16:18.716112 30633 rpcz_store.cc:276] 0709 13:16:17.156342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.696559 (+1540217us) service_pool.cc:224] Handling call
0709 13:16:18.713623 (+ 17064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.714946 (+  1323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.715350 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:16:18.719530  4556 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:16:18.723109 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 396887us
I20260709 13:16:18.723325 30626 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:18.724504 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1944) took 1520 ms. Trace:
I20260709 13:16:18.724999 30626 rpcz_store.cc:276] 0709 13:16:17.203963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.690425 (+1486462us) service_pool.cc:224] Handling call
0709 13:16:18.722961 (+ 32536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.724056 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.724396 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.725660 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:18.727088 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60823.8us
I20260709 13:16:18.729692 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:18.733282  4138 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:18.735734 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1183) took 1294 ms. Trace:
I20260709 13:16:18.736136  4547 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.736385 30628 rpcz_store.cc:276] 0709 13:16:17.441373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.901145 (+459772us) service_pool.cc:224] Handling call
0709 13:16:18.735137 (+833992us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54004a97d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:18.735453 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646863232}
I20260709 13:16:18.737028  4547 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.739228 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:18.745565  4138 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:18.748183 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1822) took 1180 ms. Trace:
I20260709 13:16:18.748057  4138 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.749130  4309 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:18.711440  4454 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:18.750262  4454 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:18.751845  4309 consensus_queue.cc:1048] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:18.682024  4445 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:18.694466  4352 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:18.738221 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:18.700083 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1184) took 1258 ms. Trace:
I20260709 13:16:18.704291  4457 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:18.759865 30553 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:18.765277 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.759215 30625 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:18.771409 30626 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:18.777737 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1947) took 1013 ms. Trace:
I20260709 13:16:18.791507 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1940) took 1635 ms. Trace:
I20260709 13:16:18.792132 30625 rpcz_store.cc:276] 0709 13:16:17.155950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.619392 (+1463442us) service_pool.cc:224] Handling call
0709 13:16:18.758801 (+139409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.791122 (+ 32321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.791427 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.780226 30629 rpcz_store.cc:276] 0709 13:16:17.441637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.591839 (+150202us) service_pool.cc:224] Handling call
0709 13:16:18.699642 (+1107803us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54005b13d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:18.699932 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1411068800}
I20260709 13:16:18.791152 30628 rpcz_store.cc:276] 0709 13:16:17.764225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.776075 (+1011850us) service_pool.cc:224] Handling call
0709 13:16:18.776622 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.777100 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.777151 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.777262 (+   111us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:18.777338 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.777436 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.777609 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.748900 31391 rpcz_store.cc:276] 0709 13:16:17.567502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.187000 (+619498us) service_pool.cc:224] Handling call
0709 13:16:18.211244 (+ 24244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.305108 (+ 93864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.747272 (+442164us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:18.747885 (+   613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214402816}
I20260709 13:16:18.795668 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1191) took 1342 ms. Trace:
I20260709 13:16:18.796294 30626 rpcz_store.cc:276] 0709 13:16:17.453001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.726136 (+1273135us) service_pool.cc:224] Handling call
0709 13:16:18.771008 (+ 44872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.795358 (+ 24350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.795604 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.775281 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:18.787870  4254 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:18.784579  4457 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:18.761982  4445 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.768947  3990 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:18.817520  4568 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:18.817991  4347 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:18.818591  4525 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:18.817659 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:18.820598 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:18.811769 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 523840us
I20260709 13:16:18.787452  4309 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:16:18.758496  4437 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.810247  4254 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:18.825889 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:18.812461  4457 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:18.849447  4438 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:18.850667  4438 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.850858 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:18.852429 30554 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:16:18.832681 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1887) took 1322 ms. Trace:
I20260709 13:16:18.856204  4346 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:18.856102 31367 rpcz_store.cc:276] 0709 13:16:17.505902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.907065 (+401163us) service_pool.cc:224] Handling call
0709 13:16:18.813298 (+906233us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b5400683b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:18.828183 (+ 14885us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384330112}
I20260709 13:16:18.830935 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:18.823397  4568 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:18.853809 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:18.879186 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104227us
W20260709 13:16:18.875762 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 182124us
I20260709 13:16:18.864723  3929 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:18.878259  4486 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:18.892593 30628 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: [21.10-21.10]
I20260709 13:16:18.879611 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1819) took 1335 ms. Trace:
I20260709 13:16:18.869491  4339 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:18.909410  4339 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:18.910813  4481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.006s	sys 0.011s
I20260709 13:16:18.876008  4103 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:18.912746  4103 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.913030  4434 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:18.914078  4434 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.915233  4309 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:18.915877  4309 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:18.914414  4490 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:18.891389  4486 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:18.919636  4563 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:18.921314  4563 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:18.921844  4563 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:18.926913 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.927277 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:18.935719  4568 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.937860  4568 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:18.868968  4575 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:18.943609  4575 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:18.912791  3929 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.902627 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1935) took 2335 ms. Trace:
I20260709 13:16:18.946775  3990 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:18.947021 30627 rpcz_store.cc:276] 0709 13:16:16.566794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:17.929004 (+1362210us) service_pool.cc:224] Handling call
0709 13:16:17.929324 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:17.929606 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:18.523310 (+593704us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.809518 (+286208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.813996 (+  4478us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:18.827637 (+ 13641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.834979 (+  7342us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.902491 (+ 67512us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30515,"prepare.queue_time_us":161261,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":10387,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":11787,"thread_start_us":790024,"threads_started":2}
I20260709 13:16:18.951192 30631 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:18.952387 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1943) took 1756 ms. Trace:
I20260709 13:16:18.952857 30631 rpcz_store.cc:276] 0709 13:16:17.196113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.685964 (+1489851us) service_pool.cc:224] Handling call
0709 13:16:18.793301 (+107337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.951964 (+158663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.952287 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.954929 30551 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:18.956027 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1942) took 1792 ms. Trace:
I20260709 13:16:18.956723 30630 rpcz_store.cc:276] 0709 13:16:17.163233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.819639 (+1656406us) service_pool.cc:224] Handling call
0709 13:16:18.955015 (+135376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:18.955455 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.955496 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:18.955590 (+    94us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:18.955665 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.955756 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.955934 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:18.959425 30596 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:18.968214 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:18.969506  4434 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:18.866158  4547 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:18.853883  4438 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:18.776764 30377 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 6 to 15, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:18.983901  4563 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.917553  4309 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.858687  4254 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.987746  4309 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:18.934607  4486 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:18.992576  4486 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:18.993436  4434 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:18.964429  4189 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:18.994572  4434 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:18.996338  4189 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:18.999159 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:18.874868  4525 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:18.880712 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:19.003823 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.005266 30557 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:19.001681 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:18.871812  4435 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:18.859158  4544 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.107s
I20260709 13:16:18.875072  4491 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:18.986393  4563 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:19.006104  4339 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.031095 30554 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:19.028540 30553 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:19.038276 30556 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:18.999135  4434 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:19.043908 30700 raft_consensus.cc:1240] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:16:19.058944 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:18.905745 31396 rpcz_store.cc:276] 0709 13:16:17.543971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.175218 (+631247us) service_pool.cc:224] Handling call
0709 13:16:18.175678 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.184959 (+  9281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.184995 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.519982 (+334987us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.706360 (+186378us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:18.706446 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.788704 (+ 82258us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.879141 (+ 90437us) raft_consensus.cc:1685] finished
0709 13:16:18.879168 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:18.879411 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42750,"raft.queue_time_us":12542,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":14563,"thread_start_us":139268,"threads_started":1,"wal-append.queue_time_us":302163}
I20260709 13:16:19.003836  4481 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:16:18.860407  4215 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 13:16:18.859763  4522 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.179s
W20260709 13:16:19.058681 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.017s	sys 0.064s
I20260709 13:16:18.986358 30377 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 8 to 11, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:19.070071 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.018s	sys 0.064s
I20260709 13:16:19.075659 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:19.077441 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 348023us
I20260709 13:16:19.047719  4103 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.044384  3929 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:19.039392  4525 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.057765  4347 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:19.095291 30624 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.971s	user 0.014s	sys 0.010s
I20260709 13:16:19.095903 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.105240 30556 raft_consensus.cc:2368] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:19.092598  4438 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:16:19.115113 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 234530us
W20260709 13:16:19.130476 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 102309us
W20260709 13:16:19.121431 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 151544us
I20260709 13:16:19.068029  4491 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.126470 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1825) took 1437 ms. Trace:
I20260709 13:16:19.146026 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.157346 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:19.127207 30624 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:19.174989 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.174952 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:19.138310  4575 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:19.181315 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.190330 30627 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:19.144364 31389 rpcz_store.cc:276] 0709 13:16:17.573188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.202475 (+629287us) service_pool.cc:224] Handling call
0709 13:16:18.202891 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.676585 (+473694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.676637 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.677687 (+  1050us) log.cc:844] Serialized 26 byte log entry
0709 13:16:18.791453 (+113766us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:18.791541 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.803229 (+ 11688us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:18.960722 (+157493us) raft_consensus.cc:1685] finished
0709 13:16:18.960755 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.005678 (+ 44923us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39346,"raft.queue_time_us":86231,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":530202,"threads_started":2,"wal-append.queue_time_us":244944}
I20260709 13:16:19.189260 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:19.153576  4438 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:19.191282 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:19.209776 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:19.213646 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.125651  4347 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:19.208315 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 249171us
I20260709 13:16:19.219642 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1948) took 1245 ms. Trace:
I20260709 13:16:19.220288 30624 rpcz_store.cc:276] 0709 13:16:17.974029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.029019 (+ 54990us) service_pool.cc:224] Handling call
0709 13:16:19.200871 (+1171852us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:19.211040 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:19.232976 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.222373 30630 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:19.152400  4103 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:16:19.234216 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:19.229202 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 260366us
I20260709 13:16:19.210577  4489 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:19.244639 30554 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:19.247432 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.249526 30631 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:19.253346  4575 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.248497 30556 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:16:19.255064 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 1.06504e+06us
I20260709 13:16:19.264638  4455 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:19.265204 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:19.254578 30707 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.096s	user 0.006s	sys 0.015s
I20260709 13:16:19.271919 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:19.257380  4575 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:19.275679 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:19.276961 30629 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:19.275940  4575 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:19.281548  4489 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:19.282874 30702 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:19.282687  4489 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:19.284821 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:19.285778 31389 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:19.288545  4372 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.941s	user 0.004s	sys 0.015s
I20260709 13:16:19.288686  4434 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:19.289402  4372 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.289810 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:19.290931  4434 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:19.244249  4254 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:19.293047 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.293419 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:19.294790  4561 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:19.295804 30627 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:16:19.299149  4577 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.205s	user 0.000s	sys 0.002s
I20260709 13:16:19.301767 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1821) took 1747 ms. Trace:
I20260709 13:16:19.303191  4586 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:19.303756  4586 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:19.302253 31383 rpcz_store.cc:276] 0709 13:16:17.554635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.183773 (+629138us) service_pool.cc:224] Handling call
0709 13:16:18.184324 (+   551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:18.684243 (+499919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.684299 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:18.685443 (+  1144us) log.cc:844] Serialized 52 byte log entry
0709 13:16:18.801149 (+115706us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:18.874139 (+ 72990us) log.cc:844] Serialized 14 byte log entry
0709 13:16:18.874465 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:18.874646 (+   181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.301282 (+426636us) raft_consensus.cc:1685] finished
0709 13:16:19.301306 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.301525 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7451,"prepare.queue_time_us":61182,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":79131,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":306,"thread_start_us":576491,"threads_started":2,"wal-append.queue_time_us":349616}
I20260709 13:16:19.305058  4586 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:19.285668  4403 raft_consensus.cc:2764] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:16:19.307404  4544 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:19.308072  4544 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:19.309118  4544 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:19.311020  4319 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:19.311714  4319 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:19.311204  4438 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:19.313019  4319 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:19.315584  4525 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:19.308092 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 72) took 29199 ms (client timeout 15000 ms). Trace:
I20260709 13:16:19.318065 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1938) took 2701 ms. Trace:
W20260709 13:16:19.319118  4580 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.198s	user 0.002s	sys 0.002s
I20260709 13:16:19.318639 30632 rpcz_store.cc:276] 0709 13:16:16.616466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.603468 (+1987002us) service_pool.cc:224] Handling call
0709 13:16:18.682815 (+ 79347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.683597 (+   782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.683629 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.806961 (+123332us) log.cc:844] Serialized 26 byte log entry
0709 13:16:19.099403 (+292442us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:19.099475 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.099569 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.317788 (+218219us) raft_consensus.cc:1685] finished
0709 13:16:19.317814 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.317984 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14249,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":144,"thread_start_us":213702,"threads_started":1,"wal-append.queue_time_us":473556}
W20260709 13:16:19.307605  4250 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.012s
I20260709 13:16:19.322235  4250 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.322707  4457 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:19.324240 30559 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:19.325366  4250 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.329276  4457 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:19.330273  4319 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:19.330750 30707 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:19.331284 30557 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:19.331521  4119 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.332130  4119 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:19.333101  4544 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:19.283139  4575 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.253830 30377 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 7 to 9, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 9 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:16:19.341188 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 145012us
W20260709 13:16:19.356884 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.369648 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:19.367173 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.372568 30557 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
I20260709 13:16:19.381238 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:19.362357  4403 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.384270 31389 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:19.363286  4586 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:19.389346 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:19.397881 31383 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:16:19.407603 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1764) took 4797 ms (client timeout 1595 ms). Trace:
I20260709 13:16:19.405683 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:19.408174 30707 rpcz_store.cc:269] 0709 13:16:14.609988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:14.613275 (+  3287us) service_pool.cc:224] Handling call
0709 13:16:19.407221 (+4793946us) spinlock_profiling.cc:243] Waited 3.52 s on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:19.407488 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8094284672}
I20260709 13:16:19.411499 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:19.397197  4522 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:19.380836  4575 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:19.434335 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44354us
I20260709 13:16:19.417698 30557 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:19.420239 30705 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:19.415768 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:19.424437  4491 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.425804 30550 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:19.428023 31367 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:19.438987  3929 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:16:19.456161 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 70499.7us
I20260709 13:16:19.450811 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:19.426353  4119 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:19.416128  4522 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:19.399909  4403 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.478245  4403 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:19.471645  4156 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:16:19.472401  4435 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.468488  4522 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:19.447305  3929 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:19.316445 30378 rpcz_store.cc:269] 0709 13:15:49.995557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:49.995905 (+   348us) service_pool.cc:224] Handling call
0709 13:16:19.182303 (+29186398us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:08.503404 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:08.504292 (+   888us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:08.504375 (+    83us) write_op.cc:435] Acquired schema lock
0709 13:16:08.504413 (+    38us) tablet.cc:662] Decoding operations
0709 13:16:08.517365 (+ 12952us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:08.517479 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:08.517520 (+    41us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:16:08.538975 (+ 21455us) tablet.cc:701] Row locks acquired
0709 13:16:08.539040 (+    65us) write_op.cc:260] PREPARE: finished
0709 13:16:08.539303 (+   263us) write_op.cc:270] Start()
0709 13:16:08.539516 (+   213us) write_op.cc:276] Timestamp: P: 1783602968539203 usec, L: 0
0709 13:16:08.539566 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 13:16:08.650596 (+111030us) log.cc:844] Serialized 10544 byte log entry
0709 13:16:10.121003 (+1470407us) op_driver.cc:464] REPLICATION: finished
0709 13:16:10.416141 (+295138us) write_op.cc:301] APPLY: starting
0709 13:16:10.430125 (+ 13984us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:10.797010 (+366885us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:10.797202 (+   192us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:11.317750 (+520548us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:11.324194 (+  6444us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:16:11.324259 (+    65us) write_op.cc:312] APPLY: finished
0709 13:16:11.324900 (+   641us) log.cc:844] Serialized 235 byte log entry
0709 13:16:11.641669 (+316769us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:11.663220 (+ 21551us) write_op.cc:454] Released schema lock
0709 13:16:11.663344 (+   124us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:11.663871 (+   527us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:11.663905 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3437203,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":169229,"apply.run_cpu_time_us":139957,"apply.run_wall_time_us":1332758,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117257,"num_ops":27,"prepare.queue_time_us":433616,"prepare.run_cpu_time_us":63177,"prepare.run_wall_time_us":557054,"raft.queue_time_us":743895,"raft.run_cpu_time_us":11035,"raft.run_wall_time_us":46776,"replication_time_us":1581228,"thread_start_us":1240261,"threads_started":5,"wal-append.queue_time_us":840208,"wal-append.run_cpu_time_us":24821,"wal-append.run_wall_time_us":2842338}]]}
W20260709 13:16:19.479046 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 231113us
I20260709 13:16:19.479585  4455 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:19.470077 30626 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:19.507223 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.496475 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.492442 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1945) took 2168 ms. Trace:
I20260709 13:16:19.490801 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:19.498497 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 94971.3us
I20260709 13:16:19.516413 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1781) took 3715 ms. Trace:
I20260709 13:16:19.508687  3929 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:19.535032  4348 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.087s	user 0.020s	sys 0.007s
W20260709 13:16:19.536288 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.536437 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:19.537315  4575 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.513286 30633 rpcz_store.cc:276] 0709 13:16:17.324348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:18.717286 (+1392938us) service_pool.cc:224] Handling call
0709 13:16:18.717911 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:18.777853 (+ 59942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:18.777898 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:18.778891 (+   993us) log.cc:844] Serialized 26 byte log entry
0709 13:16:19.226699 (+447808us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:19.226769 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.226882 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:19.492095 (+265213us) raft_consensus.cc:1685] finished
0709 13:16:19.492130 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:19.492358 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24201,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":234,"thread_start_us":447158,"threads_started":1,"wal-append.queue_time_us":588699}
I20260709 13:16:19.487949 31383 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:19.542491 30705 rpcz_store.cc:276] 0709 13:16:15.788111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:16.763425 (+975314us) service_pool.cc:224] Handling call
0709 13:16:19.419541 (+2656116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:19.435535 (+ 15994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.501455 (+ 65920us) spinlock_profiling.cc:243] Waited 2.64 s on lock 0x7b54003d22d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:19.503540 (+  2085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6083021056}
I20260709 13:16:19.544552  4592 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:19.545500 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.545706 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.546281 30550 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:19.547825 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.548604 31388 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:19.548926 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.550215 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:19.553375 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.555765  4346 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:19.555830 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.557169 31383 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:19.558234 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.558485  4215 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:19.562343 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:19.563161 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.565395 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.452744  4189 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:19.531476  4454 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.780s	user 0.014s	sys 0.003s
I20260709 13:16:19.502239  4486 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:19.586141 30553 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:19.586719 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.590548 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:19.590906 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.532897  4347 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:19.596688 30556 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:19.599538 30552 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:19.599721  4489 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:19.585538  4454 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:19.586905 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:19.603194  4156 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:19.577265  4491 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:16:19.523890  4372 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:19.582559  4455 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:19.516831  4435 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:19.607721  4309 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.619s	user 0.013s	sys 0.006s
I20260709 13:16:19.627590 30706 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:19.558290  4346 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:16:19.621521 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:19.610174  4250 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:16:19.643286 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:19.648528 30707 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:19.611583  4347 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:19.632510  4486 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:19.652751 31361 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:16:19.653235 31361 pending_rounds.cc:85] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 12
I20260709 13:16:19.653335  4435 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:19.653532 31361 pending_rounds.cc:107] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:16:19.653867 31361 raft_consensus.cc:2889] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:19.589013  4189 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:19.655830 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47084.3us
I20260709 13:16:19.615979  4438 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:19.662361 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 109244us
I20260709 13:16:19.656737 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:19.661798 30559 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:19.663625 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:19.636543  4348 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.670907 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:19.668104 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:19.673529 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 81947.2us
I20260709 13:16:19.677336 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:19.613663  4156 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:19.660537  4486 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:19.694255 30707 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:19.696528 30705 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:19.674871  4568 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.714s	user 0.014s	sys 0.002s
I20260709 13:16:19.702520 30557 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 13:16:19.663559  4575 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:16:19.684835  4352 consensus_meta.cc:287] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.921s	user 0.014s	sys 0.004s
I20260709 13:16:19.745537 30627 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:19.748700 31378 raft_consensus.cc:2410] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:16:19.676537  4489 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:19.754328 31396 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:19.685717  4372 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:19.757720 30552 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 9. (index mismatch)
I20260709 13:16:19.762588 30550 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:19.663216  4347 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:19.742465 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.751744  4575 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:19.764423  4484 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:19.756249  4591 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260709 13:16:19.769456  4595 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:19.803109 30557 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 13:16:19.763592 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 59959.6us
I20260709 13:16:19.768370  4457 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:16:19.773409 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:19.774700 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:19.780052 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.683786  4438 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:19.787102  4593 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:16:19.762555  4372 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:19.824371 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:19.756104  4489 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:19.836720  4489 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:19.773594  4563 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.710s	user 0.013s	sys 0.004s
W20260709 13:16:19.810237 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.842157  4489 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:19.809657 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.832754 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:19.742388  4348 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.834695 31292 leader_election.cc:400] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:19.850843 31292 leader_election.cc:403] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:19.851713 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 92421.1us
I20260709 13:16:19.854840  4568 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:19.855880  4568 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:19.857489  4568 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:19.857903  4568 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:19.862898  4348 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:19.864712  4568 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:19.827934 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:19.875298 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.870041  4568 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:19.798327  4484 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:19.800716  4595 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:19.886572 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 229897us
I20260709 13:16:19.808636  4457 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:19.896548 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:19.897244 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 138947us
W20260709 13:16:19.902349 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 144474us
I20260709 13:16:19.874004  4372 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:19.910797 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:19.873736  4435 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.916220 30557 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:19.893747  4484 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:19.938862 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:19.981140 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.987308 30700 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:19.990260 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:19.937060  4454 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:20.002707 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:20.010207  4219 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:19.813743  4455 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.018081 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:20.022972 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:20.033207 30556 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:19.989667  4481 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:20.040599  4103 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:20.032445  4138 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:16:20.041558  4219 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.044240  4403 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.045433  4219 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.045159  4403 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.019140  4532 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:20.018481  4347 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.048365  4403 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.051663 31391 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:20.053432 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:20.054684 31396 raft_consensus.cc:2410] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 13:16:20.055970  4602 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:16:20.058123  4590 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.253s	user 0.000s	sys 0.000s
W20260709 13:16:20.071748  4600 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.314s	user 0.000s	sys 0.001s
I20260709 13:16:20.072047  4156 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:20.072324  4457 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:20.074003  4457 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:20.077718  4138 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.078678  4138 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.079464  4613 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.080925  4613 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:19.989192  4189 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:20.094848 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:20.097992 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:19.956372  4435 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.106179 31378 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:20.015587  4348 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.124369 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:20.112692 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:20.092667  4103 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:20.134570 31391 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:20.140601 30552 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:20.142661 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 124693us
I20260709 13:16:20.110854  4219 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:20.144768 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 336828us
I20260709 13:16:20.154495 30550 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:20.165177 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:20.170135 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.142457  4250 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:16:20.149808  4544 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.816s	user 0.013s	sys 0.010s
I20260709 13:16:20.140055  4103 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:20.179997 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:20.151767  3929 consensus_meta.cc:287] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.615s	user 0.016s	sys 0.000s
I20260709 13:16:20.181155 31396 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:20.189613  4594 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:20.190404  4594 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:20.191263  4352 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:20.191934  4352 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:20.189960 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.189772  4481 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
W20260709 13:16:20.195461  4608 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:16:20.034525  4575 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.196353  4219 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:20.196763  4219 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:20.197134  4352 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.211021 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:20.212309 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.215605 30559 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:16:20.216821 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:20.217806 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.218523 30557 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:20.218377  4189 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:20.219781 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:20.221210 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.220402 30557 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:20.241118 30702 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:20.248783  4609 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:16:20.197729  4219 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.272274  4491 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:20.311715 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.271368  4250 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:20.263842  4532 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:20.230896  4481 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:16:20.250407  4598 log.cc:927] Time spent T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.445s	user 0.000s	sys 0.001s
W20260709 13:16:20.334340 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 233070us
I20260709 13:16:20.334753 30700 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:16:20.191082  4604 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:16:20.286201 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.126235  4595 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.159719  4348 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.318954 30556 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:20.314839 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97178.6us
W20260709 13:16:20.327220 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 494548us
I20260709 13:16:20.246402  4138 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:20.301074 30705 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.594s	user 0.005s	sys 0.007s
I20260709 13:16:20.220418  3929 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.197666 30377 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 5 to 10, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 10 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:16:20.336406 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:20.349488 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.351351 30705 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:20.171751  4525 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 13:16:20.333653  4250 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:20.359413 30706 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:16:20.355736 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:20.363581  4372 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.367075 31367 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:16:20.368587  3929 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.369361 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:20.370676 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:20.371435  4625 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:16:20.379878 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.410537 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:20.380170  4613 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:20.383562  4522 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.864s	user 0.016s	sys 0.000s
W20260709 13:16:20.384951 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.384766 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:20.318524  4491 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:20.332402  4532 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:20.316704  4319 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:16:20.385998 30377 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 2 to 7, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 7 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:20.390781 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:20.393556 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.399822 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:20.402704 30559 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:20.438179 30707 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
W20260709 13:16:20.395210  4609 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:16:20.416041 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:20.434067 31378 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
W20260709 13:16:20.414410 31294 leader_election.cc:336] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.629s (SENT)
I20260709 13:16:20.430655 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.413120 31394 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:20.450235 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.456812 31364 raft_consensus.cc:2410] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 23 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:16:20.426575  4347 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:20.395795  4138 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 13:16:20.467196 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 111604us
I20260709 13:16:20.468014 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:20.405478  4525 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
W20260709 13:16:20.474527 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.476935 30707 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:20.477764 30707 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:20.459721 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.443545  4621 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:20.484659  4403 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.466476  4592 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:20.470808  4613 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:20.434067  4522 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.491725 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.443964  4484 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.512287 30707 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:20.510119 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.463187  4103 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:20.507226 30699 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:20.518435 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:20.518891 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:20.523197 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 152758us
I20260709 13:16:20.495360  4319 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:20.528189 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.528918  4372 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.530359 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.491792  4403 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.533576  4103 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:20.496855  3929 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.537617 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 130557us
I20260709 13:16:20.539882 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:20.555737 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 154744us
I20260709 13:16:20.499549  4156 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:20.500020  4347 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:20.500434  4592 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:20.508414  4525 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:20.549081 30701 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:20.549544 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.556565 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:20.569804 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59816.7us
I20260709 13:16:20.572917 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:20.582852 30698 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:20.553104  4455 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:16:20.520558  4613 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.549149  4632 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.557483  4626 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:20.584774 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.500782  4491 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.572679  4525 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:20.579033  4592 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:20.578629  4372 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:20.558984  4309 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:16:20.559285  4215 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
W20260709 13:16:20.611821 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 64097.6us
I20260709 13:16:20.602504  4613 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:20.614982 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:20.614984 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.617565 30704 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:20.620472 31389 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:20.589613  4138 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.572659  3929 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:20.623170 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.625414 30556 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:16:20.622925 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 253703us
I20260709 13:16:20.627352 30701 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:20.629396 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.573668  4156 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.632800 30554 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 13:16:20.634842 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 122468us
I20260709 13:16:20.635286 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.636318 30556 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:20.622238  4346 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:20.682868 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.682894 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:20.607651  4632 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:20.757391 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 73897.2us
I20260709 13:16:20.764539  4372 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.765377  4372 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.638121  4348 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.766547 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:20.646170  4621 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:20.770853 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 214358us
W20260709 13:16:20.776868 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.777298 30670 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.772123 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.774247 30554 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 12.
I20260709 13:16:20.631855  4138 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:20.793337 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 214137us
W20260709 13:16:20.795140 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.795061 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:20.625332  4189 consensus_queue.cc:1175] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "acf852f85f47483d99cbe4020baee627" responder_term: 17 status { last_received { term: 17 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 17 index: 10 } }
I20260709 13:16:20.776022  4621 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.776736  4605 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:20.643294  4347 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.801438 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2526) took 1071 ms. Trace:
I20260709 13:16:20.807776 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:20.808475 30626 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:20.808777 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59134.1us
I20260709 13:16:20.809909 30624 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:16:20.811067 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:20.811334 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:20.811867 30631 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:20.812140 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:20.812364 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:20.812955 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:20.813158 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:20.813736 30628 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:20.813994 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:20.814843 30625 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:20.814754  4644 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:20.815399  4644 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.815766  4644 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:20.818013 30626 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:20.818987 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:20.821688  4372 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.813670 30631 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
W20260709 13:16:20.814186  4630 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 13:16:20.799556  4348 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:20.830677 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:20.840457 30633 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:20.808835 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:20.778398  4610 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.265s
I20260709 13:16:20.846338 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:20.843444 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:20.803763  4621 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:20.846364 30628 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:20.855873  4648 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.804448  4484 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:20.826751  4522 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:20.859571 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64625us
I20260709 13:16:20.871810 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.861057  4637 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.861549 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:20.861685  4403 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.794008  4525 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:20.863183  4640 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:20.863610  4641 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:16:20.864070  4586 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.475s	user 0.017s	sys 0.001s
W20260709 13:16:20.865041  4639 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:16:20.866627  4632 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.804338 30555 rpcz_store.cc:276] 0709 13:16:19.511636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.511861 (+   225us) service_pool.cc:224] Handling call
0709 13:16:19.542457 (+ 30596us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:19.569928 (+ 27471us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:19.582093 (+ 12165us) log.cc:844] Serialized 14 byte log entry
0709 13:16:19.813665 (+231572us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:19.912827 (+ 99162us) log.cc:844] Serialized 26 byte log entry
0709 13:16:19.913076 (+   249us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:19.913130 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:19.913241 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:20.247987 (+334746us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:20.577899 (+329912us) raft_consensus.cc:1685] finished
0709 13:16:20.582839 (+  4940us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.583049 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12719,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":10056,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":15341,"spinlock_wait_cycles":12226304,"thread_start_us":183042,"threads_started":1,"wal-append.queue_time_us":401441}
I20260709 13:16:20.827790  4605 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.862036  4642 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:20.879252  4435 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:20.884519  4628 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
W20260709 13:16:20.884950  4629 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.231s	user 0.001s	sys 0.001s
W20260709 13:16:20.885335  4634 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 13:16:20.885645  4631 log.cc:927] Time spent T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.283s	user 0.001s	sys 0.001s
W20260709 13:16:20.886243  4627 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.317s	user 0.000s	sys 0.002s
W20260709 13:16:20.945767 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 100817us
I20260709 13:16:20.840085  4352 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:16:20.863622  4138 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:20.907896  4156 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:20.908730  4522 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:20.916405 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:20.867455  4643 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:20.924250 30633 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
W20260709 13:16:20.918969  4489 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.074s	user 0.011s	sys 0.007s
W20260709 13:16:20.945205 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 323886us
W20260709 13:16:20.947748 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 76083.5us
W20260709 13:16:20.955761  4568 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.066s	user 0.016s	sys 0.000s
W20260709 13:16:20.951454 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 400493us
I20260709 13:16:20.893718 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:20.944337  4642 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.946309  4645 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:20.952574  4435 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:20.968634 31361 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:20.967104 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:20.971697 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 194492us
I20260709 13:16:20.968005 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:20.973801 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2045) took 1049 ms. Trace:
I20260709 13:16:20.918040  4637 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:20.965747  4522 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:20.968323  3929 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:20.969058  4645 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:20.968688  4525 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:20.985811 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:20.960956  4643 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:20.966531  4435 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:20.989078 30628 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:20.966877  4642 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:20.983049  4156 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:20.996042  4250 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.590s	user 0.015s	sys 0.000s
I20260709 13:16:20.978011  4647 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:20.996767 30629 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:16:20.982785  4649 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:21.015025  4250 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.015262  4649 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.017493  4647 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.017757  4250 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:21.018095  4647 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:20.998993 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.018349 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2029) took 1481 ms. Trace:
I20260709 13:16:21.018875 31367 rpcz_store.cc:276] 0709 13:16:19.537343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.553562 (+ 16219us) service_pool.cc:224] Handling call
0709 13:16:19.553958 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.410737 (+856779us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.017997 (+607260us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540002add4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:21.018273 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869272320}
I20260709 13:16:21.019709 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:21.020450 31361 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:20.999891 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:20.978732  4655 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:16:20.987969  4641 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:21.003626  4346 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:20.874626  4648 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.032496 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.033253  4648 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:21.034493 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.036378 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.013236  4156 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.030779  4250 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:21.039379  4643 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.040933  4643 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:21.020360 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:20.978256  4611 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:16:21.022984 31396 raft_consensus.cc:2376] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16. Re-sending same reply.
I20260709 13:16:21.001523 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:20.975008 30551 rpcz_store.cc:276] 0709 13:16:19.924308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:19.957907 (+ 33599us) service_pool.cc:224] Handling call
0709 13:16:20.012724 (+ 54817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:20.025768 (+ 13044us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:16:20.025820 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:20.115585 (+ 89765us) log.cc:844] Serialized 26 byte log entry
0709 13:16:20.354789 (+239204us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:20.354842 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:20.355113 (+   271us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:20.973553 (+618440us) raft_consensus.cc:1685] finished
0709 13:16:20.973577 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:20.973732 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12389,"raft.queue_time_us":26966,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":134,"thread_start_us":195556,"threads_started":1,"wal-append.queue_time_us":353128}
I20260709 13:16:21.038812  4156 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:21.022289  4403 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:21.046773 31367 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:21.012380 31389 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:21.013557  4435 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:21.037544 31378 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:21.055296 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:21.057228 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 71498us
W20260709 13:16:21.057863 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.050009 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:21.063892 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.064980 30704 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:21.065976 30700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:21.063208  4352 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:16:21.066949 30700 raft_consensus.cc:606] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:21.052345  4568 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.067366 30700 raft_consensus.cc:614] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:21.069137 31396 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:21.069804 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:21.070966 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.055032 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:21.052345  3929 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.072433  4189 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:16:21.067792  4568 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.074666  3929 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.075659 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:21.076756  4568 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.077112  3929 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.081993 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:21.083530 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.084571 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:21.078452 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:21.079533 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:21.050720  4403 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:21.085774 31396 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:16:21.088924 31389 raft_consensus.cc:2410] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:16:21.089968 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.095351  4403 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.097102  4403 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:21.114857 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.090956  4568 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:21.129292 31367 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:21.098448  4481 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:21.128755 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.110517  4189 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:21.121134  4352 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:21.124528  4347 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.160415  4347 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.138972 30627 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:21.144693  4605 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.145382  4189 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:21.168512  4189 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.146315 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:21.170219  4647 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.135383  4481 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:21.175417 31396 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 10. (term mismatch)
I20260709 13:16:21.166505 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.175932 31396 pending_rounds.cc:85] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:21.176232 31396 pending_rounds.cc:107] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:16:21.176494 31396 raft_consensus.cc:2889] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.136878  4348 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:21.177954 31367 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:21.138386  4651 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:21.188540 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50735.5us
I20260709 13:16:21.195588  4372 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.195981 31367 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:21.196403  4372 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.198999  4647 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.191030  4651 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.201443  4491 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:21.178252  4605 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:21.216450 31396 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 9. (term mismatch)
I20260709 13:16:21.181221  4348 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:21.217432 31396 pending_rounds.cc:85] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 8
I20260709 13:16:21.214529 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:21.221295 31396 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 13:16:21.221693 31396 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.222131 31396 pending_rounds.cc:107] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.10
I20260709 13:16:21.222437 31396 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:21.223100 31361 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:21.219362 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.202674  4651 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:21.225773 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0090416409998397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:16:21.216302  3929 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:21.227311 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.228514 30551 raft_consensus.cc:2376] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21. Re-sending same reply.
I20260709 13:16:21.229034  4438 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.229195 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.231307 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.233067 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.233865 30555 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:21.223739  4138 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:21.235108 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.235211  4138 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.235433  4644 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.236418 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.237795  4138 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.237864  4644 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:21.238282 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.238432  4481 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.227607 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.239889 30552 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:21.240288  4481 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:16:21.242859 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.246008 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.246692 30554 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:21.193593  4352 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:21.250890 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.251183 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.256269 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:21.257064 30557 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:16:21.267812 30707 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:21.271199 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.272439 30628 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:16:21.288187 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.290352 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.291704 30553 raft_consensus.cc:2410] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:16:21.301930 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.302343 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:21.224285  4348 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:21.303740 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.304232 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.303813 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.289551 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.305143 30554 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 12.
W20260709 13:16:21.310007 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 345849us
W20260709 13:16:21.309898 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.320640  4671 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:21.326004 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 353502us
I20260709 13:16:21.328636 30632 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:21.257812  4347 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:21.340281 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:21.357050 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.347692 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.360217 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.361158 30552 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:21.327970 30523 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.132:35425 timed out after 1.791s (SENT) [suppressed 32 similar messages]
I20260709 13:16:21.354789 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:21.362119  4651 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.362628 30628 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:21.362258 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:21.362907  4651 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.364701 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.365459  4651 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.366396 30552 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:21.368595 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.372332 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.375555 30627 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:21.271464  4372 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.378223 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:21.373297 30551 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:21.398651 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:21.399031 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.399528 30551 raft_consensus.cc:2376] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21. Re-sending same reply.
I20260709 13:16:21.321991  4671 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:21.310279  4632 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:21.410902 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.411437 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:21.409020 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 214695us
I20260709 13:16:21.412443 30551 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
W20260709 13:16:21.378085  4592 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.752s	user 0.011s	sys 0.003s
I20260709 13:16:21.416252 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:21.419255 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.420710 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.421517 30707 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:21.421720 30627 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:21.425196 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 51034.7us
W20260709 13:16:21.426653 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 99107.9us
I20260709 13:16:21.423298 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:21.428843 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:21.420542  4532 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.970s	user 0.010s	sys 0.006s
I20260709 13:16:21.407788  3929 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:21.432813 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.437136 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.438305 30627 raft_consensus.cc:2368] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
I20260709 13:16:21.443084 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.444217 30632 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:21.446929  4671 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.448194 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.449918 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:21.454337 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:21.455714 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:21.456339 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.458869 30553 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:21.454316 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.458572 30557 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:21.461643 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:21.463580 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47433us
W20260709 13:16:21.448289  4660 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:16:21.469197 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.467614 31396 raft_consensus.cc:2410] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:16:21.425985  4372 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:16:21.420668  4592 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.436067  4189 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:21.473191 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:21.454967  4103 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:21.483009 30552 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
W20260709 13:16:21.484264 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:21.494910 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.496502  4119 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.497121  4119 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.498801  4654 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:21.500108  4672 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.499697  4654 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.500681  4672 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.497136  4632 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.452049  4594 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.502705  4632 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.503989 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.505345  4632 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:21.505673 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.460363  4674 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:21.496568 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.514837 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.469390  4606 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:21.520047 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.435477  4532 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.527928 30703 raft_consensus.cc:2393] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:21.530124 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.532464 30707 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:21.534555 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.542780 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.447062  4352 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.546288 30698 raft_consensus.cc:2410] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:16:21.543601 30704 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:21.547595 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:21.544392 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.512135  4674 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:21.538297  4621 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
I20260709 13:16:21.560557 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.529318  4632 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.571019 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.572615 30703 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:21.574183 30699 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:21.506896  4103 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.579627 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:21.525485  4618 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:16:21.581681 30626 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:16:21.588229 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.511684  4189 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.592157 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:21.593048 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.593461 30699 raft_consensus.cc:606] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:21.593856 30699 raft_consensus.cc:614] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:21.594116 30703 raft_consensus.cc:2410] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:16:21.595618 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.596578 30706 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:21.561239  4674 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:21.598062 30630 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:21.511260  3929 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.607322 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.644038 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:21.645207 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.645968 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:21.646427 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.609844 30701 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:21.603726 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.648699 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.649662 30626 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
W20260709 13:16:21.651358 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47730.9us
W20260709 13:16:21.657768  4676 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:16:21.659781 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.596956  4372 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:21.577427  4610 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:16:21.597453  4672 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:21.644897  4156 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.584s	user 0.012s	sys 0.002s
I20260709 13:16:21.580210  4632 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.673317 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:21.574911  4621 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.665205 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.675145 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.680142 30630 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:21.682886 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 79849.4us
I20260709 13:16:21.683092 30626 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:21.683291 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.684525 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.608891  4489 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:16:21.661546  4119 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:21.661288  4618 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:21.696928 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.697975 30704 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:21.687525  4435 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.605s	user 0.006s	sys 0.009s
I20260709 13:16:21.675328  4372 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:21.688387  4653 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:21.681529  4637 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.702994 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.703678 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:21.706826  4621 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.707206 30707 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:16:21.708070  4667 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
I20260709 13:16:21.724157 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1852) took 1127 ms. Trace:
I20260709 13:16:21.725104  4665 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:21.725627  4669 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.424s	user 0.003s	sys 0.000s
I20260709 13:16:21.725806  4665 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:21.731593  4662 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 13:16:21.732134  4654 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:21.733393  4668 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.433s	user 0.000s	sys 0.002s
I20260709 13:16:21.733973 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.734230  4653 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.734692 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:21.731630  4592 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:21.712055  4250 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.673s	user 0.009s	sys 0.004s
I20260709 13:16:21.735746  4649 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.736030  4250 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:21.736435  4649 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:21.733623 30701 raft_consensus.cc:606] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:21.737030  4632 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.715550  4103 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.724628 31378 rpcz_store.cc:276] 0709 13:16:20.596369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.086830 (+490461us) service_pool.cc:224] Handling call
0709 13:16:21.111818 (+ 24988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.133979 (+ 22161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.134020 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:21.178584 (+ 44564us) log.cc:844] Serialized 26 byte log entry
0709 13:16:21.354675 (+176091us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:21.354727 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:21.354801 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:21.723901 (+369100us) raft_consensus.cc:1685] finished
0709 13:16:21.723928 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:21.724085 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8841,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":2259,"thread_start_us":130363,"threads_started":1,"wal-append.queue_time_us":245115}
I20260709 13:16:21.703960  4435 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.747591 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:21.747908 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.748322 30706 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:21.749053  4435 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.695940  4618 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.751302  4618 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:21.750388 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:21.753089 30558 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 12.
I20260709 13:16:21.753286 30625 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:21.754730  4671 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.755505  4671 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.728844  4665 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:21.758463 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:21.759598  4491 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.007s
I20260709 13:16:21.760301 30555 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:21.682832  4156 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:21.762508 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.763115 30631 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:21.717383  4663 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 13:16:21.734822  4653 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:21.712729  4672 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:21.737221 30701 raft_consensus.cc:614] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:16:21.738936  4642 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.696s	user 0.008s	sys 0.006s
I20260709 13:16:21.739781  4649 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:21.741323 30670 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:21.743034  4532 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:21.743466  4655 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:21.744233  4610 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.744746  4215 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.745069  4219 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:21.708420  4372 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.714399  4189 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:21.713898  4594 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:21.721755  4403 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.624s	user 0.014s	sys 0.000s
I20260709 13:16:21.747094  4348 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:21.789007 31378 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:21.781142  4672 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:21.803329 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62114.2us
I20260709 13:16:21.799616  4491 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:21.806824 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 139762us
I20260709 13:16:21.832082 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:21.799633 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.781136  4671 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.799642  4665 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:21.790270 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:21.823477 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.827185 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:21.833067 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.807147 30550 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:21.833606 30701 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:21.835242 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.808706  4632 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:21.858354 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.862305 30599 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:21.801028  4642 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.808768  4215 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.811281  4610 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.892931 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.893813  4346 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:21.893821  4671 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:21.863595  4641 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:21.913872 30700 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
W20260709 13:16:21.888310  4679 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
I20260709 13:16:21.902863  4671 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.911322  4653 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.891420  4625 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:21.860896  4603 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:16:21.915987 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:21.863358  4632 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:21.948210  4632 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.949188  4674 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.842386  4491 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:21.951715 30629 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:21.951865  4642 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:21.953408  4491 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:21.896450  4649 raft_consensus.cc:2804] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:21.842098  4403 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
W20260709 13:16:21.924728 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:21.968251 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:21.968896 30701 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:21.969404 30629 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:21.944986 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.945948  4532 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:21.861150  4454 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:16:21.975030 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:21.912850  4641 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:21.882889  4672 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.940591  4671 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:21.855598  4665 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:21.882807  4670 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.149s
I20260709 13:16:21.983654 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:21.985298 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89515.9us
I20260709 13:16:21.980247 30597 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:21.987221  4481 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.746s	user 0.015s	sys 0.001s
I20260709 13:16:21.973634  4532 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:21.986124 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.056366 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.026456 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:22.057734 30701 raft_consensus.cc:2376] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15. Re-sending same reply.
I20260709 13:16:21.976737  4491 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:22.037822 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57708.5us
I20260709 13:16:22.061440 30630 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:21.987339  4641 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:22.082878 30559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1155) took 1867 ms (client timeout 1991 ms). Trace:
W20260709 13:16:22.083372 30559 rpcz_store.cc:269] 0709 13:16:20.215729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.216527 (+   798us) service_pool.cc:224] Handling call
0709 13:16:22.082650 (+1866123us) spinlock_profiling.cc:243] Waited 79.3 ms on lock 0x7b540028a0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:22.082774 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":182477568}
I20260709 13:16:22.021382  4250 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:22.087134  4665 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.087630  4637 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:22.088101  4637 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:22.088243  4189 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.089120  4189 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.090513  4215 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:22.092442  4189 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.094153  4653 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:22.096629 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:22.089155  4637 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:22.059387 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 73389.6us
I20260709 13:16:22.103528 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:22.103844 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:22.022300  4346 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
W20260709 13:16:22.066953 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 210697us
I20260709 13:16:22.104387 31396 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:22.067122  4348 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:22.065268 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:22.112073 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46914us
I20260709 13:16:22.113485 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:22.070744  4103 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:22.034775  4156 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.059939  4532 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:22.072595  4622 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:22.060199  4347 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:22.079114 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.164740 30701 raft_consensus.cc:2393] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:22.079708  4684 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
W20260709 13:16:22.170261 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 62338.2us
I20260709 13:16:22.080282  4674 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:22.081700 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.104277 30523 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.027026  4610 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:22.119490 31364 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:22.120779 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:22.118588  4189 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:22.155828 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:22.166108 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:22.166504 31396 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:22.168846 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:22.171437 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:22.176780 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:22.117873  4481 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.158636  4622 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:22.179378  4347 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:22.180223 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:22.221520 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:22.222555 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:22.184414 30624 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:22.187533  4666 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.902s	user 0.004s	sys 0.000s
I20260709 13:16:22.187400 31394 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:16:22.119532  4618 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:22.195149 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 1962) took 1149 ms. Trace:
I20260709 13:16:22.184602 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:22.183521 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:22.121675  4119 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:22.205480 30558 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:22.216701 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45538.7us
I20260709 13:16:22.185400  4532 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.250730 30699 raft_consensus.cc:2376] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15. Re-sending same reply.
I20260709 13:16:22.233640 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:22.250084 30633 rpcz_store.cc:276] 0709 13:16:21.045281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.045894 (+   613us) service_pool.cc:224] Handling call
0709 13:16:21.060780 (+ 14886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.141716 (+ 80936us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:21.141751 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:21.141797 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:21.141829 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.194783 (+1052954us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b540007d0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:22.194853 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:22.195039 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185552768}
I20260709 13:16:22.204182  4189 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.266068 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.202162  4103 consensus_queue.cc:1059] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:16:22.272734 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.224050  4605 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.269586 30554 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:22.268399 30627 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:22.280426 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.286232 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:22.256860  4119 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.254132  4522 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:22.251109  4649 raft_consensus.cc:697] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:22.293087 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.303812 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.300596 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.305030 30629 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:22.295321 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.221428  4481 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:22.271696  4189 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:22.300590 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:22.300997 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.313769 30633 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:22.301971 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.302186 30632 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:16:22.302821 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.320645 30626 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:22.305430 30627 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:22.270524  4454 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:22.309185 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1162) took 1074 ms. Trace:
I20260709 13:16:22.322548 30555 rpcz_store.cc:276] 0709 13:16:21.234729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.234862 (+   133us) service_pool.cc:224] Handling call
0709 13:16:22.308946 (+1074084us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54001a9854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:22.309102 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206282624}
W20260709 13:16:22.327410 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 223215us
I20260709 13:16:22.295735  4674 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:22.294389  4119 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:22.330118 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:22.344934 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.346658 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:22.334434  4250 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:22.336835 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:22.338270 30550 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:22.339254 30553 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:22.339341 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:22.347421 30557 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:16:22.330581 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.348594 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.308876  4348 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.351029 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.351270 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.354168 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.354996 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:22.375525 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.392089 30554 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:22.392341 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:22.326797  4454 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.288746  4605 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:22.429772 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.401830 30699 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:22.399351  4697 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.407243 30557 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:16:22.407902 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 77403us
I20260709 13:16:22.409437  4649 consensus_queue.cc:237] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.410950  4481 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:22.411458 30633 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:16:22.309034  4696 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.142s
W20260709 13:16:22.440243 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 320611us
I20260709 13:16:22.412495  4641 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.413038 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:22.414343  4594 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.591s	user 0.005s	sys 0.008s
I20260709 13:16:22.414155  4671 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:22.393491  4403 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:22.414206 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:22.426791 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 162307us
I20260709 13:16:22.465200 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.404628  4348 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:22.431344  4454 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:22.436100 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.453263 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:22.454960 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 53371.8us
I20260709 13:16:22.455652 30701 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:22.474895 30701 raft_consensus.cc:606] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:22.414505 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.474990 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.459914 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:22.461923 30558 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:22.428165  4698 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:22.476711 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.505889 31378 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:22.507477 30630 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:22.498202 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.442058 30522 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:22.469720  4532 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.517899 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:22.322904  4622 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:22.437798  4647 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
W20260709 13:16:22.523455 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58397.1us
I20260709 13:16:22.471836  4642 raft_consensus.cc:2764] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:16:22.509351  4372 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:22.520747  4676 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:16:22.450054  4522 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:16:22.505227  4481 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:22.505591  4665 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:22.511598  4696 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:22.525785 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.587751 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.400527  4347 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.514971  4568 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:22.449519  4697 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:22.535497  4403 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:16:22.642688 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117010us
W20260709 13:16:22.636268 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 183091us
I20260709 13:16:22.644914 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.615984  4696 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:22.654940 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 258871us
I20260709 13:16:22.680087 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.653837 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.597133  4250 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:16:22.620687  4706 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:22.682385 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 343107us
I20260709 13:16:22.629966  4622 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:22.688417 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:22.694545  4728 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:16:22.694648 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50166.3us
I20260709 13:16:22.641019  4568 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:22.674749  4457 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.030s
I20260709 13:16:22.640027  4647 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:22.678346  4665 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:16:22.706776  4457 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.002s	sys 0.030s
I20260709 13:16:22.651732  4403 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:22.703143 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49439.6us
I20260709 13:16:22.654132  4219 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:22.710093 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 182735us
I20260709 13:16:22.716756 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.624179  4618 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:22.703939  4728 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:22.719988  4728 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:22.718791  4219 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.718852  4403 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:22.699592  4491 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:22.686980  4696 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:22.708113  4568 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:22.748039  4454 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.717011  4103 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:22.759811 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:22.772089  4708 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:16:22.717886  4665 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:22.762589 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45945.7us
I20260709 13:16:22.767807  4712 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:22.768627  4219 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.731451  4491 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:22.779711 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 337783us
I20260709 13:16:22.781065 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.716269  4647 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.751919  4522 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:22.786513 30523 leader_election.cc:400] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
I20260709 13:16:22.791057 30523 leader_election.cc:403] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:22.774859  4704 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.139s	user 0.001s	sys 0.001s
I20260709 13:16:22.751430  4724 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:22.783389  4665 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:22.795101 30625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1434) took 1544 ms (client timeout 1983 ms). Trace:
W20260709 13:16:22.802652 30625 rpcz_store.cc:269] 0709 13:16:21.250704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.250921 (+   217us) service_pool.cc:224] Handling call
0709 13:16:22.794855 (+1543934us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54007229d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:22.795030 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1143379200}
I20260709 13:16:22.809207  4723 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:16:22.790105  4671 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:22.811223  4671 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:22.814679  4403 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:22.818121  4403 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:22.790242  4641 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.811618  4707 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:22.819036  4403 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:22.819772  4403 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:22.820541  4403 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:22.820641  4647 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.821043  4403 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:22.821632 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48026.8us
I20260709 13:16:22.822965 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.824337 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:22.822602  4403 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:22.790103  4491 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:22.835726 30702 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:22.839144 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:22.840167 30556 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:22.836396  4673 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.801244  4348 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:22.821692  4720 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:22.866036 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:22.871220 30553 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
W20260709 13:16:22.873027 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 193217us
I20260709 13:16:22.830422  4457 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.835278  4250 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
I20260709 13:16:22.884814 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:22.889020 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 108030us
I20260709 13:16:22.890762 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.859349  4711 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:22.865010  4723 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:22.893620 30700 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:22.868927  4707 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:22.870922  4720 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:22.898741 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 198423us
I20260709 13:16:22.880812  4647 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:22.907210 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:22.859490  4673 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:22.915236  4348 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:22.907181  4641 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:22.923468 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99210.1us
I20260709 13:16:22.888581  4457 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:22.897567  4723 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:22.911151  4707 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:22.925335 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49969.3us
I20260709 13:16:22.913121  4677 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:22.927446 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:22.921222  4348 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:22.933830 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.914716 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.929749 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:22.936369 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:22.953269 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.917035  4730 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:22.914515  4647 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:22.924785  4697 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:22.932896  4723 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:22.957561 30632 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:22.964423  4734 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:22.976017  4250 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:22.976696  4457 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:22.961156  4735 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:16:22.972267 30552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1180) took 1715 ms (client timeout 2014 ms). Trace:
W20260709 13:16:22.969211  4674 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.000s
W20260709 13:16:22.982950 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 49245.9us
W20260709 13:16:22.983744 30624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1207) took 2169 ms (client timeout 1657 ms). Trace:
I20260709 13:16:22.986793  4724 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:22.998234 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45103.2us
I20260709 13:16:23.009298 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:22.936702  4641 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:22.961354 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46901.2us
I20260709 13:16:22.956661  4348 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:22.935318  4677 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:23.000905 30552 rpcz_store.cc:269] 0709 13:16:21.257044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.367933 (+110889us) service_pool.cc:224] Handling call
0709 13:16:22.972023 (+1604090us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400293954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:22.972186 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259829632}
W20260709 13:16:23.005671 30624 rpcz_store.cc:269] 0709 13:16:20.807993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.811043 (+  3050us) service_pool.cc:224] Handling call
0709 13:16:22.977639 (+2166596us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:22.977780 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3155895040}
I20260709 13:16:23.007097 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.015678  4700 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:23.016604  4700 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.020905  4707 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.023530  4454 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.025408 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.000746  4735 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.003315  4250 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:22.957376  4730 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.028821 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:23.006007  4674 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.010807 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.003218  4729 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:23.063495 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:22.981004  4481 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:23.078502 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 67774.4us
W20260709 13:16:23.033648  4119 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.633s	user 0.014s	sys 0.000s
I20260709 13:16:23.028622  4734 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:23.030182  4457 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:23.032027  4717 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:23.092679 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:23.043074  4677 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.093809 31378 raft_consensus.cc:2410] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 19 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:16:23.032178  4735 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:23.066648  4700 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:22.959267  4720 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.094322  4697 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:16:23.100334 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 164055us
I20260709 13:16:23.055907  4647 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.076421  4730 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:23.085603  4724 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:23.111045 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:23.110172 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.116204 31394 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 24: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:23.053161  4568 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.103041  4674 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:23.123663 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.126879  4734 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:23.130792 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.131605 30624 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:23.129940 30625 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:23.134663 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 105892us
I20260709 13:16:23.135820 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1444) took 1579 ms. Trace:
I20260709 13:16:23.136140 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.137703 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.136411 30628 rpcz_store.cc:276] 0709 13:16:21.556221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.566781 (+ 10560us) service_pool.cc:224] Handling call
0709 13:16:21.567429 (+   648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:21.567843 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:21.598160 (+ 30317us) log.cc:844] Serialized 14 byte log entry
0709 13:16:22.991414 (+1393254us) spinlock_profiling.cc:243] Waited 1.53 ms on lock 0x7b1800410da0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:16:23.135314 (+143900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.135400 (+    86us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:23.135477 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.135582 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.135747 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10132,"prepare.queue_time_us":27200,"prepare.run_cpu_time_us":6273,"prepare.run_wall_time_us":38646,"spinlock_wait_cycles":3530112,"thread_start_us":126309,"threads_started":1}
I20260709 13:16:23.140380 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.142648 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.097508  4119 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.146453 30633 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:23.148649 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.151993 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.153030 30703 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:23.131438  4649 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:23.156337 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46275.3us
I20260709 13:16:23.157289 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:23.157766 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 94392.6us
I20260709 13:16:23.159498 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.156670 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:23.145007  4438 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:16:23.173908 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.179607  4696 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.184396  4649 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:23.109352  4641 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.164096  4741 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.164317  4347 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:16:23.191542  4347 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.192080  4347 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:23.164714  4738 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:23.144291  4457 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:23.165561  4697 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.167636 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.167884  4732 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:23.168394  4700 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:23.169515  4723 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.144064  4717 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:23.170814 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.161198  4522 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:23.140157  4734 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.223932  4119 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:23.226312 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58777.3us
I20260709 13:16:23.227730 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:23.202301 30625 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:23.209404 30703 raft_consensus.cc:3060] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:23.148178  4250 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.164203  4665 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.149008  4481 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:23.247225  4746 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:16:23.247073  4603 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:23.251780  4603 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.252704  4603 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.253733  4603 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:23.256422  4723 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:23.258419  4726 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.324s	user 0.004s	sys 0.000s
I20260709 13:16:23.259188 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.260116 30626 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:23.237548  4522 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:23.265025 30631 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 1965) took 2178 ms (client timeout 1539 ms). Trace:
W20260709 13:16:23.265574 30631 rpcz_store.cc:269] 0709 13:16:21.086992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.087195 (+   203us) service_pool.cc:224] Handling call
0709 13:16:23.264757 (+2177562us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54004a9cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.264933 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487941376}
I20260709 13:16:23.266757  4454 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:23.267416  4454 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.269781  4454 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.234797  4649 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:23.212430  4741 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.251293  4696 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.218364  4734 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.282485 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:23.212850  4738 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:23.285666 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:23.286414 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:23.239248  4189 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.893s	user 0.016s	sys 0.000s
I20260709 13:16:23.291318  4723 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:16:23.306370 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.292855 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:23.303871 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.284260  4697 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.276713  4119 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.296241 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.283385  4698 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:23.297334  4749 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:23.283752  4642 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.306583  4690 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.271s	user 0.009s	sys 0.008s
I20260709 13:16:23.292164 31378 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:23.275712  4741 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.312026 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.307888 31391 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:23.281677  4454 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.281955  4438 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:23.315380  4649 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.313565 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.330376 30673 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:23.204084  4728 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.333364 31378 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:16:23.307298  4532 raft_consensus.cc:3055] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:23.352846 30673 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:23.239524  4491 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:23.339130 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.343333  4438 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:23.304656  4348 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.347787 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:23.327466  4643 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:23.348865 30703 raft_consensus.cc:2468] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
W20260709 13:16:23.364930 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52387.5us
I20260709 13:16:23.308259  4732 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:16:23.314072  4457 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:23.323194  4723 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:16:23.334901  4454 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.353222  4696 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:23.423784  4696 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.425242  4696 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.356914  4612 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:23.374751 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:23.431275 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56703.9us
I20260709 13:16:23.431699  4651 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.432215 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:23.432394  4651 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.433779 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.352996  4568 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.323339  4119 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.435120 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.436095  4731 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.437006  4119 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.318054  4697 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.438565  4728 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.439173  4697 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.313683  4189 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.357425  4532 raft_consensus.cc:740] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:23.441579  4348 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.336401  4642 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.338296  4698 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.387615 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.367777  4649 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:23.398494  4254 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.343645  4717 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:16:23.420701  4454 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.436698  4731 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.381541  4738 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.365783  4309 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:23.446103 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.462584 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.493420 30631 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:23.493644 30703 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:23.383623  4712 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:23.504200  4672 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:23.441601  4189 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.449016  4741 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:23.449397  4729 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:23.519121 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.453620  4700 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:23.528714  4720 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:23.493005  4309 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.494545  4712 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.495317  4651 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.538831 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.452414  4457 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:16:23.488035  4254 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.512326  4649 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 13:16:23.546589 30707 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2279) took 1833 ms (client timeout 1817 ms). Trace:
W20260709 13:16:23.547464 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 178028us
W20260709 13:16:23.549204 30707 rpcz_store.cc:269] 0709 13:16:21.713682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.546341 (+1832659us) service_pool.cc:211] Skipping call since client already timed out
0709 13:16:23.546501 (+   160us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:16:23.543170 30703 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:23.442406  4532 consensus_queue.cc:260] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.551144 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:23.553483 31378 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:23.540329  4309 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:23.537876  4700 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:23.542408  4712 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:23.559888 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:23.543973  4665 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.539517  4723 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:23.568960  4347 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.575139  4700 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.552229  4612 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:23.579131 30704 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1056) took 1992 ms (client timeout 1924 ms). Trace:
I20260709 13:16:23.579317  4612 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:23.579762 30704 rpcz_store.cc:269] 0709 13:16:21.586593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.592643 (+  6050us) service_pool.cc:224] Handling call
0709 13:16:23.578866 (+1986223us) spinlock_profiling.cc:243] Waited 50.6 ms on lock 0x7b5400184cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.579029 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116337920}
I20260709 13:16:23.581089 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.583310  4403 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:23.568919  4723 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:23.588366 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2286) took 1312 ms. Trace:
I20260709 13:16:23.588929 30700 rpcz_store.cc:276] 0709 13:16:22.275963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.300170 (+ 24207us) service_pool.cc:224] Handling call
0709 13:16:23.588131 (+1287961us) spinlock_profiling.cc:243] Waited 573 ms on lock 0x7b540042f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.588289 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1317309312}
I20260709 13:16:23.582444 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.591723 30631 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:23.592343 30704 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:23.593743 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.593811 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.594585 30631 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:23.594923 30703 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
W20260709 13:16:23.601402 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.607405 30631 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:16:23.610930 30671 connection.cc:256] RPC call timeout handler was delayed by -1.3268030090000593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
I20260709 13:16:23.610821 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.615095 30633 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:23.614918 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.615319 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:23.616170 31391 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 24: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:23.618672 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.618444 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.620261 30631 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:23.595628  4347 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:23.611671  4735 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.500s	user 0.012s	sys 0.000s
I20260709 13:16:23.612118  4457 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:23.629063 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.586290  4723 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.664557 30633 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:23.646680 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.614480  4643 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:23.660661 30631 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:23.645171 30625 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:16:23.633030  4457 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.680210 30707 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:16:23.680341 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 235368us
I20260709 13:16:23.622879  4730 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.682536 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.681699 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.698204 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:23.699174 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1818) took 1793 ms. Trace:
I20260709 13:16:23.704134 30707 rpcz_store.cc:276] 0709 13:16:21.906142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.652354 (+1746212us) service_pool.cc:224] Handling call
0709 13:16:23.679668 (+ 27314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.698829 (+ 19161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.699108 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:23.706548 30624 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:23.705960 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.700115 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:23.696786 30698 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1802) took 2174 ms (client timeout 2363 ms). Trace:
I20260709 13:16:23.678968  4215 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:23.709300 30698 rpcz_store.cc:269] 0709 13:16:21.504020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.506498 (+  2478us) service_pool.cc:224] Handling call
0709 13:16:23.671104 (+2164606us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b54005f00d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.678434 (+  7330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3699456}
I20260709 13:16:23.637037  4735 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.710212  4723 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:23.729715  4254 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.671381  4491 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:23.724011  4731 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.678596  4643 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.679911  4717 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:23.725101 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.697283  4457 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:23.736052 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2287) took 1459 ms. Trace:
W20260709 13:16:23.737841 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.667796  4438 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.745807 30698 rpcz_store.cc:276] 0709 13:16:22.276311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.735056 (+1458745us) service_pool.cc:224] Handling call
0709 13:16:23.735480 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.735697 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.735729 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.735771 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:23.735806 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.735861 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.735981 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:23.744274 30626 raft_consensus.cc:2410] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:16:23.740442 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.703496  4698 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.706784  4642 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.758736 30596 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:23.738303  4757 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:16:23.759094 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2288) took 1482 ms. Trace:
W20260709 13:16:23.731344  4761 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:16:23.741356  4603 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.736228  4435 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:23.742583  4347 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:23.730569  4735 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.765077 30707 rpcz_store.cc:276] 0709 13:16:22.276618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.740882 (+1464264us) service_pool.cc:224] Handling call
0709 13:16:23.744146 (+  3264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.744416 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.744456 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.744526 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:23.744574 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.744654 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.759019 (+ 14365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:23.804806 30624 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
W20260709 13:16:23.803175 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44610.1us
I20260709 13:16:23.727635  4309 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.807312 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.762734  4625 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:23.809042 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.813356 30626 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:23.737172  4215 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.792217  4435 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:23.808568  4741 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:23.752416  4717 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.821381 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.824963 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.803898  4603 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.830004  4215 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.833351  4605 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.834841 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2281) took 2103 ms. Trace:
I20260709 13:16:23.837865  4491 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.838637  4491 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.842300  4677 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:23.842818 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:23.821651  4347 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.835425 30706 rpcz_store.cc:276] 0709 13:16:21.730981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.748988 (+ 18007us) service_pool.cc:224] Handling call
0709 13:16:23.671873 (+1922885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:23.672118 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.672141 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:23.672188 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:23.672217 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.672281 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.834543 (+162262us) spinlock_profiling.cc:243] Waited 1.92 s on lock 0x7b54005f00d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.834765 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4421347712}
I20260709 13:16:23.849171 30704 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
W20260709 13:16:23.844126  4760 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:16:23.853334 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.853328 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.853894 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:23.856913 30626 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:16:23.857388 30626 pending_rounds.cc:85] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 7
I20260709 13:16:23.857621 30626 pending_rounds.cc:107] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:16:23.857839 30626 raft_consensus.cc:2889] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:23.860404 30707 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:23.864261  4309 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.866601 31394 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:16:23.867197 31394 pending_rounds.cc:85] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:23.867465 31394 pending_rounds.cc:107] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:16:23.867720 31394 raft_consensus.cc:2889] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:23.869197  4770 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:16:23.868932 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.870127 30706 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:23.872223 30706 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:23.872958  4720 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:23.873322 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34250 (request call id 48) took 1981 ms. Trace:
I20260709 13:16:23.873723  4720 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.873867 30701 rpcz_store.cc:276] 0709 13:16:21.891777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.407004 (+515227us) service_pool.cc:224] Handling call
0709 13:16:23.872862 (+1465858us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b540042bba0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5a520 00007f1806ed7652 00007f180bea560a 00007f180542b7d4 00007f180542b762 00007f180542b691 00007f180542b60c 00007f180542a8c2 00007f180521848c 00007f18052178c6
0709 13:16:23.873215 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10108160}
I20260709 13:16:23.876974  4594 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:23.877535  4594 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.877601 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:23.848793  4491 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.878537  4594 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.899966  4594 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:23.901470  4709 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.208s	user 0.002s	sys 0.002s
I20260709 13:16:23.902256 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2285) took 2012 ms. Trace:
I20260709 13:16:23.902776 30700 rpcz_store.cc:276] 0709 13:16:21.890334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:23.619950 (+1729616us) service_pool.cc:224] Handling call
0709 13:16:23.631864 (+ 11914us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:23.632283 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:23.632313 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:23.674119 (+ 41806us) log.cc:844] Serialized 26 byte log entry
0709 13:16:23.808598 (+134479us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:23.808644 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:23.808717 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:23.902017 (+ 93300us) raft_consensus.cc:1685] finished
0709 13:16:23.902040 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:23.902183 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4126,"raft.queue_time_us":1972,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":14294,"thread_start_us":79956,"threads_started":1,"wal-append.queue_time_us":149455}
I20260709 13:16:23.904943  4730 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:23.885810 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.907064 31396 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:23.888005  4189 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:16:23.890800 30702 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1044) took 3538 ms (client timeout 2000 ms). Trace:
I20260709 13:16:23.895427  4438 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:23.856070  4643 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.896848 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.896225  4637 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:23.896258  4435 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.887270  4720 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:23.907119 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 747646us
W20260709 13:16:23.912904 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:23.917459 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.925592 30704 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:23.919811 30707 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:16:23.914870 30702 rpcz_store.cc:269] 0709 13:16:20.352935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.353132 (+   197us) service_pool.cc:224] Handling call
0709 13:16:23.890555 (+3537423us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540008be54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:23.890713 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4705992960}
I20260709 13:16:23.928179 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.928500 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.929334 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:23.928364 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:23.930244 30552 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:23.922842  4723 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.900153  4717 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.932178 31364 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 13: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:23.931226  4612 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:16:23.932679 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.914501  4189 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:23.933144 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:23.934669 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.952605 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.939666 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.955518  4674 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:23.958344 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.928054  4738 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:23.957909 30559 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:23.934732  4435 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:23.954533 30558 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:23.934796  4720 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:16:23.934821  4730 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:23.961196 31364 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:16:23.971555 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.972201 30558 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:23.961699 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:23.963037 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:23.956346 30671 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:23.969796  4491 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:23.975446 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:23.971573 30552 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:23.957209  4723 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:23.970710  4712 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.981395  4215 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:23.982539  4309 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:23.982770  4674 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:23.983036  4309 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.983469  4674 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:23.984092  4712 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:23.984174  4309 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:23.984483  4674 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:23.985769  4309 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:23.960336  4438 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:23.978751  4643 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:23.979408  4717 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:23.987661 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:23.991305  4643 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.992511  4643 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:23.989897  4438 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:23.979753  4723 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:23.999161  4723 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:24.000808  4438 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:24.001420  4725 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.093s	user 0.003s	sys 0.000s
I20260709 13:16:23.961223  4765 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.004818 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.005853 30558 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 13: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:24.003703 30552 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:24.005208 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1813) took 2188 ms. Trace:
I20260709 13:16:23.988322 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:24.008323 30699 rpcz_store.cc:276] 0709 13:16:21.817128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.455387 (+638259us) service_pool.cc:224] Handling call
0709 13:16:22.504675 (+ 49288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:22.514891 (+ 10216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:22.514931 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.515587 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 13:16:22.714549 (+198962us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:22.714637 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.714751 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.000714 (+1285963us) spinlock_profiling.cc:243] Waited 8.01 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:24.003733 (+  3019us) raft_consensus.cc:1685] finished
0709 13:16:24.003763 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.005119 (+  1356us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7792,"raft.queue_time_us":11969,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4433,"spinlock_wait_cycles":18419072,"thread_start_us":112425,"threads_started":1,"wal-append.queue_time_us":332472}
I20260709 13:16:24.010914  4765 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.017616  4457 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.018932 30554 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:16:24.019892  4687 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.990s	user 0.005s	sys 0.012s
I20260709 13:16:24.020382  4765 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.023459  4595 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:24.027806 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:24.032764 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:24.035563 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54603.8us
I20260709 13:16:24.037236 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:24.032208  4610 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:24.038462  4610 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:23.990886  4612 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:16:24.039482  4610 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:24.043594 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1907) took 3064 ms. Trace:
I20260709 13:16:24.050086 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.046345 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:24.051106 30629 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
W20260709 13:16:24.051934 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.046494 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:24.048472 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.054473 30628 raft_consensus.cc:2368] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23.
I20260709 13:16:24.048956 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:24.049233 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:24.055933 30625 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:24.056391 30631 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:24.049695 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.044131 31389 rpcz_store.cc:276] 0709 13:16:20.967941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:21.192398 (+224457us) service_pool.cc:224] Handling call
0709 13:16:21.223020 (+ 30622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:21.410151 (+187131us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:21.524873 (+114722us) log.cc:844] Serialized 14 byte log entry
0709 13:16:22.032512 (+507639us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:22.064472 (+ 31960us) log.cc:844] Serialized 26 byte log entry
0709 13:16:22.110037 (+ 45565us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:22.110112 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:22.110229 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:24.018969 (+1908740us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:24.027961 (+  8992us) raft_consensus.cc:1685] finished
0709 13:16:24.032410 (+  4449us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:24.032593 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28843,"prepare.queue_time_us":28874,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":129476,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":13991,"spinlock_wait_cycles":5186304,"thread_start_us":706875,"threads_started":4,"wal-append.queue_time_us":367202}
I20260709 13:16:24.060053 31378 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:24.062212 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.063148 31378 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:24.063380 30628 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:16:24.063879 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.064604 30631 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:24.066395 30554 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:24.067371 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.068508 31378 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:24.070837 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.071079  4438 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:24.071652 31364 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:24.072463  4491 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:24.074477  4706 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:24.076136  4717 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:24.076840  4717 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:24.077863  4707 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:24.078460  4707 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:24.036114  4454 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:16:24.067155  4612 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:24.080509  4612 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:24.081532  4612 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:24.083067 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.083861 31378 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:24.083060 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.084652 30699 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:24.058264  4671 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:24.087369  4671 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:24.087762  4671 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:24.092283 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.092345 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.094249 31364 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:24.077973  4717 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.035054  4189 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.095875 31396 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: [21.9-21.9]
I20260709 13:16:24.098186  4522 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.098976  4522 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.101610  4522 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:24.103938 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.104548 31388 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:24.034957  4765 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:16:24.108148 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.109038 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.145239 31361 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:16:24.090204 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.112005 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:24.147122 30624 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:24.117043  4673 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.150713 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.151259 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.151687 31388 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:24.122381 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:24.126389 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.127010 30625 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
W20260709 13:16:24.153290 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.105741 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.154568 31367 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:24.132498 31383 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
W20260709 13:16:24.156415 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.137509 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.158859 30558 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:24.138890 30559 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:16:24.160720 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.161531 30559 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:24.143374 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.150245 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.079484  4707 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.167675 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:24.173511 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51246.1us
I20260709 13:16:24.065145  4724 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.148550 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.166754 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.181145 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.156407 31367 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:24.182060 31364 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:24.181137 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.178680 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.184643 30556 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:24.185355 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.186188 30558 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:24.190984  4724 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:24.193432 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 45034.1us
I20260709 13:16:24.193588 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.065104  4403 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:24.194806 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:24.196336 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.196816 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:24.198164 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.164777 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:24.203413 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.211035 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.211963 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.190100  4723 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.213573 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:24.217159 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 52504.5us
I20260709 13:16:24.204561 30632 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:24.220885  4724 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.221521 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.223711  4765 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:24.224934  4705 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:24.225183 31367 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:16:24.226575 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.226926  4156 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:24.183516  4491 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.228206  4457 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.213289 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.263396 31394 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:24.217912 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.196327  4673 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:24.246987 30554 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:24.195565  4759 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:24.254659 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.261118 30558 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:24.263214 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.210048  4707 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:24.244601 31361 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:24.248631 30559 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:24.213862  4403 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:24.271227 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:24.274375 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56564.9us
I20260709 13:16:24.275377 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.241869  4711 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:24.280007 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.293391 30559 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:24.252163 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:24.282539 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.250099  4705 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.291774 30558 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 14: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:24.269981  4673 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.278544  4648 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:24.307813 31378 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:24.303858 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.308789 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.305936 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.310737 30559 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:24.316072 30704 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:16:24.311901 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.324604  4403 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.299428  4156 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:24.325997  4156 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:24.276235  4491 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.327440  4156 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.328991  4491 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.307963  4673 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:24.302265  4649 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:16:24.340937 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.342739 30554 raft_consensus.cc:2393] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:24.343163  4637 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:24.343748  4637 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:24.343909 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:24.344282  4637 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:24.345356 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.346027 30554 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:24.347839  4723 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.348269 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:24.349438  4643 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:24.349972  4643 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:24.350944  4643 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.358748 30704 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:24.360708  4730 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:24.302557  4705 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:24.361984 31383 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:24.362838  4723 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.363078 30704 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:24.363438  4723 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.365792  4723 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.375734  4649 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:24.377168 30705 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:24.377797  4621 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.377547  4649 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:24.378403  4621 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:24.379920 30554 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:24.381417 30554 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:24.382001  4648 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:24.368579 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.284379  4759 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.383657  4733 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:24.395669  4733 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.396070 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:24.397542  4603 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.397917  4733 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.382596 30554 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:24.392864  4522 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:24.399186  4522 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:24.399356 30550 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:24.399748  4522 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:24.403569 30698 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:24.388799 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.394834 30699 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:24.406697 30558 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:24.409162 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:24.395011  4481 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:16:24.405767  4730 raft_consensus.cc:476] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Not starting pre-election -- already a leader
I20260709 13:16:24.412612 30556 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:24.412729 31388 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
W20260709 13:16:24.412724 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 369939us
I20260709 13:16:24.421139 31378 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:24.420360 31394 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:24.394392  4759 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.398528  4219 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 13:16:24.457417 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61476.6us
I20260709 13:16:24.384974  4648 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.049s
I20260709 13:16:24.459679 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:24.481410 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48172.9us
I20260709 13:16:24.494691  4637 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.497331  4637 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.498006  4637 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:24.509196 30553 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:24.511629  4219 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:24.512209  4219 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.512656  4219 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:24.513029  4768 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.703s	user 0.004s	sys 0.000s
W20260709 13:16:24.520776  4779 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
W20260709 13:16:24.523005 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.524184 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.528088 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:24.456929  4481 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 13:16:24.518357  4435 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.000s
I20260709 13:16:24.543463 31396 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:24.545207 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.547389 30550 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:24.531184  4438 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:24.537107  4774 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:16:24.559906 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:24.582753 30705 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1045) took 4183 ms (client timeout 2049 ms). Trace:
W20260709 13:16:24.579317 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 986763us
W20260709 13:16:24.565574 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 370869us
I20260709 13:16:24.565651 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:24.602962 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.581568 31389 raft_consensus.cc:1275] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:16:24.602196 30705 rpcz_store.cc:269] 0709 13:16:20.399226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.399444 (+   218us) service_pool.cc:224] Handling call
0709 13:16:24.582444 (+4183000us) spinlock_profiling.cc:243] Waited 3.93 s on lock 0x7b5400342454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:24.582663 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9040759552}
W20260709 13:16:24.608800 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44870.6us
I20260709 13:16:24.621165 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:24.612433 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:24.627004 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.627465 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.627972 31394 raft_consensus.cc:2393] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:24.628665 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.628536 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.629371 30702 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:24.624078 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.635749 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.636914 31388 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:24.639283 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.640115 30705 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:24.639289 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.641165 31389 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:24.566413  4696 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:24.562676  4457 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.577638  4532 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:24.649870 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.651113 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.669605 31378 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:24.630350  4730 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.652009 30706 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
I20260709 13:16:24.554657  4671 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.652962 31361 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:16:24.654394 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.656206 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.657567 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.662935  4733 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:16:24.580971  4645 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.642207 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:24.668577 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:34250
W20260709 13:16:24.671645 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.578997  4706 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:16:24.672657 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.652184 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.667328  4457 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.667369  4724 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:24.675707  4705 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.623997  4673 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.703441 30559 raft_consensus.cc:2410] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:16:24.704954 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.661075  4738 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:24.706362 31396 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:24.707600 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:24.727532 30702 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:24.642844  4603 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:16:24.659628 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.674044  4765 consensus_queue.cc:1048] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 13:16:24.720484 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.718210 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:24.716079 30698 raft_consensus.cc:606] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:24.737139 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.739403 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.740056 30706 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:24.741256 30706 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:16:24.701052  4730 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.720738  4454 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.746122 30704 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:24.743497 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:24.738293 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:24.751151 31378 raft_consensus.cc:2468] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:24.750308 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:24.771852 31364 raft_consensus.cc:1240] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:16:24.770577 30702 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:24.746203  4403 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:24.745405  4738 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:24.783658 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 359130us
I20260709 13:16:24.729790  4724 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:24.786545 31391 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.031s	sys 0.033s
W20260709 13:16:24.787117 31391 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.031s	sys 0.034s
I20260709 13:16:24.788354 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:24.805794 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.809161 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:24.811913 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:24.804450 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:24.758502  4645 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:16:24.822062 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.754392  4454 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.845470  4454 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:24.849978  4784 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:16:24.785506  4403 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.851498  4723 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:16:24.851986  4403 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:24.852480  4730 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.853205  4730 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:24.853696  4730 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:24.781876  4787 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:24.854861  4787 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.855249  4787 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:24.785890  4612 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:24.859230  4788 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:16:24.861436  4655 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:24.861886  4655 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:24.862128  4711 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.566s	user 0.007s	sys 0.004s
I20260709 13:16:24.862658  4711 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:16:24.862831  4655 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.864426  4655 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:24.867069 30557 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:24.866974  4522 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:24.868312 30630 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:24.874809  4403 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:24.879036 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:24.821924 31388 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:24.829847  4764 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:16:24.830978  4780 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:16:24.834431 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:24.835345  4603 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.835561  4705 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.839910  4696 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:24.904222  4696 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:24.904587  4673 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.883421  4764 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:24.906841  4764 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:24.841907  4724 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.908241  4724 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:24.840117 30625 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:24.842924  4457 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.912091  4621 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:24.858996  4612 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.881203 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:24.783314  4309 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:16:24.921034 31396 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:24.821370 31378 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:16:24.890931  4711 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:24.891842  4603 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:24.894848 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 53845.7us
I20260709 13:16:24.902916  4738 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.813342  4671 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:25.000494 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76637.5us
I20260709 13:16:24.905529  4696 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.005378  4696 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:25.006476  4759 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:24.870359  4522 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:25.007113  4759 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:24.902319  4219 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.009562  4759 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.009931  4219 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:25.010802  4765 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.010181  4759 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:25.011356  4765 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:25.011659  4403 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:24.917696 31361 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:24.891938 30555 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 13.
W20260709 13:16:25.012643 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.030s	sys 0.056s
I20260709 13:16:24.991833  4481 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.014569 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.030s	sys 0.056s
I20260709 13:16:25.015537 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:25.015879 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 556306us
I20260709 13:16:25.000383  4612 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.016253 31361 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:25.001273  4250 consensus_queue.cc:1059] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:25.001788  4787 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.002483  4532 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.039088 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:25.047829 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.081213 31361 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:25.082010 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:25.092566 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:25.092796 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:25.118659 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82356.3us
I20260709 13:16:25.121917 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:25.012849  4711 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.130509 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:25.131579 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:25.143764 30631 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:25.145139 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:25.130525 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 107838us
I20260709 13:16:25.133213 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:25.140636 30553 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:25.116967  4671 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:25.159731 30624 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:25.159626  4671 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.164662 30625 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:16:25.139339  4720 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.168336  4720 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.169723  4647 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.170439  4647 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.173574  4647 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.174489  4787 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:25.180228  4765 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:24.885181  4730 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.122948  4532 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.186093  4532 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:25.187067  4309 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:25.188068  4711 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:25.194522 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72038.7us
I20260709 13:16:25.147851 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.216849 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:25.138396  4648 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.022s	sys 0.034s
I20260709 13:16:25.037706  4481 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:25.221127 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73421.7us
I20260709 13:16:25.206805 30559 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:25.222478 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.226693 30625 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:25.230340 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:25.231552 30625 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:25.194031  4643 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.221760  4648 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.022s	sys 0.034s
I20260709 13:16:25.237726 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.258894  4707 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:25.211558  4457 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.264665 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 501804us
I20260709 13:16:25.121727  4603 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:24.916893  4764 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:25.265420 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:25.241151  4717 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.019s	sys 0.034s
I20260709 13:16:25.261693 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:25.220381  4673 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:25.285970 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 461260us
I20260709 13:16:25.235566  4532 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:25.260870 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 480513us
W20260709 13:16:25.302131  4791 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.302s	user 0.008s	sys 0.000s
I20260709 13:16:25.235216  4671 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:25.238862  4765 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:25.264397  4491 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:25.347151  4491 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.347688  4491 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:25.314061 30557 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:25.353247  4765 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:25.354012  4765 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.354557  4765 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:25.266397  4643 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.288206  4457 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:25.333899 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.345854  4309 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:25.327085  4720 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:25.358541  4799 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.308939  4586 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:16:25.343585 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:25.314993 30633 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:25.266085  4648 consensus_queue.cc:1243] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:16:25.368383 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:25.369297 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.363364 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:25.368598 31391 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.024s
W20260709 13:16:25.373762 31294 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.011s	sys 0.028s
W20260709 13:16:25.375742 31294 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.012s	sys 0.029s
W20260709 13:16:25.344043  4717 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.019s	sys 0.035s
I20260709 13:16:25.344637  4532 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:25.373914 31391 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.011s	sys 0.024s
I20260709 13:16:25.373370  4671 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:25.361299  4799 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:25.380044 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 641803us
I20260709 13:16:25.368476  4673 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:25.386255 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:25.380770 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.386554 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1880) took 1282 ms. Trace:
W20260709 13:16:25.327131  4347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.005s	sys 0.030s
I20260709 13:16:25.370915 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:25.399566 30628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.018s
W20260709 13:16:25.402423 30628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.007s	sys 0.018s
I20260709 13:16:25.384083  4532 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:25.407388  4309 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:25.402915 31383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.032s	sys 0.049s
W20260709 13:16:25.417352 31383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.032s	sys 0.049s
W20260709 13:16:25.431147 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60364us
I20260709 13:16:25.432536 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:25.401428  4347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.005s	sys 0.030s
I20260709 13:16:25.418184 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.446291 31292 leader_election.cc:400] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23.
I20260709 13:16:25.446780 31292 leader_election.cc:403] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:25.399729 31391 rpcz_store.cc:276] 0709 13:16:24.104293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.214292 (+109999us) service_pool.cc:224] Handling call
0709 13:16:24.250339 (+ 36047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.250585 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:24.250908 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.386014 (+1135106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:25.386098 (+    84us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:25.386171 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.386288 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.386470 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":114,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":120256,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":1133116,"threads_started":2,"wal-append.queue_time_us":935310}
I20260709 13:16:25.389189  4454 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:25.398190  4671 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:25.403002  4717 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.466572 30551 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:25.412026  4677 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:16:25.441056  4403 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
W20260709 13:16:25.474382 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 236780us
W20260709 13:16:25.528561 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.457469  4804 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:25.531118 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:25.532246 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.535046 31361 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:25.545589 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.479570  4806 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:25.547039  4648 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.547464  4806 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.548038  4806 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:25.547735  4648 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.550534 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:25.553256 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2304) took 1187 ms. Trace:
I20260709 13:16:25.553870 30704 rpcz_store.cc:276] 0709 13:16:24.366104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.366330 (+   226us) service_pool.cc:224] Handling call
0709 13:16:24.367090 (+   760us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:24.749317 (+382227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.552848 (+803531us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54000c7454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:25.553165 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":870858112}
W20260709 13:16:25.555470  4800 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:16:25.557466  4481 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.560247  4648 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:25.561847 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1202) took 1413 ms. Trace:
I20260709 13:16:25.576174 30555 rpcz_store.cc:276] 0709 13:16:24.148068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.213290 (+ 65222us) service_pool.cc:224] Handling call
0709 13:16:25.561591 (+1348301us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54001a9d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:25.561754 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438513024}
I20260709 13:16:25.577885 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:25.578249 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.567138 30556 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 12. Current term is 13. Ops: []
I20260709 13:16:25.581326  4759 raft_consensus.cc:3055] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:25.581837  4759 raft_consensus.cc:740] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:25.582687 30555 raft_consensus.cc:2393] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:25.582801  4759 consensus_queue.cc:260] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.584555 30555 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:25.584566  4759 raft_consensus.cc:3060] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:25.586192  4454 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.535436  4801 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:25.589259  4454 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.587284 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:25.592722  4673 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:25.593868  4649 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:25.594569  4724 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23.
I20260709 13:16:25.595966  4764 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:25.596294  4696 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:16:25.596728  4696 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:25.597620  4696 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.598140  4799 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:25.601567  4454 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:25.604524  4671 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.607321  4696 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:16:25.461510  4717 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.616870  4717 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.617708  4804 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:25.618719  4805 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:25.619597  4805 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.620060  4805 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.623942  4637 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:25.624559  4637 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:25.538477  4603 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.591234 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.632936 31364 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:25.634224  4799 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:25.613998  4491 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:25.642263 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:25.642868 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 97377us
I20260709 13:16:25.595968  4801 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:25.643756  4809 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:25.622351  4805 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:25.657606 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.637595  4799 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:25.667376 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 258202us
I20260709 13:16:25.671824 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:25.672730 30555 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:25.665987  4795 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:25.645620  4103 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:25.670718 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.651955  4738 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:25.683063 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 426610us
I20260709 13:16:25.685664 30553 raft_consensus.cc:1275] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:25.682555 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:25.685760 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 293431us
I20260709 13:16:25.689682 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.690819 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:25.693529 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.694542 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.701172 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:25.666103  4481 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:25.703773 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1835) took 1054 ms. Trace:
I20260709 13:16:25.692589  4673 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:25.702651  4689 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:25.687618  4655 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.823s	user 0.009s	sys 0.007s
I20260709 13:16:25.681165  4103 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.712770 30670 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:25.675906  4811 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:25.643013  4637 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.718483 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:25.721577 30627 raft_consensus.cc:2410] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:16:25.717729  4811 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.710573  4250 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:16:25.723506  4811 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:25.723716  4250 consensus_queue.cc:1243] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:16:25.710248  4717 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 13:16:25.725163  4814 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:16:25.713871  4673 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:25.727103  4603 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:25.731724 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:25.732476 30550 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:25.661689  4804 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.738090  4801 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.726362  4673 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.742904  4816 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.743227  4603 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:25.743597  4799 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.744979  4765 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.746567  4309 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:25.717701  4103 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:25.682057  4795 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.716526 31361 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:16:25.747946  4717 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:25.748396  4717 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:25.712853  4481 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.748764 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.749172  4795 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.749388  4717 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:25.714748  4702 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:25.750633  4481 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.667394  4809 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.750978 30670 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:25.720139  4594 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:25.749835  4610 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:25.731814 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:25.698755  4805 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:25.752628 31367 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:25.752846 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.753098 31367 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:25.746785  4655 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:25.753726 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:25.716233 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:25.759377 30523 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:25.715516  4764 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:25.756270 31367 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.713294  4798 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:25.769706 31367 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:25.721035 31294 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:25.697412  4738 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:25.773689 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54131.5us
I20260709 13:16:25.776736 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2080) took 1435 ms. Trace:
I20260709 13:16:25.789265 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:25.765654  4717 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:25.790657 30556 rpcz_store.cc:276] 0709 13:16:24.341659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.442087 (+100428us) service_pool.cc:224] Handling call
0709 13:16:25.566797 (+1124710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.579053 (+ 12256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.776469 (+197416us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001a9d50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:25.776665 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2529352960}
I20260709 13:16:25.767146  4250 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:25.790970 31391 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:25.764815  4655 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.721391  4689 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:25.796612 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 348986us
I20260709 13:16:25.771443  4798 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:25.811748 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.758157  4454 raft_consensus.cc:2804] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:25.772536  4765 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:25.848006 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.829061 30707 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:16:25.870033 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:25.743628  4816 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.891705 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2089) took 1011 ms. Trace:
I20260709 13:16:25.834919 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.892272 31383 rpcz_store.cc:276] 0709 13:16:24.879876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.880547 (+   671us) service_pool.cc:224] Handling call
0709 13:16:24.895995 (+ 15448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.417876 (+521881us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.417909 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.418468 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:16:25.664145 (+245677us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:25.664203 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.664312 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.891430 (+227118us) raft_consensus.cc:1685] finished
0709 13:16:25.891456 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.891629 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19042,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":744459,"threads_started":2,"wal-append.queue_time_us":399474}
I20260709 13:16:25.826817  4610 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:25.838223 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.828588 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:25.770691  4723 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:25.870371 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 117582us
I20260709 13:16:25.873742 30556 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:25.832158 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:25.878118 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.896258 30703 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:25.884895 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.818504  4798 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:25.888862  4655 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:25.819648  4711 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.592s	user 0.008s	sys 0.006s
I20260709 13:16:25.897475 31394 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:25.901561 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.902436 31388 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:25.759207  4702 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:25.897922  4738 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:25.910365  4711 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:25.759438  4805 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.918177 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.894513  4603 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:25.924094 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:25.873037  4825 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:16:25.947352 31383 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:25.923730 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:25.929523 30550 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:25.888929  4457 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:25.973448 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:25.973352 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:25.983187 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:25.984555 30558 raft_consensus.cc:2368] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25.
I20260709 13:16:25.988210 30698 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:16:25.988909 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:25.903450  4454 raft_consensus.cc:697] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:25.990020 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:25.992067 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:25.992838 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:25.970270  4532 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.001s
I20260709 13:16:25.993696 31389 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:26.000397 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:25.899399  4723 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:25.995502 31383 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:16:25.997354 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:26.001183 30703 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:26.000281 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.001147 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:25.912777  4491 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.944170  4738 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:25.966748  4822 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.006743 31388 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:25.965049  4689 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:25.965909  4403 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:25.969635  4603 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:26.009806 30628 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:26.010258 31394 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:26.023270 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.024044 30632 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:26.026057 30698 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:25.990026  4825 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:26.031931 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:26.018478 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.836s (SENT)
I20260709 13:16:26.012703 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:25.993954  4823 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:26.028681 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:25.915309  4610 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.005507  4809 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:26.032982 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.046321 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.036103 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.040371 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:26.048233 30624 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:26.032912 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:26.043977 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.049963 30629 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:26.053272 30629 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:26.038512 30698 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:26.058079 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.060164 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:26.067341 30698 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:26.002666  4455 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:16:26.071585 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.071475 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.078008 31394 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:26.074287 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2002) took 1235 ms. Trace:
I20260709 13:16:26.027488  4603 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:26.078954 30630 rpcz_store.cc:276] 0709 13:16:24.838435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.838554 (+   119us) service_pool.cc:224] Handling call
0709 13:16:24.838897 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:24.868677 (+ 29780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.073902 (+1205225us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400577954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:26.074200 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67245824}
I20260709 13:16:26.079527 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.083861 31388 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:26.066043 30624 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:26.023797  4491 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:26.082901 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64449.8us
I20260709 13:16:26.089325 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.009172  4532 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.094344 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:26.042522  4823 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:26.095533 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:26.114286 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.120621 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.122996 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.123176 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.123811 31394 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:26.020576  4738 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.023007  4822 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.125661 30631 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 13:16:26.088835  4689 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:26.087736  4655 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.137916 30701 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:26.139191 30559 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:26.132843 30698 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:26.135298 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.146927 31388 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:16:26.126268 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 144411us
I20260709 13:16:26.149448 31388 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:26.145088 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.151306 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.153671 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:26.078305  4649 consensus_queue.cc:1048] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:16:26.154680 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.158002 30629 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:26.146449 30631 pending_rounds.cc:85] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 10
I20260709 13:16:26.165508 30631 pending_rounds.cc:107] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 21.11
I20260709 13:16:26.044672  4806 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:26.109941  4823 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:26.166740 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.175552 31383 raft_consensus.cc:2410] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 24 because replica has last-logged OpId of term: 21 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 9.
I20260709 13:16:26.169660 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.126341  4703 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:26.169660 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.180949 30701 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:26.182207 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.172693 30631 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:26.138163  4621 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:26.099740  4795 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:16:26.176101 31394 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:26.176033 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:26.095986  4491 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.174155  4825 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.231419  4720 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:26.232077  4720 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:26.232973  4720 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.234112  4720 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:26.122260  4811 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:26.237051 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.237535 30554 raft_consensus.cc:2393] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:26.195396 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.199064 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.193298  4703 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.210820  4711 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:26.211083  4455 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:26.213459  4806 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:26.214201 31388 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:26.157498  4655 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:26.167024  4828 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:16:26.215404  4835 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:26.215483  4843 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:26.216508  4822 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:26.398051 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:26.220590 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.222977  4730 raft_consensus.cc:2764] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:16:26.101100  4454 consensus_queue.cc:237] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:25.704348 30705 rpcz_store.cc:276] 0709 13:16:24.649278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.649427 (+   149us) service_pool.cc:224] Handling call
0709 13:16:24.666066 (+ 16639us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 13:16:24.702045 (+ 35979us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:24.702086 (+    41us) raft_consensus.cc:1539] Triggering prepare for 5 ops
0709 13:16:24.843109 (+141023us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b2c00285df8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:24.844117 (+  1008us) log.cc:844] Serialized 130 byte log entry
0709 13:16:25.303187 (+459070us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:25.651950 (+348763us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.652666 (+   716us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.662860 (+ 10194us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.688820 (+ 25960us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.702930 (+ 14110us) log.cc:844] Serialized 14 byte log entry
0709 13:16:25.703265 (+   335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:25.703379 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:25.703435 (+    56us) raft_consensus.cc:1685] finished
0709 13:16:25.703454 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:25.703693 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21924,"prepare.queue_time_us":309565,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":2753,"raft.queue_time_us":59084,"raft.run_cpu_time_us":807,"raft.run_wall_time_us":30450,"spinlock_wait_cycles":31661312,"thread_start_us":800402,"threads_started":2,"wal-append.queue_time_us":617526}
I20260709 13:16:26.193967  4621 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:26.238591 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.409941 30555 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:26.248095 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:26.252219 31383 raft_consensus.cc:2376] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23. Re-sending same reply.
I20260709 13:16:26.258294 31391 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:26.282876 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:26.286973 31367 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.003s
W20260709 13:16:26.287658 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 125087us
I20260709 13:16:26.411653 31367 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:16:26.412093 31367 pending_rounds.cc:85] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:26.273561  4703 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:26.419345 30556 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:16:26.423354 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 175420us
I20260709 13:16:26.424847 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.426921  4841 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:26.427139  4724 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:26.427829  4637 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:26.429081 30631 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:26.339182 30624 pending_rounds.cc:85] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 8
I20260709 13:16:26.331611  4532 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:26.434738 30624 pending_rounds.cc:107] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:16:26.435220 30624 raft_consensus.cc:2889] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:26.243389  4837 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:16:26.326946  4491 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:26.127671  4723 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.341691  4805 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:26.239738  4830 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:16:26.249078  4831 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:16:26.341872 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1203) took 1565 ms. Trace:
I20260709 13:16:26.452683 30551 rpcz_store.cc:276] 0709 13:16:24.776697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.776905 (+   208us) service_pool.cc:224] Handling call
0709 13:16:24.786792 (+  9887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:25.471488 (+684696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.341458 (+869970us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b540028a0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:26.341775 (+   317us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1562866048}
I20260709 13:16:26.336664 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.248359  4702 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:26.237063  4811 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:26.370138 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.340651  4655 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:26.383505 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.412320 31367 pending_rounds.cc:107] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:16:26.475687 31367 raft_consensus.cc:2889] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:26.478010 31367 pending_rounds.cc:85] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 10
I20260709 13:16:26.478327 31367 pending_rounds.cc:107] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:16:26.414288  4677 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:26.438025 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:26.358038  4816 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:26.362285  4219 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:26.443990 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.444113 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:26.448467 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 303487us
I20260709 13:16:26.414036  4823 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:26.460803 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.462529 31388 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: [21.9-21.9]
W20260709 13:16:26.465740 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.478605 31367 raft_consensus.cc:2889] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:26.480562 31389 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:26.458204  4643 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:26.490819 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46841.9us
I20260709 13:16:26.460415 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.499429 31394 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:26.494541 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 56631us
I20260709 13:16:26.491032 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:26.506948 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:26.506155  4621 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.481784  4677 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:26.511234 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.512682 31389 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:16:26.492282  4717 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.004s
I20260709 13:16:26.517452 31388 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:26.517515 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:26.522114 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.523762 30706 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:26.503417  4219 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.504022  4828 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:26.524317 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:26.528875 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.516893  4621 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:26.518194  4595 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:26.543005 31394 raft_consensus.cc:2376] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17. Re-sending same reply.
I20260709 13:16:26.514261  4610 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:26.567750  4844 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:16:26.531101  4724 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:16:26.523594  4586 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:26.547272 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:26.524098  4805 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:16:26.533795  4798 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.627s	user 0.015s	sys 0.000s
I20260709 13:16:26.550484  4823 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:26.540665  4854 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.532923  4738 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:26.546730  4595 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.541376  4828 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:26.552629  4841 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:26.552757  4219 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:26.553820  4849 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.130s	user 0.003s	sys 0.000s
W20260709 13:16:26.556772  4839 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:16:26.557572  4603 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:26.721936  4648 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:26.517645  4822 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 13:16:26.665779 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.648244  4717 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:26.650674 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:26.812248 31364 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:26.648553 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.531209  4677 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:26.689445  4836 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.405s	user 0.000s	sys 0.010s
W20260709 13:16:26.754455  4847 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.343s	user 0.000s	sys 0.002s
W20260709 13:16:26.761464 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:26.676918 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:26.758826  4838 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.306s	user 0.002s	sys 0.000s
I20260709 13:16:26.754663  4843 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:26.772030 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 224827us
I20260709 13:16:26.783223 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.785141 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:26.795095 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:26.831724 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46681.2us
I20260709 13:16:26.763332  4804 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.802673 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.805199 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.813416 30551 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:26.832756 30558 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:26.832968 31294 leader_election.cc:400] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25.
I20260709 13:16:26.792392  4671 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:26.791929  4828 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:26.820590 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:26.800601  4403 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:26.828303 30702 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:26.784564  4595 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:26.833369 31294 leader_election.cc:403] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 13:16:26.835577 30559 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2085) took 1483 ms (client timeout 1630 ms). Trace:
I20260709 13:16:26.772323  4798 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:26.805414  4648 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:26.836201 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:26.838701 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:26.832072  4850 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.376s	user 0.003s	sys 0.000s
W20260709 13:16:26.844272 30559 rpcz_store.cc:269] 0709 13:16:25.351900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.550158 (+198258us) service_pool.cc:224] Handling call
0709 13:16:26.835370 (+1285212us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b5400242554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:26.835515 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350830464}
I20260709 13:16:26.780699  4854 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.834277  4671 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:26.837078  4811 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:26.857766 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:26.808825  4738 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:16:26.823006  4730 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:16:26.838725  4841 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:26.887928 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.894729 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:26.900127 30596 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:26.904062 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:26.887121 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:26.823676  4717 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:26.912110 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2106) took 1157 ms. Trace:
W20260709 13:16:26.860927  4103 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.113s	user 0.017s	sys 0.000s
I20260709 13:16:26.914894 31361 rpcz_store.cc:276] 0709 13:16:25.755201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.755978 (+   777us) service_pool.cc:224] Handling call
0709 13:16:25.756334 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:25.756576 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:25.756602 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:25.765404 (+  8802us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.089827 (+324423us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:26.089890 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.089964 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.555865 (+465901us) spinlock_profiling.cc:243] Waited 8.54 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:26.880733 (+324868us) raft_consensus.cc:1685] finished
0709 13:16:26.880776 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.911992 (+ 31216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19641728,"thread_start_us":295906,"threads_started":1,"wal-append.queue_time_us":655211}
I20260709 13:16:26.853849  4805 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:26.839577  4491 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:16:26.837718  4843 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.858533  4724 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:26.879231  4671 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:26.929308 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.856233  4816 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:16:26.838191  4804 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:26.909097  4825 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:26.853416  4829 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:16:26.959129 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:26.919776  4219 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:26.961261 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.967084 30703 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:26.951896 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:26.972878 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.975749 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:26.993448 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1925) took 1079 ms. Trace:
I20260709 13:16:26.994654 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:26.995508 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:26.996770 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45087.4us
I20260709 13:16:26.995657 30625 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:27.001652  4189 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:26.993052  4481 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.993269  4347 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.007125 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.007839  4347 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.009536 30703 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:27.010215  4347 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:27.010951 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.011662 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:27.015748 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.015604  4738 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:26.928776  4103 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.016327  4825 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:16:27.033953 30624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.053s	sys 0.095s
W20260709 13:16:27.051980 30624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.053s	sys 0.095s
I20260709 13:16:27.049899 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1472) took 1919 ms. Trace:
I20260709 13:16:27.018795  4717 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:27.019018 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:27.020545  4843 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.021148 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.023054 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:27.024340  4845 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.608s	user 0.000s	sys 0.003s
I20260709 13:16:27.031342 30631 raft_consensus.cc:2410] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 25 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:16:27.028143 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:27.029361 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:26.994115 31378 rpcz_store.cc:276] 0709 13:16:25.914052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.916145 (+  2093us) service_pool.cc:224] Handling call
0709 13:16:25.916735 (+   590us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:26.120375 (+203640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.120420 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:26.121427 (+  1007us) log.cc:844] Serialized 52 byte log entry
0709 13:16:26.246388 (+124961us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b48000f3fd8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180be06aa1 00007f180bdfdb37 00007f180bdfd9ca 00007f180bef526f 00007f180bef4f37 00007f180bf0955c 00007f180bf09508 00007f180bf09488 00007f180bf09443 00007f180bf086ec 00007f180ae32a96
0709 13:16:26.438674 (+192286us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:26.438723 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.438796 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:26.780269 (+341473us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:26.993192 (+212923us) raft_consensus.cc:1685] finished
0709 13:16:26.993218 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:26.993382 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91433,"raft.run_cpu_time_us":2976,"raft.run_wall_time_us":233093,"spinlock_wait_cycles":32903680,"thread_start_us":439717,"threads_started":2,"wal-append.queue_time_us":304250}
I20260709 13:16:27.017133 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.053711 30627 rpcz_store.cc:276] 0709 13:16:25.130054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.130192 (+   138us) service_pool.cc:224] Handling call
0709 13:16:27.032168 (+1901976us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:27.049828 (+ 17660us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329274752}
W20260709 13:16:27.033107 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.033442 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:26.970134  4816 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:27.050396  4825 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:27.091573 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.092478 30630 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:27.093542 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.094264 30553 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:27.095319  4860 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 13:16:27.095441  4603 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:26.935418  4648 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.091835  4825 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:27.098166 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.097849 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:27.098869 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69607.5us
I20260709 13:16:26.987088  4724 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:26.993893  4861 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.223s	user 0.000s	sys 0.002s
I20260709 13:16:27.119642 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:27.056082  4799 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:27.100282 30625 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:27.100648 31361 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.123142 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:27.109967 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81914.4us
I20260709 13:16:27.123826 31388 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:27.060370 30628 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:27.117460 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.113718 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:27.125926 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.099951 31391 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.129592 30630 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:27.118563 30629 raft_consensus.cc:2368] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:16:27.121094 30633 raft_consensus.cc:3060] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:27.098745  4720 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.862s	user 0.016s	sys 0.000s
I20260709 13:16:27.140509 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.141736 30555 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
W20260709 13:16:27.134795  4862 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:16:27.135180  4833 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:27.127910 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:27.142460 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.144613 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.151851 31364 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:27.153273 30630 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:27.099388 30703 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:16:27.119602  4858 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:16:27.163483 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:27.032582  4851 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:27.195729  4851 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:27.195861 30632 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:27.196455  4851 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.197000  4851 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.197432  4671 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.203205  4671 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.204161  4816 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.204768 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.205646 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 1940) took 1030 ms. Trace:
I20260709 13:16:27.205998 31396 rpcz_store.cc:276] 0709 13:16:26.175678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.329343 (+153665us) service_pool.cc:224] Handling call
0709 13:16:27.204579 (+875236us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800330398. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:16:27.205575 (+   996us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5521280}
I20260709 13:16:27.147403  4854 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.208454  4854 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.209012  4854 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.212159 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2308) took 1247 ms. Trace:
I20260709 13:16:27.212630 30702 rpcz_store.cc:276] 0709 13:16:25.964348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.976673 (+ 12325us) service_pool.cc:224] Handling call
0709 13:16:26.005577 (+ 28904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.839730 (+834153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.211882 (+372152us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b540008cad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:27.212092 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1891420416}
I20260709 13:16:27.213114  4805 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25.
I20260709 13:16:27.164125  4833 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.214365  4833 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:27.152174 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:27.220148 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88527.1us
I20260709 13:16:27.151851 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:27.221696 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69951.2us
I20260709 13:16:27.223091 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:27.007241  4481 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.171069 30633 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:27.113917  4219 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:27.173321 30631 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:27.119621  4347 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:16:27.177022  4103 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:27.177508  4637 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.001s
I20260709 13:16:27.177994  4828 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.180060  4595 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:27.244866  4595 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:27.183938  4804 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.056126  4738 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.187199  4798 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.188076 30625 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:27.263149 30707 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:27.129796  4648 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:27.263960  4637 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:27.264561  4637 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:27.125301  4724 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:27.165155  4703 consensus_meta.cc:287] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.746s	user 0.014s	sys 0.002s
W20260709 13:16:27.265471 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 132073us
I20260709 13:16:27.175499 31391 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
W20260709 13:16:27.234057 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.162242 31361 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:27.246001 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1483) took 1154 ms. Trace:
I20260709 13:16:27.267793 30624 rpcz_store.cc:276] 0709 13:16:26.091306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.338276 (+246970us) service_pool.cc:224] Handling call
0709 13:16:26.338844 (+   568us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:26.436166 (+ 97322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.436202 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:26.436862 (+   660us) log.cc:844] Serialized 52 byte log entry
0709 13:16:27.052384 (+615522us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:27.052433 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.052526 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:27.245729 (+193203us) raft_consensus.cc:1685] finished
0709 13:16:27.245754 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:27.245931 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":128,"thread_start_us":615033,"threads_started":1,"wal-append.queue_time_us":806027}
I20260709 13:16:27.119840  4764 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:27.247337 30626 raft_consensus.cc:2410] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:16:27.257289 31388 raft_consensus.cc:2468] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
W20260709 13:16:27.230755 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 346603us
I20260709 13:16:27.276026 30631 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:27.273957 30627 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:27.264955  4637 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.282958  4841 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:27.285732 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62737us
I20260709 13:16:27.236397  4491 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:27.229221  4219 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:27.308388 31394 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:27.294543 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:27.292014 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 392355us
I20260709 13:16:27.310635 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:27.263156  4798 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:27.311379 30702 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:27.313227 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:27.309441 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:27.315490 30559 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:27.316676 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.236188  4720 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:16:27.334247 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:27.327900 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.339468 30703 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.339990 31388 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 24: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:27.353879 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.322733 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:27.355394 31388 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:16:27.356082 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61621.1us
I20260709 13:16:27.349634 30702 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:27.357939 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.360383 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.363723 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.364784 31361 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:27.354638 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44127.2us
I20260709 13:16:27.318949 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.279731  4703 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:27.312817  4219 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:27.373409 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:27.294144  4861 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:27.374761 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.375828 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:27.380827 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.381541 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.391511 30702 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:27.381614 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.382699 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:27.383569 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.392303 31396 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:27.316109  4798 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:27.388022 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:27.397408 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:27.333515  4870 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:16:27.359937  4843 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:27.400673 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.408371 30701 raft_consensus.cc:2393] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:27.404071 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:27.410053 30554 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:27.406447 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:27.360143  4491 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:27.406555 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.390672 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.437119 31364 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:16:27.439213 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.433557 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.441152 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:27.437103 30702 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
W20260709 13:16:27.415370 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 125841us
I20260709 13:16:27.447901 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:27.449596 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:27.431177 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:27.377828  4854 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:27.374022  4866 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:27.457134 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.369619  4882 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.471187 30673 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.130:39473 timed out after 1.630s (SENT) [suppressed 19 similar messages]
I20260709 13:16:27.401144  4798 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:27.476811 31364 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:27.374810  4532 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 13:16:27.401341  4805 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:27.407996  4738 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:27.455897  4877 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:27.399665  4841 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.461433  4347 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.481241  4677 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:27.524685 30556 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:16:27.531065 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81529.4us
I20260709 13:16:27.461802  4828 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.543000 30703 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:27.511348  4871 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.524537  4829 raft_consensus.cc:2764] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 13:16:27.578183 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 241678us
I20260709 13:16:27.528505  4764 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:27.574976 30522 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:27.531360  4886 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:27.585770 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:27.637136 30550 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:16:27.645848 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.646693 31364 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:16:27.557267  4805 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.595135  4724 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:27.669137 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.660316 31364 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:27.526784  4882 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:27.575672  4347 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:27.635298  4803 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.606s	user 0.019s	sys 0.011s
I20260709 13:16:27.632997  4481 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:27.675995 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:27.677842 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 304983us
I20260709 13:16:27.637909  4764 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:27.652493  4738 raft_consensus.cc:2804] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:27.585654  4871 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.665980  4829 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.685271 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 254082us
I20260709 13:16:27.713703 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.692655  4347 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:27.698546 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.700786 30624 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:16:27.702189 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:27.722024 30625 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:27.722301 31292 leader_election.cc:400] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:16:27.704452 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:27.662175  4886 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:27.639537  4828 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:27.556239  4491 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.683544  4703 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:27.636664  4833 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:27.668061  4816 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.715894 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:27.679329  4803 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.664s	user 0.020s	sys 0.012s
I20260709 13:16:27.723636 31292 leader_election.cc:403] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:27.759641 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 47148.3us
I20260709 13:16:27.726404  4724 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:27.726452 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2319) took 1682 ms. Trace:
I20260709 13:16:27.738788 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:27.711413 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:27.781002 30706 rpcz_store.cc:276] 0709 13:16:26.044159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.071020 (+ 26861us) service_pool.cc:224] Handling call
0709 13:16:26.071600 (+   580us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:26.524212 (+452612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.726119 (+1201907us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b540008b454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:27.726360 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1036377344}
W20260709 13:16:27.782407 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 71132.7us
I20260709 13:16:27.720271 30629 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
W20260709 13:16:27.770532 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.769894 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:27.788259 30554 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:27.741379  4347 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.791971 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 69813.3us
I20260709 13:16:27.745208  4103 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.731428  4738 raft_consensus.cc:697] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:27.795082 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:27.797367 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.805279 31396 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:27.721378  4829 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.790400  4720 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:27.748386  4828 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:27.820422 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.842589  4805 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:27.848241  4803 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.854705 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 464098us
I20260709 13:16:27.832446 30551 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:16:27.826092 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56239.7us
I20260709 13:16:27.821130 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:27.856452 30552 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:27.861162 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.861243 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:27.775257  4816 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.869419 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:27.874925 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:27.855556  4885 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
I20260709 13:16:27.872822  4712 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:27.872666 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.865005 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.874349 30706 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:27.892244 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97241.9us
I20260709 13:16:27.899956 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:27.875134  4881 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
W20260709 13:16:27.876073  4899 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 13:16:27.875720  4904 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:27.888315 30558 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 13:16:27.932163 30558 pending_rounds.cc:85] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 13
I20260709 13:16:27.932612 30558 pending_rounds.cc:107] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:16:27.933081 30558 raft_consensus.cc:2889] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:27.934034 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:27.875095  4880 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:16:27.899062 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:27.910094  4892 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:16:27.876211  4884 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:27.912226  4895 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:16:27.851528  4481 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:16:27.822161  4347 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:27.916684  4816 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:27.875792  4868 raft_consensus.cc:2764] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:16:27.882512 30554 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 13:16:27.929675 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:27.875622  4874 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
W20260709 13:16:27.895845  4825 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.005s
I20260709 13:16:27.890430  4720 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:27.875177  4894 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:27.945884 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 80984.9us
W20260709 13:16:27.917630  4872 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
I20260709 13:16:27.823148  4103 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.889540  4867 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:27.821709  4764 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.904932  4712 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:27.875255  4897 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.336s
I20260709 13:16:27.881582  4803 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.844373  4886 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:27.901651  4738 consensus_queue.cc:237] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:27.910203  4890 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 13:16:27.983311 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 512097us
W20260709 13:16:27.990859 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 129807us
W20260709 13:16:28.149348 30626 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2013) took 2112 ms (client timeout 1763 ms). Trace:
W20260709 13:16:28.064548 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 165640us
W20260709 13:16:28.249440 30626 rpcz_store.cc:269] 0709 13:16:26.036688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.073084 (+ 36396us) service_pool.cc:224] Handling call
0709 13:16:28.148993 (+2075909us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54004a83d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.149245 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2647846656}
W20260709 13:16:28.244185 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.167995 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.961s (SENT)
I20260709 13:16:28.251305 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:28.016670 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 219400us
I20260709 13:16:28.252046 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:28.252327 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:28.243623 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:27.982491 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:28.132367 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1245) took 1081 ms. Trace:
W20260709 13:16:28.227464 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 293542us
I20260709 13:16:27.880792  4888 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:28.263144 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:27.918023  4724 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:28.304800 30625 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:28.221266  4904 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:28.361184  4904 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:28.361711  4904 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:28.015462  4828 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:28.074265  4899 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:16:28.084877  4876 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 13:16:28.125074  4798 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.593s	user 0.009s	sys 0.003s
W20260709 13:16:28.122911  4637 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.007s
I20260709 13:16:28.300836  4867 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.393776  4637 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.138965  4871 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.309353 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:28.267742 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:28.092984  4907 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:28.245353  4825 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.348345 31396 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 13:16:28.349215  4896 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:28.370762  4874 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:28.350894  4927 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:27.933142  4804 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.358381 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 2112) took 1339 ms. Trace:
W20260709 13:16:28.360077  4853 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.826s	user 0.008s	sys 0.005s
I20260709 13:16:28.426582 31394 rpcz_store.cc:276] 0709 13:16:27.019143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.144312 (+125169us) service_pool.cc:224] Handling call
0709 13:16:28.358114 (+1213802us) spinlock_profiling.cc:243] Waited 67.4 ms on lock 0x7b540031ddd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.358259 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155020032}
I20260709 13:16:28.360237  4103 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:28.084218  4910 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:28.197258 30706 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:28.083748  4920 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:28.221194  4930 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:28.255522 30628 rpcz_store.cc:276] 0709 13:16:27.050725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.174176 (+123451us) service_pool.cc:224] Handling call
0709 13:16:27.174733 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.175050 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.175089 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.193762 (+ 18673us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.356547 (+162785us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:27.356585 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.356657 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.131994 (+775337us) raft_consensus.cc:1685] finished
0709 13:16:28.132033 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.132268 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":162302,"threads_started":1,"wal-append.queue_time_us":287214}
I20260709 13:16:28.007548  4884 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.148273  4900 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.059056  4882 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.138484  4833 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:28.294750  4923 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:16:28.294296  4586 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.524s	user 0.015s	sys 0.003s
I20260709 13:16:28.120322  4916 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:28.296453 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:28.401630 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 133975us
I20260709 13:16:28.470016 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:28.265776  4894 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:28.293617  4915 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:28.305202  4829 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.350770  4677 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.299563  4696 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:28.382156  4888 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.475322  4696 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.477310  4888 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.477669  4696 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:28.404472  4219 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.005s	user 0.013s	sys 0.002s
I20260709 13:16:28.456892  4867 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:28.478482  4219 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.479064  4867 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.479660 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:28.480839  4219 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.481549  4871 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:28.483353  4926 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:16:28.401072  4872 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:16:28.418337 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:28.403708  4717 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.481567  4867 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.487308  4803 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.487694 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1492) took 1393 ms. Trace:
W20260709 13:16:28.475324  4909 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.254s	user 0.000s	sys 0.001s
I20260709 13:16:28.488135 30627 rpcz_store.cc:276] 0709 13:16:27.098328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.117066 (+ 18738us) service_pool.cc:224] Handling call
0709 13:16:28.487483 (+1370417us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54005b13d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.487623 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264448256}
W20260709 13:16:28.471833  4648 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.208s	user 0.014s	sys 0.002s
W20260709 13:16:28.435016 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172815us
I20260709 13:16:28.490370  4904 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.491567 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:28.492684  4803 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:28.493247  4803 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:28.494019 30630 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:28.495422 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1248) took 1402 ms. Trace:
I20260709 13:16:28.495826 30630 rpcz_store.cc:276] 0709 13:16:27.092871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.191467 (+ 98596us) service_pool.cc:224] Handling call
0709 13:16:28.493606 (+1302139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.494462 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:28.495165 (+   703us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54005b13d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.495346 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2993236864}
I20260709 13:16:28.497640 31389 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:28.498766 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2312) took 2503 ms. Trace:
I20260709 13:16:28.499439 30700 rpcz_store.cc:276] 0709 13:16:25.995863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.998101 (+  2238us) service_pool.cc:224] Handling call
0709 13:16:25.998544 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.146580 (+148036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.146633 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.153043 (+  6410us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.375430 (+222387us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:16:26.375495 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.375620 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.359536 (+1983916us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b2400329d38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:16:28.498400 (+138864us) raft_consensus.cc:1685] finished
0709 13:16:28.498424 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.498693 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8338,"raft.queue_time_us":76346,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":9764,"spinlock_wait_cycles":23936896,"thread_start_us":320475,"threads_started":2,"wal-append.queue_time_us":310566}
W20260709 13:16:28.499841 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 246180us
I20260709 13:16:28.504318 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:28.422303 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 254382us
I20260709 13:16:28.122093  4925 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:28.399085  4798 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.401153  4703 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:16:28.419402  4934 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.460566 30598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.021s	sys 0.041s
I20260709 13:16:28.430172  4907 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:28.428565  4874 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.459071  4930 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:28.461620  4851 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.264s	user 0.015s	sys 0.005s
I20260709 13:16:28.471671 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1087) took 1421 ms. Trace:
I20260709 13:16:28.472005  4894 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:28.474340  4931 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:16:28.496290  4904 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.507725 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:28.510900 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:28.494264  4803 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.433319  4804 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.482427  4720 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:28.518540 30598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.022s	sys 0.041s
W20260709 13:16:28.565413 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:28.524451 30671 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.763s (SENT)
W20260709 13:16:28.527041 30701 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2321) took 2427 ms (client timeout 2047 ms). Trace:
W20260709 13:16:28.567324 30701 rpcz_store.cc:269] 0709 13:16:26.099550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.194708 (+ 95158us) service_pool.cc:224] Handling call
0709 13:16:28.526737 (+2332029us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400342454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.526951 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659272576}
I20260709 13:16:28.530236 30699 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:28.430583  4927 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.499451  4586 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:28.540011 30706 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:28.569603 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.548617 30633 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:28.511842 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:28.536810 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:28.573393 31394 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:28.505618  4103 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:28.534907 30705 rpcz_store.cc:276] 0709 13:16:27.050200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.050984 (+   784us) service_pool.cc:224] Handling call
0709 13:16:27.114283 (+ 63299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.115477 (+  1194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.115514 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.127666 (+ 12152us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.273105 (+145439us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:27.480841 (+207736us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.495425 (+ 14584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.495677 (+   252us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.471426 (+975749us) raft_consensus.cc:1685] finished
0709 13:16:28.471451 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.471605 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12809,"prepare.queue_time_us":565282,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":105641,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6427,"thread_start_us":295495,"threads_started":2,"wal-append.queue_time_us":680170}
W20260709 13:16:28.579326 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 77140.3us
I20260709 13:16:28.581363 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:28.587093 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:28.587147 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50414.5us
I20260709 13:16:28.466228  4900 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.597167 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.598173 30706 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:28.599974 30698 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:28.564484  4720 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:28.603796 31389 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:28.564831  4925 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:28.605880 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.558688  4648 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 13:16:28.613445 30553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2579) took 2929 ms (client timeout 2972 ms). Trace:
I20260709 13:16:28.593004  4103 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.591941  4347 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.557615  4851 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:28.614064 30553 rpcz_store.cc:269] 0709 13:16:25.679209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.712209 (+ 33000us) service_pool.cc:224] Handling call
0709 13:16:28.608393 (+2896184us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54009ca4d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.608581 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3084602368}
I20260709 13:16:28.620050 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:28.597512  4637 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.531944  4934 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.623481 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:28.593631  4936 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:28.609704  4720 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:28.602694  4875 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:28.630414 30700 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:16:28.635677 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.640674 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:28.626281  4907 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:16:28.664908 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:28.657018 30630 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:28.667765 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:28.636420 30701 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:28.642514 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.670042 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:28.672794 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:28.674743 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.671015 30705 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:28.675096 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.660554  4934 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.662209  4103 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.677058 30630 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:28.682824  4851 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.684216 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:28.685922 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:28.626325  4940 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:28.686841 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:28.657987  4875 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.657702  4900 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.656035  4930 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.687731  4673 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:28.648399  4874 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.626497  4717 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:16:28.633316  4347 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:28.690205  4877 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.126s	user 0.017s	sys 0.000s
I20260709 13:16:28.690747 30627 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:28.690994  4936 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:28.631908  4825 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.643050  4898 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.642319  4918 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:28.712417 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44313.7us
I20260709 13:16:28.714161 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:28.671617  4833 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:28.718132  4454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.254s	user 0.011s	sys 0.014s
I20260709 13:16:28.696668  4219 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:16:28.736138 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:28.728940  4940 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.723685  4738 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:28.742647 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:28.733157  4454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.269s	user 0.011s	sys 0.014s
I20260709 13:16:28.731741  4930 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:28.755183 30671 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:28.729975  4877 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.731307  4874 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:28.763845 30704 raft_consensus.cc:1240] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:28.738503  4918 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.751040  4940 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:28.769673  4939 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:28.770280  4939 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.761543  4930 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.771157  4930 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:28.726686  4347 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:28.748106  4798 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.747881  4884 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.775689  4939 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:28.751047  4738 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:28.798146  4738 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:28.800362  4347 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.801513  4594 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:28.802410  4594 consensus_queue.cc:1243] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:16:28.781637  4918 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:28.782425  4823 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:28.803140  4823 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:28.783429  4875 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.719794  4925 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.778340  4936 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:28.793040  4671 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:28.786942  4874 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:28.811129  4896 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:28.812414  4940 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.733748  4898 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:28.807591  4703 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 13:16:28.809962  4948 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:16:28.816036  4859 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.888s	user 0.002s	sys 0.004s
I20260709 13:16:28.816144  4948 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:28.816596  4898 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.817731 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1905) took 2658 ms. Trace:
I20260709 13:16:28.817412  4948 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:28.818869  4948 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:28.804948 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1082) took 1910 ms. Trace:
I20260709 13:16:28.812909  4874 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:28.807827 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:28.809371  4403 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:28.813257  4671 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:28.807382  4825 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:28.818150 31367 rpcz_store.cc:276] 0709 13:16:26.159002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.477249 (+318247us) service_pool.cc:224] Handling call
0709 13:16:26.477776 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.517729 (+ 39953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:26.517768 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:26.524992 (+  7224us) log.cc:844] Serialized 26 byte log entry
0709 13:16:26.798200 (+273208us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:26.798260 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:26.798491 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:28.815421 (+2016930us) spinlock_profiling.cc:243] Waited 5.32 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:28.817368 (+  1947us) raft_consensus.cc:1685] finished
0709 13:16:28.817396 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:28.817656 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12245888,"thread_start_us":240439,"threads_started":1,"wal-append.queue_time_us":300256}
I20260709 13:16:28.844769 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:28.760775  4822 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.823464  4703 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:28.824254  4904 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:28.847558  4703 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:28.806793  4871 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:28.807654  4804 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:28.825734  4867 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:28.806895  4720 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.826329  4896 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:16:28.827526  4868 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:28.831871  4724 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:28.829586 31394 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:28.833127 31389 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:28.847615  4904 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:28.818774  4823 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.847813 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:28.810587  4841 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:28.848837  4703 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.027789  4939 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.842810  4677 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:29.037788  4833 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:28.905179  4825 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.060346  4677 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.058657  4673 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:29.060765  4724 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.069005  4724 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:29.069265  4825 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.071123  4677 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:29.062052  4703 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:29.019739  4712 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.855548  4927 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.063597  4841 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.063792  4904 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.007602  4720 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.073618  4904 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:28.966988  4671 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.074652  4841 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:28.841012  4930 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:28.910415  4738 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.076480  4938 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:29.060177  4939 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:29.076969  4930 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.075127  4671 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:29.077005  4738 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.077042  4938 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:29.084569  4938 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.085702  4823 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:29.086004  4938 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:29.089090  4915 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:29.089923  4915 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:29.128579 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.010s	sys 0.002s
W20260709 13:16:29.129740 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.011s	sys 0.002s
I20260709 13:16:28.393517 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:16:29.130650 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 510578us
W20260709 13:16:29.137719 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.152415 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:29.162781 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.164920 30552 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1187) took 5027 ms (client timeout 1887 ms). Trace:
I20260709 13:16:29.166472 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.167472 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.167706 30557 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:29.168640 30553 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:29.168556 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:29.165659 30552 rpcz_store.cc:269] 0709 13:16:24.137468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.137591 (+   123us) service_pool.cc:224] Handling call
0709 13:16:29.164195 (+5026604us) spinlock_profiling.cc:243] Waited 3.3 s on lock 0x7b54003b38d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:29.164567 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7594600192}
W20260709 13:16:29.177361 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:29.177583 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2098) took 2417 ms (client timeout 2089 ms). Trace:
W20260709 13:16:29.178238 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.175918 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2584) took 1816 ms. Trace:
W20260709 13:16:29.178958 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 373212us
I20260709 13:16:29.178957 30550 rpcz_store.cc:276] 0709 13:16:27.358830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.359007 (+   177us) service_pool.cc:224] Handling call
0709 13:16:27.383352 (+ 24345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.637696 (+254344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.174912 (+1537216us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b5400112954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.175559 (+   647us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":582694784}
I20260709 13:16:29.182279 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.183912 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1210) took 2382 ms. Trace:
I20260709 13:16:29.183959 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:29.184733 30556 rpcz_store.cc:276] 0709 13:16:26.801787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.801972 (+   185us) service_pool.cc:224] Handling call
0709 13:16:26.804336 (+  2364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.565318 (+760982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.182986 (+1617668us) spinlock_profiling.cc:243] Waited 720 ms on lock 0x7b54001aa254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.183591 (+   605us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1655848704}
I20260709 13:16:29.185663 30630 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:29.185515 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1243) took 2189 ms. Trace:
I20260709 13:16:28.806782 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:29.189078 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2023) took 1479 ms. Trace:
I20260709 13:16:28.819945 30707 rpcz_store.cc:276] 0709 13:16:26.894301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.975338 (+ 81037us) service_pool.cc:224] Handling call
0709 13:16:28.804741 (+1829403us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400343854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:28.804884 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620611840}
W20260709 13:16:29.178035 30555 rpcz_store.cc:269] 0709 13:16:26.760382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.762833 (+  2451us) service_pool.cc:224] Handling call
0709 13:16:29.177046 (+2414213us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b5400626454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:29.177361 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713175936}
W20260709 13:16:29.204792 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:28.836750 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:28.843729  4403 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.189745 30633 rpcz_store.cc:276] 0709 13:16:27.709215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.723408 (+ 14193us) service_pool.cc:224] Handling call
0709 13:16:27.745450 (+ 22042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.572647 (+827197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.188328 (+615681us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400055ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.188810 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1846476544}
I20260709 13:16:29.266872 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.268235 30625 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:29.186111 30632 rpcz_store.cc:276] 0709 13:16:26.996206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.050591 (+ 54385us) service_pool.cc:224] Handling call
0709 13:16:27.091942 (+ 41351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.196640 (+104698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.184708 (+1988068us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400052154. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.185196 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238094720}
W20260709 13:16:29.271636 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.205493 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.226756  4532 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.227258 31367 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:29.228224 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1079) took 2338 ms. Trace:
I20260709 13:16:29.229740 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1901) took 3071 ms. Trace:
I20260709 13:16:29.232686  4712 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:29.235414  4219 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:29.235585  4829 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.236398  4720 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:29.237663 30382 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 19, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.243674 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:29.244462  4738 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.245354 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77430.2us
I20260709 13:16:29.245510  4894 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:29.246377  4764 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.247184 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.247359 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:29.250896  4869 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.262554 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.267998 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:29.202507 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50460.1us
I20260709 13:16:29.053349 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:28.911350 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.280887 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.282579 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 445774us
I20260709 13:16:29.287221 30707 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:29.281248 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.297914 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 150246us
I20260709 13:16:29.281884 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:29.290908 30699 rpcz_store.cc:276] 0709 13:16:26.890035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.999439 (+109404us) service_pool.cc:224] Handling call
0709 13:16:28.528298 (+1528859us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.568663 (+ 40365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.227559 (+658896us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400342450. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:29.227911 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3289553280}
I20260709 13:16:29.286453  4841 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:29.279301  4891 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:29.275116  4882 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.361280  4882 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.362067  4936 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:29.363055  4936 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.362139 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2111) took 2343 ms. Trace:
I20260709 13:16:29.362150 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 1947) took 1052 ms. Trace:
W20260709 13:16:29.284235 30377 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 145) took 57514 ms (client timeout 15000 ms). Trace:
I20260709 13:16:29.365656 31364 rpcz_store.cc:276] 0709 13:16:28.309153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.309282 (+   129us) service_pool.cc:224] Handling call
0709 13:16:28.309738 (+   456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.481963 (+172225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.482002 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:28.482053 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:28.482090 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.361516 (+879426us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b5400717ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.361587 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.361874 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395461504}
I20260709 13:16:29.363078 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:29.368608 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.368814 30699 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:29.369139  4738 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.370877  4219 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.375048 30550 raft_consensus.cc:2393] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:29.376364 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.377180 30550 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:29.378925  4703 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:16:29.379750 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 132854us
I20260709 13:16:29.276538  4904 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:16:29.346892 30630 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:29.291605  4712 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.383427  4904 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:29.383875  4904 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.384363  4904 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:29.348593 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:29.386181 30633 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:29.342484 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.350908  4869 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.352169 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.352701 30552 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:29.354096 31388 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:29.356925  4403 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.357446 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:29.359786 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.365263 31378 rpcz_store.cc:276] 0709 13:16:27.018827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.201181 (+182354us) service_pool.cc:224] Handling call
0709 13:16:27.201711 (+   530us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.328538 (+126827us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.328594 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.355253 (+ 26659us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.480284 (+125031us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:27.480410 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.361246 (+1880836us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400145fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.361358 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.361437 (+    79us) raft_consensus.cc:1685] finished
0709 13:16:29.361459 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.361936 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20620,"raft.queue_time_us":33458,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":158,"spinlock_wait_cycles":242689408,"thread_start_us":124065,"threads_started":1,"wal-append.queue_time_us":296320}
I20260709 13:16:29.369760 31364 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:29.383671  4891 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.382390 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.349519  4829 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.343350 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:29.378463 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.408977 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82183.3us
I20260709 13:16:29.350292 30627 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:16:29.351263 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:29.391917 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.354131 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.414083 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.287916  4532 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:29.427434 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76482.8us
I20260709 13:16:29.429078  4532 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:29.430053  4532 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:29.422802 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.420578 30627 pending_rounds.cc:85] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
W20260709 13:16:29.437999 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.434543 30558 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:29.437165  4532 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:29.374387  4219 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.434151 30627 pending_rounds.cc:107] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 13.10
W20260709 13:16:29.452567 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 117481us
I20260709 13:16:29.466882 30627 raft_consensus.cc:2889] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:29.474431 31378 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:29.448078  4841 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:29.477384 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.482542 30554 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:29.486469 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.358420  4764 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.488425 30631 raft_consensus.cc:2393] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:29.492831 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.493256 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47707.2us
I20260709 13:16:29.493955 30551 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
W20260709 13:16:29.494207 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 671311us
I20260709 13:16:29.494398  4927 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:29.489523  4894 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:29.496172  4894 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.497977  4455 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:29.291033 31383 rpcz_store.cc:276] 0709 13:16:26.157821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.443668 (+285847us) service_pool.cc:224] Handling call
0709 13:16:26.449522 (+  5854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:26.452979 (+  3457us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:26.489369 (+ 36390us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.019526 (+530157us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b1800439f00. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:16:27.281602 (+262076us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.282319 (+   717us) log.cc:844] Serialized 26 byte log entry
0709 13:16:27.282488 (+   169us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:27.284886 (+  2398us) log.cc:844] Serialized 14 byte log entry
0709 13:16:27.285130 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:27.285213 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.080984 (+1795771us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b240052aa68. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2efb0 00007f1806aa5488 00007f1806aa5be8 00007f1806a7d398 00007f1806a7876a 00007f1806a77bb6
0709 13:16:29.228935 (+147951us) raft_consensus.cc:1685] finished
0709 13:16:29.228989 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.229367 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2855,"prepare.queue_time_us":40166,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":112,"raft.queue_time_us":14578,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":211,"spinlock_wait_cycles":32509056,"thread_start_us":261884,"threads_started":1,"wal-append.queue_time_us":1024008}
I20260709 13:16:29.499153 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.500069 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.471302  4712 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.500182 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:29.500779 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.503546 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:29.504985 30707 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:29.501158 31383 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:16:29.507414 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:29.507783 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.508550 31391 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:29.509351 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.512660  4869 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:29.366883 30377 rpcz_store.cc:269] 0709 13:15:31.769450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:31.769769 (+   319us) service_pool.cc:224] Handling call
0709 13:16:29.283881 (+57514112us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:21.309064 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:21.484850 (+175786us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:21.484927 (+    77us) write_op.cc:435] Acquired schema lock
0709 13:16:21.484951 (+    24us) tablet.cc:662] Decoding operations
0709 13:16:21.488139 (+  3188us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:21.488252 (+   113us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:21.488285 (+    33us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:16:21.489015 (+   730us) tablet.cc:701] Row locks acquired
0709 13:16:21.489068 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:16:21.489270 (+   202us) write_op.cc:270] Start()
0709 13:16:21.489441 (+   171us) write_op.cc:276] Timestamp: P: 1783602981489192 usec, L: 0
0709 13:16:21.489480 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:16:21.490348 (+   868us) log.cc:844] Serialized 2463 byte log entry
0709 13:16:22.673658 (+1183310us) op_driver.cc:464] REPLICATION: finished
0709 13:16:23.036326 (+362668us) write_op.cc:301] APPLY: starting
0709 13:16:23.036500 (+   174us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:23.041486 (+  4986us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:23.041629 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:23.220160 (+178531us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:23.220501 (+   341us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:16:23.220576 (+    75us) write_op.cc:312] APPLY: finished
0709 13:16:23.220956 (+   380us) log.cc:844] Serialized 64 byte log entry
0709 13:16:23.221193 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:23.221441 (+   248us) write_op.cc:454] Released schema lock
0709 13:16:23.221530 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:23.222079 (+   549us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:23.222125 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1364536,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":330519,"apply.run_cpu_time_us":13812,"apply.run_wall_time_us":194347,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29267,"num_ops":6,"prepare.queue_time_us":219397,"prepare.run_cpu_time_us":102516,"prepare.run_wall_time_us":471735,"raft.queue_time_us":350221,"raft.run_cpu_time_us":21848,"raft.run_wall_time_us":392645,"replication_time_us":1131761,"thread_start_us":705185,"threads_started":4,"wal-append.queue_time_us":309399,"wal-append.run_cpu_time_us":19859,"wal-append.run_wall_time_us":2459604}]]}
I20260709 13:16:29.521111 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.484543 31361 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.027s	sys 0.070s
I20260709 13:16:29.491845 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.522877 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:29.475759  4882 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.531889 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:29.532140 31361 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.029s	sys 0.070s
I20260709 13:16:29.431313  4720 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.528764 30632 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:29.540410  4912 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:29.538229 30557 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 13.
I20260709 13:16:29.280036  4915 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.526680  4455 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:29.526576 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:29.553637  4403 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.556445 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:29.557265 30555 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:29.557523 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:29.559594 30552 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
W20260709 13:16:29.555073 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 79321.2us
I20260709 13:16:29.545310  4869 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:29.549763  4455 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:29.554229 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:29.568264 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:29.585579 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.578405  4829 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:29.586964 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.591164  4703 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.593420 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.592207  4703 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:29.601059 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.578545  4219 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.601686 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 244572us
I20260709 13:16:29.600768 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:29.607532 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.608870  4884 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:29.609786  4884 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.610908  4703 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.592651  4712 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:29.619252  4712 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:29.619782  4712 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:29.608384 30558 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:29.624925  4696 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:29.625026  4882 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:29.600881  4764 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:29.576030  4532 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.598335  4869 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:29.643913 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.627223  4882 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:29.657019  4309 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:29.631863 30382 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 12 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.658000  4309 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:29.658937 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 104946us
I20260709 13:16:29.657037 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:29.595539  4403 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.663280  4309 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:29.664234 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:29.665616  4403 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:29.666921  4894 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.667053 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.669883  4884 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.673834 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.682837 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.679879 30633 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:29.649773 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.692298 31388 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:16:29.694900 30699 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:29.704051 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 215460us
I20260709 13:16:29.601227  4841 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.706983  4916 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:29.712698  4869 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:29.655179  4219 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.691653 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:29.698567 30555 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:29.685436 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:29.702919 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:29.737149 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:29.740442 30552 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:29.741215 31388 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:29.722500  4219 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.722597  4720 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:29.738451 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 101929us
I20260709 13:16:29.747763 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.723608  4738 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:29.720464  4455 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.752346 31367 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
W20260709 13:16:29.756863 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.737142  4309 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:16:29.759500  4829 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.767779  4829 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.752099  4869 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:29.774752 31396 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:29.775127  4869 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:29.775522  4869 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:29.779160  4309 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:29.779738 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:29.779841  4309 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:29.781193  4927 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:29.781785  4927 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:29.782349  4927 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:29.782058  4309 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.784183  4309 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:29.767782 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 91386.5us
I20260709 13:16:29.787711 30382 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 12, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 12 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:29.789928  4738 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.790820  4738 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.791273  4455 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:29.791553 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 127500us
I20260709 13:16:29.794059 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:29.744385  4894 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:29.742600  4891 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:29.795221  4829 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.852732 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:29.853991 30555 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:29.790450 30382 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:29.867051  4966 log.cc:927] Time spent T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:16:29.877828  4965 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.291s	user 0.003s	sys 0.000s
I20260709 13:16:29.743934  4882 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.886790  4971 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:29.887570  4971 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:29.889619 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.890164  4971 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:29.820642  4891 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:29.910646 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:29.891901 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:29.886277  4823 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.800s	user 0.009s	sys 0.012s
I20260709 13:16:29.893900 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1842) took 4071 ms. Trace:
I20260709 13:16:29.921833 30550 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:29.890471 30557 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:16:29.886816  4964 log.cc:927] Time spent T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.570s	user 0.004s	sys 0.000s
W20260709 13:16:29.894443  4967 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.576s	user 0.004s	sys 0.000s
W20260709 13:16:29.894049  4938 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.807s	user 0.007s	sys 0.015s
I20260709 13:16:29.940639 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:29.817713  4927 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:29.890756  4934 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:29.950907 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:29.932240 30704 rpcz_store.cc:276] 0709 13:16:25.821890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:25.822027 (+   137us) service_pool.cc:224] Handling call
0709 13:16:25.822356 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.764575 (+2942219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.893185 (+1128610us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b540021fa54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:29.893601 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6764340096}
I20260709 13:16:29.857368  4829 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 13:16:29.891335 30382 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:16:29.980252 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 186327us
I20260709 13:16:29.972911 30700 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:29.990670 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2583) took 2658 ms. Trace:
W20260709 13:16:29.991524  4904 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.607s	user 0.017s	sys 0.000s
I20260709 13:16:29.991786 30553 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:29.992123  4904 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:29.991279 30559 rpcz_store.cc:276] 0709 13:16:27.332093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:27.333255 (+  1162us) service_pool.cc:224] Handling call
0709 13:16:27.333692 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:27.459733 (+126041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:27.459773 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:27.475777 (+ 16004us) log.cc:844] Serialized 26 byte log entry
0709 13:16:29.172800 (+1697023us) spinlock_profiling.cc:243] Waited 3.58 ms on lock 0x7b18003cbf80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:29.426332 (+253532us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:29.441345 (+ 15013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.472696 (+ 31351us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.885749 (+413053us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:29.990172 (+104423us) raft_consensus.cc:1685] finished
0709 13:16:29.990199 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:29.990436 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26293,"raft.queue_time_us":140237,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":12047,"spinlock_wait_cycles":23643392,"thread_start_us":222135,"threads_started":2,"wal-append.queue_time_us":1839496}
I20260709 13:16:30.006706 31383 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:30.007582  4869 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:29.950840  4891 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.013139  4891 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:30.015172  4671 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.937s	user 0.015s	sys 0.010s
I20260709 13:16:30.016618 31364 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:30.017776 30629 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:30.019866 30382 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:16:29.998205 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.005467  4904 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:30.023860 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 620511us
I20260709 13:16:30.027312  4971 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:30.022094 30382 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.037206 30382 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 11 to 16, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 16 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:30.038056 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.039856  4894 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.039752 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:30.046406 30704 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:30.046200  4894 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:30.040644  4927 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.041566 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:30.042469 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.043624 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.042301 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:30.043675 30702 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
I20260709 13:16:30.010385  4869 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:30.067080 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.068264 30629 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
I20260709 13:16:30.073624 30707 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
W20260709 13:16:30.093372 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 311965us
W20260709 13:16:30.121364 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.070024  4823 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:16:30.082286  4894 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:30.124713 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46041.4us
I20260709 13:16:30.126542 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.127609 30626 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:30.127161 30382 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 11 to 15, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.121893  4455 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:30.129436  4455 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.131541 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.132938 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:30.135339  4938 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:30.135790  4938 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.136116  4938 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:30.137579 31394 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:30.139036  4934 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:30.140337 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2333) took 1431 ms. Trace:
I20260709 13:16:30.140799 30705 rpcz_store.cc:276] 0709 13:16:28.638048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.741314 (+103266us) service_pool.cc:224] Handling call
0709 13:16:28.754292 (+ 12978us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:28.762394 (+  8102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:28.762429 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:28.767399 (+  4970us) log.cc:844] Serialized 52 byte log entry
0709 13:16:29.317458 (+550059us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:29.318451 (+   993us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.495280 (+176829us) log.cc:844] Serialized 14 byte log entry
0709 13:16:29.495552 (+   272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:29.495686 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:29.893395 (+397709us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:30.069330 (+175935us) raft_consensus.cc:1685] finished
0709 13:16:30.069375 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.069599 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175833,"prepare.queue_time_us":4916,"prepare.run_cpu_time_us":226,"prepare.run_wall_time_us":4474,"raft.queue_time_us":186556,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":5402,"spinlock_wait_cycles":48196992,"thread_start_us":41332,"threads_started":1,"wal-append.queue_time_us":42034}
I20260709 13:16:30.146965 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.148524  4455 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:30.151222  4936 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:30.152184  4936 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:30.129984 30382 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 13, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 13 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:16:30.168195 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.168982 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:30.170104 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.173655 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.171583 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.177915 31394 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:16:30.178512 31394 pending_rounds.cc:85] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 11
I20260709 13:16:30.182518 31394 pending_rounds.cc:107] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:16:30.154183  4811 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:30.183039 31394 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.187095 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:30.183168 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.187876 30704 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:30.193403 30557 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:30.198495  4916 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:30.202940 31383 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:30.191356  4811 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.146950  4927 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:16:30.218472 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:30.211573 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78755.1us
I20260709 13:16:30.221583 31378 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:30.227286 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.217603  4811 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.226364 30382 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 6 to 17. New cstate: current_term: 17 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.240579 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.243937 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.201804  4532 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:16:30.245930 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.251643 30627 pending_rounds.cc:85] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:16:30.257895 30627 pending_rounds.cc:107] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:16:30.258293 30627 raft_consensus.cc:2889] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.259791 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.267280 31383 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:30.266170 31378 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:16:30.268730 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 226587us
I20260709 13:16:30.272059 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:30.281939 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.272605  4823 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:30.277590 30382 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:30.286070 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:30.278769  4712 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.658s	user 0.019s	sys 0.000s
I20260709 13:16:30.291698 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1929) took 1458 ms. Trace:
I20260709 13:16:30.300297 30626 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:30.301227 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73999.4us
I20260709 13:16:30.300127 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.308717 31388 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:16:30.290148  4309 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:30.309505 30382 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 10 to 16, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:16:30.294787  4823 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:30.315325  4730 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.316287  4730 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.318229 30631 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:30.292455 31361 rpcz_store.cc:276] 0709 13:16:28.832822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:28.832948 (+   126us) service_pool.cc:224] Handling call
0709 13:16:28.833235 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:28.833464 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:29.551314 (+717850us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.265748 (+714434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.272742 (+  6994us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:30.272799 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.280126 (+  7327us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.291629 (+ 11503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6727,"prepare.queue_time_us":983073,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":50487,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":1318754,"threads_started":3,"wal-append.queue_time_us":546490}
W20260709 13:16:30.308856 30599 connection.cc:256] RPC call timeout handler was delayed by -1.0141638850000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2972s for a response
I20260709 13:16:30.285584  4869 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:16:30.300961  4712 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:30.335477  4985 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.336246  4985 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.311947  4927 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.340670 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:30.342839  4870 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.343156 30555 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:30.344480  4986 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:30.365037 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 1870) took 1167 ms. Trace:
I20260709 13:16:30.345036 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.350772  4915 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.351047  4934 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.354813  4916 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:30.355041 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.355580 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.330284  4309 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:30.356990  4977 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:16:30.373487  4870 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.379351 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.379648  4696 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:30.380158 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:30.357496  4730 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.320883  4823 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:30.371718  4916 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.366947 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1932) took 1094 ms. Trace:
I20260709 13:16:30.367831  4986 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:30.368767  4974 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:30.368964 30626 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:30.369621 31378 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:30.396991 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:30.369524 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:30.355847 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55620us
I20260709 13:16:30.374287 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2104) took 1091 ms. Trace:
I20260709 13:16:30.399226 30556 rpcz_store.cc:276] 0709 13:16:29.278398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.281394 (+  2996us) service_pool.cc:224] Handling call
0709 13:16:30.339086 (+1057692us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b5400081ca8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccba924 00007f180bea3227 00007f180bea2ab8 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6
0709 13:16:30.369908 (+ 30822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19075840}
I20260709 13:16:30.380187  4696 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:30.385247 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 2131) took 1112 ms. Trace:
I20260709 13:16:30.398542  4916 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:30.404572 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.383620 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2112) took 1008 ms. Trace:
I20260709 13:16:30.407310  4309 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.424705 30551 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:30.425344 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1933) took 1151 ms. Trace:
I20260709 13:16:30.389837  4988 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:30.392517 30628 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:30.391623  4738 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.426402  4988 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.427631 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.451855 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.382998  4927 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:30.389268  4934 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:30.404932  4981 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:16:30.455399 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:30.405488  4976 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
W20260709 13:16:30.424515  4970 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.516s	user 0.001s	sys 0.001s
I20260709 13:16:30.365717 30700 rpcz_store.cc:276] 0709 13:16:29.193202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.204727 (+ 11525us) service_pool.cc:224] Handling call
0709 13:16:29.542513 (+337786us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:16:30.360554 (+818041us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b540032b254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:30.360744 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601271552}
I20260709 13:16:30.456785 30555 raft_consensus.cc:2410] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 23 because replica has last-logged OpId of term: 21 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:16:30.368976 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.458403 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.459666 30630 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 23: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:30.460783 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.393373 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.415787 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:30.413360  4455 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.413959 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.415547  4696 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:30.408466 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.415423 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:30.395385 31391 rpcz_store.cc:276] 0709 13:16:29.273131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.577289 (+304158us) service_pool.cc:224] Handling call
0709 13:16:29.616678 (+ 39389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:29.672914 (+ 56236us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:30.054968 (+382054us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.366136 (+311168us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.366489 (+   353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.366550 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:30.366634 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.366722 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.366879 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38067,"prepare.queue_time_us":233122,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":1975,"raft.queue_time_us":47332,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5216,"spinlock_wait_cycles":48818944,"thread_start_us":559878,"threads_started":2,"wal-append.queue_time_us":257774}
I20260709 13:16:30.425756 31389 rpcz_store.cc:276] 0709 13:16:29.273577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.623424 (+349847us) service_pool.cc:224] Handling call
0709 13:16:29.635728 (+ 12304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:29.671169 (+ 35441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:29.671203 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:29.672079 (+   876us) log.cc:844] Serialized 52 byte log entry
0709 13:16:30.072580 (+400501us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:30.072643 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.073453 (+   810us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.424957 (+351504us) raft_consensus.cc:1685] finished
0709 13:16:30.424982 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.425138 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":392127,"threads_started":1,"wal-append.queue_time_us":598197}
I20260709 13:16:30.470422  4219 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:30.416531 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 89177.6us
I20260709 13:16:30.419344  4870 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:30.415740 31364 rpcz_store.cc:276] 0709 13:16:29.272894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.500398 (+227504us) service_pool.cc:224] Handling call
0709 13:16:30.375806 (+875408us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400460ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:30.385176 (+  9370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1183691008}
I20260709 13:16:30.423650 30557 rpcz_store.cc:276] 0709 13:16:29.368385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.951849 (+583464us) service_pool.cc:224] Handling call
0709 13:16:30.189880 (+238031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.196350 (+  6470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.376268 (+179918us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b540028a5d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:30.376447 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505014144}
I20260709 13:16:30.417215 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.439983  4738 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:30.444258  4841 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:30.480587 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:30.360828  4971 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.453819 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:30.460897 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.387428  4730 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:30.431110 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44020.1us
I20260709 13:16:30.483532 30626 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:30.484169  4971 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.484545 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:30.488984  4886 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:30.489609  4886 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.490000  4886 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.489835  4491 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:30.491659  4869 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:30.488670 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.493631  4219 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:30.494045  4219 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.481523  4841 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:30.494514  4219 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:30.484225  4986 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.495348  4841 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.496949  4841 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:30.498541  4971 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:30.501616  4927 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:30.502874  4712 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:30.405916 30382 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 18, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:16:30.506178 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90537.3us
I20260709 13:16:30.507253 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:30.508222 30630 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:30.504456 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:30.521466 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 176661us
I20260709 13:16:30.530647 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:30.530433 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 82004us
I20260709 13:16:30.525636  4730 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.534349 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:30.534194  4730 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.535364 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.536006 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.536733  4730 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:30.537356  4730 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.537856 30559 pending_rounds.cc:85] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 9
I20260709 13:16:30.538345 30559 pending_rounds.cc:107] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:16:30.463028  4455 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.533496  4870 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:30.531834 30382 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 11 to 15, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:30.538691 30559 raft_consensus.cc:2889] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.547199  4403 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.547994  4403 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.551584  4712 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:30.552009  4455 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:30.552325  4712 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:30.553542  4712 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.541478  4934 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:30.556375  4870 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.557003  4870 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:30.539000 30628 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:30.559054 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.559348 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.561219 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.559140  4982 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:30.562908 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.563966 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:30.564257  4982 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:30.564674 31396 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.564751  4982 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.568459 31294 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:30.568761  4986 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.569085  4671 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.553969  4985 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.569736  4988 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.579324  4833 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.540767 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:30.589196 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48528.2us
I20260709 13:16:30.575472  4934 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:30.575635  4986 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.576778  4649 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:30.577358  4904 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:30.526217  4971 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:30.580412 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73246.8us
I20260709 13:16:30.577297 31391 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:16:30.537959  4730 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.589766  4671 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.593370  4833 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.595080 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.595856  4936 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:30.571336  4673 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 13:16:30.590109 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:30.611198 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.609879 30557 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:30.591483 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2116) took 1221 ms. Trace:
I20260709 13:16:30.604626 30382 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 7 to 11, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 11 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:16:30.614667  4927 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:30.600306  4971 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:30.619982 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51670.9us
I20260709 13:16:30.598553  4986 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.608341  4455 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:30.623718  4927 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:30.610515  4936 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.625128 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:30.583848  4829 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
W20260709 13:16:30.654974  4938 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.518s	user 0.014s	sys 0.005s
I20260709 13:16:30.658257  4938 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.659287  4833 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.660727  4938 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:30.662917 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61959.7us
I20260709 13:16:30.664381  4980 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:30.655428 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:30.655730  4971 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:30.655927  4927 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.616870  4649 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:30.613945 30550 rpcz_store.cc:276] 0709 13:16:29.370214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.959760 (+589546us) service_pool.cc:224] Handling call
0709 13:16:29.984215 (+ 24455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.057924 (+ 73709us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:30.266340 (+208416us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.590937 (+324597us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:30.591012 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:30.591086 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.591195 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.591395 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":261761,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":8428,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"thread_start_us":469312,"threads_started":2}
W20260709 13:16:30.685225 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 60050.6us
I20260709 13:16:30.686751 31396 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:30.683817 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1507) took 1397 ms. Trace:
I20260709 13:16:30.646973  4455 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:30.667987  4936 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:30.697192 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.698191 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:30.712507 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 151419us
I20260709 13:16:30.718320 31367 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:30.695343 30625 rpcz_store.cc:276] 0709 13:16:29.286817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.299611 (+ 12794us) service_pool.cc:224] Handling call
0709 13:16:30.126769 (+827158us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b580050a998. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:16:30.127136 (+   367us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:30.127842 (+   706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.127873 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:30.128635 (+   762us) log.cc:844] Serialized 52 byte log entry
0709 13:16:30.291356 (+162721us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:30.611954 (+320598us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.612817 (+   863us) log.cc:844] Serialized 14 byte log entry
0709 13:16:30.618319 (+  5502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.618438 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.648721 (+ 30283us) raft_consensus.cc:1685] finished
0709 13:16:30.648752 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.683687 (+ 34935us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2865,"prepare.queue_time_us":107750,"prepare.run_cpu_time_us":114,"prepare.run_wall_time_us":3222,"raft.queue_time_us":8963,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":3539,"spinlock_wait_cycles":10070144,"thread_start_us":464185,"threads_started":2,"wal-append.queue_time_us":291324}
I20260709 13:16:30.714306 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:30.734354 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.685138  4649 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:30.622434  4988 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 13:16:30.690790  4738 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:30.747697 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84015us
I20260709 13:16:30.751965 30698 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:16:30.752521 30698 pending_rounds.cc:85] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 11
I20260709 13:16:30.752832 30698 pending_rounds.cc:107] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:16:30.753129 30698 raft_consensus.cc:2889] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:30.743947 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:30.758996  4730 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.759904  4870 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.735556  4696 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.762753  4673 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:30.764946  4696 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:30.766022 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.766865 30706 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:30.767287  4455 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.767974 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:30.768544  4990 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:16:30.768697 31367 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:30.769125  4870 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:30.769842 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.772073 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1222) took 1158 ms. Trace:
I20260709 13:16:30.772570 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.772536 30553 rpcz_store.cc:276] 0709 13:16:29.613177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:29.992930 (+379753us) service_pool.cc:224] Handling call
0709 13:16:29.993403 (+   473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:29.994646 (+  1243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.005483 (+ 10837us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:30.352627 (+347144us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:30.353195 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 13:16:30.603589 (+250394us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:30.603630 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.603702 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:30.771681 (+167979us) raft_consensus.cc:1685] finished
0709 13:16:30.771706 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:30.771995 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":481,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":157,"spinlock_wait_cycles":97255680,"thread_start_us":249897,"threads_started":1,"wal-append.queue_time_us":411492}
I20260709 13:16:30.773903 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.779260 31367 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:30.781642 31367 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:16:30.744499  4738 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:30.783491  4738 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:30.791492 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.792349 30706 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:30.792701  4986 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:30.793217 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.793350  4986 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:30.805994 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:30.742488  4649 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:30.742259  4869 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:30.830291 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 207637us
I20260709 13:16:30.757371 31389 raft_consensus.cc:2376] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20. Re-sending same reply.
I20260709 13:16:30.711038  4829 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.112s
I20260709 13:16:30.829814  4980 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:30.864226 30632 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:16:30.843847  4869 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:30.854312  4811 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.008s
W20260709 13:16:30.873060 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 304544us
I20260709 13:16:30.880216 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.831162  4982 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:30.833726  4916 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:30.892716  4403 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:30.897029 30706 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:30.861056  4986 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.908059 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:30.910959 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 221380us
I20260709 13:16:30.852841  4823 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.914601 30703 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:30.918179 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.861578  4730 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:16:30.931070 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.897524  4671 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:30.881467  4811 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:30.946161 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.947695 30705 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 14: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:30.951087 30700 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:30.953122 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.954166 31396 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
W20260709 13:16:30.950021 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 229756us
I20260709 13:16:30.962173 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.971827 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:30.972856 31389 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:30.978576  5005 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:16:30.979545  5002 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 13:16:30.981395  5001 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:16:30.981880 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:30.983916  4823 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:30.984130 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.985893 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.987154 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.989080 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:30.979070 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:30.967057 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:30.997958 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.004225 31396 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:30.999495 31394 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:31.010159 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.010576 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:30.938701  4886 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:31.024093 30698 raft_consensus.cc:2368] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23.
I20260709 13:16:30.978848  4979 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:31.050634 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 257730us
I20260709 13:16:31.076483 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.088485  4904 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.107331 30706 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:16:31.060702 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 254970us
I20260709 13:16:31.094703 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:30.964177  4980 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:30.976612  4869 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:31.013171 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.037104 31394 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:16:31.030309 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.116871 30704 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:31.041314 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:31.077913 30698 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:16:31.007723  4219 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.004s
W20260709 13:16:31.097604  5013 log.cc:927] Time spent T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:16:31.114003 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:31.112982 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:31.129287  4829 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:31.129961  4829 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:31.130414  4829 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:31.130852  5009 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:16:31.134816  4309 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.074013  4977 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.077003  5015 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:31.033047  5016 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.122779  4673 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:31.163690 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113749us
W20260709 13:16:31.165614 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 252257us
W20260709 13:16:31.177294 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 136103us
I20260709 13:16:31.148782  4915 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:31.135555  4982 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:31.155715  5015 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.198685 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.198770 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:31.194703 31389 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:31.202693 30705 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:31.202833 30553 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:31.205062 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.211280 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:31.211735 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:31.216477 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:31.122720  4904 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.218473 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:31.220567 30554 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:31.178727  4403 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:31.195381  4916 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:31.137432  4219 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.223153 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:31.199834  4915 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.155546  4309 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.228216 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:31.231570 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:31.207003  5015 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:31.276569 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 46043.6us
I20260709 13:16:31.178098  4882 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
W20260709 13:16:31.228531  5007 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.002s
I20260709 13:16:31.201524  4823 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:16:31.198864  4971 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:16:31.225338  4811 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:31.303470 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76668.5us
I20260709 13:16:31.229831  4915 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:31.163193  4977 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.320273 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:31.305565 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:31.172499  5016 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:31.316860 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:31.340274 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 143212us
I20260709 13:16:31.341535 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.233141  4403 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.232868  4936 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.230582  4738 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:31.345031 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 187004us
I20260709 13:16:31.347200 31391 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:31.348827 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:31.350935 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:31.334781  4886 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:31.316964  4870 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:16:31.338550  5006 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:31.316362  4916 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:16:31.338359  4982 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.371791 30552 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:31.375247 30627 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:31.386860 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:31.387761 31378 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:31.391659  4309 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.354677  5043 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:31.397030  4738 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:31.399252  5020 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 13:16:31.401373  5028 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:16:31.402544  4934 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:31.403283  4934 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:31.403476  5027 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:31.403771  4934 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:31.405741  4649 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.565s	user 0.006s	sys 0.006s
W20260709 13:16:31.374401  5014 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 13:16:31.415349  4904 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.369342  5006 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.420454 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2041) took 1149 ms. Trace:
I20260709 13:16:31.429457 30632 rpcz_store.cc:276] 0709 13:16:30.270860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.270997 (+   137us) service_pool.cc:224] Handling call
0709 13:16:30.283778 (+ 12781us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:30.864793 (+581015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.417687 (+552894us) spinlock_profiling.cc:243] Waited 574 ms on lock 0x7b54004aabd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:31.420350 (+  2663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1320264576}
W20260709 13:16:31.428412 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50548.4us
I20260709 13:16:31.403304  4977 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:31.448624 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:31.438252 30523 leader_election.cc:336] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.512s (SENT)
I20260709 13:16:31.374949  4982 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.406410  4649 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.434746  5006 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:31.493187  5051 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:31.495040  5034 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:16:31.493378 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55196.3us
I20260709 13:16:31.499449 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:31.500473 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52018.4us
I20260709 13:16:31.500800  4977 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:31.502763  5046 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:31.504757  5042 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:31.505285  5042 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:31.506534  5042 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.509723  5042 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:31.510298  4936 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:31.511521  5030 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:31.512082  4403 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:31.513756  5016 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:31.490566  4219 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.502328 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:31.517283  4934 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.519470 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:31.520857  4219 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:16:31.512320 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:31.517963  5019 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 13:16:31.514490 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:31.530552 31378 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:31.531404 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:31.532480  5030 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:31.532816  4712 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:31.545825 30554 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:31.561069 30553 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:31.586186 30523 leader_election.cc:400] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23.
I20260709 13:16:31.588344 30523 leader_election.cc:403] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:16:31.529807  4219 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.548288  4936 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:16:31.595173 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 58588.5us
I20260709 13:16:31.560968  4916 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:31.610695 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 91375.2us
I20260709 13:16:31.591833  5053 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:31.612430 30703 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.020s	sys 0.029s
W20260709 13:16:31.610404 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 274426us
W20260709 13:16:31.614219 30703 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.020s	sys 0.030s
I20260709 13:16:31.605643 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:31.616576 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:31.618417 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:31.616999  4455 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:31.620837  5008 log.cc:927] Time spent T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.285s	user 0.000s	sys 0.002s
I20260709 13:16:31.622242  4886 raft_consensus.cc:2764] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 13:16:31.623293  4886 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:31.593518  4988 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23.
I20260709 13:16:31.626379 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2158) took 1084 ms. Trace:
I20260709 13:16:31.630460 31383 rpcz_store.cc:276] 0709 13:16:30.542063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.556279 (+ 14216us) service_pool.cc:224] Handling call
0709 13:16:30.556659 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:30.570378 (+ 13719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.570415 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:30.571235 (+   820us) log.cc:844] Serialized 52 byte log entry
0709 13:16:30.797322 (+226087us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:30.797390 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:30.797505 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:31.626035 (+828530us) raft_consensus.cc:1685] finished
0709 13:16:31.626064 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:31.626297 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13147,"raft.queue_time_us":13335,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":225531,"threads_started":1,"wal-append.queue_time_us":538804}
I20260709 13:16:31.593144  4219 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:31.636255 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.629703  4738 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:31.654742 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:31.655102 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.659191 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:31.646049 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:31.660244 30704 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:31.614367  4916 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:31.667327 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 361810us
I20260709 13:16:31.644505  4936 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:31.651706  4738 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.650476  4649 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:31.678750 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:31.627557  5053 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.687861 30705 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:31.695204 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49266.3us
I20260709 13:16:31.697781 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:31.702823 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 397749us
I20260709 13:16:31.704190 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:31.709367 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:31.651005  4915 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.706249  4982 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.685014  4738 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:31.682790  4904 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.716602 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:31.686745  4980 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:31.721627 30704 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:31.724638 30706 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
W20260709 13:16:31.722330 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 100088us
I20260709 13:16:31.731062 30628 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 15: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:31.728181 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:31.731046 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:31.710472  4823 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:31.743494  4649 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:16:31.746901 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 49272.6us
I20260709 13:16:31.734150  4982 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:16:31.754573 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:31.755492 31389 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:31.758503 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.759500 30704 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:31.760224  5017 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:31.760761  5017 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:31.761711  5017 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.763975 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.765054 30705 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 21 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:16:31.767259 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.768550 30704 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:16:31.771561 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.772421 30704 raft_consensus.cc:2368] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:31.772179 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.776314 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:31.772986 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.778342 31391 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:31.737397  4927 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:31.779757  4927 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:31.784188 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.785070  4649 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:31.725914  5025 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:31.753839  4673 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.003s
I20260709 13:16:31.720831  5038 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:31.784173 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.788018 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:31.789582 30624 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:31.790227 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.811944 30707 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:16:31.815636 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 109836us
I20260709 13:16:31.729815  4980 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:31.807116 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:31.817152 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 89067.6us
I20260709 13:16:31.817584 30705 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:31.726111  4936 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:31.819479 30628 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:31.824117 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76322.8us
I20260709 13:16:31.810792  4673 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:31.827451  5017 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:31.827654  4927 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:31.730950  4904 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.789871  4971 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:31.832501 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:31.831427  4916 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.855087 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.857178 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.858071 30701 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:16:31.858937 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.820040  4980 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:31.862895 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:31.867594 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 128269us
I20260709 13:16:31.869503 30701 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:31.869927 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.782511  5058 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.105s
I20260709 13:16:31.883212 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.879097 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.825078  4938 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:31.883212 30554 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:31.884730 30553 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:16:31.878196 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46870.8us
W20260709 13:16:31.889858  5059 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 13:16:31.891714  5015 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.582s	user 0.006s	sys 0.006s
I20260709 13:16:31.892314  5015 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.893610 30629 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:31.893901 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:31.896746 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.897624 30555 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:31.898049 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:31.899164 30556 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:16:31.894691 31367 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:31.898770 31394 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:31.901752 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.861085  4971 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:31.866698  5065 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:31.873235  4870 raft_consensus.cc:2764] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 13:16:31.908252 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:31.931484 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:31.907159 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:31.908752 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:31.909073 30557 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:16:31.932760 30555 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:31.910833  5054 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.175s	user 0.000s	sys 0.002s
I20260709 13:16:31.949772 30629 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:31.916391  4904 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:31.920840  5010 log.cc:927] Time spent T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.706s	user 0.004s	sys 0.001s
I20260709 13:16:31.951095 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1880) took 1024 ms. Trace:
I20260709 13:16:31.951282  4971 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.923192 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:31.952232 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:31.951553 30703 rpcz_store.cc:276] 0709 13:16:30.926314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.926484 (+   170us) service_pool.cc:224] Handling call
0709 13:16:30.939418 (+ 12934us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:30.956792 (+ 17374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.956856 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:30.994254 (+ 37398us) log.cc:844] Serialized 26 byte log entry
0709 13:16:31.616903 (+622649us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:31.616956 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.617030 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:31.909883 (+292853us) spinlock_profiling.cc:243] Waited 7.85 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:31.950794 (+ 40911us) raft_consensus.cc:1685] finished
0709 13:16:31.950824 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:31.951029 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18061184,"thread_start_us":550299,"threads_started":1,"wal-append.queue_time_us":655081}
I20260709 13:16:31.953373  4823 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:31.954834  4916 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:31.955456  4916 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:31.956511  4916 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.9, Last appended by leader: 9, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:31.957645 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:31.859347  4673 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.049s
I20260709 13:16:31.910146  5015 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.928320  5053 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:31.960000 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.960450 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2124) took 1430 ms. Trace:
I20260709 13:16:31.961026 30625 raft_consensus.cc:2410] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:16:31.935531 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:31.859735  4649 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:31.964180 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.938934  5006 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:31.939810  5065 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:31.943733  4829 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.813s	user 0.015s	sys 0.000s
I20260709 13:16:31.943956  5071 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:31.944438  5055 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.945068  5058 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:32.003234 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.947860  4882 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.003357 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.948426 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:31.949179 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:32.005875 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72282.9us
I20260709 13:16:32.006850 31378 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:16:31.939877  5025 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:31.967684 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.983028 30555 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:32.012775 30631 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:31.954409 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:31.960855 30559 rpcz_store.cc:276] 0709 13:16:30.530214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:30.537169 (+  6955us) service_pool.cc:224] Handling call
0709 13:16:30.537592 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:30.581945 (+ 44353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:30.581997 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:30.582806 (+   809us) log.cc:844] Serialized 26 byte log entry
0709 13:16:30.899448 (+316642us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:31.109233 (+209785us) log.cc:844] Serialized 14 byte log entry
0709 13:16:31.248328 (+139095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:31.248462 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:31.920339 (+671877us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b240045d058. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803c769c4 00007f1803c74acb 00007f1803e2fdc9 00007f1803e2fa55 00007f1803e2f09d 00007f1806a6110e 00007f1806a7888c 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221
0709 13:16:31.960109 (+ 39770us) raft_consensus.cc:1685] finished
0709 13:16:31.960225 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:31.960385 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7697,"prepare.queue_time_us":266585,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3249,"raft.queue_time_us":231469,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":13052,"spinlock_wait_cycles":44805376,"thread_start_us":622428,"threads_started":3,"wal-append.queue_time_us":555267}
I20260709 13:16:31.992195 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.004272 30632 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:31.992057 30705 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:31.999583 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.000692 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:31.947113 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.015591 31396 raft_consensus.cc:2410] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:16:32.017905 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.019467 30633 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:32.019953 30698 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 13:16:32.021262 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 204526us
I20260709 13:16:31.899130  4938 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.022210 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.029407 31388 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:32.038036 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.038996 30554 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
W20260709 13:16:32.039773 30523 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.913s (SENT)
I20260709 13:16:32.040472 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.030000 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.031299 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.032222 30555 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:32.035418 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81088.3us
I20260709 13:16:32.054217  4455 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:32.054908  4455 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.055287  4455 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.059530  4730 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:32.059713 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.063169 30629 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:32.063946 31378 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:32.066972 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.066504 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.071223 30632 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:31.996626  5053 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:16:32.078047 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.083889 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.001746  4829 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.020946  5055 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.092365 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.094681 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.061189  4870 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:32.109962 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.072263  5046 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:32.105365 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.107403 30550 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:32.110993  5032 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 13:16:32.113260  4915 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:32.115988 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 86096.8us
W20260709 13:16:32.139075 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99334.7us
I20260709 13:16:32.139377 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.097299 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.140672 30629 raft_consensus.cc:2376] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17. Re-sending same reply.
I20260709 13:16:32.141933 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.132632 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:32.017722  5058 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.144068 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.144294  5048 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:32.144842  5048 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:32.145226  5058 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.145645 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.146818  4979 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:32.146647  5031 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:32.147174 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.147835  4979 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:32.086234  4841 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.034209  4882 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.132611 30625 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:32.134835 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 161358us
I20260709 13:16:32.142836  5032 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:32.011052  5071 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.068275  5016 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:16:32.150805 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.149930 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.172435 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.172729 30626 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:32.152293 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.173619 30555 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:32.125931 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.170203  5012 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:32.176146  5012 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.176487  5065 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:32.176779  5032 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:32.176939  4934 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.178815  5058 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.179771  5006 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.181226  4829 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.181879  5055 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.176520 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.184922  4882 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.185467  5032 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:32.186290  5032 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:16:32.186988 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.187958 30624 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:32.188046  4833 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:32.124414  5029 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.188988 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.189087  5029 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.161710  4915 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:32.190588 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.190828  4915 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.191220  4915 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:32.179876  5038 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.192226  5038 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:32.193042 30555 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:32.024488  5025 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.164162 30559 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:16:32.167172  5071 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.169404 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.169709 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.187151  5032 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:32.188458 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.151676  4673 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:32.148231  4979 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:32.184948 31292 leader_election.cc:400] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:32.162442  4938 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.156457 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:32.197266 30632 raft_consensus.cc:2376] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25. Re-sending same reply.
I20260709 13:16:32.148118  5031 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:32.261027 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.219871 30633 raft_consensus.cc:2368] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:32.157998 30554 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:32.219084 30629 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:32.226796 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.159165  5056 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:32.240046 31292 leader_election.cc:403] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:16:32.265796 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 81005.6us
I20260709 13:16:32.178467  5012 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.268359 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.172611 30671 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.269905  5012 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:32.271231 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 98684.6us
I20260709 13:16:32.272337 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.273464  5012 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:32.275025 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.261816 30631 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
W20260709 13:16:32.279815 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 201883us
I20260709 13:16:32.283004 30628 raft_consensus.cc:2393] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:32.283821 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.218797  5051 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.285810 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 2170) took 1074 ms. Trace:
I20260709 13:16:32.301847 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.300614 31391 rpcz_store.cc:276] 0709 13:16:31.202411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:31.209747 (+  7336us) service_pool.cc:224] Handling call
0709 13:16:32.264504 (+1054757us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400460ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:32.276249 (+ 11745us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267432704}
I20260709 13:16:32.291585 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:32.223760  5032 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:32.305809 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.306210 31383 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:32.258667  4979 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:32.261027  5030 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.006s
W20260709 13:16:32.309190 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 83020.1us
I20260709 13:16:32.316562 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.344228 30626 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:32.317852 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.335285 30625 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:32.260095  5071 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:32.303360 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.364534 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.365772 30629 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:32.260644  4673 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:32.261238  5031 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:32.371258 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 65599.2us
I20260709 13:16:32.382807 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.373523 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.297039  4841 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:32.374665 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:32.377990 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 78607.6us
I20260709 13:16:32.285612  4829 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:32.319988  4882 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.340814 30550 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:32.396203 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:32.385602  4219 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.000s
I20260709 13:16:32.381664  5072 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:32.394775  4738 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.003s
I20260709 13:16:32.401010 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.417740 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.380793  4927 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:32.327562  5051 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.377701  5030 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.405947 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.402220 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:32.400842 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:32.415305 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:32.469575 30698 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:32.325470  4938 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:16:32.383117  4455 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.391785  5055 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:32.380991  4649 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:16:32.394213  4869 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:32.403447  5042 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.893s	user 0.013s	sys 0.001s
I20260709 13:16:32.396191  4730 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.467377 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.546931 30705 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
W20260709 13:16:32.471853 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 79629.3us
W20260709 13:16:32.474910 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 69093.6us
I20260709 13:16:32.549386  5017 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:32.549964  5017 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:32.370592  5016 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:32.550832  5016 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.551462  5016 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:32.551071  5017 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.552681  5017 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:32.557564  4980 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
I20260709 13:16:32.557081  5032 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.559561 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.489516  5091 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.560676  5012 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:32.560985  5091 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.564078  5091 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.496347 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.509538 30556 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:32.417047  5029 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.460843  4829 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.096s
I20260709 13:16:32.535125 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.573894 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.575007 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.575064 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.566815  4730 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.576736 30627 raft_consensus.cc:1275] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:32.578879 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.579303 30701 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:32.517408  4869 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:32.580510 30629 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:16:32.580511  4869 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.581952 30629 raft_consensus.cc:1275] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:16:32.583262 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.584594 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.585246  5016 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.585816  5051 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.586710  4927 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.587515  4927 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.561259 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.493673 30632 raft_consensus.cc:2368] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
I20260709 13:16:32.414983  4916 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.592521  4916 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.404744  4882 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.592883  4823 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:32.571655 30703 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:16:32.605263  4455 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.593189  5032 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:32.596237  4870 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:32.595360  4916 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.606595  4870 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:32.607196  5025 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.415459  4219 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.607807  4870 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.609653  4870 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:32.515326  4673 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.575191 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.611886 30628 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:32.603797  4730 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.571787  5048 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.419822  4738 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.613529 30559 raft_consensus.cc:606] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.614029 30559 raft_consensus.cc:614] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:32.615056 30553 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:16:32.592926 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.558053  4980 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.616880 30630 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:32.597846 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.621093 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.494159  5071 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:16:32.626441 30670 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.017s
W20260709 13:16:32.626936 30670 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.009s	sys 0.017s
W20260709 13:16:32.627643 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 543813us
I20260709 13:16:32.630300 30552 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:16:32.633422 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55470.8us
I20260709 13:16:32.628278  5025 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:32.635077 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 478717us
I20260709 13:16:32.637008 30555 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:16:32.642820 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 258013us
W20260709 13:16:32.636514 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 337574us
I20260709 13:16:32.649605 30624 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:32.605046  4927 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:32.632277  5039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.023s	sys 0.074s
I20260709 13:16:32.644939 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:32.652885 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:32.641084 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.549s (SENT) [suppressed 26 similar messages]
I20260709 13:16:32.654325 30631 raft_consensus.cc:2376] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17. Re-sending same reply.
I20260709 13:16:32.631392  4916 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:32.664988 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.659515 30597 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:32.671491 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:32.672209 31383 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:32.678885 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.671253  4927 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:32.687563  4927 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:32.688742  4927 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.689548  4649 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:16:32.655212  5039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.024s	sys 0.074s
I20260709 13:16:32.694970 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:32.695492 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.695839 30707 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:32.697098  5053 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:32.697531 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.697755  5053 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.699661  5016 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:32.673688  4915 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.701879  4938 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:32.703337 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:32.703591 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:32.704490 31396 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:32.704573 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.651655  5048 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.721899 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.723348 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:32.724177 30702 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:16:32.731673 30555 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:32.724977 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.726662 30705 raft_consensus.cc:1275] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 13:16:32.735013 30700 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 18: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:32.737879 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.738746 31396 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:32.743333 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.746213 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.744277 30706 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:32.700158  5053 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.637573  4936 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:16:32.744218 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:32.717186  5071 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:16:32.759013 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 114784us
I20260709 13:16:32.730526  4938 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.603315  5031 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.747037 30556 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:32.655326  4730 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:16:32.779711 30625 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:16:32.781080 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.723127  5025 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:32.800621  4979 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.800964  5025 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.803046  4986 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:32.803539  4986 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:32.690203  4869 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:32.804422  4986 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.805848  4986 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:32.806214  4869 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.727797  4916 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:32.808213 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:32.730213  5016 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:32.808964 31361 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.809067  5016 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.811833  5016 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.717558  5051 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:16:32.776382  4738 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:32.807885  4916 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:32.813597  4915 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:32.823468  4938 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.824079  5031 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.825796 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.830284 31367 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:32.798859 30629 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:16:32.833673 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98012.7us
I20260709 13:16:32.834115 30632 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:16:32.834908 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.844318  4455 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:32.845960 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:32.847136 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59186.8us
I20260709 13:16:32.847128 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.899812 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.848136  4455 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:32.848728  5046 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:32.850538  4980 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:32.852375 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.853263  4882 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:32.854071 30706 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "72181014679a4c538f96f2d305a5474f"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:32.957705 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.854876  4219 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:32.856702 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:32.960832 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:32.961458 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104842us
I20260709 13:16:32.962291 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:32.787482  5071 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:32.858882  5038 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.003s
W20260709 13:16:32.800997  5081 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:16:32.862205  4936 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:32.867108  5098 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:32.830873 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86742.5us
I20260709 13:16:32.876252 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:32.878455  4979 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:32.879069  5096 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:32.882279 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.001464 31391 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:32.888168 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.963881  5012 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:32.896715 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:32.876478 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:32.920492 30707 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
W20260709 13:16:32.920686 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 74816.6us
I20260709 13:16:33.012311 30699 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:33.012941 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:33.014127 30707 raft_consensus.cc:2410] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:16:33.004614 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 128525us
I20260709 13:16:32.983371  5071 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:32.987111  4870 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:33.054515 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:32.995298  4936 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:33.060878 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 419797us
I20260709 13:16:33.018543 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.017872  4829 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:32.965906 31394 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.020s	sys 0.033s
W20260709 13:16:33.069609 31394 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.021s	sys 0.033s
I20260709 13:16:33.009467  5012 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.957787 30706 raft_consensus.cc:606] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:32.967806 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.977154 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:33.075676 30705 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:33.077636 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:33.078955  5012 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:33.079967 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:32.982432 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:32.983363 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82798.8us
I20260709 13:16:32.985603 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:32.985942 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:33.085772 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.086122 30625 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:32.992718 30557 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:32.936299  5046 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:33.003026 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:33.045742  4870 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:32.967720  5065 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.042006 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.099233 30632 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:33.033984 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:33.044848 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.032684  5032 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:32.966161 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:32.954797 31361 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:33.073800  4979 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:33.080960  4738 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:32.980080 30559 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:33.083896 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:32.987434  5038 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:32.888638 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.017345  5096 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:32.951259 30698 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:32.931303 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:33.131037 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 199863us
I20260709 13:16:33.050274 31383 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:16:33.101570 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67723.2us
I20260709 13:16:33.115906 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:33.132687 31389 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 13:16:33.125242 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 109257us
I20260709 13:16:33.133838 31292 leader_election.cc:400] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
W20260709 13:16:33.127784 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.119638 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.129221 30630 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:33.096613  5051 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:33.145608 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:33.147065 30673 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.099589  4870 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:33.007277  5098 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:33.174606 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58799.2us
I20260709 13:16:33.176899 31383 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:33.109113  4979 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:33.187641 30556 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 9. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
I20260709 13:16:33.195779 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.179967 31394 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 23 index: 10. (index mismatch)
I20260709 13:16:33.182044 30632 raft_consensus.cc:1240] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:33.139448  5042 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:33.165235  4649 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:16:33.180899 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44372.4us
I20260709 13:16:33.232477 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:33.232928 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:33.235188 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.106840 30705 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:33.198024  4979 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:33.242431 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.113693  5065 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:33.130277  5032 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:33.199150  5056 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:33.198900  4934 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:33.112298  4309 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.010s	sys 0.012s
W20260709 13:16:33.257928 30633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.008s	sys 0.023s
I20260709 13:16:33.170008  4833 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 13:16:33.172953  4882 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:16:33.220980  5039 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:33.254021  5016 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:33.260854 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:33.267485 30633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.009s	sys 0.023s
I20260709 13:16:33.271266 30625 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:33.277760 31391 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:33.244217  4986 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:16:33.227543  5017 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.674s	user 0.019s	sys 0.001s
I20260709 13:16:33.263151  4730 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:33.260807  4649 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:33.252233  4884 consensus_queue.cc:1048] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:16:33.206240  4936 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.250720  4673 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:16:33.281340  4915 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:33.263603  4712 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 13:16:33.274731  5091 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.000s	sys 0.020s
I20260709 13:16:33.323356  5016 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:33.264295  5042 consensus_queue.cc:1048] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:16:33.281080  4980 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:33.334754 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.269562  4934 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:33.286515  4986 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:33.324509  4927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.024s	sys 0.063s
W20260709 13:16:33.341087  5091 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.001s	sys 0.020s
W20260709 13:16:33.364619 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.336241  4915 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:33.270016  5046 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:33.387861 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:33.342521  4936 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:33.402566 30698 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
W20260709 13:16:33.381035  4927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.025s	sys 0.064s
I20260709 13:16:33.373603  4986 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:33.432328  5051 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:33.383023  4696 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:33.431782 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:33.452919 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:33.445552  5093 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:16:33.383574  5017 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
W20260709 13:16:33.445907  5102 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:16:33.445328  5097 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:33.442111  4884 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:33.460358  5101 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.277s	user 0.001s	sys 0.001s
I20260709 13:16:33.430537  5029 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:16:33.467312 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 240596us
I20260709 13:16:33.461313  5055 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:33.488323 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 234893us
I20260709 13:16:33.461156  5095 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:33.481314  5105 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.157s	user 0.001s	sys 0.001s
I20260709 13:16:33.462496  5096 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:33.523990 30670 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:33.532431  5095 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:33.479295  5038 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:33.488755  5097 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:33.537177 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 109682us
W20260709 13:16:33.545300 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.496879  4884 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:33.547844  4884 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:33.500689 30523 leader_election.cc:400] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
W20260709 13:16:33.536799 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 105095us
I20260709 13:16:33.552138 30523 leader_election.cc:403] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:16:33.491592  5098 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:33.554731 30627 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.012s
W20260709 13:16:33.557881 30627 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.005s	sys 0.012s
I20260709 13:16:33.450516  4649 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:33.559753 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:33.560111 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63047.6us
I20260709 13:16:33.535008  5032 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:33.551085  5107 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 13:16:33.540881  5055 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:33.568127 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.552109  5097 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:33.555815  5048 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.815s	user 0.012s	sys 0.003s
W20260709 13:16:33.573658 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.573577 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:33.579459  5116 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:16:33.579226 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.577762 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1522) took 1580 ms. Trace:
I20260709 13:16:33.565968  5096 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21.
I20260709 13:16:33.561273  5065 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:33.567194  5006 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:33.594336 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.594691 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:33.579694  5120 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:16:33.582129  5085 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:33.590735  5056 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:33.601136 30632 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:33.584097  4988 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:16:33.638200  5129 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:33.785458  5129 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:33.787951  5113 log.cc:927] Time spent T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:16:33.762455  5048 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.762483  5055 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.622546  4870 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.627146  5085 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:33.796890 30705 raft_consensus.cc:1240] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:33.796546  5085 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.798758  5129 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:33.796577  4870 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:33.800513  5121 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
I20260709 13:16:33.618242 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.802589 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:33.803010 30630 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:16:33.799221 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.747956  5096 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:16:33.804131 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:33.804829  4738 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:33.805279  4738 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:33.806248  4738 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:33.639652  5116 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:16:33.619438 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:33.636510  5071 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.004s
W20260709 13:16:33.649295  5111 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:16:33.609325  5006 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:33.672732  5118 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.001s
I20260709 13:16:33.749871 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:33.628758 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.633991 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:33.767267  5122 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 13:16:33.870064 30557 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:33.776336  5133 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.872421 31364 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:16:33.836241  4738 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:33.781246  4829 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.686s	user 0.011s	sys 0.002s
I20260709 13:16:33.790136  5048 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:33.799053  5114 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 13:16:33.759636  5038 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.799227  5032 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:33.606554  4934 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:33.630980 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.807055 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.804839 30631 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:33.581485 30627 rpcz_store.cc:276] 0709 13:16:31.997279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.577585 (+580306us) service_pool.cc:224] Handling call
0709 13:16:32.577883 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:32.578114 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:32.785589 (+207475us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.577094 (+791505us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.577341 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:33.577408 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:33.577467 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.577556 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.577690 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4332,"prepare.queue_time_us":463892,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":15892,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7552,"thread_start_us":846013,"threads_started":2}
I20260709 13:16:33.832559 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:33.897783 30630 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:16:33.753194  5109 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 13:16:33.905323 30627 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:33.906841  4971 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:33.857079 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:33.877363  5133 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:33.910497  5119 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.383s	user 0.004s	sys 0.001s
I20260709 13:16:33.886641  4829 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:33.913587  5133 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:33.914179 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:33.914438  5100 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.915105 30701 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:33.915164  5100 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.916792 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:33.917788  4833 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:33.918227  4833 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.918605  4833 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:33.922726  5032 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:33.925616  5032 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:33.928584  4980 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.929323  4980 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:33.932449 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 75488.9us
I20260709 13:16:33.855240 30670 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:33.819180  5115 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
W20260709 13:16:33.867508 30553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.006s	sys 0.014s
W20260709 13:16:33.870352 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 120645us
I20260709 13:16:33.871986 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:33.934633 30553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.014s
I20260709 13:16:33.776420  5097 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:33.872135 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.838675  5095 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:33.842491  5096 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:33.878870 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.005s	sys 0.005s
I20260709 13:16:33.891793 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:33.864585  5071 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:33.938845  4934 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:33.939385  4934 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:33.939741  4934 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:33.940989  5097 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:33.941430  5135 consensus_queue.cc:1059] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:33.873256  5129 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:33.941892 30559 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:33.938865 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:33.942698 30628 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.028s	sys 0.045s
W20260709 13:16:33.957345 30628 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.029s	sys 0.046s
W20260709 13:16:33.878455  5012 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.797s	user 0.015s	sys 0.001s
W20260709 13:16:33.949057 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.005s	sys 0.005s
I20260709 13:16:33.958040 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2068) took 1986 ms. Trace:
W20260709 13:16:33.960830 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:33.962448 30633 rpcz_store.cc:276] 0709 13:16:31.972034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.576162 (+604128us) service_pool.cc:224] Handling call
0709 13:16:32.576520 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:32.613645 (+ 37125us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:32.613695 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:32.614309 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 13:16:33.279786 (+665477us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:33.279854 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.279964 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:33.957713 (+677749us) raft_consensus.cc:1685] finished
0709 13:16:33.957748 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:33.957952 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36241,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"thread_start_us":664981,"threads_started":1,"wal-append.queue_time_us":854309}
W20260709 13:16:33.937301  5122 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:16:33.964571 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 703773us
I20260709 13:16:33.974172 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:33.976336 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 121246us
I20260709 13:16:33.977159 30626 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 24. Ops: []
I20260709 13:16:33.983568  5095 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:33.984429  5095 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:33.984965  5095 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:33.994298  5089 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:33.996881  5089 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.001199  5053 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:34.009728 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.012789 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:34.016134 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:33.956616  5006 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:34.026557 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.027395 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:33.895213  5038 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.029897 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.033093 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.037528 30631 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:34.038429 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.040243 30633 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:34.039274 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:34.040741 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2371) took 1065 ms. Trace:
I20260709 13:16:34.043651 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.046452 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.047667 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.049002 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:33.954169  5096 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.056954 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:34.041538 30700 rpcz_store.cc:276] 0709 13:16:32.975696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.975909 (+   213us) service_pool.cc:224] Handling call
0709 13:16:32.976471 (+   562us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:33.001516 (+ 25045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.001557 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:33.032552 (+ 30995us) log.cc:844] Serialized 52 byte log entry
0709 13:16:33.353896 (+321344us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:33.353930 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.353986 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.040494 (+686508us) raft_consensus.cc:1685] finished
0709 13:16:34.040520 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.040677 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26396,"raft.queue_time_us":38421,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":27553,"thread_start_us":260690,"threads_started":1,"wal-append.queue_time_us":473634}
W20260709 13:16:34.034173  5031 consensus_meta.cc:287] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.209s	user 0.007s	sys 0.010s
I20260709 13:16:34.062036 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.030707  5016 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:34.057097 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:34.062893 30629 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:33.973881  5012 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.006137  5097 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:16:34.074883  5137 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:16:34.031841  5100 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.081315  4833 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.082901  5038 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.063843  4730 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:34.086835  5142 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:34.089809 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.040228  5032 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:16:34.054281  5046 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:16:34.099002 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51442.2us
I20260709 13:16:34.052897  4829 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:34.101814 30700 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:34.108479 30555 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:34.120483 30700 raft_consensus.cc:606] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:34.121083 30700 raft_consensus.cc:614] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:34.088050  5135 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:34.122934 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:34.100865  4986 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.654s	user 0.013s	sys 0.001s
I20260709 13:16:34.116839 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:34.096876 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.111876 30702 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 20. Ops: []
I20260709 13:16:34.124879 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:34.104930  5136 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:16:34.124497 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:34.051715  4927 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:16:34.107924  5142 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.137760 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:34.138698 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.102159  5031 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:16:34.126358  5135 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.141158 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.147336 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.152045 30700 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:34.146484 30596 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:34.140424 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.153707 30558 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:34.126320  4980 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.130918  4730 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:34.196848 30699 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:16:34.176147 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:34.192618 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 58709.1us
I20260709 13:16:34.132501  4988 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:34.200075 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:34.146484  5142 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:34.129908  4986 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.188653  5129 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.153833  5048 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.196452  5100 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:34.153947  5135 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:34.199337  5097 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:34.150028  4829 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:34.207366  4730 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.188228  5089 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.237620 31383 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:34.245045 30552 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f"
dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
mode: GRACEFUL
new_leader_uuid: "acf852f85f47483d99cbe4020baee627"
 from {username='slave'} at 127.0.0.1:49580
I20260709 13:16:34.245823 30552 raft_consensus.cc:606] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Received request to transfer leadership to TS acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.217360  4884 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:34.222780  5071 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.242554  4730 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:34.259909 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44399us
W20260709 13:16:34.254529  4979 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.990s	user 0.007s	sys 0.009s
I20260709 13:16:34.250670  4884 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.221819  5129 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.244386  5100 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.280068 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:34.281059  4986 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.281854  4986 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:34.281941 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.282656  5100 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.283197  4986 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:34.285069  5032 consensus_queue.cc:1048] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:16:34.286275  5146 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 13:16:34.288156 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:34.290984 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:34.271526  4884 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:34.295037 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:34.305545 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:34.290691  5145 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:16:34.278920  4927 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.275094  4829 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.277830  5012 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.289496  5032 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:34.316876 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:34.404582 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.289721  5096 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.270893  4979 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:34.464948 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 154088us
I20260709 13:16:34.467510 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.468456 31394 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:34.472151 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 155515us
I20260709 13:16:34.306043  4833 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.308300  5156 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:34.495373 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.496964 31391 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:34.459051  5031 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:34.501154 31361 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
W20260709 13:16:34.478857  5109 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:16:34.517453 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:34.509588  5123 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.523813  5123 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.528735  5129 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.465026  4829 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.543304 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1897) took 1842 ms. Trace:
I20260709 13:16:34.544051 30704 rpcz_store.cc:276] 0709 13:16:32.707129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.729377 (+ 22248us) service_pool.cc:224] Handling call
0709 13:16:32.729791 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:32.760563 (+ 30772us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:16:32.760939 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 13:16:33.181267 (+420328us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.275720 (+1094453us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b2c00285df8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:34.297124 (+ 21404us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.297420 (+   296us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:34.301322 (+  3902us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.301760 (+   438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.301881 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.542955 (+241074us) raft_consensus.cc:1685] finished
0709 13:16:34.542987 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.543198 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76498,"prepare.queue_time_us":61399,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":118,"raft.queue_time_us":52694,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":25054,"spinlock_wait_cycles":15046912,"thread_start_us":393981,"threads_started":1,"wal-append.queue_time_us":590063}
I20260709 13:16:34.480465  5100 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.553779  5100 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.557308  5100 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.479579  5158 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:34.558378  5158 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.559064  5158 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:34.559365 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.560648 31391 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:34.561177 31294 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.024s	sys 0.058s
W20260709 13:16:34.561599 31294 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.024s	sys 0.058s
I20260709 13:16:34.561833 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.563512  4986 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:34.564414  4986 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.481230  5042 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:34.567986  5042 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:34.569381  5042 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.479350  5032 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:34.571535  5032 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.479276  5153 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
W20260709 13:16:34.561877 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 518687us
I20260709 13:16:34.460705  4927 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.587774 31391 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:34.501242  5018 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.606244  5018 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:34.599215  5095 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.614s	user 0.016s	sys 0.000s
I20260709 13:16:34.615576 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.584219  5129 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:34.624912  4934 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.685s	user 0.016s	sys 0.000s
W20260709 13:16:34.628914 30624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.023s	sys 0.023s
W20260709 13:16:34.629513 30624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.023s	sys 0.023s
I20260709 13:16:34.633167 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.635454 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:34.638424  4915 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.639731  4915 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:34.641048 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.642174 31364 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:34.643760 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.644472 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2374) took 1667 ms. Trace:
I20260709 13:16:34.644966 30702 rpcz_store.cc:276] 0709 13:16:32.976926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.977166 (+   240us) service_pool.cc:224] Handling call
0709 13:16:32.977557 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.126213 (+1148656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.644138 (+517925us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540032a854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:34.644386 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607963520}
I20260709 13:16:34.647842 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1276) took 1718 ms. Trace:
W20260709 13:16:34.648255 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 586778us
I20260709 13:16:34.648371 30628 rpcz_store.cc:276] 0709 13:16:32.929401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.168675 (+239274us) service_pool.cc:224] Handling call
0709 13:16:33.169151 (+   476us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:33.183094 (+ 13943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.183138 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:33.228973 (+ 45835us) log.cc:844] Serialized 52 byte log entry
0709 13:16:33.957944 (+728971us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:34.010239 (+ 52295us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.041796 (+ 31557us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.647086 (+605290us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b18000c9ee0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f1803df2a40 00007f1803ded4e4 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:16:34.647338 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.647434 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.647572 (+   138us) raft_consensus.cc:1685] finished
0709 13:16:34.647589 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.647770 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":625073,"prepare.queue_time_us":96245,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":5373,"raft.queue_time_us":626943,"raft.run_cpu_time_us":486,"raft.run_wall_time_us":30296,"spinlock_wait_cycles":6659328,"thread_start_us":687033,"threads_started":1,"wal-append.queue_time_us":827856}
I20260709 13:16:34.645032 31364 raft_consensus.cc:2393] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:34.651101  4219 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:34.651706  4219 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.652092 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.652899  5100 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.653573 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.654155  4219 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:34.656261 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 509852us
I20260709 13:16:34.657049  4219 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:34.657238 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:34.657516  4219 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:34.657939  4219 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:34.658857  4980 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:34.659628 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.660336 30628 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:34.655411 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 531088us
I20260709 13:16:34.654764  4936 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:34.664063 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:34.653530  5100 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.662353  5155 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:34.676678 30630 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:16:34.655089 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.674728  4696 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:34.677767 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1539) took 1075 ms. Trace:
I20260709 13:16:34.618768  5095 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.688145  5085 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.688930  5085 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.689733  5095 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.690742 30698 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:34.686206  4696 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:34.691208  5048 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:34.691932  4738 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.814s	user 0.015s	sys 0.001s
I20260709 13:16:34.691870  5048 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.694255  5048 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.670764  4833 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:34.696028  4833 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.696501 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:34.697888 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:34.698182  4833 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.667548  4455 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:34.698949  5048 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:34.699055  4455 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.699751  5048 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:34.700289  5048 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.701448  4455 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.701975  4455 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.629810  5123 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.709326 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:34.684373 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.678496 30629 rpcz_store.cc:276] 0709 13:16:33.593503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.593678 (+   175us) service_pool.cc:224] Handling call
0709 13:16:34.668404 (+1074726us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b54004ab0d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:34.668531 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517956480}
I20260709 13:16:34.629882  5129 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.704911 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.703796  5048 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.702457  4455 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:34.721331 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1553) took 1071 ms. Trace:
I20260709 13:16:34.630280  4934 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.725245 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:34.724303 31364 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:16:34.724035 30627 rpcz_store.cc:276] 0709 13:16:33.650440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.907574 (+257134us) service_pool.cc:224] Handling call
0709 13:16:33.909398 (+  1824us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:33.911048 (+  1650us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:34.032426 (+121378us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.683193 (+650767us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:34.686977 (+  3784us) log.cc:844] Serialized 52 byte log entry
0709 13:16:34.687198 (+   221us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:34.687443 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.715234 (+ 27791us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.720890 (+  5656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.721014 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:34.721077 (+    63us) raft_consensus.cc:1685] finished
0709 13:16:34.721096 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.721271 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447140,"prepare.queue_time_us":34980,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":11496,"raft.queue_time_us":443737,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":304,"thread_start_us":321313,"threads_started":2,"wal-append.queue_time_us":437628}
I20260709 13:16:34.750099 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2377) took 1387 ms. Trace:
I20260709 13:16:34.750854 30705 rpcz_store.cc:276] 0709 13:16:33.363141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.363424 (+   283us) service_pool.cc:224] Handling call
0709 13:16:33.363877 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.797402 (+433525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.749719 (+952317us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b5400233554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:34.749996 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":995255552}
I20260709 13:16:34.753835 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2192) took 1189 ms. Trace:
I20260709 13:16:34.714507  5051 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:34.715020  5071 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:16:34.766960 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.754525 31396 rpcz_store.cc:276] 0709 13:16:33.564116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.564364 (+   248us) service_pool.cc:224] Handling call
0709 13:16:33.789820 (+225456us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.790045 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.790075 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:33.790130 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:33.790167 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:33.790234 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.753553 (+963319us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540002bcd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:34.753751 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514787200}
I20260709 13:16:34.715958  4927 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.762200 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.724320  4980 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:34.720180  5018 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.772843  4884 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.802605  4884 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.776393  5085 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.778228  4870 raft_consensus.cc:3055] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:34.806473  4870 raft_consensus.cc:740] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:34.807521  4870 consensus_queue.cc:260] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.809111  4870 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:34.779143  4730 consensus_meta.cc:287] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.001s
I20260709 13:16:34.810914  4730 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.813311  4730 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.757944  5048 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:34.785060  4696 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.814175  5048 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.788317  4934 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:34.792466 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 919611us
I20260709 13:16:34.820015 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.704165  4936 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.824002 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.825884 30627 raft_consensus.cc:2368] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:34.796738 30628 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:34.799109 30629 raft_consensus.cc:2368] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:34.775388  4455 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:34.832623 30630 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:34.792335  5129 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.805154  4219 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:34.840566  4219 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:34.841039  5142 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.620s	user 0.006s	sys 0.010s
I20260709 13:16:34.819101 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.841636  5142 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:34.841964 30556 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:34.831267 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.855532 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.857856 30557 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:34.858587 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.859391 30629 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:34.865429 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.866428 30627 raft_consensus.cc:2393] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:34.867551 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:34.868898 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.868892 30628 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:34.872148 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 145775us
I20260709 13:16:34.874058 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:34.876825 31396 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:16:34.881899  5166 consensus_queue.cc:1059] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:34.883234  5166 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:34.883842  5166 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.886920  5165 raft_consensus.cc:3055] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:34.887096 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:34.887687  5165 raft_consensus.cc:740] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:34.888974  5165 consensus_queue.cc:260] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:34.895033  4936 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.860514  4696 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:34.897024  5159 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:16:34.897753  4979 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:34.874883  5048 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.900367  5100 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.903968 30556 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:34.904578 30559 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2146) took 1720 ms. Trace:
I20260709 13:16:34.905081 30559 rpcz_store.cc:276] 0709 13:16:33.184109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.197469 (+ 13360us) service_pool.cc:224] Handling call
0709 13:16:33.197891 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.962346 (+764455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.904273 (+941927us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400293454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:34.904502 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710570752}
I20260709 13:16:34.906904 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:16:34.907075 30555 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:16:34.907917 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:34.908557 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.549s (SENT)
I20260709 13:16:34.909170 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:34.909788 31378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2200) took 1072 ms. Trace:
I20260709 13:16:34.910255 31378 rpcz_store.cc:276] 0709 13:16:33.837958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.849555 (+ 11597us) service_pool.cc:224] Handling call
0709 13:16:33.850046 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.924332 (+ 74286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.924395 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:33.924464 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:33.924514 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.909473 (+984959us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b5400145fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:34.909533 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:34.909718 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170038144}
W20260709 13:16:34.911221 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:34.914970  5048 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:34.908723 30632 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:34.922988  5165 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:34.880271 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.860991 31394 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:34.935884 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.877960 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.943578 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.946638 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.947903 30705 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:34.949231 31396 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:16:34.948997 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.953727 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.954128 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.954571 30705 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:34.954933 31394 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:34.957952 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:34.959161 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:34.960918 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:34.961201 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:34.964144  5100 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:34.962826 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:34.962550 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:34.972250 31294 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:34.949576  5166 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:34.981535 30702 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:16:34.979508 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.976035 30628 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:34.984506 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:34.986096 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:34.995558 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:34.996904 30628 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:34.998932 30552 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:34.976660  5171 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:35.011057 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:35.011417 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.011890 30551 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:35.011288  5171 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:35.014127  5171 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:34.985644  5169 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:35.015255  5169 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:34.979947  4979 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:35.016014  4979 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:35.016489  4979 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:35.016222  5169 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.016827  4455 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:35.012140 30558 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:35.020846 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1555) took 1349 ms. Trace:
I20260709 13:16:35.020465 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.035377 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.034046 30633 rpcz_store.cc:276] 0709 13:16:33.671161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.056348 (+385187us) service_pool.cc:224] Handling call
0709 13:16:34.073640 (+ 17292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.757505 (+683865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.757558 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.771323 (+ 13765us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.953470 (+182147us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:16:34.989931 (+ 36461us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.990500 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.990648 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.020489 (+ 29841us) raft_consensus.cc:1685] finished
0709 13:16:35.020526 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.020760 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":579831,"prepare.queue_time_us":37527,"raft.queue_time_us":582357,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":24141,"thread_start_us":288281,"threads_started":2,"wal-append.queue_time_us":229464}
I20260709 13:16:34.952041  5006 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:16:35.040262 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.043884 30554 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:35.034129 30632 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:34.988731  5142 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:35.042272 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.044200 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2093) took 1033 ms. Trace:
I20260709 13:16:35.047559 30702 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:35.047653 30633 rpcz_store.cc:276] 0709 13:16:34.011021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.036822 (+1025801us) service_pool.cc:224] Handling call
0709 13:16:35.043645 (+  6823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.043856 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.043884 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.043931 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:35.043964 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.044022 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.044139 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.033269  4455 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.051865 30627 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:35.054215 30629 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:35.050629 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2089) took 1096 ms. Trace:
I20260709 13:16:35.052908 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2090) took 1093 ms. Trace:
I20260709 13:16:35.054966 30628 rpcz_store.cc:276] 0709 13:16:33.953907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.049588 (+1095681us) service_pool.cc:224] Handling call
0709 13:16:35.049917 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.050165 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.050203 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.050264 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:35.050312 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.050389 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.050524 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.055711 30627 rpcz_store.cc:276] 0709 13:16:33.959804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.033388 (+1073584us) service_pool.cc:224] Handling call
0709 13:16:35.047875 (+ 14487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.052375 (+  4500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.052847 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.057507 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2094) took 1032 ms. Trace:
I20260709 13:16:35.058259 30628 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:16:35.059350 30628 pending_rounds.cc:85] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 9
I20260709 13:16:35.059705 30628 pending_rounds.cc:107] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:16:35.059504 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.060038 30628 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:35.064980  5150 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.873s	user 0.012s	sys 0.010s
I20260709 13:16:35.058063 30632 rpcz_store.cc:276] 0709 13:16:34.014762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.045895 (+1031133us) service_pool.cc:224] Handling call
0709 13:16:35.046176 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.046552 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.046587 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.046657 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:35.046704 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.046773 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.046906 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:35.068477 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.070586 30554 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:35.072495 30628 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:16:35.053323  4455 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:35.077955 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:35.078743 30702 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:35.044476  5158 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:35.082598 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:35.091449 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.092235 30554 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 14.
W20260709 13:16:35.102399  5174 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:16:35.107371  4829 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:35.110845  5162 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.421s	user 0.001s	sys 0.002s
I20260709 13:16:35.114082 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2088) took 1169 ms. Trace:
I20260709 13:16:35.114666 30630 rpcz_store.cc:276] 0709 13:16:33.944369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.690449 (+746080us) service_pool.cc:224] Handling call
0709 13:16:34.830379 (+139930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.833336 (+  2957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.113809 (+280473us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540007d0d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:35.114004 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320872832}
I20260709 13:16:35.116202 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1554) took 1463 ms. Trace:
I20260709 13:16:35.116683 30624 rpcz_store.cc:276] 0709 13:16:33.652565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.972579 (+320014us) service_pool.cc:224] Handling call
0709 13:16:34.028811 (+ 56232us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.038866 (+ 10055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.038901 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.067846 (+ 28945us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.629992 (+562146us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:34.637139 (+  7147us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.705533 (+ 68394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.705633 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.109467 (+403834us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b1800023f80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db
0709 13:16:35.110112 (+   645us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:35.115962 (+  5850us) raft_consensus.cc:1685] finished
0709 13:16:35.115988 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.116138 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":675478,"prepare.queue_time_us":32167,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":94,"raft.queue_time_us":92450,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":31772,"spinlock_wait_cycles":43527808,"thread_start_us":524506,"threads_started":1,"wal-append.queue_time_us":588835}
I20260709 13:16:35.118064 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1249) took 1110 ms. Trace:
I20260709 13:16:35.126451 30557 rpcz_store.cc:276] 0709 13:16:34.008052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.009061 (+  1009us) service_pool.cc:224] Handling call
0709 13:16:35.117876 (+1108815us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b54000208d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:35.118001 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1497876864}
I20260709 13:16:35.131263 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1984) took 1750 ms. Trace:
I20260709 13:16:35.132004 31383 rpcz_store.cc:276] 0709 13:16:33.380653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.400928 (+ 20275us) service_pool.cc:224] Handling call
0709 13:16:33.436264 (+ 35336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.238380 (+802116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.130818 (+892438us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54000032d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:35.131129 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1842185600}
I20260709 13:16:35.134961  5158 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:35.086009  5171 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:35.136063  5171 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.134457  4455 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:35.141436 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 21 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:35.095451  5135 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.846s	user 0.020s	sys 0.000s
I20260709 13:16:35.143051 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:35.143155  5135 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:35.143939 30702 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:35.146066  5135 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:35.147572 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.148438 31396 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:16:35.164608 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:35.165527 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:35.166253 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:35.167400 30699 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:35.166213 31396 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:35.168001 30702 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:35.168241 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.171339 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.138442  5171 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:35.149466 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:35.151153 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.181749 30556 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:35.151670 31394 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:35.182322 31394 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:35.152602 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:35.152781 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 1988) took 1288 ms. Trace:
I20260709 13:16:35.155498 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1268) took 2999 ms. Trace:
I20260709 13:16:35.147593 31389 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 25. Current term is 26. Ops: []
I20260709 13:16:35.174887 30554 raft_consensus.cc:2410] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:16:35.185287 30626 rpcz_store.cc:276] 0709 13:16:32.134779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:32.779538 (+644759us) service_pool.cc:224] Handling call
0709 13:16:32.811361 (+ 31823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:33.977936 (+1166575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.134287 (+1156351us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54009434d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:35.134547 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2680397568}
I20260709 13:16:35.190145 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.183630 31364 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 14.
I20260709 13:16:35.164881  4673 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:16:35.195073 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:35.148870  4455 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:35.160646  5042 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:35.203377  4455 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:35.211758  4455 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:35.203413  5100 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:35.212857  4455 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:35.214159  5098 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:35.214497  4455 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:35.214831  5098 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:35.215050 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.170565  4870 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:16:35.215804  5098 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.204082  5135 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 13:16:35.220952  5166 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.221801  5166 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:35.193612 31394 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.223870 31394 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:35.224349 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.231091 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2096) took 1183 ms. Trace:
I20260709 13:16:35.184060 31388 rpcz_store.cc:276] 0709 13:16:33.864844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.866763 (+  1919us) service_pool.cc:224] Handling call
0709 13:16:33.867195 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:33.867647 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:33.867676 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:33.879156 (+ 11480us) log.cc:844] Serialized 52 byte log entry
0709 13:16:34.063856 (+184700us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:34.094070 (+ 30214us) log.cc:844] Serialized 14 byte log entry
0709 13:16:34.220362 (+126292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.220486 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.064273 (+843787us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:35.125968 (+ 61695us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b1800424360. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0557 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:35.152421 (+ 26453us) raft_consensus.cc:1685] finished
0709 13:16:35.152462 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.152692 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34046,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":12533,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":14173,"spinlock_wait_cycles":26148992,"thread_start_us":232851,"threads_started":2,"wal-append.queue_time_us":226137}
I20260709 13:16:35.231678 30629 rpcz_store.cc:276] 0709 13:16:34.047511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.055005 (+1007494us) service_pool.cc:224] Handling call
0709 13:16:35.055342 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.056813 (+  1471us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:16:35.071973 (+ 15160us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.230248 (+158275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.230373 (+   125us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:35.230454 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.230669 (+   215us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.231009 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":27199,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":3022,"raft.queue_time_us":235,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":168,"thread_start_us":157678,"threads_started":1}
I20260709 13:16:35.250660 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:35.251648 31383 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:35.254251 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.262240 30703 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:35.191165  5085 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 13:16:35.228758  5098 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:35.280341 30551 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:35.291579 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:35.297384 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:35.309698 30702 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:35.311482 30703 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:35.314205 31383 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:35.230415  5042 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:35.315737 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2095) took 1277 ms. Trace:
I20260709 13:16:35.349817 30633 rpcz_store.cc:276] 0709 13:16:34.038801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.048844 (+1010043us) service_pool.cc:224] Handling call
0709 13:16:35.058119 (+  9275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.058418 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:35.083853 (+ 25435us) log.cc:844] Serialized 14 byte log entry
0709 13:16:35.315286 (+231433us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:35.315370 (+    84us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:35.315419 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.315515 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.315669 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13375,"prepare.queue_time_us":27343,"prepare.run_cpu_time_us":272,"prepare.run_wall_time_us":273,"raft.queue_time_us":10477,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6791,"thread_start_us":220350,"threads_started":1}
I20260709 13:16:35.324249 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.324658 31394 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:35.282651  4870 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:35.332794 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.320816  5155 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:35.359862 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1125) took 1219 ms. Trace:
I20260709 13:16:35.361096 30704 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:35.327929  4934 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:35.354550 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:35.360435 30698 rpcz_store.cc:276] 0709 13:16:34.140645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.140819 (+   174us) service_pool.cc:224] Handling call
0709 13:16:35.359623 (+1218804us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540032ad54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:35.359798 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241341184}
I20260709 13:16:35.366657 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.371616 30703 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:35.374292  5155 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.377125  5155 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:35.392078 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:35.392925 30631 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:35.393440 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:35.395128 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:35.385015 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:35.354233  4673 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:35.394721 30624 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:35.394688 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:35.418213 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2185) took 2337 ms. Trace:
I20260709 13:16:35.434244 31361 rpcz_store.cc:276] 0709 13:16:33.080430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:33.178596 (+ 98166us) service_pool.cc:224] Handling call
0709 13:16:33.237545 (+ 58949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.505925 (+1268380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.417864 (+911939us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540002b2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:35.418130 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2905383168}
I20260709 13:16:35.442921 30625 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:35.365173  4934 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:35.433811 30629 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:16:35.466696 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.436400 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:35.416292 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:35.458029 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.423046  5177 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:16:35.389076  5018 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:35.430378 30631 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:35.396130  5166 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:35.424245  4979 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:35.467096  4934 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:35.497773 30556 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:35.476215  5018 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:35.393515  5100 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:16:35.503654 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:35.510875 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:35.516009 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 268772us
I20260709 13:16:35.505389  5018 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:35.525177 31378 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:35.538501 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.539198  5170 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:35.528482  4870 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:35.547909 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1558) took 1480 ms. Trace:
I20260709 13:16:35.551991 30627 rpcz_store.cc:276] 0709 13:16:34.067590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.056599 (+989009us) service_pool.cc:224] Handling call
0709 13:16:35.075797 (+ 19198us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:35.076504 (+   707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.076565 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:35.092518 (+ 15953us) log.cc:844] Serialized 52 byte log entry
0709 13:16:35.470562 (+378044us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:16:35.471403 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.471495 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.547578 (+ 76083us) raft_consensus.cc:1685] finished
0709 13:16:35.547606 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.547817 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":786,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":10583,"thread_start_us":277695,"threads_started":1,"wal-append.queue_time_us":375087}
W20260709 13:16:35.551306 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.113s (SENT)
W20260709 13:16:35.557463 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 172578us
W20260709 13:16:35.558775 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.563833 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.551738  4870 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:35.579175  5181 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
W20260709 13:16:35.593169 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49065.5us
I20260709 13:16:35.593490 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.596824 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:35.602799 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.611274  5099 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:35.611969  5099 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:35.612329  5179 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:35.612387  5099 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:35.613278 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.613803 30559 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:35.613006 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.584177  5165 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:35.611510  5161 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.004s	user 0.004s	sys 0.000s
I20260709 13:16:35.586167  4870 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:35.590973  5042 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:35.623924 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.837s (SENT)
I20260709 13:16:35.627767 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:35.621191 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 84897.7us
I20260709 13:16:35.631493 30557 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
W20260709 13:16:35.615665  5182 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.183s	user 0.002s	sys 0.000s
W20260709 13:16:35.673807 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49920.5us
I20260709 13:16:35.699754 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:35.700680 30559 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:35.675948 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:35.614872  5193 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:35.704929 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.588279  5187 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
W20260709 13:16:35.715041 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 123515us
I20260709 13:16:35.723917 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:35.783496 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:35.794834 30627 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:35.677871  5017 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:35.809758  5198 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:35.798414 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 176463us
W20260709 13:16:35.811760  5185 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:16:35.813481 31383 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:35.813874  5192 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 13:16:35.817138  5196 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:35.819936  4649 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:35.623193  5165 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.761425  5193 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:35.828482 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:35.845746 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1250) took 1837 ms. Trace:
I20260709 13:16:35.846410 30550 rpcz_store.cc:276] 0709 13:16:34.008351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.008576 (+   225us) service_pool.cc:224] Handling call
0709 13:16:34.008904 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.033000 (+ 24096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.033036 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.046207 (+ 13171us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.471732 (+425525us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:34.471777 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.471861 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.536402 (+1064541us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:35.820628 (+284226us) raft_consensus.cc:1685] finished
0709 13:16:35.820657 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.845624 (+ 24967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6064,"raft.queue_time_us":6741,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8194,"spinlock_wait_cycles":6711040,"thread_start_us":384358,"threads_started":1,"wal-append.queue_time_us":538109}
I20260709 13:16:35.848683 31389 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
W20260709 13:16:35.855109 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:35.707691  5125 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 13:16:35.701023  5042 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.107s
I20260709 13:16:35.802155  5171 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.807432  4979 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:35.875154  4696 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:35.814455  5085 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:35.851599  4649 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:35.888768  4455 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.674s	user 0.015s	sys 0.000s
W20260709 13:16:35.915987 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 192215us
W20260709 13:16:35.910861 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 196829us
I20260709 13:16:35.821168  5195 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:16:35.929452  4649 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:35.934441 30559 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:35.910259  5201 consensus_queue.cc:1059] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:16:35.954871  5058 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:35.959748  5012 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:16:35.960104  5176 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.837s	user 0.000s	sys 0.022s
I20260709 13:16:35.960421  5012 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:35.961530 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1248) took 1954 ms. Trace:
I20260709 13:16:35.961910 30553 rpcz_store.cc:276] 0709 13:16:34.007714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.007871 (+   157us) service_pool.cc:224] Handling call
0709 13:16:34.008250 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.008642 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.008674 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.023304 (+ 14630us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.915926 (+892622us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b180025d4a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:35.045986 (+130060us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:35.046037 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.046105 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.959352 (+913247us) spinlock_profiling.cc:243] Waited 31.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:35.961289 (+  1937us) raft_consensus.cc:1685] finished
0709 13:16:35.961312 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:35.961464 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2091,"raft.queue_time_us":5364,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":16375,"spinlock_wait_cycles":88894592,"thread_start_us":128680,"threads_started":1,"wal-append.queue_time_us":1099536}
I20260709 13:16:35.836880  5198 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:35.974440 30553 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:16:35.977437 30559 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:35.980877 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 305008us
W20260709 13:16:35.986944 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62842.3us
I20260709 13:16:35.988804 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:35.896363  5171 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:35.991686 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:35.999343  5210 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:36.001479 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:35.910552  5085 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:35.981305  5163 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.138s	user 0.004s	sys 0.000s
I20260709 13:16:35.961769  5012 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.028879 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:36.008714  5053 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:36.041992  5198 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:36.009850 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.003458  5087 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:16:36.034718  5193 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:36.051010 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:36.051592 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 190887us
I20260709 13:16:36.038920  5012 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:36.052245 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:36.010633 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:36.057930 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.058246 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:36.054245 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 72375.6us
I20260709 13:16:36.063169 31292 leader_election.cc:400] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21.
I20260709 13:16:36.063354 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.017617  5210 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:36.060285 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49777.4us
I20260709 13:16:36.064821 31292 leader_election.cc:400] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:36.039649  5195 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:16:36.072461 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1129) took 1566 ms. Trace:
I20260709 13:16:36.082284 30700 rpcz_store.cc:276] 0709 13:16:34.505950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.506372 (+   422us) service_pool.cc:224] Handling call
0709 13:16:34.506905 (+   533us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:34.512395 (+  5490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:34.512451 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:34.580378 (+ 67927us) log.cc:844] Serialized 26 byte log entry
0709 13:16:34.723064 (+142686us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:34.723111 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:34.723180 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:35.799233 (+1076053us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:36.072045 (+272812us) raft_consensus.cc:1685] finished
0709 13:16:36.072081 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.072370 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4903,"raft.queue_time_us":6643,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":119,"spinlock_wait_cycles":4186880,"thread_start_us":103975,"threads_started":1,"wal-append.queue_time_us":217698}
I20260709 13:16:36.087710  5203 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:36.088812  5099 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:36.078485  4455 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:36.078558  5018 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.003s
I20260709 13:16:36.062065  4696 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:36.059623  4979 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:36.104979  5133 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:36.105159  5085 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:36.078935  5206 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:36.109143 31292 leader_election.cc:403] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:16:36.130786 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66185.5us
I20260709 13:16:36.121827  5099 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:36.132653  5153 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:36.121791  5210 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:36.123049  4986 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:36.055657  5053 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:36.134738  5053 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:36.135281  5205 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:16:36.135833  5053 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.071084  5098 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:36.139312  4455 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:36.140018  5135 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:36.140511  5135 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:36.140550  4455 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.141543  5135 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.142830  4696 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:36.143357  5135 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:36.130175  4979 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:36.144227 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1257) took 1474 ms. Trace:
I20260709 13:16:36.143970  4696 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.144650 30552 rpcz_store.cc:276] 0709 13:16:34.669940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.670112 (+   172us) service_pool.cc:224] Handling call
0709 13:16:34.699042 (+ 28930us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:34.999750 (+300708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.143918 (+1144168us) spinlock_profiling.cc:243] Waited 299 ms on lock 0x7b54002891d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:36.144157 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688796544}
I20260709 13:16:36.146116  4979 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:36.131731 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.148686 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:36.080538  5165 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:36.151170 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.153709 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:36.130106  4934 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.618s	user 0.015s	sys 0.000s
I20260709 13:16:36.159031 30627 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 13:16:36.159741 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.161177 30631 raft_consensus.cc:2410] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:16:36.161609 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.155332 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:36.164366 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 111417us
I20260709 13:16:36.168967 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:36.170421 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.170675 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.173060 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:36.173454 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.175354 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.177659 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:36.179283 30626 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:36.181861 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.153398  5155 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:36.174117 30624 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:36.169024 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:36.183928 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:36.181747  5085 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:16:36.229830  5098 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.230093 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.237666 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.238947 30628 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:36.236119  5194 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.425s	user 0.003s	sys 0.000s
I20260709 13:16:36.184723 30599 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.235538 30630 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:16:36.241876 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:36.187624 30627 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:36.211184 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.240989 31364 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:36.216534 30700 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:16:36.130414  5018 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.242244 30630 pending_rounds.cc:85] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 11
W20260709 13:16:36.184305 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 192763us
I20260709 13:16:36.240701 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.245381 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.245981 31367 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:36.246263 30700 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:36.245285 30630 pending_rounds.cc:107] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 13:16:36.246989 30630 raft_consensus.cc:2889] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:36.248545 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 63973.6us
I20260709 13:16:36.251165 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.253633 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:36.170164  4979 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:36.257764 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:36.257903 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 89006.2us
I20260709 13:16:36.260690 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:36.267050 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.269610 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.268211 31391 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:36.163275  4934 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:36.177561  5209 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:36.169788  5211 raft_consensus.cc:2764] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:16:36.271763 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:36.276075 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:36.269044  4870 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.583s	user 0.014s	sys 0.000s
I20260709 13:16:36.280092  5217 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:36.262753  4986 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.230554  5098 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.268616  5171 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:36.242962  5200 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:16:36.255409  5205 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:36.267594  5165 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:36.285450 30626 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:36.292300  5187 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:16:36.295315 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 258026us
I20260709 13:16:36.296402  4934 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:36.297538  5046 raft_consensus.cc:3055] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:36.304299 30702 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:36.333235 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:36.314329 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:36.277395  5213 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:36.271739  4979 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:36.324208 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.324668 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.327950 30673 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:36.292423  5018 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:36.334296 31364 raft_consensus.cc:2376] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22. Re-sending same reply.
W20260709 13:16:36.339545 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64451.2us
I20260709 13:16:36.323375  5217 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:36.336426  5205 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:36.326718  5187 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:36.312988  4870 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.358809 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.347748 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:36.360050 31367 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:36.333618  5046 raft_consensus.cc:740] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:36.339174  5213 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:36.376175 30703 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:16:36.376658  5213 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.374788  5187 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:36.324983  4986 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.340737  4934 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:16:36.363736  5046 consensus_queue.cc:260] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:36.385757 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:36.364403  4870 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:36.365816  5098 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.366205 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:36.366806  5217 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.343284  4979 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.361126  5042 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:16:36.372154 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.373881  5125 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:36.372213 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.403146 30554 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:36.413710 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.341619  5053 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:36.386651 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:36.378598  5187 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:36.426090 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:36.428197 30553 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:36.423345 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:36.429646 30552 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:36.431309 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:36.444726 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.426597  5046 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:36.390021  5141 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:16:36.452880 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:36.464046 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 91684.5us
I20260709 13:16:36.441093  5012 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.460449 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:36.467262 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49913.3us
I20260709 13:16:36.401420  5200 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:16:36.473820 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:36.475435 30552 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
W20260709 13:16:36.456791  5169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.324s	user 0.017s	sys 0.038s
I20260709 13:16:36.444077  5053 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:36.479257 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.475472 30553 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:36.480126 31367 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:36.481724 30556 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:36.482447 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.484048 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:36.478502  5012 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:36.491726  5224 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:16:36.478461  5153 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 13:16:36.493379  5058 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:36.495164  5184 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.145s	user 0.010s	sys 0.000s
I20260709 13:16:36.476882  5171 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:36.500361 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.502235 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.503311 30626 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:36.503789 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.501189 30703 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
W20260709 13:16:36.482865  5169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.350s	user 0.018s	sys 0.038s
I20260709 13:16:36.500900 30700 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:36.512413 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.517318 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1561) took 1805 ms. Trace:
I20260709 13:16:36.518050 30632 rpcz_store.cc:276] 0709 13:16:34.711455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:35.068906 (+357451us) service_pool.cc:224] Handling call
0709 13:16:35.084830 (+ 15924us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:35.085148 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:35.085167 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:35.093931 (+  8764us) log.cc:844] Serialized 26 byte log entry
0709 13:16:35.345298 (+251367us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:35.345400 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:35.345510 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:36.494354 (+1148844us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:36.499952 (+  5598us) raft_consensus.cc:1685] finished
0709 13:16:36.500003 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:36.517191 (+ 17188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5005,"raft.queue_time_us":120,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":3763,"spinlock_wait_cycles":7774208,"thread_start_us":231090,"threads_started":1,"wal-append.queue_time_us":239878}
I20260709 13:16:36.517966 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.530321 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:36.531603 30627 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:36.540750 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:36.545193 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:36.559727 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:36.556335 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.567153 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.568092 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.523579  5206 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:36.464283  5205 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.547281  5133 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:36.577320 30596 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.572319 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.572621 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.511286  5058 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:16:36.558238  5216 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:16:36.643486 30703 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:36.565300  5195 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:36.644204 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67040.7us
W20260709 13:16:36.645468 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 73233.5us
I20260709 13:16:36.655507 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:36.656374 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.571496  5227 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:36.668680 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.579854  5165 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:36.641618  4649 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:36.650924  5042 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:36.648996  5206 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:36.648272  5216 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:36.671954 30523 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:36.666399  5232 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:36.689903 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.682318  5195 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.679996  4986 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.677078  5129 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:36.669685  5133 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:36.685101  5135 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.006s
I20260709 13:16:36.687860  5227 raft_consensus.cc:2764] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:16:36.688848  5216 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:36.648510  5053 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.692045  5218 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:36.739327 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 83928.6us
W20260709 13:16:36.755223 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.684934  5165 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:36.717702  5238 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:36.758579  4649 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.688100  5206 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.723042  5232 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:36.725821  4986 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:36.753810  5032 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.907s	user 0.000s	sys 0.023s
W20260709 13:16:36.783082 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.722136  5195 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.753748  5098 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.798776  4649 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.799242  5098 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.802120  5098 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.740150  5153 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:36.791167  5165 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:36.810200  5230 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:16:36.784494  5032 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.947s	user 0.000s	sys 0.024s
I20260709 13:16:36.810859  4649 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:36.816640  5229 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:16:36.783740  5238 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.784826  5239 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:36.835131 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 145729us
W20260709 13:16:36.830359  5231 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:16:36.863286 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.819370  5165 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:36.832576  5223 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:36.874080 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:36.874928 31396 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:36.867421 31367 raft_consensus.cc:1240] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:36.873826  5046 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:36.852106  5218 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.811900  4986 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.851989  4649 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.885900 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1909) took 1923 ms. Trace:
I20260709 13:16:36.886445 30704 rpcz_store.cc:276] 0709 13:16:34.962111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:34.962650 (+   539us) service_pool.cc:224] Handling call
0709 13:16:35.360939 (+398289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:35.361514 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:36.885643 (+1524129us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b540032ad50. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:36.885821 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915191040}
I20260709 13:16:36.875165  5223 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:36.896541 31396 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:36.885228 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:36.866230  5213 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:16:36.861485  5239 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.861864  5238 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:36.874742  5165 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:36.902779  5241 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.886086  5218 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.886817  4649 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:36.928997 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:36.950492 30704 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:36.912508  4882 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:36.914804  5213 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:36.919657  5242 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:36.959128  5240 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:16:36.927107  5211 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:36.954574  5046 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:36.964242  4882 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:36.970943  5046 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:36.977775 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 405218us
I20260709 13:16:36.952402  5217 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:36.968204  5236 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
I20260709 13:16:36.980469 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:36.981501  5165 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:36.983021  5239 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.983361 30703 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:16:36.969449  5211 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:37.006096 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:36.989847 30704 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
I20260709 13:16:36.990931 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.992722  5245 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:36.993796  5012 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:36.999353  5244 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:16:36.979187  5206 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.999497  5208 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:37.012565 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:37.002048  4882 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:36.967451  5166 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:16:37.021152 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:36.925899  5241 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:37.019948 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:37.006568  5235 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.321s	user 0.002s	sys 0.002s
I20260709 13:16:37.024250  5253 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:36.941525  5223 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.027503  5206 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.026206  5213 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:37.028424  5213 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:37.027401  5255 consensus_queue.cc:1175] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6b393e79ded4461ea5da7b6b6063305f" responder_term: 22 status { last_received { term: 19 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.13-21.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:16:37.029956  5241 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:37.029618  5213 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.030494 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.028223  5206 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.014114  5155 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.032116  5155 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:37.032843  5206 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.027444  5223 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:37.035732  4649 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:37.035975  5196 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:16:37.036446  5196 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:37.037369  5196 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.039361 30630 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:16:37.041052 30624 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:16:37.043774  5211 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.045228  5211 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:37.048207  5243 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:16:37.046356 30631 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:37.003810  5217 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.051335  5238 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.051575 30556 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:37.056340  5217 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:37.057272 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:37.058108 30633 raft_consensus.cc:2393] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:37.058944  5237 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 13:16:37.060568  4986 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:37.018464  5218 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:37.063452  5216 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:37.065554 31427 heartbeater.cc:646] Failed to heartbeat to 127.21.5.190:45121 (4 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.5.190:45121 timed out after 15.000s (SENT)
I20260709 13:16:37.034276  5155 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:37.054450 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.027374  5204 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:37.078015  5204 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.078533  5204 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:37.083483  5251 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.084131  5195 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:37.084218  5251 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:37.086475 30596 leader_election.cc:336] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.757s (SENT)
I20260709 13:16:37.086995 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.087720  5251 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:37.094054 30558 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:16:37.096807 30555 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:16:37.071192 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:36.979930  5242 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.103884 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.104904  5242 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:37.104797 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.106374 30702 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:37.106701 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:37.023929 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:37.019649  5239 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:37.016266  5208 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:37.120131 30633 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:16:37.122432 31367 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:37.133790 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.135429 31396 raft_consensus.cc:2410] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 21 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:37.137837 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:37.141314 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:37.139936  4649 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:37.150274 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.171947 30704 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:37.151324 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.172477 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:37.152619 30624 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 25. Current term is 26. Ops: []
W20260709 13:16:37.156410  5171 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.657s	user 0.014s	sys 0.000s
I20260709 13:16:37.071590  5196 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:37.179399 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 155495us
I20260709 13:16:37.181138 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.185645 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.130756  5129 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:37.129122  5220 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:37.187232 30699 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:37.159642  5202 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:16:37.144207  4934 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:37.159066  5087 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.196282 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:37.181743  5171 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:37.211287 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.209322 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:37.189159  5207 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:37.141819  5239 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.196863  5209 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:37.231112 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.231477 31378 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:37.196134  5085 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:37.207190  5153 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:37.174757  5135 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 13:16:37.215600  4986 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:37.190056  4673 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 13:16:37.260849 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 48202.4us
I20260709 13:16:37.230381  5207 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:37.292426 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 83233.9us
I20260709 13:16:37.295608 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.297969 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.298987 30705 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:37.302273 30625 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 13:16:37.302850 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:37.303460 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.305780 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:37.306772 30625 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:37.307144  5212 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:37.307767  5212 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:37.308914  5212 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.310268 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:37.314123 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:37.315042 30625 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:16:37.324087 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.326125 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:37.328090  5242 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:37.330135 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:37.294214 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.332718  5223 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:37.333287  5223 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:37.310293  5212 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:37.333783  5223 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:37.334914 30698 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:37.345659 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:37.211776  5238 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.078s
I20260709 13:16:37.350035 30597 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:37.248545  5171 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:37.263348  5032 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:16:37.262224  5153 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:37.365721 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:37.366992 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 220542us
I20260709 13:16:37.212713  5087 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.205632  5220 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.239585  4455 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 13:16:37.346803  5207 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:37.394316 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.348500  5209 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:37.363663  5171 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:37.348328  5135 consensus_queue.cc:1243] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:16:37.409211 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 197173us
W20260709 13:16:37.366062  5235 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:16:37.354398  5155 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:37.402920 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 191715us
I20260709 13:16:37.229678  5208 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.379803  5238 consensus_queue.cc:1243] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:16:37.399858  4934 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.374246  5048 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:37.450045 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:37.476822 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.479951 30701 raft_consensus.cc:2410] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:16:37.450291 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:37.374464  5153 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:37.591988 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 300784us
I20260709 13:16:37.513697 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.467228 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:37.469311 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 54699.4us
I20260709 13:16:37.457774 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.641817 31378 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:37.597680 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:37.453838 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:37.481395  5187 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.044s	user 0.013s	sys 0.004s
I20260709 13:16:37.454993 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.452466 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:37.546988  5253 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.520s	user 0.005s	sys 0.007s
I20260709 13:16:37.480024 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.525233  5246 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:37.489356  4979 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:37.695297 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47527.3us
I20260709 13:16:37.700333 31396 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:37.701326 31383 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 17.
W20260709 13:16:37.704150 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 53615.6us
I20260709 13:16:37.522131  5203 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:37.627566  5085 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:37.656334  5208 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:37.645020  5171 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:37.707065 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52486.2us
I20260709 13:16:37.543720  5258 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:37.530344  5234 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.412966  5213 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:37.530686  5247 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:16:37.704713 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 107161us
I20260709 13:16:37.715659  5216 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.650604  4934 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:37.718125  5211 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.671s	user 0.012s	sys 0.000s
I20260709 13:16:37.648103  5048 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:37.705765 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 238708us
W20260709 13:16:37.737666 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.636445  5155 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:37.734275  5272 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:16:37.710193  5253 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:37.712257  4986 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:37.757802 30556 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:37.655833  4455 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
W20260709 13:16:37.747016 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 372598us
I20260709 13:16:37.713503  5006 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:37.732403  5239 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.657130  4870 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 13:16:37.760565 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.830996 30559 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:37.761449 30555 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 13:16:37.777019 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:37.758529 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:37.738933  5279 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:37.718787  5135 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:16:37.719481  4979 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.813855 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.821769  5220 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:37.855274 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:37.748821  5268 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:37.764765  5247 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:37.817217  5276 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:16:37.817901  5204 consensus_meta.cc:287] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.739s	user 0.013s	sys 0.000s
I20260709 13:16:37.753140  5048 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:37.821230 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.826256 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:37.742274  5213 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:37.787954 30700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.012s	sys 0.037s
W20260709 13:16:37.882242 30700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.018s	sys 0.042s
I20260709 13:16:37.814205  5253 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:37.832552  5085 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.895900  5253 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:37.896513  5253 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:37.836850 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.899056 30550 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:37.837759 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.900208 30559 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:37.838837  4934 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:37.839857  5267 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:16:37.844872 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 86488.3us
I20260709 13:16:37.815197  5006 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:37.823612  5087 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:37.786480  5017 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:16:37.922127 31388 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:37.731451  5203 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:37.855819  4649 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:37.881485  5270 raft_consensus.cc:2764] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 13:16:37.903987 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.930496 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:37.898404  5259 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.528s	user 0.000s	sys 0.001s
I20260709 13:16:37.828930  5211 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:37.934818 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.936578 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:37.937168 30624 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:16:37.854180  5239 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.074s
I20260709 13:16:37.949326 30559 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:37.930516  5087 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.969482 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.970556 30706 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:16:37.971228 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:37.966789  5212 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.633s	user 0.012s	sys 0.002s
I20260709 13:16:37.931502  4934 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:37.968237 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:37.979300 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:37.983000 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.987627 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:37.943250  5247 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:37.945399  5218 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:38.018064 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.019842 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:37.935127  4649 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.037003 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.038960 30553 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 17: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:38.027841 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 289303us
I20260709 13:16:38.031232 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.043265 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:37.923602  4986 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:37.985080  5255 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:38.053730 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 85579.8us
W20260709 13:16:38.052598 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 65036.2us
I20260709 13:16:38.056018 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.993021  5289 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.059139 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.949345  5211 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.095680 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.104957 30631 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:38.109479 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:37.986861  4934 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.982543  5087 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.027329  5294 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.025599  5218 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:37.713721  5246 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.146890  5246 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.147120  4934 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.920575  5204 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.147768  5218 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:38.149816  5204 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.053269  5187 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.150959  5155 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:37.991501  5006 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.151677  5155 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:38.152072  5155 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.152415  5155 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:38.150996  5187 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.155527  5290 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:38.156606  5290 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:38.157480  5187 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.130970  5216 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.158267  5216 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.139184  5018 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:16:38.159158 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.159873 30559 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:37.919335  5268 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:38.141458  5196 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.957s	user 0.015s	sys 0.000s
I20260709 13:16:38.029381  5247 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.131780 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:38.109004  5223 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.774s	user 0.017s	sys 0.000s
I20260709 13:16:38.132242 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:38.110906  5275 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 13:16:38.101727  5085 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:37.848630  5279 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.062390  5255 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:38.111047  5289 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:37.738837  5234 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:37.741367  5258 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:37.524494  5209 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.173647 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.125449  5284 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:38.175293 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:38.167660 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:38.170574 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.169428 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.169150 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:38.181095 30559 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:16:38.197134 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.166396  5268 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:38.202934 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.167866  4649 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.161191  5216 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:38.234273  5273 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
I20260709 13:16:38.203725  4979 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.223744  5085 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.234179  5281 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.215s
W20260709 13:16:38.241279  5208 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.004s
I20260709 13:16:38.210465  5247 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:38.251067  5260 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.852s	user 0.003s	sys 0.000s
I20260709 13:16:38.253953 31364 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:38.200325  5255 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:38.255383 30624 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:38.212051  5098 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:16:38.269652 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.164219  5046 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:38.201922  5254 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
W20260709 13:16:38.260355 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 85237.6us
I20260709 13:16:38.147821  5246 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:38.257766 30707 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
W20260709 13:16:38.246431  5293 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:16:38.321511 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:38.290007  5171 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.004s
I20260709 13:16:38.282068 30625 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:38.238188 31394 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:38.176657  5223 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:38.279574 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 111993us
W20260709 13:16:38.259629 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 132633us
I20260709 13:16:38.328203 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:38.136140  5294 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:38.244666  5129 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.049s	user 0.013s	sys 0.000s
I20260709 13:16:38.234334  5291 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:38.337218 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 294561us
W20260709 13:16:38.182180  5153 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.002s
I20260709 13:16:38.315678 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:38.316021 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 379526us
W20260709 13:16:38.233044  5288 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 13:16:38.212643  5286 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:38.307406  5275 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 13:16:38.313330  5296 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.143s	user 0.001s	sys 0.001s
I20260709 13:16:38.166801  5218 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.329742 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:38.235137  5279 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.285583 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.330488 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.290503  5268 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:38.333194 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.334188 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:38.311841 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 206996us
I20260709 13:16:38.268982  5285 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.340133 30554 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:38.370080 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.343552  5291 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.318389  5246 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:38.381547  5246 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:38.384557  5294 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.347673 30630 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:38.374305  5289 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.353387 30704 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:38.386525 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.387912 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.309233  5283 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:16:38.367978 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:38.389989 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:38.390853 31364 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:38.347138  5153 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.369097 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.370877 30629 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:38.391557  5285 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.394196  5285 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.340438 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:38.371516 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.339073  5216 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:38.395325  5216 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:38.395399 30625 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:38.395803  5216 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:38.396695 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.396895  5308 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:38.397480  5308 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.397856  5308 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:38.399178  5234 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.341728  5129 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:38.400384  5153 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.400655 30558 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:38.402856  5085 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.404055  5218 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.358316  5286 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.370232  4649 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.406262  5251 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:38.409638  5207 consensus_meta.cc:287] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.941s	user 0.011s	sys 0.005s
I20260709 13:16:38.415470 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2416) took 1272 ms. Trace:
I20260709 13:16:38.416016  5208 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:38.416579  5208 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.422271 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.422701 30706 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:38.422874 30629 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:38.416043 30700 rpcz_store.cc:276] 0709 13:16:37.143024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.143221 (+   197us) service_pool.cc:224] Handling call
0709 13:16:37.149425 (+  6204us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.160901 (+ 11476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.160938 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.174589 (+ 13651us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.882928 (+708339us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:37.883016 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:37.883139 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.415243 (+532104us) raft_consensus.cc:1685] finished
0709 13:16:38.415267 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.415397 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11015,"raft.queue_time_us":32720,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":3883,"thread_start_us":670753,"threads_started":1,"wal-append.queue_time_us":754687}
I20260709 13:16:38.430241 30705 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:16:38.431968 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:38.433517 30707 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 14. Current term is 15. Ops: []
I20260709 13:16:38.433679 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.333271  5171 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.291023  5284 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.439291 30671 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:38.440943 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.371585  5268 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.451746 30706 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 15: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:38.453802 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.410202  5207 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.378950  5305 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.417279  5208 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:38.407181  5291 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:38.446240  5245 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:38.503319 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 173657us
I20260709 13:16:38.505203 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:38.478438  5048 consensus_meta.cc:287] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.001s
I20260709 13:16:38.455219  5279 raft_consensus.cc:2804] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:38.407851  4649 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.455960  5046 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.457335  5268 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:38.526202 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2120) took 1488 ms. Trace:
I20260709 13:16:38.481122  5087 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.528658 30627 rpcz_store.cc:276] 0709 13:16:37.037732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.037878 (+   146us) service_pool.cc:224] Handling call
0709 13:16:37.038320 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.071964 (+ 33644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.072022 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:37.072635 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 13:16:37.850000 (+777365us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b180012c8e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:37.939609 (+ 89609us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:37.994345 (+ 54736us) log.cc:844] Serialized 14 byte log entry
0709 13:16:38.017459 (+ 23114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.017561 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:38.525722 (+508161us) raft_consensus.cc:1685] finished
0709 13:16:38.525865 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:38.526111 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56570,"prepare.queue_time_us":50490,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":3774,"raft.queue_time_us":35137,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":52076,"spinlock_wait_cycles":10786688,"thread_start_us":89340,"threads_started":1,"wal-append.queue_time_us":1013228}
I20260709 13:16:38.523921  5212 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:38.541908  5212 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.544703  5046 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.545392  5212 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.449091  5217 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 13:16:38.547436 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.547613 30628 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:38.456496  5283 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:16:38.449972  5227 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
W20260709 13:16:38.562938 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57897.1us
I20260709 13:16:38.566818 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:38.569607  5046 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.569991  5286 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.511562  5245 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.582192 30627 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:38.584244 30627 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:38.586558 31367 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:38.587106 31367 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:38.588020 31367 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.589131 31367 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:38.527089  5048 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.567281  5279 raft_consensus.cc:697] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:38.548760  5255 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.626416 31367 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:16:38.632211 31367 pending_rounds.cc:85] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 13
I20260709 13:16:38.632838 31367 pending_rounds.cc:107] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:16:38.575834  5258 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.633183 31367 raft_consensus.cc:2889] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:38.599465  5306 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:16:38.508643  5305 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.637434 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2013) took 1186 ms. Trace:
I20260709 13:16:38.639060  5317 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 13:16:38.623991  5312 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:16:38.627771  5223 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:38.657608  5253 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.761s	user 0.017s	sys 0.000s
W20260709 13:16:38.662822  5304 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 13:16:38.669667  5279 consensus_queue.cc:237] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.670078  5308 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.671011  5320 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:38.672493  5321 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:38.673622 31367 rpcz_store.cc:276] 0709 13:16:37.451031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.452786 (+  1755us) service_pool.cc:224] Handling call
0709 13:16:37.453211 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.586417 (+1133206us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400697420. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:16:38.637035 (+ 50618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.637342 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411215616}
W20260709 13:16:38.639822  5169 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
W20260709 13:16:38.677874 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 101722us
W20260709 13:16:38.683503  5311 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:16:38.711967  5305 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.712674  5171 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.712893  5048 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.713426  5207 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.689796  5253 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.697469  5317 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:38.714963  5246 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.690356  5223 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:38.716189  5223 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.716754  5203 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.717444  5246 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.698828  5318 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.722956 31361 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:38.725018  5274 log.cc:927] Time spent T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.966s	user 0.000s	sys 0.004s
I20260709 13:16:38.719025  5255 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.726514  5211 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:38.738059 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 171330us
I20260709 13:16:38.718590  5223 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.724246  5253 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.742908 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:38.736156  5155 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.002s
I20260709 13:16:38.727363  5318 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.707113  5321 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.776516  5232 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:38.777076  5232 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:38.777333  5321 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.768810  5308 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.778038  5232 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.781342  5232 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:38.751279  5155 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.747202  5317 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.787132  5317 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.788491  5155 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.767777  5289 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.766054  5291 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.791091  5289 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.793725  5289 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.778309  5268 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:38.768496 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 381789us
I20260709 13:16:38.794975 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:38.795851  5268 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:38.796547 30630 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:38.769439  5234 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:38.769322 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.770097  5153 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:38.770663  5290 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:38.789673  5317 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:37.447470 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:38.740481  5216 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.796461 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:38.768894 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 429121us
W20260709 13:16:38.797627 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55049.3us
I20260709 13:16:38.813735 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.815652 30633 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:16:38.816854 30629 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:16:38.806378  5218 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:38.820513 31383 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:38.824626 30625 raft_consensus.cc:2393] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 23: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:38.827509 31383 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:38.803408  5291 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:38.806507  5310 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:38.830631 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:38.831753 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:38.837231  5290 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:38.845391 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.852995 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.853034  5216 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.855818  5285 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:38.856355  5208 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:38.856508  5285 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:38.859063 30627 raft_consensus.cc:2410] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:16:38.861033  5285 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.863373 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.808218  5234 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.867590 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:38.867580 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.869938 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:38.869788 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:38.872663 30630 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:16:38.873193 30630 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 12
I20260709 13:16:38.873278 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:38.873449 30630 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:16:38.874459 30630 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:38.876384 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.878580 31388 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 2235) took 1052 ms. Trace:
I20260709 13:16:38.879098 31388 rpcz_store.cc:276] 0709 13:16:37.825772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.825916 (+   144us) service_pool.cc:224] Handling call
0709 13:16:38.878405 (+1052489us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b540044e654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:38.878522 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92545920}
I20260709 13:16:38.880384 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.881196 30556 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:38.882002 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 2018) took 1418 ms. Trace:
I20260709 13:16:38.882429 31394 rpcz_store.cc:276] 0709 13:16:37.465868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.479663 (+ 13795us) service_pool.cc:224] Handling call
0709 13:16:38.881813 (+1402150us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400013954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:38.881931 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235480960}
I20260709 13:16:38.896910 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:38.888643 30633 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:38.884361 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.916447 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.917312 31394 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:38.918455 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.895859 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:38.922353 30559 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:38.897661 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:38.896382 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:38.898288 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.926167 30550 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:38.880384 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.928817 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:38.929646 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.906416 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.937232 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:38.938087 30554 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:38.940138  5213 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.937321 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.944207 31394 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:38.943686 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.946209 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.946811 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:38.948269 30630 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:16:38.892736  5327 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.949194 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:38.884208  5208 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.914405  5216 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:38.959388 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.959468  5327 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:38.959651  5216 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:38.961879  5327 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:38.947954 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.889770  5234 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:38.965857 31388 raft_consensus.cc:2393] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:38.967236 30559 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:38.967468 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:38.964071  5310 consensus_queue.cc:1059] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:16:38.970008 31388 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:38.954094 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.960253 30630 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:38.976300 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.957646 30552 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:38.978153 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.980947 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:38.981914 30630 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:38.951284 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:38.987573 31389 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:38.971051 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:38.884744  5320 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:16:38.999210 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.004148 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:39.003664 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.004375 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.004932 31394 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:39.015295  5208 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:16:39.000964 31389 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:39.016125 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.016985 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.017468  5320 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:38.962527  5216 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:39.008709  5327 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.008603 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.016052 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.017510 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.012674 30630 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:39.016032  5208 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:39.017949  5320 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.006439 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:39.019960 31388 raft_consensus.cc:3060] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:39.018167 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.006211 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.021158 30627 raft_consensus.cc:2393] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:39.021744  5327 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.015311 30629 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:39.024678 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.026077 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.026727 30629 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:39.028195 31394 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:39.020931 31389 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:39.029446 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.031685 30557 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:39.033774 30632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2132) took 1306 ms. Trace:
I20260709 13:16:39.008109 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:39.035913 30596 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.040333 31394 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:39.042730 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.043159 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.044086 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:39.045431 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.038460 30632 rpcz_store.cc:276] 0709 13:16:37.727425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.727604 (+   179us) service_pool.cc:224] Handling call
0709 13:16:37.741618 (+ 14014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.742119 (+   501us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:37.781317 (+ 39198us) log.cc:844] Serialized 14 byte log entry
0709 13:16:37.965116 (+183799us) log.cc:844] Serialized 14 byte log entry
0709 13:16:38.399728 (+434612us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b1800442e40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0557 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:38.892571 (+492843us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b1800442e40. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:16:38.949260 (+ 56689us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.954566 (+  5306us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.955161 (+   595us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:38.955210 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:38.955277 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.033028 (+ 77751us) raft_consensus.cc:1685] finished
0709 13:16:39.033063 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.033687 (+   624us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10242,"prepare.queue_time_us":46644,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":1716,"raft.queue_time_us":11636,"raft.run_cpu_time_us":488,"raft.run_wall_time_us":34761,"spinlock_wait_cycles":16690176,"thread_start_us":216664,"threads_started":2,"wal-append.queue_time_us":237996}
I20260709 13:16:39.048986 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.046185 31389 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:39.051347 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.041249  5303 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:39.052608 31383 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:16:39.054531 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.056056 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.056921  5289 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.057102  5169 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:39.057484  5289 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.057483 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.058413  5333 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.058216  5153 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:39.038931  5218 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:39.061378  5289 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.061749  5317 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:39.047106  5155 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:39.030258  5320 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:39.066581 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.067257 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.050568  5216 raft_consensus.cc:2804] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:39.071323  5323 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:39.067560 30599 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:39.071686  5310 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:39.067804  5208 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:39.087358 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:39.195072 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 127916us
I20260709 13:16:39.196836 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:39.201061 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 113840us
I20260709 13:16:39.205511  5153 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:39.205866  5255 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:39.208858  5327 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:39.071956  5333 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.077018  5317 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.211484  5317 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:39.213376 30627 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:16:39.231314 31388 raft_consensus.cc:2468] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:39.227456 30599 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.236722  5255 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:39.236759  5153 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:39.237704  5153 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.238237  5153 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:39.238118  5255 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.241523  5308 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:39.242163  5308 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:39.243634 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.243522  5308 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:39.246559 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49771us
I20260709 13:16:39.249742 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.250730 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:39.255041  5289 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:39.260246 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.082283  5323 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.288864 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.289359 30630 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:39.269133  5269 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:39.272089  5332 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:16:39.272533 30703 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:39.272861  5279 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.269613  5321 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.275593  5333 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:39.191924  5218 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:39.191476  5216 raft_consensus.cc:697] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:39.277599 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.315675 30559 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:16:39.277797  5318 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:39.317373 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.280742 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2136) took 1185 ms. Trace:
I20260709 13:16:39.300788  5323 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:39.303521 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2011) took 1820 ms. Trace:
I20260709 13:16:39.291441 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:39.325875 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:39.267349  5270 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:39.328689 30703 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:39.309479  5321 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.311885  5284 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.304653  5269 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.307709  5279 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.318710 30624 rpcz_store.cc:276] 0709 13:16:38.095453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.095593 (+   140us) service_pool.cc:224] Handling call
0709 13:16:38.095935 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.290239 (+194304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.280386 (+990147us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54004a8dd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.280663 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365787520}
I20260709 13:16:39.328087  5289 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.335323 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.335779  5289 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.184000  5208 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.338249 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:39.321337 31391 rpcz_store.cc:276] 0709 13:16:37.450195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.450358 (+   163us) service_pool.cc:224] Handling call
0709 13:16:37.516904 (+ 66546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:38.249388 (+732484us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:38.249423 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:38.249972 (+   549us) log.cc:844] Serialized 26 byte log entry
0709 13:16:38.307067 (+ 57095us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:38.307592 (+   525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.235354 (+927762us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b5400521cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.235445 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.235530 (+    85us) raft_consensus.cc:1685] finished
0709 13:16:39.235552 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.270891 (+ 35339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":883,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":1683356544,"thread_start_us":56563,"threads_started":1,"wal-append.queue_time_us":78317}
I20260709 13:16:39.304657  5315 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:39.324692 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:39.323153  5333 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:16:39.340044 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.322734  5218 raft_consensus.cc:697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:39.352394 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.327798  5234 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:39.328636  5270 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:39.355938 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:39.343370  5208 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:39.343811  5308 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:39.347623  5333 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.345883  5318 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:39.396772 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.353647  5315 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:39.358139  5234 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.366991  5032 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.365803  5166 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.420240  5141 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:16:39.420802  5141 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:39.395952  5318 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.419952  5166 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.395547  5333 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:39.422458  5318 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:39.357381  5153 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.404968  5234 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:39.402980  5313 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.408344  5308 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:39.421283  5218 consensus_queue.cc:237] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.421854  5141 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.424726  5310 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.453706 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2419) took 2268 ms. Trace:
I20260709 13:16:39.454298 30701 rpcz_store.cc:276] 0709 13:16:37.185100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.185337 (+   237us) service_pool.cc:224] Handling call
0709 13:16:39.453391 (+2268054us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:39.453590 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653190912}
I20260709 13:16:39.369422  5216 consensus_queue.cc:237] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.461022 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.468569  5087 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:39.469671  5087 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.476039  5006 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:39.490855  5321 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.491765  5321 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:39.492036 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.494253 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.494814  5042 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.495113 30559 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:39.495532  5042 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.496321  5166 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.499603  5279 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.424801  5032 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.502067 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.503021 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.503736 31389 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:39.462325  5310 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.506809  5141 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:39.442839  5270 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.507228 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:39.509663  5298 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.366s	user 0.006s	sys 0.000s
I20260709 13:16:39.479928  5269 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.484402  5315 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.498234  5042 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.487569  5284 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:39.486560 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:39.511487  5238 raft_consensus.cc:3055] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:39.512012  5238 raft_consensus.cc:740] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:39.512737  5202 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:39.513275  5202 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:39.512880  5238 consensus_queue.cc:260] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.513702  5202 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:39.514062  5238 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:39.515960  4649 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:39.516443 30559 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:39.511550  5284 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.521462  5284 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.521836 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.522807 30557 raft_consensus.cc:2410] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:16:39.524359  5270 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:39.444270  5313 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.532213  5245 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.533743  5317 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.544657 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:39.545722 30705 raft_consensus.cc:3060] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:39.546072 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.546934 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.547068 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:39.547809 30707 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:39.548384 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.549853 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:39.552821 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2144) took 1172 ms. Trace:
I20260709 13:16:39.553342 30625 rpcz_store.cc:276] 0709 13:16:38.380587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.396353 (+ 15766us) service_pool.cc:224] Handling call
0709 13:16:39.552592 (+1156239us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54004ab5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:39.552753 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981992832}
I20260709 13:16:39.559072 30633 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:16:39.567091 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1575) took 1444 ms. Trace:
I20260709 13:16:39.567584 30628 rpcz_store.cc:276] 0709 13:16:38.122712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.399654 (+276942us) service_pool.cc:224] Handling call
0709 13:16:38.399918 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.556166 (+156248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.566812 (+1010646us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54007aa654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.567018 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":338738560}
I20260709 13:16:39.569764 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:39.573128 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.577262 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.588459 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.582060 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.575021 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.607620 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.626214 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2131) took 1903 ms. Trace:
I20260709 13:16:39.627767 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.627084 30626 rpcz_store.cc:276] 0709 13:16:37.722547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.724183 (+  1636us) service_pool.cc:224] Handling call
0709 13:16:37.739489 (+ 15306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:37.752131 (+ 12642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.752177 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.281963 (+1529786us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:39.282478 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 13:16:39.565751 (+283273us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:39.565828 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.565955 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:39.566013 (+    58us) raft_consensus.cc:1685] finished
0709 13:16:39.566034 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.626117 (+ 60083us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":865,"raft.queue_time_us":6546,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4690,"spinlock_wait_cycles":4162304,"thread_start_us":282661,"threads_started":1,"wal-append.queue_time_us":254508}
I20260709 13:16:39.595817 30522 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:39.619107 30705 raft_consensus.cc:2468] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:39.632242 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2237) took 1737 ms. Trace:
I20260709 13:16:39.640054 31396 rpcz_store.cc:276] 0709 13:16:37.895052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.900186 (+  5134us) service_pool.cc:224] Handling call
0709 13:16:37.900566 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:37.921417 (+ 20851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:37.921454 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:37.921505 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:37.921546 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.631805 (+1710259us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b54002b07d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.631866 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:39.632173 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47320832}
W20260709 13:16:39.653890 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46401.8us
I20260709 13:16:39.663643 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:39.663314 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.665191 30704 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:39.671769 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:39.666970 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:39.570797  5290 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.592450  5017 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:39.698082 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.591781  5042 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:39.718151 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.719197 31388 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:39.702575 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:39.633088  4649 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:39.722244 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.721526 30705 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:39.663069  5327 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:39.732973 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:39.725200 30707 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
W20260709 13:16:39.727155 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.020s (SENT) [suppressed 12 similar messages]
I20260709 13:16:39.729231 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.594182  5292 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 13:16:39.738530 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:39.688213  5213 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:39.747711 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.701819  5255 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:39.748699 31378 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:39.646196  5320 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:39.733064  5287 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:39.761690  5289 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.748504  5213 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:39.761101  5098 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.723562  5042 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.708490  5290 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.772688  5287 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.792416  5287 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.777164  5213 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:39.809474  5313 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.799393  5234 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:39.815536  5317 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:39.824581  5290 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:39.818198  5234 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.839735  5320 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:39.879462 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 211245us
I20260709 13:16:39.888218 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:39.891582 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:39.892889 30557 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:39.903278 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:39.921059 30596 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.006s	sys 0.018s
W20260709 13:16:39.921923 30596 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.006s	sys 0.018s
W20260709 13:16:39.923241 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.017s	sys 0.008s
W20260709 13:16:39.924012 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.017s	sys 0.008s
W20260709 13:16:39.924849 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 569002us
W20260709 13:16:39.927912 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.928345  5318 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:39.831648  5323 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.003s
I20260709 13:16:39.931710  5323 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:39.792840  5042 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:39.916076 30557 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:39.794276  5098 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:39.959277 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.960532 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2191) took 2864 ms. Trace:
I20260709 13:16:39.960685  5042 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:39.961383  5042 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:39.961941  5042 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:39.961199 30558 rpcz_store.cc:276] 0709 13:16:37.095700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.119097 (+ 23397us) service_pool.cc:224] Handling call
0709 13:16:37.147874 (+ 28777us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.401153 (+1253279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.959778 (+1558625us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54000cb2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.960241 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880584960}
I20260709 13:16:39.962144  5098 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:39.967619 31388 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 16: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:39.968410 30624 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:39.967713 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:39.981036  5320 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:39.924608 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:39.949800  5333 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.511s	user 0.007s	sys 0.009s
W20260709 13:16:39.984233 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.983762  5202 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:39.989307 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2241) took 1736 ms. Trace:
W20260709 13:16:39.991075 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 355416us
W20260709 13:16:39.992300 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:39.992151 30701 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:39.982033  5318 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:39.993388 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90528.1us
I20260709 13:16:40.002420 31378 raft_consensus.cc:2393] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:40.003149  5320 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.005561 31378 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:16:39.997349 31361 rpcz_store.cc:276] 0709 13:16:38.252192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.252660 (+   468us) service_pool.cc:224] Handling call
0709 13:16:38.253075 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:38.728591 (+475516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:39.988351 (+1259760us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400680454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:39.988947 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1080141568}
I20260709 13:16:40.013907  5323 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.014147  5333 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:40.022665 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98611.6us
I20260709 13:16:40.022939 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.023070  5025 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:40.038057  5025 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.042896  5025 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:40.043949 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 316719us
I20260709 13:16:40.050331  5287 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:16:40.053217  5287 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:40.053911  5287 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:40.061091  5287 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.062083  5166 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.068586  5166 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.037286  5320 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.068651  5279 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:16:40.077621 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.080992  5310 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.081229 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.083405 31361 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:40.081017  5202 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.084172  5166 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:40.078605 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.089638  5287 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:40.094395 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:40.095245  5202 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:40.095875 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:40.099458 30698 raft_consensus.cc:1217] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Deduplicated request from leader. Original: 19.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 13:16:40.100361 30698 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:16:40.106304  5349 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:40.112439  5268 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:40.116322  5269 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:40.116544 30673 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.117235  5269 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:40.126149 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 49586.4us
I20260709 13:16:40.099335  5320 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:40.137482  5269 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.139909  5087 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.140282  5310 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:40.141593  5310 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:40.141983  5310 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:40.142794  5245 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.143623  5245 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:40.183336 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 82166.5us
I20260709 13:16:40.139922  5269 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:40.135843  5268 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:40.187877 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 66986.8us
W20260709 13:16:40.194203 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 96251.2us
I20260709 13:16:40.196077 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.196661  5042 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.199008  5017 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.199693  5017 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:40.197371  5270 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.014s
I20260709 13:16:40.202016 30558 raft_consensus.cc:2368] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:40.185724  5245 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:40.206662 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.208043  5042 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.209260 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:40.209527 30704 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.016s
W20260709 13:16:40.222465  5006 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.733s	user 0.003s	sys 0.026s
I20260709 13:16:40.223474 31364 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
W20260709 13:16:40.226797 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.227514 30702 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 13:16:40.227890  5303 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:16:40.229628 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.227981 30702 pending_rounds.cc:85] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 14
I20260709 13:16:40.232966 30702 pending_rounds.cc:107] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 13:16:40.244840 30702 raft_consensus.cc:2889] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:40.204231  5349 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:40.240327 30704 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:40.250708 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.251223 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.251772 30551 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.252828 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.253701 30552 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:40.255237  4696 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.255885 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:40.256312 30597 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:40.206714  5310 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.259002  5166 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:40.259727  5310 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:40.261751 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.262540 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.263278 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.263449 30627 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:40.264361 31367 raft_consensus.cc:2393] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:40.266113 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.268055 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.271682  5320 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.272563  5320 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.279237 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:40.281136  5208 consensus_meta.cc:287] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.014s
I20260709 13:16:40.281376  5187 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:40.281805  5208 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.258826 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:40.298090 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 177998us
I20260709 13:16:40.288697 30633 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
W20260709 13:16:40.305014 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75660.9us
I20260709 13:16:40.305151 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.306506  5202 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:40.308599 30554 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:40.309566  5279 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:40.311576  5364 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:40.311903  5323 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.312098  5364 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:40.312610 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1148) took 2248 ms. Trace:
I20260709 13:16:40.312829  5323 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.313122 30698 rpcz_store.cc:276] 0709 13:16:38.063836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:38.063995 (+   159us) service_pool.cc:224] Handling call
0709 13:16:39.983796 (+1919801us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b5800628e98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180beb6fc0 00007f180bea1929 00007f180bea13d2 00007f1805418bd4
0709 13:16:40.099096 (+115300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:40.100771 (+  1675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.312329 (+211558us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54005f05d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:40.312514 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271263360}
I20260709 13:16:40.313261 30599 leader_election.cc:400] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:40.314759 30599 leader_election.cc:403] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:16:40.314925 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:40.318904  5141 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.811s	user 0.006s	sys 0.019s
I20260709 13:16:40.318960  5213 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.259757  5166 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:40.319540  5364 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:40.321398 30559 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.014s
I20260709 13:16:40.313896  4649 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:40.358414  5169 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.358712  4649 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.359459  5169 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.361550  4649 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:40.365777 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 137) took 42645 ms (client timeout 15000 ms). Trace:
I20260709 13:16:40.367384 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.369791 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.337132 31361 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:40.339258  5153 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.339987  5251 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:40.340093  5032 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.343750  5333 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.346455 30555 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:40.350446  5135 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:40.418881 30550 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:40.419842 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.377432 30627 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:40.395540 30552 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:40.386662 30557 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:40.402493 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2422) took 2430 ms. Trace:
I20260709 13:16:40.351380 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:40.352344 31378 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:40.352716 31389 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:16:40.354411 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.357862  5323 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:40.436633 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 133350us
I20260709 13:16:40.427609 30702 rpcz_store.cc:276] 0709 13:16:37.972164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:37.972326 (+   162us) service_pool.cc:224] Handling call
0709 13:16:37.972664 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:40.245902 (+2273238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.402048 (+156146us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b54002c4cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:40.402395 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5183838592}
I20260709 13:16:40.410712  5169 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.415017 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.385090 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.255937  4696 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:40.447611 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 134631us
I20260709 13:16:40.447402 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.413270 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.354274  5242 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.384142  5349 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.452172  5242 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.451478 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.454563 31378 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:40.376364  5208 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.454937 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.455648 31383 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:40.395974 30559 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:40.405611  5364 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.385679  5320 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:40.461755 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.463315 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.464347 30633 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:40.464695 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.465543 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.451463  5324 raft_consensus.cc:3060] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:40.460428 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.454813 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:40.437915  5251 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:40.462819 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:40.462850 30624 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 13:16:40.461826 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.446898 30551 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:40.462127 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.476356 30555 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:40.470363 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.482108 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.485047 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.488350 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.490734  5135 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:16:40.494764 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.404289  5025 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:40.497473 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.503111 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.503806 30626 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:40.512897 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.491396  5135 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:40.514266 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.509429 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.514392 31364 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:40.476250  5242 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:40.529841 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.533878 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.534970 30557 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:40.540241 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.541468 30628 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:40.529342  5169 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.543228  5333 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:16:40.544580  5238 consensus_meta.cc:287] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.988s	user 0.012s	sys 0.012s
I20260709 13:16:40.533486 30552 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:40.550766 30552 raft_consensus.cc:1240] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:40.553320  5166 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.556964  5251 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:40.558689  5363 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:40.558845 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46551.2us
I20260709 13:16:40.559494  5363 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:40.560145 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:40.560985 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.511726  5318 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
W20260709 13:16:40.567757 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 154818us
I20260709 13:16:40.569082 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.569697  5213 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:40.570917  5213 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:40.571312 30702 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 13:16:40.572861 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 221671us
I20260709 13:16:40.574306 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:40.575105 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:40.575350 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 101412us
I20260709 13:16:40.576538 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.543355  5169 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:40.577126  5284 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:40.577803 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:40.554100  5367 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.333s	user 0.000s	sys 0.004s
I20260709 13:16:40.562173  5363 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.580663 30552 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:40.577450 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.562284 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.522181  5135 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.586724 30550 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 27.
I20260709 13:16:40.525098  5042 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:40.588912 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:40.599471 30555 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:40.600044 30551 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:40.601105 30704 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:40.581223  5238 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:40.583510  5284 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.604592 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:40.607861 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.619714 30701 raft_consensus.cc:2410] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:16:40.622985 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.604242  5032 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:40.366312 30382 rpcz_store.cc:269] 0709 13:15:57.720506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:57.726822 (+  6316us) service_pool.cc:224] Handling call
0709 13:16:40.365504 (+42638682us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:31.132239 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:31.133002 (+   763us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:31.133058 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:16:31.133076 (+    18us) tablet.cc:662] Decoding operations
0709 13:16:31.407182 (+274106us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:31.436388 (+ 29206us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:31.441188 (+  4800us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:16:31.520078 (+ 78890us) tablet.cc:701] Row locks acquired
0709 13:16:31.520137 (+    59us) write_op.cc:260] PREPARE: finished
0709 13:16:31.520353 (+   216us) write_op.cc:270] Start()
0709 13:16:31.520615 (+   262us) write_op.cc:276] Timestamp: P: 1783602991520261 usec, L: 0
0709 13:16:31.520661 (+    46us) op_driver.cc:348] REPLICATION: starting
0709 13:16:31.521552 (+   891us) log.cc:844] Serialized 6698 byte log entry
0709 13:16:32.115024 (+593472us) op_driver.cc:464] REPLICATION: finished
0709 13:16:32.523094 (+408070us) write_op.cc:301] APPLY: starting
0709 13:16:32.523285 (+   191us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:32.697911 (+174626us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:32.698035 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:33.777493 (+1079458us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:33.777996 (+   503us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:16:33.778082 (+    86us) write_op.cc:312] APPLY: finished
0709 13:16:33.778731 (+   649us) log.cc:844] Serialized 155 byte log entry
0709 13:16:33.779051 (+   320us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:33.779262 (+   211us) write_op.cc:454] Released schema lock
0709 13:16:33.779347 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:33.796882 (+ 17535us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b18000a64e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:33.874350 (+ 77468us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:33.876689 (+  2339us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2536132,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":407618,"apply.run_cpu_time_us":36672,"apply.run_wall_time_us":1329420,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75357,"num_ops":17,"prepare.queue_time_us":301729,"prepare.run_cpu_time_us":34288,"prepare.run_wall_time_us":533098,"raft.queue_time_us":189245,"raft.run_cpu_time_us":30482,"raft.run_wall_time_us":288421,"replication_time_us":594125,"spinlock_wait_cycles":9872256,"thread_start_us":619969,"threads_started":4,"wal-append.queue_time_us":217914,"wal-append.run_cpu_time_us":23017,"wal-append.run_wall_time_us":2270222}]]}
I20260709 13:16:40.612607  5284 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:40.623780 30624 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:40.614266  5251 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.635560 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:40.625695  5042 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:40.644237 31383 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:40.665076  5308 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:40.666023 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:40.667960 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154741us
I20260709 13:16:40.669106 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:40.670727 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.672320 30556 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 26. Current term is 27. Ops: []
I20260709 13:16:40.644332 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.684729 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:40.684973 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.694523 30624 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 13:16:40.638725  5202 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:40.702453 30626 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
W20260709 13:16:40.705924  5017 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.506s	user 0.016s	sys 0.000s
I20260709 13:16:40.706781  5017 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.707537  5279 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:40.708266  5279 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:40.712163  5169 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.713721 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:40.674384  5141 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:16:40.730775 30703 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:40.693414  5324 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:40.710320  5017 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.722378  5213 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:16:40.722079  4979 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:40.757723  4979 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:40.722898  5333 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:40.731745 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 141938us
I20260709 13:16:40.733445 30557 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:40.662904  5042 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:40.724931 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.725500 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.738724 30551 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
W20260709 13:16:40.743032 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 180906us
W20260709 13:16:40.758083 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 114417us
W20260709 13:16:40.730847 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 155901us
I20260709 13:16:40.709667  5279 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.760834 30559 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:40.762377 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:40.725010  5032 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.764959 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.766551 30599 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:40.769513 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:40.780125  5213 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:40.780994 30555 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.761607  5042 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:40.787465 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:40.788964 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:40.791844 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:40.795706  5169 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:40.796033 30522 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:40.796422  5169 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.797772  5355 raft_consensus.cc:2804] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:40.798355  5355 raft_consensus.cc:697] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:40.799183  5169 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.799962  5355 consensus_queue.cc:237] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.803033 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.805006 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:40.805742 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:40.806802  5017 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:40.807159 31391 raft_consensus.cc:2368] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24.
W20260709 13:16:40.811069 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 136319us
I20260709 13:16:40.770157  5141 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:40.821043 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:40.827059 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49764us
I20260709 13:16:40.831519 30522 leader_election.cc:400] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24.
I20260709 13:16:40.832216 30522 leader_election.cc:403] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 13:16:40.813472  5310 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.002s
I20260709 13:16:40.833526 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:40.835814 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:40.812546  5372 raft_consensus.cc:2764] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:16:40.817123  5017 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:40.809226  5032 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.861019  5384 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24.
I20260709 13:16:40.873028  5141 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.866645  5320 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:40.877518  5320 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:40.868146  5308 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:40.881052 30624 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:40.856714  5363 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:16:40.882329  5363 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:40.885699  5363 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:40.886585 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:40.886917  5363 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.887614 30625 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:40.888371  5363 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:40.887292 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1313) took 1863 ms. Trace:
I20260709 13:16:40.846076  5279 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:40.873638  5032 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.894089 30626 raft_consensus.cc:1240] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:40.867367  5333 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:40.926941 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 109127us
W20260709 13:16:40.932993 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 65657.4us
W20260709 13:16:40.944970 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 179793us
I20260709 13:16:40.941772 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:40.896345  5169 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.950695 30701 raft_consensus.cc:1275] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:40.891145 30629 rpcz_store.cc:276] 0709 13:16:38.905492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.027364 (+121872us) service_pool.cc:224] Handling call
0709 13:16:39.027758 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:39.030379 (+  2621us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:39.030415 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:39.720559 (+690144us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b1800173740. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a
0709 13:16:40.348126 (+627567us) spinlock_profiling.cc:243] Waited 4.4 ms on lock 0x7b2c00323c88. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:40.348922 (+   796us) log.cc:844] Serialized 26 byte log entry
0709 13:16:40.622259 (+273337us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:40.622325 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:40.638506 (+ 16181us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:40.740731 (+102225us) raft_consensus.cc:1685] finished
0709 13:16:40.740765 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:40.768599 (+ 27834us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82,"raft.queue_time_us":5259,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":1286,"spinlock_wait_cycles":16972416,"thread_start_us":268861,"threads_started":1,"wal-append.queue_time_us":378141}
I20260709 13:16:40.897547  5320 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:40.948329  5153 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:40.873135  5017 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.963203  5381 consensus_queue.cc:1059] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:40.964890  5381 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:40.965317  5381 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:40.935988  5006 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:16:40.968197  5135 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:40.977802 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:40.984108  5369 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:16:40.988687 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:40.990101  5349 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:40.990667  5349 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:40.991338  5349 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:40.994664 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:40.997946 30523 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:40.999168  5378 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 13:16:41.001523 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46160.2us
I20260709 13:16:40.956676  5169 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.005653  5169 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:40.966209  5381 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:40.951661  4696 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:41.007982  5381 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:40.963265  5153 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:40.990489 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:41.016008 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 61988.3us
I20260709 13:16:41.006223  5379 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:40.969373  5380 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:16:40.954039  5166 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:40.949359  5042 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.025808 30555 raft_consensus.cc:2468] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:41.016832  5153 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:41.033599  5238 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.030516  5006 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:41.033659  5213 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:41.044641 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54250.4us
I20260709 13:16:41.045992  5238 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.074796 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:41.082151  5042 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:41.064188  5245 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:41.059250  5310 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:41.059592  5153 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:41.041191  5166 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.018972  5349 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:41.097019 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 150766us
I20260709 13:16:41.103996  5153 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.101361 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.116025 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.117270 31383 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:41.123023  5283 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:41.126012  5360 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:41.112229  5310 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.120788  5153 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:41.126791  5141 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:41.142350  5141 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:41.143482  5141 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:41.126549  5377 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.178s	user 0.000s	sys 0.003s
W20260709 13:16:41.149353 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 90993.4us
I20260709 13:16:41.127974  5202 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:16:41.150765  5320 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:16:41.150916 30597 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.154057 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.128517  5238 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:41.157786  5317 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:41.154834 30525 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.001s	sys 0.048s
W20260709 13:16:41.158607 30525 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.001s	sys 0.048s
I20260709 13:16:41.158766  5317 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:16:41.160269 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.161311 30597 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.162240  5317 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:41.162344  5379 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:41.163139  5379 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:41.164048 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 85502.4us
I20260709 13:16:41.165613  5379 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:41.166265  5379 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:41.166944 30555 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:41.137270  5310 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:41.167739 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.137764 30599 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:41.141247 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 48286.5us
I20260709 13:16:41.169803  5349 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.170047 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.141497  5284 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.164162 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:41.170570 31292 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:41.166903 30378 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 24. New cstate: current_term: 24 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:41.155426  4979 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.172497 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.180334 31389 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:41.171804 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.173309 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.183084 31361 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:41.183857  5284 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:41.185199 30551 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:41.154763  5320 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:41.172988  5025 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.191937 31391 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
W20260709 13:16:41.195374 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57688.3us
I20260709 13:16:41.201436 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:41.202723  5387 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.094s	user 0.000s	sys 0.002s
I20260709 13:16:41.202922  5242 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:41.204074 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.204133  4979 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:41.204985 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.205478  4979 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.137465  5166 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.205896  4979 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:41.201506 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:41.210979 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:41.211889 30551 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 15.
I20260709 13:16:41.218719 31383 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:41.135601  5135 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.246481 30551 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:16:41.247112 30551 pending_rounds.cc:85] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 11
I20260709 13:16:41.247352 30551 pending_rounds.cc:107] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:16:41.247609 30551 raft_consensus.cc:2889] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:41.247985  5251 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:41.249529 30550 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:41.225224  5333 raft_consensus.cc:2804] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:41.264993 30629 raft_consensus.cc:1275] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:16:41.262214 30522 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:41.262063 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.266436 31378 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:41.259989 30625 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:41.270931  5333 raft_consensus.cc:697] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:41.272398  5333 consensus_queue.cc:237] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:41.283730 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:41.281440 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.287454 31292 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.288789 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.289160 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.236877  5213 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:16:41.248781  5324 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:41.296989 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.298451 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.299413 31383 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:41.292675 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:41.249302 30378 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 23, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 23 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:41.313644 30378 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:41.314916 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:41.316352 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:41.317273 30625 raft_consensus.cc:2368] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24.
I20260709 13:16:41.293016  5284 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:41.319552 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.319710  5317 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.320513  5317 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.319553  5284 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:41.313985  5153 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 13:16:41.326977 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:41.326651 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:41.328306 30705 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:41.328374  5283 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:16:41.333447 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:41.333250 30700 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:41.339258 30703 raft_consensus.cc:1240] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18. Ops: []
I20260709 13:16:41.329939 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:41.296478  5169 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:41.343834 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:41.340265 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.361316 30556 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:41.358350 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:41.365319 30705 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:41.305935  5238 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:41.356804 30378 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 19, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 19 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:41.367383 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:41.336895 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0290309420001904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1698s for a response
W20260709 13:16:41.368366 30523 connection.cc:256] RPC call timeout handler was delayed by -1.0056309420001526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:16:41.369484 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:41.370932 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:41.371560 30704 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
I20260709 13:16:41.376129 30553 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:41.393937 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:41.343884  5310 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:41.340847  5283 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:41.328173  5393 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:41.405005 30378 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 15, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 15 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:41.398639 30702 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:16:41.404866 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.357149  5169 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:41.368443  5355 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:41.412590  5317 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:41.412725 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 108456us
I20260709 13:16:41.420243 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:41.425037 30554 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:41.424638 30378 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 17, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 17 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:41.374147  5238 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:41.403825  5025 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:41.356773  5032 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 13:16:41.357352  5320 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:16:41.322172  5284 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.427167 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.366520  5324 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.403838  5310 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:41.443605 30378 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 20 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:41.406834  5153 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:41.430938 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.407886  5393 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:41.444604 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 331145us
I20260709 13:16:41.424489  5169 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:41.462298 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:41.421707  5355 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:41.433167  5238 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:41.474591 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.472199 30554 raft_consensus.cc:2376] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16. Re-sending same reply.
I20260709 13:16:41.463708  5310 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:41.518877 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 52489.5us
I20260709 13:16:41.519307 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:41.469563  5395 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:16:41.431973  4979 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.510773 30378 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 19 to 21, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:41.529882 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:41.464671  5379 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:41.461781  5317 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
W20260709 13:16:41.539398 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 92012.8us
W20260709 13:16:41.546295 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 119216us
W20260709 13:16:41.548321 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:41.541467 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 209295us
I20260709 13:16:41.470495  5393 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:41.544638 31361 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:41.473105  5153 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:41.479224  5042 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:41.551674 30671 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.552721 30378 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 20 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:41.555372 30597 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:41.475688  5355 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:41.558192 30523 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.502888  5320 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.510244  5032 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:41.600574 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 202901us
I20260709 13:16:41.528645  5025 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:16:41.556258  5135 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:41.600988  5403 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:16:41.561348 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 217060us
I20260709 13:16:41.545392  5363 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 13:16:41.600539  5279 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:41.637660 30378 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 20 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:41.614675  5006 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:41.553906  5379 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.676235 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:41.553073  5283 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.652851  5135 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:41.677600  5042 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:41.656775  5403 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:41.671975  5087 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:41.714967 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 172261us
W20260709 13:16:41.715797 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 157704us
I20260709 13:16:41.719125 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.719137 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:41.729312 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117276us
W20260709 13:16:41.749396 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:41.730937  5418 raft_consensus.cc:3055] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:41.741263  5408 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:41.703956  5213 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:41.657500  5320 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:41.740592  5398 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:16:41.748658  5379 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:41.755347  5287 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:41.720194  5406 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:16:41.711935  5135 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:41.736146 30378 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:41.783344 30627 raft_consensus.cc:2468] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:41.767973  5363 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:41.721083  5006 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:16:41.781260  5418 raft_consensus.cc:740] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:41.804695 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 146742us
I20260709 13:16:41.786988  5408 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:41.830525 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 111473us
W20260709 13:16:41.837188 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 281967us
I20260709 13:16:41.841571 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.799925  5042 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:41.804239  5324 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.847549 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.849588 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.737128  5153 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.853824 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.851125 30378 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:41.853866 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.859019 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.855019 31364 raft_consensus.cc:1275] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:16:41.788108  4696 consensus_queue.cc:1059] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:16:41.862780 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:41.863611 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:41.803267  5287 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:41.803208  5213 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:41.843402  5408 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:41.880169 31367 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:16:41.880717 31367 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:41.881690 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:41.870839 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 87079.4us
I20260709 13:16:41.861627 30378 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 19, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:16:41.891714 31378 consensus_meta.cc:287] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.007s
I20260709 13:16:41.892257 31378 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:41.858155  4979 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:41.895325 31389 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:41.896502 30378 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:41.898298 30378 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 10 to 13, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 13 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:41.894671 30522 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:41.901996 30378 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:41.872686  5324 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.893594 31367 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.914911  5419 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.844726  5403 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:41.940194  5410 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:41.940902  5409 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.940074  5419 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.872918  5363 consensus_queue.cc:1059] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:16:41.900513  5042 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:41.947208 30705 consensus_meta.cc:287] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.579s	user 0.012s	sys 0.006s
I20260709 13:16:41.949635 30378 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 14, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 14 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:41.952988 30705 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:41.935809  5379 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:16:41.963774 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 308427us
W20260709 13:16:41.971441 30556 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.006s
I20260709 13:16:41.974519  5166 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:41.977236  5166 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.979775  5166 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:41.955156  5413 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
I20260709 13:16:41.972601 30556 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:41.992923 31367 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:16:41.954744 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62191.3us
W20260709 13:16:41.968029 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 87729.3us
I20260709 13:16:41.959388  5287 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.029304 30378 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 5 to 18, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 18 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:41.982987  5414 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
W20260709 13:16:42.013967  5417 log.cc:927] Time spent T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:16:41.959872  5423 raft_consensus.cc:2804] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:16:42.035496 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59228us
I20260709 13:16:42.036296 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:41.970299  5242 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:16:41.965767  5324 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.042158 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:41.966109  5042 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.044109 30378 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 18 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:16:41.941541  5409 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:41.972949  5393 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.085916  5032 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:42.086875  5421 consensus_queue.cc:1048] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:42.088666 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:42.083593 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47339.6us
I20260709 13:16:42.039605  5166 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:16:42.051133  5423 raft_consensus.cc:697] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.116226 30378 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 19, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:42.096544  5213 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:42.106729  5411 consensus_queue.cc:1048] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 13:16:42.107664 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 267498us
I20260709 13:16:42.119434 30378 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 11 to 12, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 12 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:42.108527  5087 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:42.110366  5042 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:42.111629  5424 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:16:42.113301  5324 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.122051 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.121137 30378 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 18, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:42.079147  5320 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:42.132135 31364 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
W20260709 13:16:42.140729 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:42.141558 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 117617us
W20260709 13:16:42.144022 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 101978us
I20260709 13:16:42.145390 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:42.147775 31391 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:42.094888  5360 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:16:42.131439  5238 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.625s	user 0.014s	sys 0.000s
I20260709 13:16:42.151721 30378 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 8 to 16, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 16 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:42.155979 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.157342 30378 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:16:42.123149  5087 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.163357 30378 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:42.187529 31367 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 13:16:42.188174 31367 pending_rounds.cc:85] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 14
I20260709 13:16:42.188426 31367 pending_rounds.cc:107] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 13:16:42.188702 31367 raft_consensus.cc:2889] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:42.190140  5439 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:42.191939  5429 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.192845  5429 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.183808 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:42.197297 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:42.199400 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43522.6us
I20260709 13:16:42.161406  5042 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:42.165024  5087 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.214905 30552 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:42.215490 30556 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:16:42.182930  5213 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:16:42.217972  5408 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.219074 30625 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:42.220768 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.221920 30633 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:42.224692  5032 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.224864  5422 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.225418  5032 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.225576  5422 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.226984  5318 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.227638  5318 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.230026  5318 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:42.169502  5360 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:16:42.159930  5238 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.234543 31378 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:42.236562  5360 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:42.237133  5360 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.239452  5423 consensus_queue.cc:237] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.243149  5349 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:42.205529  5169 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.702s	user 0.004s	sys 0.010s
I20260709 13:16:42.245117  5169 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:42.250149  5355 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.005s
I20260709 13:16:42.253804  5202 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:42.218482  5441 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:42.257129  5442 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:16:42.183933  5310 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.626s	user 0.005s	sys 0.010s
W20260709 13:16:42.261422 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 168616us
I20260709 13:16:42.254163 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:42.184671  5410 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:42.172041  5333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.007s	sys 0.011s
I20260709 13:16:42.273412 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.278101 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.278887 30705 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:42.257756  5245 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 13:16:42.257229  5403 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:42.282780 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.284881 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.295002 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.260679  5202 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.298506 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:42.291857 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 201892us
I20260709 13:16:42.304917 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:42.284294  5333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.007s	sys 0.012s
I20260709 13:16:42.328488 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.296067 30707 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:42.267189  5279 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:42.327430 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44735us
I20260709 13:16:42.339857 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.292512  5317 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:42.264379  5355 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.351346 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:42.353343 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.353724 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.356328 30624 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:42.275063  5310 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.366844  5446 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:42.367120 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.341651  5025 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:42.361505 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.361856 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:42.362583  5396 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 13:16:42.362735 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.363106 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.363718 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.346251  5213 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:16:42.386914  5202 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.388391  5386 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:42.388886  5202 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:42.388906  5386 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.390336  5386 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.392177  5386 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.392395  5087 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.395524  5025 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.397087  5025 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:42.401710  5135 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.539s	user 0.013s	sys 0.002s
I20260709 13:16:42.402571  5135 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.365650 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.359814  5333 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:42.360292 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:42.367405 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 62638.4us
W20260709 13:16:42.408330 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48136.9us
I20260709 13:16:42.408841 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:42.409559 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.361557 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.337783  5308 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:42.411865  5411 consensus_queue.cc:1048] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:16:42.367779 30701 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:42.412614  5308 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:42.362787 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.377959  5385 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.413909 30554 raft_consensus.cc:1240] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:42.348222  5317 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.414638  5317 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:42.414335  5385 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.416899  5385 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.376258 30552 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:42.379465 30523 leader_election.cc:400] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 23. Current term is 24.
I20260709 13:16:42.368139  5433 consensus_queue.cc:1059] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:16:42.343088  5279 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.418762  5213 raft_consensus.cc:3055] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:42.419246  5213 raft_consensus.cc:740] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.409478  5446 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:42.410817  5287 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:42.413998 30557 raft_consensus.cc:2376] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20. Re-sending same reply.
I20260709 13:16:42.420626  4696 raft_consensus.cc:3055] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:42.420677  5287 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.423309  5287 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.424994 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:42.426064  5042 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:42.367858 30699 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 15.
I20260709 13:16:42.426854  5042 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.429939  5419 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.430945  5419 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:42.431564  5419 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:42.443627 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.444307 30631 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:42.444519 30628 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:42.445147  5407 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.445788  5407 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.446131 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.446843  5418 consensus_queue.cc:260] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.448316  5418 raft_consensus.cc:3060] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.449258  5446 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.449811  5446 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.451942  5438 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:42.453006  5238 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.453660  5438 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:42.454109  5438 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.454843  5385 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.455441  5385 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.453691 31391 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 19. Current term is 20. Ops: []
I20260709 13:16:42.457935  5385 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.452172  5446 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:42.438272  5429 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:42.458586  5385 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.462335  5385 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.463543  5409 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:42.460897  5438 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.454270 31394 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:42.467202  5419 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.471547  5393 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.473129  5438 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.475406  5213 consensus_queue.cc:260] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.477102  5385 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.479094  5408 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:42.479908 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.480954  5407 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.495280 30705 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:16:42.507489 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.507813 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.508128 30626 raft_consensus.cc:1275] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:42.508513 30707 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:42.509055 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.509461 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.517386 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.514122 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.530651 30625 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 13. Current term is 14. Ops: []
I20260709 13:16:42.533680 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.538064 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:42.538339 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 284271us
I20260709 13:16:42.539295 30523 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.489854  4696 raft_consensus.cc:740] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.546470 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.538296 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:42.517454  5213 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:42.499780  5407 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.556474 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.546145  5006 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:16:42.558863  5431 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:16:42.559657  5433 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:42.554661 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.558997  5251 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:16:42.563407 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 105949us
I20260709 13:16:42.539891  5429 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:42.555303  5284 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:42.581434 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.582598 31394 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 27.
I20260709 13:16:42.587302  5434 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:42.588040  5434 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.588470  5434 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:42.591586  5135 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.428185  5042 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.594589 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:42.611303 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:42.612190 30626 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:42.578531  5429 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.615010 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.611438 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:42.619922 30628 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:42.568075  5407 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.636785 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:42.638053  5393 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:42.637840  5454 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:16:42.639560  5423 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:42.640031  5423 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.640676 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.641129  5423 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.642055  5393 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:42.613230  5169 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.643736 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.646198 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:42.647266 31367 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:42.627432  5355 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:42.649887  5440 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.650691  5440 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.642444  5423 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:42.658321  5438 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:16:42.659026  5438 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.659482  5438 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:16:42.662024  5032 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.663203  5427 raft_consensus.cc:3055] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:42.663795  5427 raft_consensus.cc:740] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.664800  5427 consensus_queue.cc:260] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.667842  5450 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:42.668094  5407 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:42.670815 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.666261 31367 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:42.673317 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:42.684809 30626 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:42.620630  5429 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:42.666291  5427 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.699090  5440 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:42.700464  5450 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:42.701115  5450 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:42.703567 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.704334 30627 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:42.690675  5245 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.705679  5245 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.708099  5245 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.708606  5251 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:42.716079  5283 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:16:42.608414  5433 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:16:42.720098  5087 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:42.730038 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.731899 30671 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:42.635942  5411 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.722117  5310 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:42.737993 31367 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:16:42.734172 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:42.754477 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:42.755280 31361 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:16:42.738655 31367 pending_rounds.cc:85] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 12
I20260709 13:16:42.762082 31367 pending_rounds.cc:107] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:16:42.762429 31367 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:42.764676 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.765748 31383 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 25: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
W20260709 13:16:42.767665 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 96929.1us
I20260709 13:16:42.757265 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:42.762414 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.769853 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.769893 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.770985 31391 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 13.
I20260709 13:16:42.773510 30706 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:42.770985 31388 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
W20260709 13:16:42.774765 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 58756us
I20260709 13:16:42.778581 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.783394 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:42.725013  5437 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 13:16:42.798043 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.795926 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:42.754094  5087 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:16:42.804253 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.810966  5025 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.811659  5025 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:42.812491  5455 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:16:42.762866  5135 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:42.702129  5450 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.762863  5251 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:42.818069 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.820199 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.833639 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:42.754734  5411 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.816206  5135 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.804165  5410 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:42.793169  4979 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.839656 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.840890 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:42.841574 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.835489  5251 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:42.851050 31367 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:42.845220 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.856441 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:42.822732  5187 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.823866  5384 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.852857 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:42.853609 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:42.857534 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:42.815840  5407 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.827173  5355 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.846550  5169 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:42.861189 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:16:42.862370 30555 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:42.866653 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:42.868062 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 154622us
I20260709 13:16:42.859491 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.827394  5087 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:42.837026  5453 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.847766  5410 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:42.846691  5135 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:42.844234  5032 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.889564 30552 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:42.894236 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:42.865612  5393 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:42.906435 31391 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:42.903678 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.902189 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:42.868320  5403 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.879599  5442 raft_consensus.cc:2764] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 13:16:42.957065 31294 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:42.872341  5213 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.858198  4979 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.877995  5396 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:42.960697 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50630.1us
I20260709 13:16:42.962391 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:42.963711 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:42.901026  5308 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:42.872876  5384 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:42.967684 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:42.909072  5393 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.872609  5407 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.867040  5434 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.905546  5238 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:42.979319 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:42.867580  5187 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.879050  5355 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:42.903740  5410 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:42.963151  5025 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:42.956792  5317 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.980334  5403 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.015420  5379 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:42.974895  5006 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:43.017972 30554 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:43.020479 30627 raft_consensus.cc:1240] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:42.912385  5032 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:42.899587  5429 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.990402  5393 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:43.014171  5456 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 13:16:42.994272  5310 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:42.996554  5238 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:42.968874  5448 consensus_queue.cc:1059] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:16:43.048122 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:43.092757 30698 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:43.094300 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 184027us
I20260709 13:16:43.115072 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:43.024497  5349 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:42.988620  5308 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:42.992286  5213 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:43.111187 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 71755.9us
I20260709 13:16:43.118474 30701 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:43.024232  5166 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 13:16:43.040275  5017 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:16:42.984010  5396 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.158107 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:43.166523 30706 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:43.110096  5238 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:43.167083 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.102226  5310 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:43.174058 30699 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:16:42.908789  5453 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.119068  5433 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.207805  5433 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.210268  5433 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.155040  5169 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:43.140761  5437 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:43.194240 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:43.199388  5461 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 13:16:43.199709  5308 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.203640  5407 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.264184 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.211599  5403 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:43.212194  5411 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.174907 30699 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 12
I20260709 13:16:43.213475  5187 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.103688  5379 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.083005  5087 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.278895  5360 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.280508  5360 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:43.284729  5434 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:43.285648 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:43.247416 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.252719 30553 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:43.189145  5310 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:43.204046  5317 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.157150  5349 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.266420 30699 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:16:43.300565 30699 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:43.215950  5032 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.266342 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.271407 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.325364 30706 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:43.277060  5379 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.291662 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:43.328131 30703 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:43.299165 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 160187us
W20260709 13:16:43.330463 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 154812us
I20260709 13:16:43.285946  4696 consensus_queue.cc:260] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:43.336330 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:43.329736  5461 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:16:43.340618  5284 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.000s
I20260709 13:16:43.359856 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:43.360400 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.001s	sys 0.013s
W20260709 13:16:43.364620 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.002s	sys 0.014s
I20260709 13:16:43.367024 31367 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:16:43.331635 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 439270us
I20260709 13:16:43.369985 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.372395 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:43.372785 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.370401 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.378485 30556 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:43.383235 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.385984 31364 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:43.388309 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.341492  5384 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.389497 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.384056 31394 raft_consensus.cc:2376] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24. Re-sending same reply.
I20260709 13:16:43.397379 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.398226 30550 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:43.398732 30633 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:43.383129  5427 consensus_meta.cc:287] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.690s	user 0.014s	sys 0.000s
I20260709 13:16:43.402326 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.397233 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.403267 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.403899 30559 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:43.375255  5284 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:43.381219  4979 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.387079  4696 raft_consensus.cc:3060] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:43.411044 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.414430 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.414659 30627 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:43.408978 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.420632 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.404628 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.410575 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.406527 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.424141 31388 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:43.356702  5379 consensus_queue.cc:1048] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:16:43.425043 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.428469 30558 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:43.429651 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.387277  5169 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:43.425431 30624 raft_consensus.cc:2368] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 25. Current term is 26.
I20260709 13:16:43.411583 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.423000 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.399439  5403 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.426205 30554 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:43.426666 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.396307  5251 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:43.427392  5006 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:43.407951  5438 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.746s	user 0.009s	sys 0.006s
W20260709 13:16:43.461442 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.403062  5381 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.410109  5279 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.455279 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:43.457692  5135 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.004s
I20260709 13:16:43.432269 30633 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:43.430775 31361 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:43.412742  5429 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:43.453840  5476 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:16:43.469776 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.474784 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:43.475085 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.479287 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.437446 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:43.486475 30633 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:43.487311 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49988.4us
I20260709 13:16:43.488615 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:43.486254  5135 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.464636  5438 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.491343  5422 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.491475  5135 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.492274  5135 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:43.492336  5427 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:16:43.492863  5438 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 28 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.428339  5454 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:16:43.482786 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:43.458103  5478 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 13:16:43.491500 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.428435  5453 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.498595 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.499557 30550 raft_consensus.cc:2393] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 21: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:43.458302  5187 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:43.464723  5403 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.503780 31361 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:16:43.466043  5384 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:16:43.481305  5381 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:43.512298  5418 consensus_meta.cc:287] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.063s	user 0.017s	sys 0.000s
I20260709 13:16:43.515736 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:43.516589 30559 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:43.507601  5393 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.458850  5251 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:43.497004  5467 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
W20260709 13:16:43.529217 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 124698us
I20260709 13:16:43.520354 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:43.530295 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.533087 31388 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:43.533805 30633 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:43.539386 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.464843  5006 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.542465 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.527201  5251 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:43.554075 30559 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:43.530488  5429 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:43.557917 31361 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 28.
W20260709 13:16:43.568511 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.552093  5381 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.573606 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.574304  5393 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.562243  5284 consensus_queue.cc:1059] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:16:43.556936  5349 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.626060 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.578719 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.580605 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.581498 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.582290 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.628809 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.629875 30624 raft_consensus.cc:2393] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:43.583127 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.630007  5310 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:43.630661  5284 raft_consensus.cc:3055] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:43.545840  5453 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:43.593407 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:43.620007 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.577960 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.624058 30599 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.631018 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.637199 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.634691 30628 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:43.541692  5169 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.639303 30629 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:43.645782 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.646704 30707 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:43.622915  5485 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.649572 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.652417 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:43.631086  5284 raft_consensus.cc:740] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:43.644222  5317 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:43.642690  5493 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:43.642957  5381 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:43.644547  5017 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:43.676400 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.679816 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.660337  5453 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:43.663445  5384 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:43.684105 30703 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:16:43.674722  5017 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.683607  5238 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:43.699026  5017 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:43.708596  5238 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:43.709440 31367 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:43.676883  5381 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:43.711692  5006 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.712147  5381 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.719723 30705 raft_consensus.cc:1275] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:16:43.720414 30705 pending_rounds.cc:85] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 10
I20260709 13:16:43.720709 30705 pending_rounds.cc:107] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:16:43.721005 30705 raft_consensus.cc:2889] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:43.730556 30550 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
W20260709 13:16:43.734841  5492 log.cc:927] Time spent T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:43.747654  5310 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:43.684514  5453 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:43.749413  5213 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.736449  5284 consensus_queue.cc:260] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:43.751299  5410 consensus_meta.cc:287] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.665s	user 0.012s	sys 0.000s
I20260709 13:16:43.751515  5453 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:43.751909  5410 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.753522 30555 raft_consensus.cc:1240] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:16:43.700783  5493 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.754921 31383 raft_consensus.cc:2368] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:43.754882  5493 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:43.755803  5462 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.736s	user 0.000s	sys 0.002s
I20260709 13:16:43.736398 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.757614  5493 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:43.759541 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1601) took 1208 ms. Trace:
W20260709 13:16:43.751374  5283 consensus_meta.cc:287] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 1.034s	user 0.015s	sys 0.001s
I20260709 13:16:43.760133 30625 rpcz_store.cc:276] 0709 13:16:42.550893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.551191 (+   298us) service_pool.cc:224] Handling call
0709 13:16:42.551728 (+   537us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:16:42.552269 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:42.552314 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:16:42.562936 (+ 10622us) log.cc:844] Serialized 52 byte log entry
0709 13:16:42.793946 (+231010us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:16:42.793992 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:42.794062 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.754955 (+960893us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:43.759054 (+  4099us) raft_consensus.cc:1685] finished
0709 13:16:43.759089 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.759447 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2263,"raft.queue_time_us":3113,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":116,"spinlock_wait_cycles":27839360,"thread_start_us":192736,"threads_started":1,"wal-append.queue_time_us":377297}
I20260709 13:16:43.762276 30557 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:43.763042  5470 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:43.763652  5470 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:43.739449  5495 log.cc:927] Time spent T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:16:43.693074  5384 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.766822  5384 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:43.767684 31396 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 13:16:43.769308 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.770084 31383 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:43.654934  5485 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.749429 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2059) took 1021 ms. Trace:
I20260709 13:16:43.771133 31389 rpcz_store.cc:276] 0709 13:16:42.728516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.728624 (+   108us) service_pool.cc:224] Handling call
0709 13:16:42.763906 (+ 35282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:42.803022 (+ 39116us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:16:43.280273 (+477251us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b1800095c20. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:16:43.360033 (+ 79760us) log.cc:844] Serialized 14 byte log entry
0709 13:16:43.500973 (+140940us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:43.535252 (+ 34279us) log.cc:844] Serialized 26 byte log entry
0709 13:16:43.535527 (+   275us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:43.535872 (+   345us) log.cc:844] Serialized 14 byte log entry
0709 13:16:43.537528 (+  1656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.537610 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:43.749192 (+211582us) raft_consensus.cc:1685] finished
0709 13:16:43.749218 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:43.749366 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54834,"prepare.queue_time_us":306874,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":124,"raft.queue_time_us":58071,"raft.run_cpu_time_us":508,"raft.run_wall_time_us":36469,"spinlock_wait_cycles":4976512,"thread_start_us":161101,"threads_started":2,"wal-append.queue_time_us":155617}
I20260709 13:16:43.772841  5485 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:43.773909 30627 raft_consensus.cc:2410] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:16:43.774155  5396 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.776075  5355 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.777928 30706 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:43.783708 30550 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:43.739001  5017 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.751323  5284 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:43.820457  5460 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.084s	user 0.008s	sys 0.006s
I20260709 13:16:43.806221  5410 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:43.841568  5503 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.168s	user 0.003s	sys 0.000s
W20260709 13:16:43.842279 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:43.843700 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48122 (request call id 1179) took 1102 ms. Trace:
I20260709 13:16:43.844332 30698 rpcz_store.cc:276] 0709 13:16:42.741568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.742108 (+   540us) service_pool.cc:224] Handling call
0709 13:16:43.843363 (+1101255us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400343854. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:43.843583 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567024896}
I20260709 13:16:43.845199  5384 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:43.850888  5017 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.851712 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1314) took 1012 ms. Trace:
I20260709 13:16:43.852195 30553 rpcz_store.cc:276] 0709 13:16:42.839140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.839260 (+   120us) service_pool.cc:224] Handling call
0709 13:16:43.851469 (+1012209us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b5400374954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:43.851633 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74651904}
I20260709 13:16:43.795517  5251 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.857203 30701 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1949) took 1367 ms. Trace:
I20260709 13:16:43.857399  5251 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.865880 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.866415 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.867058 30553 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:43.867762 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.857821 30701 rpcz_store.cc:276] 0709 13:16:42.489858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.499589 (+  9731us) service_pool.cc:224] Handling call
0709 13:16:42.533080 (+ 33491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.139735 (+606655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:43.856920 (+717185us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540001f9d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:43.857108 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1328889088}
I20260709 13:16:43.870639 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:43.858166  5453 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.880729 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:43.881872 30553 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:16:43.879125  5355 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:43.896967 30555 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:43.899133 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:43.904222 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.907370 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.910681 30596 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.912690 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:43.880405  5251 consensus_queue.cc:1048] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.020s
I20260709 13:16:43.896669  5355 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:43.892999  5453 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:43.915529 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:43.919399  5437 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:43.919934  5437 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:43.920863  5437 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:43.921918 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.914041  5432 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:16:43.890189  5213 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:16:43.911599  5381 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:43.916786  5017 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:43.936724  5432 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.942343  5432 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:43.917447  5408 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:43.914245  5411 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 13:16:43.925608  5355 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:43.947337 30596 leader_election.cc:400] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:43.947767 30596 leader_election.cc:403] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:43.949820  5477 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:16:43.950490  5459 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.912s	user 0.003s	sys 0.002s
I20260709 13:16:43.944957  5453 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:16:43.961413 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 161895us
I20260709 13:16:43.960335  5042 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:43.945333  5017 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:43.970587 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:43.973268 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.973661 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.975404 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.976186 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.976933 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.977161 31394 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 13:16:43.975453 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:43.977860 30627 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:43.975490 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:43.974592  5381 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:43.978575 31389 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:43.979017 30628 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:43.967574  5453 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:43.982051  5453 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:43.970005  5017 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:43.992555 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.969182  5384 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:43.993600 31396 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:43.995213 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:43.997205 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:44.003765 30632 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:43.998632 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:43.999436 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.006744 31389 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:43.971408  5437 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:43.972393  5485 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:44.012365  5485 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.013504 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:44.014220 30628 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:44.011019 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2713) took 1491 ms. Trace:
I20260709 13:16:44.015784 30557 rpcz_store.cc:276] 0709 13:16:42.512411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.512914 (+   503us) service_pool.cc:224] Handling call
0709 13:16:44.003223 (+1490309us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003a5f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:44.003355 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2813336448}
I20260709 13:16:44.021831  5453 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:44.015188  5384 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:44.015147  5411 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:44.066572  5384 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:44.067931 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.070161 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.027439  4696 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.073629 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.076026 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.076748 30705 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:16:44.077097 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.077970 30703 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:44.073629 30698 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:44.028298  5006 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.057750  5251 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.032805 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.057163  5025 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:44.021339  5437 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.022307  5408 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:16:44.026427  5485 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.104297 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.105057 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1312) took 1266 ms. Trace:
I20260709 13:16:44.105659 30558 rpcz_store.cc:276] 0709 13:16:42.838330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.840184 (+  1854us) service_pool.cc:224] Handling call
0709 13:16:42.858302 (+ 18118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:43.432610 (+574308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:44.104621 (+672011us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b540028a5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:44.104926 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310716288}
I20260709 13:16:44.108434  5384 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:44.116937 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.117729 30707 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:44.083042  4696 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.085969  5006 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:44.125334 30705 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
I20260709 13:16:44.126416 30632 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:44.067183  5458 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:16:44.129896 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.130617 30698 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:44.133564 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.134279 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.135326 30627 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:44.135232 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.026155  5441 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 23.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:16:44.067746  5434 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 13:16:44.137187 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 125996us
I20260709 13:16:44.139707 30703 raft_consensus.cc:1275] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:16:44.146898 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:44.144687 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.147987 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.089516  5251 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.153873 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.154670  5169 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:44.155470  5169 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:44.156638 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:44.155874 31292 leader_election.cc:400] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 25. Current term is 26.
I20260709 13:16:44.157831 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.157900 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:44.167831 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:44.174983  5363 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.115020  5025 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.181185 30625 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:44.187927 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:44.181780 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.080225  5411 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.188673 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:44.188653 30551 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:44.188702 30631 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:44.192826 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 106643us
I20260709 13:16:44.177160  5153 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:44.194386 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:44.175729  5363 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.193919 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:44.198549 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:44.192631  5283 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.200532  5489 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:44.200459  5283 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.201107  5489 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:44.201561  5489 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:44.207782  5363 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.210783  5403 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.211593  5403 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:16:44.210825  5507 log.cc:927] Time spent T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.309s	user 0.003s	sys 0.000s
I20260709 13:16:44.215267  5437 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.217844  5006 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:44.220171 30707 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:44.223534  4696 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.191159  5410 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:44.227979  5251 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.229485 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.185317  5441 consensus_queue.cc:1243] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 24.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:44.231940  5238 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:44.232801  5421 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:44.233253  5421 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:44.233479  5396 raft_consensus.cc:3055] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:44.233906  5396 raft_consensus.cc:740] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:44.233062  5238 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:44.237929  5411 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.232852  5025 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.186573  4979 raft_consensus.cc:2764] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:16:44.234179  5421 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.10, Last appended by leader: 10, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.243400  5355 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:44.244700  5421 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:44.245151 30627 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:44.263226  5489 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.269563 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.245198 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.270534 31367 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 13.
I20260709 13:16:44.269915  5489 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:44.243438  5017 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.274243 30624 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:44.230022  5432 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.277040  5432 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.277315 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:44.278031 30630 raft_consensus.cc:2368] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 14. Current term is 15.
I20260709 13:16:44.255679  5430 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:44.279066  5430 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:44.279537  5408 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:44.280134  5430 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:44.281590  5430 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:44.283285 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.284739  5485 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 13:16:44.285300  5485 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:44.285241  5437 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:16:44.285976  5410 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:44.259944 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.287031 31361 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:44.287313  5381 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:44.287884  5515 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.244s	user 0.000s	sys 0.004s
I20260709 13:16:44.289160 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1311) took 1451 ms. Trace:
I20260709 13:16:44.289563 30554 rpcz_store.cc:276] 0709 13:16:42.837952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.838077 (+   125us) service_pool.cc:224] Handling call
0709 13:16:44.288958 (+1450881us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b5400270954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:44.289076 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":369998080}
I20260709 13:16:44.290966  5423 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.266904  5470 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:44.271791 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.249078  5477 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:44.284790 30522 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:44.265471 30628 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:44.245724  5355 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:44.315472 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 43780.6us
W20260709 13:16:44.315689 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 119436us
I20260709 13:16:44.316360 30698 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:44.346727 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:44.302330 30631 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:44.347577 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:44.317866 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:44.348280 30559 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 14.
I20260709 13:16:44.320267 30632 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:44.352125 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:44.346773 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.354534 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:44.357461 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 72762.5us
I20260709 13:16:44.358264 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.359179  5453 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:44.359543 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:44.360091 30550 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:44.360157 30559 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:16:44.363615 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:44.286491  5485 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.409165 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:44.403596 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45419.5us
I20260709 13:16:44.441694 30559 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:44.466473 30626 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
W20260709 13:16:44.403159 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.357306  5435 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:44.367749 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:44.404309 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:44.468066 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.354894  5317 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:44.470618 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 102971us
I20260709 13:16:44.350884  5421 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:44.464216 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 275617us
I20260709 13:16:44.365645  5489 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:44.475838  4979 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:44.417517  5245 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:16:44.478055  5238 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:44.478268  5245 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:44.478518  5238 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:44.479676  5238 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.481066  5238 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:44.482544 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:44.483600 30698 raft_consensus.cc:606] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:44.484009 30698 raft_consensus.cc:614] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:44.484835  5169 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:44.402776  5153 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:44.369015  5419 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:44.487227 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.469043 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:44.490958 30703 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:44.321274  5423 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.330096  5464 consensus_queue.cc:1059] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.146s
I20260709 13:16:44.360937  5017 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.506465 30597 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:44.494999 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90791.5us
I20260709 13:16:44.495993  5169 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:44.509123  5169 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.511746  5169 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.501200 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:44.504297  5470 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:16:44.513944  5470 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:44.514415  5470 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.514846  5470 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:44.471383  5317 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:44.469789  5435 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:44.489394  5287 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.489207  5310 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:44.503854  5485 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:16:44.542829 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.502846  5437 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:44.505769  5453 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:44.506604  5153 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:44.568208 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:44.579299 30703 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:44.566475  5310 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:44.581034 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.581977 30698 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:44.581804  5310 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.562114  5384 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:44.583967  5384 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.586189  5384 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.535313  5287 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:44.493162  5421 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.531394  5435 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:44.591306 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 116095us
I20260709 13:16:44.521402  5522 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:44.617215 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:44.619850 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.621049 31361 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:44.622359 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.623881 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:44.624864 31396 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:16:44.624003 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.626457 31389 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:44.628624 30702 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 13:16:44.623126 31364 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:44.638310 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:44.560504  5437 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.649724 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:44.672747 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:44.673882 31361 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 13:16:44.674114 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:44.675614 30707 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:44.634066 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:44.677475 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 43564.6us
I20260709 13:16:44.678843 31364 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:44.680964 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:44.531778  5521 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:16:44.591212  5386 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.707576 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:44.692298 30698 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:44.592231  5489 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.693097 31391 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 20: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:44.696781  5153 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.696271 30703 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:44.700368  5287 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.702493  5410 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:44.702785  5421 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:44.702867 30558 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:16:44.704036 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:44.710781 30707 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:44.735440 30707 raft_consensus.cc:606] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:44.735996 30707 raft_consensus.cc:614] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:44.681732  5437 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:44.709206 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.713436 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:44.693694  5464 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:44.743693 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:44.739739 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 192123us
I20260709 13:16:44.614151  5042 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 13:16:44.751474 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:44.755146 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:44.756906 30557 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
I20260709 13:16:44.754320 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:44.732050  5408 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:16:44.769520 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56193.3us
I20260709 13:16:44.727453  5017 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:44.776297 31396 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 15.
I20260709 13:16:44.692855  5419 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:16:44.752686  5442 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 13:16:44.771404 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 248293us
I20260709 13:16:44.715983  5386 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:44.629347  5245 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:44.795650 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:44.727250  5489 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:44.796877 30551 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1315) took 1957 ms (client timeout 2162 ms). Trace:
I20260709 13:16:44.739705  5410 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:44.869884 30551 rpcz_store.cc:269] 0709 13:16:42.839474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:42.839588 (+   114us) service_pool.cc:224] Handling call
0709 13:16:44.796719 (+1957131us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003417d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:44.796816 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880499584}
I20260709 13:16:44.779955  5421 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:44.779171  5464 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:44.758980  5479 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:16:44.884541  5386 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:44.886931  5528 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:16:44.887760  5349 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:44.869824 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:44.996054 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 473722us
I20260709 13:16:44.867417  5448 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:44.891698  5421 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:44.786382  5017 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:45.001243 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 131563us
I20260709 13:16:44.887835  5522 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:45.018932 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:44.999504  5528 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:45.001083  5408 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:45.059078 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.034788  5421 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:45.069897 30552 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:45.000295  5423 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:45.035847  5479 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:45.037335  5287 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.017818  5448 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:45.074903  5407 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.024s	sys 0.039s
I20260709 13:16:45.037573  5169 consensus_queue.cc:1059] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:16:44.996486  5349 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.057247  5381 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:16:45.084610  5524 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.432s	user 0.001s	sys 0.001s
I20260709 13:16:45.068852  5535 raft_consensus.cc:2764] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:16:45.049535  5522 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:45.128569  5407 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.024s	sys 0.039s
I20260709 13:16:45.074414  5408 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.118717  5287 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.116510  5479 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.058454  5464 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.191891  5410 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:45.192649  5440 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:45.193166  5440 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:45.193208  5448 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:45.195315  5017 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:45.183830  5317 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.645s	user 0.014s	sys 0.000s
I20260709 13:16:45.179366  5535 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:45.177220  5532 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:45.187556  5470 consensus_meta.cc:287] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.001s
I20260709 13:16:45.180900  5423 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:45.201148 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 446087us
W20260709 13:16:45.201222  5531 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 13:16:45.218439 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 467174us
I20260709 13:16:45.178885  5407 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.022s
W20260709 13:16:45.209513 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:45.312034  5493 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.005s	sys 0.016s
I20260709 13:16:45.249213 31361 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:45.306658  5283 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:45.314790  5042 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.023s	sys 0.026s
I20260709 13:16:45.221895 30522 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:45.322738 30559 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:45.322656 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 100890us
I20260709 13:16:45.305891  5360 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.327165 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:45.237721 30670 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.741s (SENT) [suppressed 12 similar messages]
I20260709 13:16:45.201185  5534 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:45.335860 31396 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 13:16:45.319593  5042 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.024s	sys 0.026s
W20260709 13:16:45.333827 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 538335us
I20260709 13:16:45.355095 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:45.334285 31292 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.017s	sys 0.012s
W20260709 13:16:45.356202 31292 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.018s	sys 0.012s
I20260709 13:16:45.214829  5535 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:45.359148 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:45.312940  5493 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.005s	sys 0.016s
I20260709 13:16:45.337852 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:45.364015 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 347727us
I20260709 13:16:45.364202 31391 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:45.361908 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.216779  5431 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.243534  5448 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.367066 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.363653 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:45.367240 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 334268us
I20260709 13:16:45.194314  5440 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.366022 31367 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:45.202698  5317 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.315485  5410 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:45.371539 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:45.226781  5489 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:45.316000  5419 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:45.375466 31389 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:45.376269 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:45.381183 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 335920us
I20260709 13:16:45.173591  5528 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.216076  5470 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.218178  5423 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.386224 30596 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:45.364542  5533 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:45.377162  5440 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:45.223848  5537 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.167s
I20260709 13:16:45.392427 30597 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:45.403126 30629 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.006s	sys 0.031s
W20260709 13:16:45.403573 30629 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.006s	sys 0.031s
I20260709 13:16:45.346714  5360 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.382889  5553 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:45.409387 30556 raft_consensus.cc:1240] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 13:16:45.395159  5017 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:16:45.402463  5489 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.387867  5541 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:45.419987 31391 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:45.395234  5532 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:45.412223 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:45.416287 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.380487  5431 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.398554  5533 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:45.396314  5542 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:45.446846  5245 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.446846 30557 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 16.
I20260709 13:16:45.447980 31367 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:45.448565 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.449254 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.449846 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.455392 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.410157  5536 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.557706  5536 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.388532  5410 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.383661  5448 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:45.473031 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60877.2us
I20260709 13:16:45.467736 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:45.482728  5431 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.416184  5552 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:45.484154 30698 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:45.637699 30698 raft_consensus.cc:606] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:45.638254 30698 raft_consensus.cc:614] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:45.558722  5245 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.646998 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1621) took 1673 ms. Trace:
I20260709 13:16:45.500334 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:45.502208 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:45.513746 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 178226us
W20260709 13:16:45.514720 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 277044us
I20260709 13:16:45.391772  5419 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.424396  5553 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.560026 31361 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:45.547977 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:45.429016  5430 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.642052 30596 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:45.428896  5489 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.687899 31388 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:45.647470 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.444145  5541 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:45.677711 30702 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:45.675148 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:45.677269 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 129392us
I20260709 13:16:45.456218  5522 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:45.555210  5543 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.677019 30629 rpcz_store.cc:276] 0709 13:16:43.986881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:43.996062 (+  9181us) service_pool.cc:224] Handling call
0709 13:16:43.996456 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:44.009616 (+ 13160us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:44.028364 (+ 18748us) log.cc:844] Serialized 14 byte log entry
0709 13:16:44.551316 (+522952us) spinlock_profiling.cc:243] Waited 28.7 ms on lock 0x7b18004255c0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f1806ed5060 00007f1806ee7716 00007f1806ef8287 00007f1806ef8208
0709 13:16:45.405950 (+854634us) log.cc:844] Serialized 14 byte log entry
0709 13:16:45.646559 (+240609us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.646646 (+    87us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:45.646702 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.646784 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:45.646928 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17293,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":62940,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":65989888,"thread_start_us":1092310,"threads_started":3}
I20260709 13:16:45.479313  5042 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:45.638556  5556 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:45.566354  5527 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.385s	user 0.000s	sys 0.001s
I20260709 13:16:45.717262  5410 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.717796  5553 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:45.721921  5489 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.695505  5355 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:45.722872  5355 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.723294  5355 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:45.723878  5410 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:45.724512  5410 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:45.725594  5410 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.728976  5470 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:45.731429  5435 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.059s	user 0.011s	sys 0.003s
I20260709 13:16:45.731979  5435 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.735915  5419 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:45.647828  5552 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.738759  5552 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.708487  5477 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:45.675451  5403 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:45.696413  5479 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.479576  5017 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:45.458429  5555 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:45.645828 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:45.699469 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57550.6us
W20260709 13:16:45.673815 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 283062us
I20260709 13:16:45.561175  5542 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.698825 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:45.426998  5551 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:45.456158  5545 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.478858  5349 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:45.699800  5310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.114s	user 0.021s	sys 0.034s
I20260709 13:16:45.686106 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:45.639690  5547 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.636639  5408 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:45.743878 31394 raft_consensus.cc:2410] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:16:45.734333  5435 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:45.698081  5287 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.767153 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:45.781579 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 77870.7us
I20260709 13:16:45.698290  5430 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.742763  5477 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:45.784629 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 98596.5us
I20260709 13:16:45.788630 31389 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: []
W20260709 13:16:45.783432 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 84708.2us
I20260709 13:16:45.799984 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.801273 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.819705 30631 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:45.801681 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.820676 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.802228 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:45.821653 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 175997us
I20260709 13:16:45.822176 30633 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:45.803077 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:45.810408 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.780545  5042 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:45.791674  5310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.023s	sys 0.034s
I20260709 13:16:45.479573  5535 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.824014 30523 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:45.831218 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:45.848192 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.849732 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.771538  5542 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:45.852042 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.853430 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.768774  5431 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
W20260709 13:16:45.857750 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 50894.1us
I20260709 13:16:45.857825 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.872778 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:45.863219 30633 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:45.864144 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.868523 30554 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:45.870424 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.874871 30559 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:45.820955  5477 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:45.883639 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:45.714980  5543 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.815850  5408 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:45.880523 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:45.880342 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:45.881631 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:45.790864  5349 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:16:45.855520  5470 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:45.871335 30625 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:45.889462 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:45.892529 30701 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:45.858376  5522 consensus_queue.cc:1048] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:16:45.847832  5536 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.855739  5561 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:45.869863  5564 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.783290  5545 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.780015  5551 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:45.807412  5547 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:45.914408  5485 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.338s	user 0.013s	sys 0.003s
I20260709 13:16:45.886050  5549 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:45.918565  5539 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.002s
W20260709 13:16:45.938899 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 114963us
I20260709 13:16:45.913551  5470 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:45.964007 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:45.769289  5533 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:45.943715 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 123984us
I20260709 13:16:45.892562  5552 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:16:45.947423  5245 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:45.949314  5551 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:45.937815  5360 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:45.920051  5553 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:45.930866  5437 consensus_meta.cc:287] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.170s	user 0.013s	sys 0.002s
I20260709 13:16:45.916642  5568 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:45.924865  5561 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:45.967722 30554 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:45.927227  5555 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:45.971499 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:45.985321 30557 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:45.953896  5549 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:45.945521  5550 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.456s	user 0.001s	sys 0.001s
I20260709 13:16:45.970980  5238 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:45.970456  5470 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:45.984473  5485 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:45.901115  5017 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:45.928882  5566 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:16:45.998075 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 190451us
W20260709 13:16:46.011865 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 544259us
I20260709 13:16:45.896353  5042 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:45.939824  5564 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.103889 30704 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1971) took 1562 ms. Trace:
I20260709 13:16:46.099299 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:45.996378  5245 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:45.997691  5556 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:46.032034 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:45.983641  5287 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 13:16:46.118170 31361 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:46.118913 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:45.988477  5431 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:46.104468 30704 rpcz_store.cc:276] 0709 13:16:44.541842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.542912 (+  1070us) service_pool.cc:224] Handling call
0709 13:16:44.576469 (+ 33557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:44.577012 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:44.577058 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:44.595872 (+ 18814us) log.cc:844] Serialized 26 byte log entry
0709 13:16:45.114869 (+518997us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:45.114955 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:45.115088 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:46.103616 (+988528us) raft_consensus.cc:1685] finished
0709 13:16:46.103656 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.103816 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3800,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":396215,"threads_started":1,"wal-append.queue_time_us":1170488}
I20260709 13:16:46.097748 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2261) took 1462 ms. Trace:
I20260709 13:16:46.108282  5571 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:46.108999  5544 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:46.109390  5547 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.106189  5536 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:46.110009  5245 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:46.013830  5549 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:46.111779  5558 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.256s	user 0.001s	sys 0.001s
I20260709 13:16:46.111299  5555 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:46.112900 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:46.180044  5572 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.353s	user 0.000s	sys 0.002s
I20260709 13:16:46.001379  5541 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.182097  5541 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:46.159137  5557 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.305s	user 0.001s	sys 0.001s
I20260709 13:16:46.183974  5541 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:46.156150  5536 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:46.186945  5589 log.cc:927] Time spent T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:16:46.188273  5536 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.139981  5543 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.110356  5570 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:46.133589 30556 rpcz_store.cc:276] 0709 13:16:44.635795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.635925 (+   130us) service_pool.cc:224] Handling call
0709 13:16:44.703647 (+ 67722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:45.418524 (+714877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.097425 (+678901us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b54001aa254. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:46.097662 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622352768}
I20260709 13:16:46.106823  5448 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.030405  5238 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.013729  5553 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:46.157964 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 287050us
W20260709 13:16:46.173192 31292 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.592s (SENT)
I20260709 13:16:46.134526  5431 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:45.999766  5437 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:46.179818  5578 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.002713  5561 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:46.113819 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 82046.7us
I20260709 13:16:46.101161  5485 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:46.210168 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 326629us
W20260709 13:16:46.200572  5421 consensus_meta.cc:287] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.107s	user 0.009s	sys 0.006s
I20260709 13:16:46.215016  5245 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.216146  5565 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.216536  5396 consensus_queue.cc:260] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.188345  5545 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.230218 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.211814  5431 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:46.237247 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.227383 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.221380  5464 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:16:46.246155 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.240499 30627 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:46.241614 30702 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:46.239595 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:46.146008  5571 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.220439  5595 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:46.250263 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.248764 30701 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:46.247457 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:46.244386 30556 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:46.156791  5544 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.206573  5570 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.226895  5421 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.222071  5578 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:46.239705  5547 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
W20260709 13:16:46.319235 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.278573 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:46.304636  5581 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:16:46.224537  5556 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.367422  5437 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.368911  5437 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:46.370859 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:46.373781  5355 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.650s	user 0.013s	sys 0.000s
I20260709 13:16:46.292320  5448 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:46.374897  5448 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.284179  5594 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:46.375227  5547 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:46.279175  5564 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.376969  5564 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:46.257932  5595 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:46.377418  5564 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.378376  5564 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.379160  5485 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.251935  5042 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:46.278410  5593 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:46.257503  5396 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:46.382480 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 168625us
W20260709 13:16:46.387686 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 134268us
I20260709 13:16:46.298600  5310 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.379750  5564 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:46.405442 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:46.242146  5565 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.426069 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.429082 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.430083 30633 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:46.375272  5448 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.255007  5592 consensus_queue.cc:1059] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.157s
W20260709 13:16:46.437099 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.385826  5593 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.374190  5561 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.304935  5575 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.207s
I20260709 13:16:46.384970  5464 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:46.444061 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.374293  5355 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:46.445585 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.440653 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:46.447950 30555 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:46.449334 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.471482 30625 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:46.446024 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.432508  5528 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:46.468617 30633 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 13:16:46.472702 30624 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:46.476867 30559 raft_consensus.cc:2368] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22.
I20260709 13:16:46.472470 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:46.467255  5477 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.561s	user 0.009s	sys 0.005s
I20260709 13:16:46.398785  5042 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.427529  5310 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.474717 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.443823  5595 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.476931 30633 pending_rounds.cc:85] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 14
I20260709 13:16:46.490834 30625 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
W20260709 13:16:46.455606  5440 consensus_meta.cc:287] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.050s	user 0.016s	sys 0.000s
I20260709 13:16:46.450428  5601 consensus_queue.cc:1059] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:16:46.496227 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.497538 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.491058 30633 pending_rounds.cc:107] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 13:16:46.491967 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.496312 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.448942  5593 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:46.439430  5542 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.005s
I20260709 13:16:46.498860 30633 raft_consensus.cc:2889] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:46.499550 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.472936  5464 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.476727  5528 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:46.479451  5535 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:46.491392  5556 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.513091 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.493201  5585 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:46.516752 30633 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 28.
I20260709 13:16:46.517560 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.518527 30624 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:46.502823  5552 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:46.523159  5552 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:46.523257  5604 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:16:46.523674  5552 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:46.522171  5440 raft_consensus.cc:3055] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:46.526876  5440 raft_consensus.cc:740] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.527464  5421 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.527916  5440 consensus_queue.cc:260] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.528590 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.529350  5440 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:46.529415 30554 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:46.490149  5477 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.532397 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.541023 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.544706 30633 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:46.548518 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.555249 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.515497  5528 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:46.515249  5566 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:46.557359 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:46.529424  5355 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.515781  5553 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.498485  5549 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.582602 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.584347 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.589709 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.595911  5477 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.583544  5566 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:46.597247  5603 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:16:46.597430  5566 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:46.600304  5549 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.510830  5542 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.603374 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.610215  5552 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.611313  5578 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.615681  5310 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.583886  5571 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:46.619653  5470 consensus_meta.cc:287] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.526s	user 0.010s	sys 0.003s
I20260709 13:16:46.620498  5470 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.621220  5431 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:46.623150 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 365200us
I20260709 13:16:46.532735  5556 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:46.607097 30556 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:46.585924  5551 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:46.624985 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.626361 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.590679  5595 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.627461 30554 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:46.627805 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:46.627602  5577 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:16:46.632835  5571 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:46.628541  5551 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.637431 31378 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:46.638090  5435 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.638478  5551 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.639602 30550 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:46.639997 30550 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:46.638583  5435 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.642923 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.643652 31388 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:46.641027 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.645612 31361 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:46.646736 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:46.646824 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.647382 31361 raft_consensus.cc:2376] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21. Re-sending same reply.
I20260709 13:16:46.647521 31388 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:46.648388 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.648547 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:46.649029 31361 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 26.
I20260709 13:16:46.649807 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:46.647677  5408 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.742s	user 0.014s	sys 0.000s
I20260709 13:16:46.644467  5571 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.650590  5408 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:46.651233  5571 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.651844 30631 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:46.652005 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.652884 31364 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:46.653750  5571 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.655242  5470 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.661566  5435 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.663456  5549 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.664215  5549 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.667286  5561 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.630228  5595 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.640793 30550 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.654347 31364 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:46.679092  5618 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.679410  5566 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.682420  5566 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.682227  5618 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.685124 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.686650 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.687245  5430 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.674736  5549 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.688390  5593 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:46.688419  5618 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.689868 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.690995  5593 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.690776  5618 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:46.692070  5618 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:46.692607  5618 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:46.677495  5571 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.695673  5571 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.697538  5423 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.697952  5571 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.699783  5542 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.701300  5534 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:46.701920  5534 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:46.702209  5556 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.710577  5360 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:46.711151  5360 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.712057  5360 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.713116  5360 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:46.714371  5610 log.cc:927] Time spent T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:46.692512  5464 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.715464  5562 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:46.716107  5562 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.716291  5464 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.717226  5562 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.717758  5565 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.718541  5562 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:46.723511 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.739100 30670 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.743054  5608 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:46.743834  5608 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:46.744944  5608 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.747327  5570 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.748757  5618 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.749790  4979 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:46.750414  4979 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.751056 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2087) took 1384 ms. Trace:
I20260709 13:16:46.734165  5528 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.751524 31383 rpcz_store.cc:276] 0709 13:16:45.366790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.380339 (+ 13549us) service_pool.cc:224] Handling call
0709 13:16:45.391206 (+ 10867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.709144 (+317938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:45.709190 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:45.709252 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:45.709301 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.750739 (+1041438us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54004f8654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:46.750802 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:46.750981 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730504576}
I20260709 13:16:46.752950  4979 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.753620  5528 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.760473 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:46.760135  5532 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.006s	sys 0.018s
W20260709 13:16:46.763078  5532 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.844s	user 0.006s	sys 0.018s
I20260709 13:16:46.764026  5532 raft_consensus.cc:3055] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:46.764568 30553 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:46.764887  5532 raft_consensus.cc:740] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:46.765779  5534 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.761757 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.770439 30554 raft_consensus.cc:2468] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:46.772225 30553 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:46.729440  5571 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:46.772993 30558 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:46.767010  5431 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:46.784332 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:46.787190 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.787895 31396 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:46.774945  5571 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.783799  5532 consensus_queue.cc:260] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.772100  5564 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:16:46.804262 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:46.798358 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.783021  5549 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:16:46.783843  5534 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.782207  5430 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:16:46.809336  5564 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:46.814818 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.815574  5564 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:46.816951  5549 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.817507  5549 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:46.817629  5448 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:46.818871  5317 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:46.820735  5448 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:46.822072 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:46.823434 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.824570 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.825752 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.803629  5571 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.831498  5532 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:46.838325 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:46.840639 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.843782  5568 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:46.844416  5568 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.838384  5564 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:46.847206 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.849972  5564 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:46.851851 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:46.852836 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.853111 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.853814 30552 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:46.854656 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.855466 30707 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:46.851920 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2200) took 1031 ms. Trace:
I20260709 13:16:46.858026 30631 rpcz_store.cc:276] 0709 13:16:45.820060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.820623 (+   563us) service_pool.cc:224] Handling call
0709 13:16:45.865789 (+ 45166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.652251 (+786462us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:46.851629 (+199378us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b5400053554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:46.851847 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807286656}
I20260709 13:16:46.862866  5287 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:46.863488  5287 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:46.863903  5287 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:46.867203 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.868484 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.868727 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.869407 30557 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:46.869820 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.873009 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:46.873358 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.875725 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:46.877420 30599 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:46.880262 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:46.881153 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.882799 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.884929 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:46.870549 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:46.892220 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47391.9us
I20260709 13:16:46.894876 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:46.895702 30707 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:46.900349 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:46.901684 31391 raft_consensus.cc:3060] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:46.903582 30552 raft_consensus.cc:2468] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
I20260709 13:16:46.908119 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.911152 30557 raft_consensus.cc:2468] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:16:46.915343 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.917697 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:46.918792 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.920746 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.923558 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:46.923722 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.890882  5528 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:46.924598 30624 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:46.922786 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:46.931727 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:46.932771 30631 raft_consensus.cc:2376] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21. Re-sending same reply.
I20260709 13:16:46.933169 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.934078 30627 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:46.937981 30556 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:16:46.938301 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:46.947548 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:46.947590 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.912853  5448 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:46.990579 30673 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:46.964448  5622 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:16:46.966514 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:46.996708 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46862.7us
I20260709 13:16:46.967120  5554 raft_consensus.cc:2804] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:46.968078 30556 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:16:46.962757  5546 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:46.926148  5408 consensus_queue.cc:1059] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
W20260709 13:16:47.001546 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:46.979157 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.985280 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:46.986989  5549 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.002458 30627 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:46.991788 30625 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
W20260709 13:16:46.928692  5594 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.553s	user 0.012s	sys 0.001s
I20260709 13:16:46.962423 31391 raft_consensus.cc:2468] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:46.998805 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:46.937507  5431 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:46.939133  5528 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.000572 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:46.963740  5565 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.000232  5440 raft_consensus.cc:2804] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:47.012005  5594 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:47.009704 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:47.000488  5310 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:16:47.009541  5431 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.033155  5310 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.033677  5310 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:16:47.033916  5431 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:47.003046  5435 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:16:47.028627  4979 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:47.004166 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.007277 30670 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:47.013772 30701 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:16:47.015393 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:47.036212 30704 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 26. Current term is 27. Ops: []
I20260709 13:16:47.038614 30629 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:16:47.042351  5396 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:16:47.067895  5435 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.068727  5435 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:47.048820 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.072407 31394 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 2092) took 1624 ms (client timeout 1909 ms). Trace:
I20260709 13:16:47.073715 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:47.062207 30702 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:47.072072 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1321) took 1117 ms. Trace:
I20260709 13:16:47.084802 30555 rpcz_store.cc:276] 0709 13:16:45.955157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.955330 (+   173us) service_pool.cc:224] Handling call
0709 13:16:45.989834 (+ 34504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:46.470968 (+481134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.071773 (+600805us) spinlock_profiling.cc:243] Waited 455 ms on lock 0x7b540012c5d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:47.071998 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1046669568}
W20260709 13:16:47.085798 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 78618.7us
W20260709 13:16:47.083379 31394 rpcz_store.cc:269] 0709 13:16:45.448060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.448241 (+   181us) service_pool.cc:224] Handling call
0709 13:16:47.072105 (+1623864us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540031d8d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:47.072314 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422239616}
W20260709 13:16:47.088254 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.088729 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.039518 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:47.089069 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.006413  5554 raft_consensus.cc:697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:47.091008  5546 raft_consensus.cc:2804] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:47.080713  5533 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:47.039717  5523 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:16:47.041989  5440 raft_consensus.cc:697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:47.070479  5552 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.073858  5435 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:47.095351  4979 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.095729 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:47.082867 30597 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:47.089548 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50093.9us
W20260709 13:16:47.139858 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 57954.2us
I20260709 13:16:47.140728 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:47.031941  5528 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.068027  5396 consensus_queue.cc:1243] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer acf852f85f47483d99cbe4020baee627 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:47.043373  5408 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:47.037382  5565 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:47.115500 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.104869  5571 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.143734 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:47.146173  5411 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:47.147373 31361 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:47.102594  5287 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.152307 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:47.158252  4979 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.151351  5440 consensus_queue.cc:237] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.130227  5546 raft_consensus.cc:697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:47.164899 30597 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:47.174877 31391 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:47.177592  5568 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.146247  5523 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:47.179418  5523 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.159271  5411 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:47.162299  5430 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:47.172362  5528 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:47.168798 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:47.177443  5546 consensus_queue.cc:237] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.172576 30702 raft_consensus.cc:2468] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:47.194320 31394 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:47.191993  5523 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:47.223482 30702 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:16:47.166838 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.227648 30597 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:47.231478 31361 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
W20260709 13:16:47.227830 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 61097.5us
I20260709 13:16:47.193742  5411 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.254889  5549 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:47.237488 30701 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:47.256834  5555 raft_consensus.cc:2804] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:47.257454  5555 raft_consensus.cc:697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:47.224470  5528 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:47.237944 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1340) took 1024 ms. Trace:
I20260709 13:16:47.259608  5528 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.259727 30558 rpcz_store.cc:276] 0709 13:16:46.213418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.677366 (+463948us) service_pool.cc:224] Handling call
0709 13:16:46.771121 (+ 93755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.781304 (+ 10183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.237645 (+456341us) spinlock_profiling.cc:243] Waited 76.1 ms on lock 0x7b5400358750. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:47.237868 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174990592}
I20260709 13:16:47.258458  5555 consensus_queue.cc:237] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:47.262863 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.268280 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.269775 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.272995 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.273919 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.274544 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.275424 30633 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:16:47.275849 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.239223  5396 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:47.275466 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.277449  5396 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.277931  5396 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:47.279208 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.279873  5568 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:47.279763  5477 raft_consensus.cc:3055] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:47.283164 31394 raft_consensus.cc:2468] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:47.246703 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.226202  5408 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:47.226547  5585 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.293509 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:47.220054  5442 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.329s	user 0.019s	sys 0.029s
I20260709 13:16:47.297078 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.297175 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:47.298130 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.183182  5554 consensus_queue.cc:237] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.304327 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.276574  5523 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:47.300479  5442 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.428s	user 0.019s	sys 0.030s
I20260709 13:16:47.284755  5477 raft_consensus.cc:740] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:47.305377 30627 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:16:47.294767  5213 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:47.306603 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60021.7us
I20260709 13:16:47.295414  5548 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.317754 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.317859 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.325897 30626 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:47.319415 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.318292 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.325670 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.334396 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.335129 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.338562 30632 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:47.340585 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.341512 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.343433 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.343806 30556 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:47.344161 30557 raft_consensus.cc:2393] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:47.307896  5551 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:47.349097 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.350721 31394 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 26: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:47.311618  4979 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:47.304467  5628 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:47.360870 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.361749 30557 raft_consensus.cc:2368] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:16:47.357278 30556 raft_consensus.cc:2368] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22.
I20260709 13:16:47.322151  5442 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:47.369565 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.302167  5585 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.371888 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.373138 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.350479 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.335429  5533 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.377908 30627 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:47.340664  5213 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.353470  5408 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.378918  4979 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:47.352715  5631 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:47.339897  5317 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.381944 30555 raft_consensus.cc:2393] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:47.389642  5593 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:47.390300  5593 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.392208  5565 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.392748  5593 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.398005 31367 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:47.370707  5628 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.379901  5551 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.399003  5628 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.387421  5477 consensus_queue.cc:260] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.399211  5551 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.400369  5477 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.401785  5551 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.395188  5408 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.403214  5442 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.404083  5213 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.387943  5533 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.387708  5631 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.405741  5631 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:47.407095  5533 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:47.388093  4979 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:47.408572 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:47.409529 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.412111 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.411988 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.408100  4979 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:47.416813 30624 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:47.339972  5548 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.388657  5571 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:47.395526  5317 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.425541 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.408022  5523 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.396068  5287 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.427980 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.405222  5408 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:47.425062  5547 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.437736 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.417830  5587 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:47.441622 30557 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:16:47.447528 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.451124 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1356) took 1650 ms. Trace:
I20260709 13:16:47.453323 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.453585 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2089) took 2047 ms. Trace:
I20260709 13:16:47.454144 31389 rpcz_store.cc:276] 0709 13:16:45.406232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.406802 (+   570us) service_pool.cc:224] Handling call
0709 13:16:45.447548 (+ 40746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.817110 (+369562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.453226 (+1636116us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b5400145fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:47.453503 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":783691136}
I20260709 13:16:47.435002  5547 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:47.429314 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 150218us
I20260709 13:16:47.451548 30628 rpcz_store.cc:276] 0709 13:16:45.800944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.801095 (+   151us) service_pool.cc:224] Handling call
0709 13:16:45.851716 (+ 50621us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:45.851956 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:46.009501 (+157545us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.076198 (+1066697us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b18003d82e0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a 00007f1806ee7a2a
0709 13:16:47.437814 (+361616us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.450696 (+ 12882us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.450766 (+    70us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:47.450809 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.450909 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:47.451059 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38803,"prepare.queue_time_us":135489,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":35994,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":20204,"spinlock_wait_cycles":32767360,"thread_start_us":383028,"threads_started":3,"wal-append.queue_time_us":1241045}
W20260709 13:16:47.451678  5630 log.cc:927] Time spent T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:16:47.463595 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.464490 30698 raft_consensus.cc:2468] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:47.466708  5585 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.442317  5587 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.467703  5587 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:47.450971  5576 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:47.471616  5576 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.475040 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:47.477824 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.479450 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.480327 30698 raft_consensus.cc:2393] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:47.479588  5431 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:47.485994 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 137592us
I20260709 13:16:47.487905 30523 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.488955  5585 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:47.489998  5585 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.490687  5585 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:47.460421  5533 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.493896  5310 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.489254 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.497248 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.502203  5384 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:47.502743  5384 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:47.503187  5384 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:47.508376  5238 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.509131 31367 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 28. Current term is 29. Ops: []
I20260709 13:16:47.509572  5548 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:47.509871  5533 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.512172  5533 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:47.512930 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.514207 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.515203 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:47.519229 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.522910 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.523303 30627 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:47.523783 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:47.524435 30628 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:47.519634 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.520361  5590 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:16:47.521018  5609 consensus_queue.cc:1059] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:16:47.522286 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.527546 30703 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:47.529775 30698 raft_consensus.cc:2410] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:16:47.533007 30702 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:47.535315 30705 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:47.536614 30702 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:47.533777 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:47.533185 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.544315 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.554418 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.561254 30704 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:16:47.561612 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:47.561995 30704 pending_rounds.cc:85] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Aborting all ops after (but not including) 9
I20260709 13:16:47.562383 30704 pending_rounds.cc:107] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:16:47.562716 30704 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:47.568208 30627 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:47.569171  5628 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.571545  5576 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:47.572558  5396 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.573427  5544 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:47.573681 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:47.574412 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.543901  5453 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 13:16:47.584591 31294 leader_election.cc:400] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
I20260709 13:16:47.583521 30704 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:47.598932 31294 leader_election.cc:403] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.580104 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.580547 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.583420 31391 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:16:47.581096 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.601871 30556 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:16:47.605964 30559 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 22: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:47.599135 30673 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:47.612279 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50742.9us
W20260709 13:16:47.578433  5625 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.403s	user 0.003s	sys 0.000s
I20260709 13:16:47.619583 30703 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:47.637591 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.639932 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.640717 31294 leader_election.cc:400] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22.
I20260709 13:16:47.641047 31294 leader_election.cc:403] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:16:47.568243  5431 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.574520  5310 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.644093 30670 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:47.652716 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.531416  5580 consensus_queue.cc:1059] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 13:16:47.579574  5407 raft_consensus.cc:3055] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:47.646205 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.662370 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.663480 30702 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:47.610173  5638 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:47.669071 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.670089 30702 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:47.574860  5631 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.545235  5555 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 13:16:47.680379  5042 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:47.686342 30632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46168 (request call id 1627) took 1878 ms (client timeout 1881 ms). Trace:
W20260709 13:16:47.687037 30632 rpcz_store.cc:269] 0709 13:16:45.799007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.802740 (+  3733us) service_pool.cc:224] Handling call
0709 13:16:47.677583 (+1874843us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400577954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:47.677693 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3380434816}
I20260709 13:16:47.676856 30629 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1369) took 1084 ms. Trace:
I20260709 13:16:47.693487 30629 rpcz_store.cc:276] 0709 13:16:46.592220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.592421 (+   201us) service_pool.cc:224] Handling call
0709 13:16:46.604975 (+ 12554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.061909 (+456934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.676627 (+614718us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54000541d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:47.676795 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988071680}
I20260709 13:16:47.696383  5549 raft_consensus.cc:2804] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:47.697257  5549 raft_consensus.cc:697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:47.649405 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.657501 30556 raft_consensus.cc:2468] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:47.699534  5638 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:47.611653  5430 raft_consensus.cc:2804] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:47.609187  5586 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:47.566032  5578 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 13:16:47.622541  5565 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:47.575354  5636 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:16:47.674796 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:47.576602  5528 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 13:16:47.588284  5411 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.643102  5410 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.557812  5548 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 13:16:47.702139  5396 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:47.697124 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.701807 30673 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:47.782018 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 80339.4us
I20260709 13:16:47.705080 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.649302 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.650362  5544 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:47.708185 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.785454 30698 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:47.785957 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 77905.7us
I20260709 13:16:47.787231 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:47.736488 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61832.5us
I20260709 13:16:47.787758 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:47.700687  5638 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:47.666455  5407 raft_consensus.cc:740] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:47.662274 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:47.750370 30702 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:47.701834  5310 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:47.708830 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:47.793790 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 96786.4us
I20260709 13:16:47.717176  5565 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.808128 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.818250 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:47.819414 30698 raft_consensus.cc:3060] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.782091  5568 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:47.781584  5442 raft_consensus.cc:3055] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:47.823755 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.807472 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.826152 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.835551 30702 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
I20260709 13:16:47.716771  5545 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:16:47.717339  5586 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.792027  5453 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:47.842873 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:47.807966  5042 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:47.799319  5608 raft_consensus.cc:2804] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:47.830597  5568 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:47.805377  5310 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.811431  5565 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:47.841321  5442 raft_consensus.cc:740] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:47.850889  5547 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.850291  5640 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:47.885712 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:47.852787 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.891732 30699 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:47.862114  5287 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:47.895129 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:47.899606 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.739068  5549 consensus_queue.cc:237] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.900763 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.903755 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:47.870980  5568 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:47.874523  5546 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:47.910341 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:47.921830 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 247742us
I20260709 13:16:47.879695  5628 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:47.889251  5545 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:47.933429 30698 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:47.881038  5608 raft_consensus.cc:697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:47.740607  4979 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.900553  5453 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:16:47.951329 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 307311us
I20260709 13:16:47.927989 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:47.942174 30523 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:47.934381  5213 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:47.905120  5571 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:16:47.956727 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:47.870038  5407 consensus_queue.cc:260] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:47.907753  5384 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.970988 31364 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 24. Current term is 25. Ops: []
W20260709 13:16:47.962546  5642 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:16:47.955547  5453 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.979377 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50014 (request call id 2340) took 1213 ms. Trace:
I20260709 13:16:47.955958  5213 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:47.943676  5587 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:47.938169  5546 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:47.961167  5396 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:47.986896 31396 rpcz_store.cc:276] 0709 13:16:46.765903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.814333 (+ 48430us) service_pool.cc:224] Handling call
0709 13:16:47.979074 (+1164741us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400683b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:47.979288 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917254912}
I20260709 13:16:47.967288  5637 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 13:16:47.991416  5453 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:47.983223  5287 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:47.999598  5213 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:48.024403 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 211815us
I20260709 13:16:48.003919  5411 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.010555  5396 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.026921 30671 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.018855  5608 consensus_queue.cc:237] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.034454 30671 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.038784  5653 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:48.032923  5396 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:48.066058  5586 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.083842  5587 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:48.045001 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:48.089725 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55337.9us
I20260709 13:16:48.091240 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:48.046901  5585 consensus_meta.cc:287] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
W20260709 13:16:48.063254  5651 log.cc:927] Time spent T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 13:16:48.055483  5647 log.cc:927] Time spent T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:16:48.025853  5430 raft_consensus.cc:697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:48.104313 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.109318 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.092413  5646 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:48.113014 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.102723  4979 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.120393  5430 consensus_queue.cc:237] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.123751  5648 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:48.125372 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.127171  5411 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:48.127636  5649 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:16:48.127889  5411 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.128955  5618 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.027522  5287 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.135247 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:48.139881 30699 raft_consensus.cc:1240] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:48.114591  5653 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:48.142298 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.109874  5654 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:48.101547  5585 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.144002 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.141633  5384 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:48.150597  5549 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:48.152384  5546 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.154865 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.155673 31396 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:48.211894 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:48.292985 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.293635 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.294462 31388 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:48.295732 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.297482  5565 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.300729  5565 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.293843 31396 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:48.336005  5654 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.336793  5654 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.340101  5408 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:48.340716  5408 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:48.341187  5408 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:48.142593  5646 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:48.343165  5646 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:48.343657  5646 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:48.362452  5411 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:48.362833 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.398712 30703 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:48.354235  5042 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:48.451000  5410 consensus_meta.cc:287] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.698s	user 0.015s	sys 0.000s
I20260709 13:16:48.451936  5410 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19.
W20260709 13:16:48.459152  5544 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.656s	user 0.005s	sys 0.011s
W20260709 13:16:48.464425  5568 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.526s	user 0.013s	sys 0.002s
I20260709 13:16:48.512791  5213 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.313633  5453 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.460063  5544 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22.
I20260709 13:16:48.565737 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.590021 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2330) took 1827 ms. Trace:
I20260709 13:16:48.593700 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.591681 31383 rpcz_store.cc:276] 0709 13:16:46.763111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.767697 (+  4586us) service_pool.cc:224] Handling call
0709 13:16:47.072278 (+304581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.072556 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:47.224899 (+152343us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.425893 (+200994us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.425958 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:47.426012 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.426115 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.589743 (+1163628us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540031d8d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:48.589931 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247275,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":7758,"raft.queue_time_us":240,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6734,"spinlock_wait_cycles":673944832,"thread_start_us":317246,"threads_started":2,"wal-append.queue_time_us":272474,"wal-append.run_cpu_time_us":1367,"wal-append.run_wall_time_us":1001379}
I20260709 13:16:48.595139  5595 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.595180  5547 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:48.589872  5648 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:48.607187  5585 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.607913  5565 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.608556  5565 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:48.607388 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.601732  5546 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 13:16:48.611773  5565 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.574745  5568 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.613840  5213 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.614116 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:48.617185 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.617290 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.617837 31396 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:48.618121 31383 raft_consensus.cc:2393] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:48.618661  5586 raft_consensus.cc:2804] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:16:48.619064 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.619246  5586 raft_consensus.cc:697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:48.620245  5586 consensus_queue.cc:237] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.623232 30556 raft_consensus.cc:1240] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: []
W20260709 13:16:48.616022 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 673966us
I20260709 13:16:48.624850 30555 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:48.623306 30554 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:48.620927  5213 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:48.626667  5213 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.626942 31396 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 27.
I20260709 13:16:48.628170 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.628849 31396 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:48.629799 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.630475 30702 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:48.630975 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.631597 30704 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:48.631664  5648 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.632215  5648 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.635066 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:48.612849  5453 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.639940 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:48.640287 30702 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:16:48.640826 30557 raft_consensus.cc:2368] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:48.639940 30551 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:48.642700 30556 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 25. Current term is 26. Ops: []
I20260709 13:16:48.618655 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 2098) took 1003 ms. Trace:
I20260709 13:16:48.644901 31383 rpcz_store.cc:276] 0709 13:16:47.615585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.617004 (+1001419us) service_pool.cc:224] Handling call
0709 13:16:48.618555 (+  1551us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.643181 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.643767 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.646867 30552 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
I20260709 13:16:48.649750 30559 raft_consensus.cc:1240] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:48.652999 30555 raft_consensus.cc:1240] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:16:48.653367 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.654650 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:48.657109 30554 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 13:16:48.657393 31378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50024 (request call id 2093) took 3201 ms (client timeout 1509 ms). Trace:
I20260709 13:16:48.657598 31388 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:48.658979 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:48.660121 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:48.657876 31378 rpcz_store.cc:269] 0709 13:16:45.448485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:45.455024 (+  6539us) service_pool.cc:224] Handling call
0709 13:16:48.649181 (+3194157us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400113d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:48.649558 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2476500224}
I20260709 13:16:48.660579 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.661603 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.662395 31378 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 15.
W20260709 13:16:48.666522 30522 connection.cc:256] RPC call timeout handler was delayed by -1.3852105869998468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 13:16:48.667435 30522 connection.cc:256] RPC call timeout handler was delayed by -1.219344984999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
W20260709 13:16:48.674360 30522 connection.cc:256] RPC call timeout handler was delayed by -1.18631058699998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1805s for a response
W20260709 13:16:48.644513  5440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.000s	sys 0.036s
I20260709 13:16:48.673272 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.655131  5654 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.668704 30704 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
W20260709 13:16:48.678194 30522 connection.cc:256] RPC call timeout handler was delayed by -1.0960807369999657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15600000000000003s for a response
I20260709 13:16:48.663389  5657 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:48.663309  5595 raft_consensus.cc:3060] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:48.688711 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.670861  5548 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:48.691196 30706 raft_consensus.cc:1240] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 25. Ops: []
I20260709 13:16:48.673316  5648 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.691957 31378 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:48.674259  5568 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:48.687189 30522 connection.cc:256] RPC call timeout handler was delayed by -1.0621105869997791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
W20260709 13:16:48.700027 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 419318us
W20260709 13:16:48.701627 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.702417 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.692768  5654 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:48.707201 30522 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:48.698822  5213 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.699260  5548 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.675318  5646 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.706176  5169 consensus_queue.cc:1059] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:16:48.734994 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.704034  5544 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:48.718297  5396 consensus_meta.cc:287] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.002s
I20260709 13:16:48.704100  5411 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:48.742161 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 1993) took 1067 ms. Trace:
I20260709 13:16:48.752065 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:48.721395  5654 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.753530 30699 rpcz_store.cc:276] 0709 13:16:47.675000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.937772 (+262772us) service_pool.cc:224] Handling call
0709 13:16:47.938246 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.140469 (+202223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.741827 (+601358us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54005a32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.742087 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":458243840}
I20260709 13:16:48.738422  5213 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:16:48.758159 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 97027.4us
I20260709 13:16:48.761310 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "5a03a8214bba4eb1b0ff00895c565564"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:48.762310 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.762223 30699 raft_consensus.cc:606] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Received request to transfer leadership to TS 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:48.763118 30699 raft_consensus.cc:614] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:48.763827 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.765292 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.765715  5435 raft_consensus.cc:3055] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:48.766111  5435 raft_consensus.cc:740] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:48.752142  5453 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.709654  5408 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.767803 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.769222 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.766593 30702 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:16:48.753199  5640 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:48.774921 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.775435 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.776698 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.775779 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.772641 31294 leader_election.cc:400] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:48.780943 31294 leader_election.cc:403] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:16:48.758822  5654 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:48.778540 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 118057us
I20260709 13:16:48.785432 31378 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:48.760238  5638 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:48.786604  5493 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:48.789785  5435 consensus_queue.cc:260] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.792151  5317 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.789106 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.767330  5440 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.798118  5595 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:48.798703  5543 consensus_queue.cc:1059] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:48.799507  5544 consensus_queue.cc:1059] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:48.800110  5657 consensus_queue.cc:1059] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:16:48.815253 30699 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98"
dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
mode: GRACEFUL
new_leader_uuid: "015ca80eba674552b34856a4f20271f1"
 from {username='slave'} at 127.0.0.1:34250
I20260709 13:16:48.815653  5665 consensus_queue.cc:1059] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:48.817160  5544 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:48.817687  5544 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.818131  5544 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:48.819037 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2338) took 2053 ms. Trace:
I20260709 13:16:48.819586 31367 rpcz_store.cc:276] 0709 13:16:46.765309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.410822 (+645513us) service_pool.cc:224] Handling call
0709 13:16:47.411255 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.509626 (+ 98371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.818740 (+1309114us) spinlock_profiling.cc:243] Waited 97.7 ms on lock 0x7b540031ddd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.818967 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224630656}
I20260709 13:16:48.820914 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.821317  5548 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.822100 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.822731  5646 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.794189  5666 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20.
I20260709 13:16:48.823614  5493 raft_consensus.cc:3055] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:48.823794  5442 consensus_queue.cc:260] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.824414  5493 raft_consensus.cc:740] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:48.825264  5442 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:48.825802  5408 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.825397  5493 consensus_queue.cc:260] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.828605  5493 raft_consensus.cc:3060] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:48.779235  5453 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.803079  5565 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:16:48.800457 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.755913  5396 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.757874  5411 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:48.808820 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.816030 30699 raft_consensus.cc:606] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Received request to transfer leadership to TS 015ca80eba674552b34856a4f20271f1
I20260709 13:16:48.851882 30699 raft_consensus.cc:614] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:16:48.794245 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:48.855000  5585 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:16:48.855486  5585 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:48.855839  5585 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:48.857414  5643 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.996s	user 0.001s	sys 0.010s
I20260709 13:16:48.864732 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:16:48.865156  5407 raft_consensus.cc:3060] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:48.867688 30706 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1204) took 1436 ms. Trace:
I20260709 13:16:48.868299 30706 rpcz_store.cc:276] 0709 13:16:47.431523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.431677 (+   154us) service_pool.cc:224] Handling call
0709 13:16:47.432075 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.693637 (+1261562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.867257 (+173620us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400343354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.867590 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2895107840}
W20260709 13:16:48.854549 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60439.2us
I20260709 13:16:48.870882 30522 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:48.871495 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2508) took 1864 ms. Trace:
I20260709 13:16:48.871896 30705 rpcz_store.cc:276] 0709 13:16:47.007035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.007223 (+   188us) service_pool.cc:224] Handling call
0709 13:16:47.007685 (+   462us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.535739 (+528054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.871239 (+1335500us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54002233d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.871428 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212974464}
I20260709 13:16:48.872691  5396 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.874650 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2339) took 2109 ms. Trace:
I20260709 13:16:48.875036 31389 rpcz_store.cc:276] 0709 13:16:46.765596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.460648 (+695052us) service_pool.cc:224] Handling call
0709 13:16:47.508093 (+ 47445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.508280 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.508308 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:47.508355 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:47.508386 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.508439 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.874452 (+1366013us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b54008b15d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:48.874566 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84609408}
I20260709 13:16:48.813102  5440 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.876585 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.877444 31389 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:48.881078  5592 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.882055  5631 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:48.881839  5592 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.883380 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.884210 30553 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:48.884357  5592 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.884773 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1350) took 1539 ms. Trace:
I20260709 13:16:48.885208 30553 rpcz_store.cc:276] 0709 13:16:47.345038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.345174 (+   136us) service_pool.cc:224] Handling call
0709 13:16:48.883179 (+1538005us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b58001c8098. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8a22 00007f180ccd1c09 00007f180be2bee0 00007f180bef50ab 00007f180bef5131 00007f180bea30a0 00007f180bea2984 00007f180541d6d4 00007f180541d662
0709 13:16:48.884698 (+  1519us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4177152}
I20260709 13:16:48.885950  5245 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:48.886576  5245 raft_consensus.cc:697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:48.887596  5245 consensus_queue.cc:237] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.888682 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.890091 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.883526  5453 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.892020 31383 raft_consensus.cc:1240] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:48.875036  5411 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.892459  5287 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.892726 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2086) took 1975 ms. Trace:
I20260709 13:16:48.878441 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.893141 31383 rpcz_store.cc:276] 0709 13:16:46.917960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.891497 (+1973537us) service_pool.cc:224] Handling call
0709 13:16:48.891812 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.892470 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.892670 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.898011  5411 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.898746  5411 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.900403  5580 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:48.900893  5580 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:48.900980 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.901954 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1380) took 1556 ms. Trace:
I20260709 13:16:48.901778  5580 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.902346 30624 rpcz_store.cc:276] 0709 13:16:47.345026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.476404 (+131378us) service_pool.cc:224] Handling call
0709 13:16:47.477194 (+   790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.482276 (+  5082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:47.482331 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:47.499406 (+ 17075us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.671834 (+172428us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:47.671886 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.671971 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.856792 (+1184821us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:48.901670 (+ 44878us) raft_consensus.cc:1685] finished
0709 13:16:48.901693 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.901886 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4525,"raft.queue_time_us":7918,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":26942,"spinlock_wait_cycles":4091264,"thread_start_us":172000,"threads_started":1,"wal-append.queue_time_us":313788}
I20260709 13:16:48.903339  5410 raft_consensus.cc:3055] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:16:48.904965  5410 raft_consensus.cc:740] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:48.905611  5440 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.906157  5410 consensus_queue.cc:260] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:48.907665  5410 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:48.903456 31389 raft_consensus.cc:2468] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:48.910107 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2090) took 1986 ms. Trace:
I20260709 13:16:48.910487 31389 rpcz_store.cc:276] 0709 13:16:46.923889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.909200 (+1985311us) service_pool.cc:224] Handling call
0709 13:16:48.909545 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.909762 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.909786 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.909835 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:48.909868 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.909926 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.910040 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.915212 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:48.919718 30700 raft_consensus.cc:2368] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20.
I20260709 13:16:48.929041  5580 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:48.939785 30550 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:48.939787 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69007.6us
W20260709 13:16:48.938791 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:48.940958 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:48.942457 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2114) took 2099 ms. Trace:
I20260709 13:16:48.943138 31391 rpcz_store.cc:276] 0709 13:16:46.842931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.507425 (+664494us) service_pool.cc:224] Handling call
0709 13:16:47.507841 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:47.601151 (+ 93310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.942168 (+1341017us) spinlock_profiling.cc:243] Waited 75.2 ms on lock 0x7b5400145ad4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.942384 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172929792}
I20260709 13:16:48.943949  5453 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:16:48.944991  5638 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 13:16:48.945103 31391 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2095) took 1979 ms. Trace:
I20260709 13:16:48.945402  5638 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:48.943383 30597 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:48.946666 31361 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 2092) took 2019 ms (client timeout 1544 ms). Trace:
I20260709 13:16:48.946826  5638 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:48.947119 31361 rpcz_store.cc:269] 0709 13:16:46.927446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.073405 (+145959us) service_pool.cc:224] Handling call
0709 13:16:48.946404 (+1872999us) spinlock_profiling.cc:243] Waited 63.2 ms on lock 0x7b540057b054. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:48.946556 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":145373056}
I20260709 13:16:48.948302  5407 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:48.948998 31361 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:16:48.948872  5407 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:48.949823 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2096) took 1984 ms. Trace:
I20260709 13:16:48.947417  5411 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:48.950222 31361 rpcz_store.cc:276] 0709 13:16:46.965786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.948130 (+1982344us) service_pool.cc:224] Handling call
0709 13:16:48.948533 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.949492 (+   959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.949752 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.967355 30700 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:48.951781  5407 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:48.953742 30698 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 13:16:48.954111  5638 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:16:48.945629 31391 rpcz_store.cc:276] 0709 13:16:46.965377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.944236 (+1978859us) service_pool.cc:224] Handling call
0709 13:16:48.944548 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.944760 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:48.944788 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:48.944838 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:48.944871 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.944926 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.945045 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.970711 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:16:48.970953 31391 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:48.972254 31391 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:48.973722 31391 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:16:48.955919  5626 log.cc:927] Time spent T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 1.751s	user 0.004s	sys 0.010s
I20260709 13:16:48.974764 31396 raft_consensus.cc:1240] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:48.975521 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2094) took 2027 ms. Trace:
I20260709 13:16:48.975919 30707 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2500) took 2055 ms. Trace:
I20260709 13:16:48.976784 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.976641 30707 rpcz_store.cc:276] 0709 13:16:46.920888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.921112 (+   224us) service_pool.cc:224] Handling call
0709 13:16:46.921466 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:46.921845 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:46.921874 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:46.922391 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:16:47.010815 (+ 88424us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:47.197282 (+186467us) log.cc:844] Serialized 14 byte log entry
0709 13:16:47.228630 (+ 31348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:47.228721 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:48.955262 (+1726541us) spinlock_profiling.cc:243] Waited 52 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:48.975575 (+ 20313us) raft_consensus.cc:1685] finished
0709 13:16:48.975716 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:48.975853 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":292944,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":6243,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":54272,"spinlock_wait_cycles":119677696,"thread_start_us":206008,"threads_started":2,"wal-append.queue_time_us":229311}
I20260709 13:16:48.979260 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.989704 30555 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
W20260709 13:16:48.990314  5658 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.631s	user 0.000s	sys 0.011s
I20260709 13:16:48.981370 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:48.981683 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:48.997206 31394 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50010 (request call id 2093) took 2066 ms. Trace:
I20260709 13:16:48.997915  5548 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:48.998401 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:48.998646  5548 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:48.982458 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:48.999563 30630 raft_consensus.cc:2376] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21. Re-sending same reply.
I20260709 13:16:48.999657  5548 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:48.985082 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2517) took 1345 ms. Trace:
I20260709 13:16:49.000643 30703 rpcz_store.cc:276] 0709 13:16:47.639859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.652134 (+ 12275us) service_pool.cc:224] Handling call
0709 13:16:47.652463 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.399587 (+747124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.984813 (+585226us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54009c8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:48.985016 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715248896}
I20260709 13:16:48.983527 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.002017 30625 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:49.003283  5543 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:48.985714 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:48.986434 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.003904  5543 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:49.004396 30628 raft_consensus.cc:2410] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:16:49.004958  5543 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:48.978504 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.988523  5411 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:48.979625 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:49.035563 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:48.956475 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:48.994788 31383 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 23. Ops: []
I20260709 13:16:48.996263 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:48.997646 31394 rpcz_store.cc:276] 0709 13:16:46.931353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.293156 (+361803us) service_pool.cc:224] Handling call
0709 13:16:48.996984 (+1703828us) spinlock_profiling.cc:243] Waited 55.9 ms on lock 0x7b540044e654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:48.997139 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128668416}
I20260709 13:16:48.999899  5310 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:48.987824  5590 raft_consensus.cc:3055] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 13:16:49.043303 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86924.2us
I20260709 13:16:49.044021 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2088) took 2125 ms. Trace:
I20260709 13:16:49.044651 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.044590 31383 rpcz_store.cc:276] 0709 13:16:46.918795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.893912 (+1975117us) service_pool.cc:224] Handling call
0709 13:16:48.894235 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.043629 (+149394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.043938 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:48.986079  5407 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.007741 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:48.874214  5585 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:49.008365 30700 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:49.009512 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.010051 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:48.975854 31396 rpcz_store.cc:276] 0709 13:16:46.948671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.942178 (+1993507us) service_pool.cc:224] Handling call
0709 13:16:48.942738 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.975250 (+ 32512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:48.975450 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:49.022806 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:49.052734 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.053242 30629 raft_consensus.cc:2410] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:16:49.049685 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.022354 31367 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 25. Current term is 26. Ops: []
I20260709 13:16:49.033460 31391 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 15. Current term is 16. Ops: []
I20260709 13:16:49.033789 30625 raft_consensus.cc:1240] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:16:48.940958 31389 raft_consensus.cc:1240] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 13:16:49.055171 30628 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:49.056098 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2091) took 2129 ms. Trace:
I20260709 13:16:48.990726  5654 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:49.041133 30550 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 13:16:49.041325 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.048027 30632 raft_consensus.cc:3055] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:16:49.034665 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 245644us
I20260709 13:16:49.035580 30698 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:49.056713 31389 rpcz_store.cc:276] 0709 13:16:46.927146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.917138 (+1989992us) service_pool.cc:224] Handling call
0709 13:16:48.938690 (+ 21552us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.055736 (+117046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.056030 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:16:49.052376 30631 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
W20260709 13:16:49.088742 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.070082 30551 raft_consensus.cc:3055] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:49.089524 30551 raft_consensus.cc:740] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:49.089643 30552 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:49.089531 30558 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:49.089911 30632 raft_consensus.cc:740] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:49.091300 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1338) took 2910 ms. Trace:
I20260709 13:16:49.070184 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:49.069428 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:49.096879 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.063203 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2118) took 2157 ms. Trace:
I20260709 13:16:49.097966 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:49.101624  5386 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:16:49.048659  5411 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:49.091802 30632 consensus_queue.cc:260] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:49.105163 30632 raft_consensus.cc:3060] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.098822 31364 rpcz_store.cc:276] 0709 13:16:46.905455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:47.521959 (+616504us) service_pool.cc:224] Handling call
0709 13:16:47.522254 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:47.974942 (+452688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.062881 (+1087939us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b540020d354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:49.063130 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031497088}
I20260709 13:16:49.105007 31292 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.106890 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:49.090637 30551 consensus_queue.cc:260] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:49.108557 30551 raft_consensus.cc:3060] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:49.054860  5545 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:49.107260 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.060567  5587 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.111449 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:49.049618  5546 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:49.043512  5590 raft_consensus.cc:740] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:49.134415 30705 raft_consensus.cc:2410] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:16:49.135561 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:49.133955  5595 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 13:16:49.137900  5411 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:49.104776  4979 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:49.101637  5549 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:16:49.143100  5585 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:49.134374  5287 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:49.146835  5287 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:49.149225  5287 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:49.121840  5396 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:16:49.150081 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.151533 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.091971 30550 rpcz_store.cc:276] 0709 13:16:46.180891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:46.631529 (+450638us) service_pool.cc:224] Handling call
0709 13:16:46.632050 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:48.939213 (+2307163us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b58002486d0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e390aa 00007f1806ebb447 00007f1806ed3fe7 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4
0709 13:16:49.090930 (+151717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.091159 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400826624}
I20260709 13:16:49.120819 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:49.114656 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.123705 30631 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:49.124050 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:49.135438  5169 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:49.132283 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:49.155296 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:49.156080 30553 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:49.122376  5571 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:16:49.160919 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:49.058565  5407 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:49.151427  5640 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:49.168073  5654 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:49.172685 30558 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:49.175060 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:49.201136 30552 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:49.160635  5411 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.152362  5546 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:49.161252  5592 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.162529  5408 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:16:49.182410 30555 raft_consensus.cc:2393] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:49.155252  5554 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.182864  5568 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:49.183414  4979 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:49.186590 30629 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:49.190310 30631 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:16:49.188910 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:49.216141 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 424699us
I20260709 13:16:49.188107  5528 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:49.243085 30551 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:16:49.280550 30551 pending_rounds.cc:85] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 11
I20260709 13:16:49.281031 30551 pending_rounds.cc:107] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 13:16:49.281405 30551 raft_consensus.cc:2889] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:49.250509 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.285298 30552 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 26.
W20260709 13:16:49.291320 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 102490us
I20260709 13:16:49.235565  5554 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.207393  5411 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:49.208316  5580 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.222946  5586 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.016s	sys 0.030s
I20260709 13:16:49.215204  5546 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:49.166333  5544 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.296185  5669 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:49.309517 30670 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:49.292752  5571 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:49.228590  5592 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:49.316098 31391 raft_consensus.cc:1275] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:16:49.318284 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.319748 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:49.375869 31388 raft_consensus.cc:2368] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23.
I20260709 13:16:49.320564 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:49.376631 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 67226.7us
W20260709 13:16:49.322362 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.322819 31383 raft_consensus.cc:1275] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:16:49.323655 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.325484 30673 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:49.378052 30670 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
W20260709 13:16:49.378396 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53029.3us
I20260709 13:16:49.326774 30705 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:49.234338 30632 raft_consensus.cc:1275] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:16:49.380718 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 331128us
I20260709 13:16:49.331326 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:49.298231  5608 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:16:49.330312  5668 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:49.332948 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:49.333772 30551 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:16:49.400633 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:49.335655 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.401728  5669 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:16:49.401968 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66406.4us
I20260709 13:16:49.401305 31396 raft_consensus.cc:3060] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:49.402300  5669 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:49.403555  5669 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:49.403362  4979 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:49.335718 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.367895 30558 raft_consensus.cc:1240] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 13:16:49.303084  5407 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:49.326517 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:49.331763 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.408171  5169 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:49.409142 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 77928.1us
I20260709 13:16:49.370518  5396 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:16:49.412379 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.398896 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.398046 31361 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 24. Current term is 25. Ops: []
W20260709 13:16:49.400158 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 210458us
I20260709 13:16:49.436570 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.382704  5652 raft_consensus.cc:2804] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:49.385696  5287 raft_consensus.cc:2804] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:49.439850 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.444758 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.411537  5618 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:16:49.452024  5440 consensus_queue.cc:1059] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:16:49.456357  5411 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:49.454084  5431 consensus_queue.cc:1059] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 23.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:16:49.372619  5580 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:49.460426 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.460801 31367 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.466719 30624 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:49.451788  5586 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:49.449730 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:49.446535 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.380906  5571 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:49.469058 31364 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:49.440331  5169 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:49.469941 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.485900 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.474123 31378 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:49.447053 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 231711us
I20260709 13:16:49.438151  5654 consensus_queue.cc:1059] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 13:16:49.486629 30522 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.494472 31367 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:49.499886  5549 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:49.502317  5682 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:49.503337  5674 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:16:49.454334  5493 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
W20260709 13:16:49.503991 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 89546.1us
I20260709 13:16:49.505347 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:49.540318 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53816.5us
I20260709 13:16:49.542096 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:49.545522 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.546531 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.485111  5680 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:49.544899  5384 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:49.554256 31396 raft_consensus.cc:2468] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 26.
W20260709 13:16:49.570144 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.577471 31364 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 13:16:49.576315 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 135364us
I20260709 13:16:49.557452  5493 consensus_queue.cc:1243] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:16:49.581523 30670 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.599205 30698 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:16:49.568982  5408 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:49.586885 30628 raft_consensus.cc:3055] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:49.608173 30628 raft_consensus.cc:740] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:49.590430  5549 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:49.602504 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:49.610652 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.611479 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:49.612471 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:49.613186 30699 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.569950  5683 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:49.607121 30703 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:49.601624 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:49.571786  5238 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.622138 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75740.7us
I20260709 13:16:49.598738 31378 raft_consensus.cc:2468] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 16.
I20260709 13:16:49.615530 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:49.623448 30702 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.619931 30704 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:16:49.584055 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.628085  5652 raft_consensus.cc:697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:49.601153  5595 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:49.574375  5407 consensus_queue.cc:1059] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:16:49.606549  5396 raft_consensus.cc:3055] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:16:49.654170 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 166359us
W20260709 13:16:49.636215 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 132821us
I20260709 13:16:49.645419 30702 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:49.601446  5287 raft_consensus.cc:697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 26 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:49.571877  5213 consensus_queue.cc:1059] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 24.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:16:49.610518  5586 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:49.599763  5568 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:49.657529 30670 leader_election.cc:400] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23.
I20260709 13:16:49.624671  5551 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:49.629487  5544 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:49.622355  5549 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.693204 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.632688  5631 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:49.693085 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.760058 30631 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:49.759976 30627 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:16:49.695976 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.633684  5665 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.695094 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.653206  5592 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:49.648516  5595 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:49.633847  5686 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:49.613004  5684 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:16:49.711272 30670 leader_election.cc:403] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.755748 30629 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:49.693836 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.785629 30632 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:49.715526 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.174929  5590 consensus_queue.cc:260] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.679562 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.758955 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.762988 30624 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.755005  5384 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.690068  5667 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:16:49.692523  5675 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:16:49.633627  5238 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.767413 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.770313 30626 raft_consensus.cc:2393] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:49.771478 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 141401us
I20260709 13:16:49.750985 30597 leader_election.cc:400] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20.
I20260709 13:16:49.411698  5668 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:49.658550  5554 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:49.691046  5440 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:49.654942  5396 raft_consensus.cc:740] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:49.712512  5245 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:49.691882  5673 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.254s	user 0.007s	sys 0.002s
W20260709 13:16:49.789316 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 131957us
I20260709 13:16:49.713347 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.810402  5675 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:49.857350 30625 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:16:49.790928 30632 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:16:49.871078 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.872200 30632 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:49.769812  5435 raft_consensus.cc:3060] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:49.766371  5580 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:49.809835 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 130409us
I20260709 13:16:49.810920 30624 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:49.814365 30629 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:49.753621  5568 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:49.691903  5682 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:16:49.818826 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.819643 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
W20260709 13:16:49.886413 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50021.4us
I20260709 13:16:49.887156 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.820210 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:49.887702 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.888710 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.888988 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.889174 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:49.889514 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.890081 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.821550 30597 leader_election.cc:403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:16:49.822090 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.822104 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:49.765388  5693 consensus_queue.cc:1059] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 13:16:49.826769 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.828446 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:49.824354 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.769980  5665 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:49.817344  5672 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 13:16:49.847509 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:49.678025  5408 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
W20260709 13:16:49.771224  5678 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 13:16:49.815925  5590 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:16:49.850528  5685 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:16:49.901962  5684 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:49.913112  5687 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:16:49.912890 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:49.914917  5695 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:49.919178  5665 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:49.928973 30699 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:49.929852  5692 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021) with higher term. Message: Invalid argument: T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23.
I20260709 13:16:49.690829  5287 consensus_queue.cc:237] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:49.877247 30632 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:16:49.936460 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.937175  5668 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:49.937563 30632 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:49.823037  5431 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:49.809037  5384 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:49.820482 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:49.879289  5681 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:16:49.891528 31378 raft_consensus.cc:2393] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 29: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:49.948078 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:49.949420 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 60340.2us
I20260709 13:16:49.950336 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.893605 31394 raft_consensus.cc:2393] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 23: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:49.893723 30633 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:49.894125 30703 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:49.801462  5652 consensus_queue.cc:237] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:49.895471 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:49.956768 30596 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:49.895200 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 184945us
I20260709 13:16:49.896693 31364 raft_consensus.cc:2376] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24. Re-sending same reply.
I20260709 13:16:49.900537 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:49.958597 31396 raft_consensus.cc:2410] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 26 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 13:16:49.823506  5694 consensus_queue.cc:1048] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:16:49.907059 31389 raft_consensus.cc:2410] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:16:49.940824 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 29.
I20260709 13:16:49.887727  5686 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:49.943622 31388 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 25: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:49.890770  5568 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:16:49.942405  5689 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:16:49.893188  5580 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:49.866601  5245 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:49.882751  5677 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.304s	user 0.000s	sys 0.002s
I20260709 13:16:49.959883 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:49.962265 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.056135 30703 raft_consensus.cc:2410] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 13:16:49.965126 30631 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 29 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
W20260709 13:16:50.059026 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 99287us
I20260709 13:16:49.943418  5700 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
I20260709 13:16:50.007121  5554 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:16:50.015522 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 65273us
I20260709 13:16:49.962766  5684 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 19. Current term is 20.
W20260709 13:16:50.032644 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 404781us
I20260709 13:16:50.083319 30703 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:49.959578  5668 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:49.951674 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:49.940781  5675 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.036223 30633 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:50.085234 30559 raft_consensus.cc:1240] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:50.086041 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:49.898243  5396 consensus_queue.cc:260] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.086804  5238 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:50.087037  5396 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:50.087643 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.090425 31391 raft_consensus.cc:1275] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:50.085268  5675 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:50.091651  5384 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:50.092269 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.075001  5549 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:50.095263  5213 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.095609  5668 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:50.096091  5213 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.096921 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:50.097258  5571 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:50.096798  5668 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.039708 30597 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:50.099189 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 59624.6us
I20260709 13:16:50.099905 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:49.879505  5595 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.100945  5595 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:50.101163 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:50.103590  5493 raft_consensus.cc:3055] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:50.103760 31396 raft_consensus.cc:1240] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:50.037578  5665 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:50.104118  5493 raft_consensus.cc:740] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:50.104668  5665 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:50.105332  5546 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.729s	user 0.006s	sys 0.007s
I20260709 13:16:50.105111  5493 consensus_queue.cc:260] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.105649  5665 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:49.951553  5431 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.043994  5711 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:50.108505  5431 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:50.108726  5245 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:50.110491 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.111042  5430 raft_consensus.cc:3055] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:50.111239 30552 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:50.111493  5430 raft_consensus.cc:740] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.112368 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:50.048298  5587 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.913s	user 0.009s	sys 0.006s
I20260709 13:16:50.115264  5587 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:50.115413 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:50.116729 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.071089 30629 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 25. Current term is 26. Ops: [22.14-25.15]
I20260709 13:16:50.119714 30633 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:50.119828  5696 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.120445  5696 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.120958 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:50.121608 30633 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:50.122157  5169 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:50.127127  5608 raft_consensus.cc:3055] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:50.128259 31396 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:16:50.128736 31396 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:50.130791 30705 raft_consensus.cc:1275] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:16:50.131726 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.044086 30632 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:50.066509  5592 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:50.114634 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50010 (request call id 2097) took 2499 ms. Trace:
W20260709 13:16:50.117833 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 166272us
I20260709 13:16:50.140058 31361 rpcz_store.cc:276] 0709 13:16:47.615187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:48.967648 (+1352461us) service_pool.cc:224] Handling call
0709 13:16:48.968152 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.435302 (+467150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.114334 (+679032us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b540002e754. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:50.114548 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":965008640}
I20260709 13:16:50.117475 30628 consensus_queue.cc:260] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.141465 30705 raft_consensus.cc:1240] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:50.141848 30628 raft_consensus.cc:3060] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:50.106328  5493 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:50.134472 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.037065  5654 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:50.135288 30671 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:50.137501 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.152572 30703 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:50.138562 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.129609 31396 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.154301 30558 raft_consensus.cc:2393] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:50.148092 30522 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:50.055810  5580 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.151687 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.183467 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.177732 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.179808 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:50.154955 31396 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:16:50.185771 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 50593.4us
I20260709 13:16:50.189016 30630 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2232) took 1146 ms. Trace:
I20260709 13:16:50.184496 30707 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:50.194169 30633 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 26.
I20260709 13:16:50.195741 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:50.198395 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 51659.9us
I20260709 13:16:50.194418 30628 raft_consensus.cc:1275] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:16:50.189580 30630 rpcz_store.cc:276] 0709 13:16:49.042987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.046003 (+  3016us) service_pool.cc:224] Handling call
0709 13:16:49.046511 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.058711 (+ 12200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:49.058741 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.101895 (+ 43154us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.381783 (+279888us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:49.381848 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.383604 (+  1756us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.188681 (+805077us) raft_consensus.cc:1685] finished
0709 13:16:50.188718 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.188916 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":33756,"thread_start_us":279362,"threads_started":1,"wal-append.queue_time_us":436139}
I20260709 13:16:50.200282 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:50.195120  5408 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:50.141579  5554 raft_consensus.cc:2764] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 13:16:50.205796  5669 consensus_meta.cc:287] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.802s	user 0.014s	sys 0.000s
I20260709 13:16:50.204222 30671 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:50.141884  5608 raft_consensus.cc:740] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.205153  5694 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:50.105767  5546 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.148052  5440 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.217325 30628 pending_rounds.cc:85] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 11
I20260709 13:16:50.223110 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:50.147044  5699 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:50.182950  5442 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 13:16:50.112408  5430 consensus_queue.cc:260] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.146322  5543 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 13:16:50.273213 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50201.7us
I20260709 13:16:50.272687 30628 pending_rounds.cc:107] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:16:50.187705  5245 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 13:16:50.255260 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:50.187271  5042 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:16:50.177209  5683 consensus_queue.cc:1048] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:16:50.238265 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.208423  5213 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:50.283972  5608 consensus_queue.cc:260] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.281769  5590 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:50.272656  5238 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:50.287206  5440 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.286338 30700 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:50.298677 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:50.299825 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.334070 30628 raft_consensus.cc:2889] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:50.382248 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.382354 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46186 (request call id 2239) took 1174 ms. Trace:
I20260709 13:16:50.382824 30628 rpcz_store.cc:276] 0709 13:16:49.207544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.207698 (+   154us) service_pool.cc:224] Handling call
0709 13:16:49.208079 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:49.586734 (+378655us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540064f3a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:16:50.102504 (+515770us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b540064f3a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e5b06f 00007f1806ed3f60 00007f1806ed57de 00007f1806edfb49 00007f1806ee0416 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62
0709 13:16:50.382037 (+279533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.382287 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945989632}
W20260709 13:16:50.334297 30596 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.544s (SENT)
I20260709 13:16:50.337831 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.294361 31378 raft_consensus.cc:1240] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:50.341719 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:50.384168 30596 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:50.388932 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.390364 30555 raft_consensus.cc:2410] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
W20260709 13:16:50.281749  5709 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:16:50.279359  5697 consensus_queue.cc:1059] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:16:50.376818  5700 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:50.515293 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.355916 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:50.226882  5592 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:50.279512  5699 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.283456  5652 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:16:50.352732  5673 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:16:50.240693 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:50.581166 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 377053us
I20260709 13:16:50.274029  5571 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:50.393759 30551 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:16:50.348722  5554 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:50.356037  5430 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:50.398656 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.621410 30556 raft_consensus.cc:2393] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 26: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:50.595228 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.328917 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:50.402666 30631 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:50.359740  5590 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.404856 30628 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:50.400822  5317 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:16:50.450476  5595 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:50.453117  5042 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:50.485895  5684 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:50.497197  5697 raft_consensus.cc:3055] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:50.581033  5686 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:50.558544  5690 raft_consensus.cc:3055] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:50.561671  5699 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:50.553823  5592 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:50.562968  5696 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:50.584072 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 383883us
I20260709 13:16:50.585091  5710 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:50.587098  5580 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:50.587038  5711 consensus_queue.cc:1059] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:50.582259 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:50.586903 31396 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
W20260709 13:16:50.618165  5568 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.006s
I20260709 13:16:50.282833  5384 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:16:50.620690  5213 raft_consensus.cc:2804] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:16:50.633735  5554 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:50.642225  5679 log.cc:927] Time spent T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.187s	user 0.004s	sys 0.002s
I20260709 13:16:50.641222  5590 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:50.633424  5407 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.570537  5700 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:50.645849  5688 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.028s	sys 0.061s
I20260709 13:16:50.645850  5697 raft_consensus.cc:740] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:50.598501  5669 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.620464  5571 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.651849  5690 raft_consensus.cc:740] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.656816  5686 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.703538  5690 consensus_queue.cc:260] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.704577  5571 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:50.704627  5690 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:50.705612  5686 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:50.647390  5618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.013s	sys 0.016s
I20260709 13:16:50.669090  5592 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:50.647363  5595 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.709618  5700 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:50.692420  5711 raft_consensus.cc:3055] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:50.711577  5711 raft_consensus.cc:740] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.712467  5697 consensus_queue.cc:260] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.712718  5711 consensus_queue.cc:260] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.714017  5697 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:50.714236  5711 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:16:50.714975  5618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.014s	sys 0.016s
I20260709 13:16:50.715377  5618 raft_consensus.cc:697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.716325  5695 raft_consensus.cc:3055] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:50.716953  5695 raft_consensus.cc:740] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:50.700227  5688 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.029s	sys 0.062s
I20260709 13:16:50.716454  5618 consensus_queue.cc:237] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.717912  5695 consensus_queue.cc:260] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.721469  5695 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:50.734403  5675 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.643s	user 0.016s	sys 0.000s
I20260709 13:16:50.588459 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:50.777731 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 189119us
I20260709 13:16:50.408133 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.781742 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2314) took 1628 ms. Trace:
W20260709 13:16:50.783032 30554 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.011s	sys 0.040s
W20260709 13:16:50.783722 30554 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.011s	sys 0.040s
I20260709 13:16:50.783779 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.785684 30553 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
W20260709 13:16:50.786928 30559 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.017s	sys 0.028s
I20260709 13:16:50.669350 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:50.589493 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 255247us
I20260709 13:16:50.679975 31378 raft_consensus.cc:2393] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 25: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:50.620072 30625 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
W20260709 13:16:50.642795 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.792522  5440 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:50.793083 30625 raft_consensus.cc:1275] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:50.793311  5440 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:50.793965  5440 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:50.679757 31396 pending_rounds.cc:85] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Aborting all ops after (but not including) 9
I20260709 13:16:50.795639 30628 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:50.795925 31396 pending_rounds.cc:107] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:16:50.565574 30629 raft_consensus.cc:1240] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
I20260709 13:16:50.796629 31396 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:50.582338 30630 raft_consensus.cc:1240] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:50.799978 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.803124  5213 raft_consensus.cc:697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:16:50.626097 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.648355  5042 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.812595 30552 raft_consensus.cc:1240] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:16:50.640491  5310 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.690235  5568 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:50.815820  5310 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.560711 30627 raft_consensus.cc:1240] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:50.819195  5407 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.821553 30626 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:50.823436  5675 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:50.824787  5675 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:50.774240 30523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.010s	sys 0.006s
W20260709 13:16:50.828161 30523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.010s	sys 0.006s
I20260709 13:16:50.827912  5682 consensus_queue.cc:1059] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:16:50.828946 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 681331us
I20260709 13:16:50.829864  5682 raft_consensus.cc:3055] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:16:50.829612  5654 consensus_queue.cc:1048] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:50.830541  5682 raft_consensus.cc:740] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:16:50.787429 30559 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.017s	sys 0.029s
I20260709 13:16:50.820129 30627 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1392) took 1062 ms. Trace:
I20260709 13:16:50.805465  5213 consensus_queue.cc:237] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.786204 30671 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:50.833659  5568 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:50.822345  5592 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.863943  5310 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:50.865967 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.845428  5554 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:50.875020 30624 raft_consensus.cc:2368] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20.
W20260709 13:16:50.882808 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 52512.4us
I20260709 13:16:50.832756 30627 rpcz_store.cc:276] 0709 13:16:49.757177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.760858 (+  3681us) service_pool.cc:224] Handling call
0709 13:16:49.776846 (+ 15988us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.560319 (+783473us) spinlock_profiling.cc:243] Waited 25.2 ms on lock 0x7b540084d620. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806e54dc0 00007f1806edf1c0 00007f1806ee0231 00007f1806edcd8f 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c
0709 13:16:50.819264 (+258945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:50.819858 (+   594us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58029952}
I20260709 13:16:50.827836  5688 consensus_queue.cc:1048] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:16:50.884308  5042 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:50.887495  5682 consensus_queue.cc:260] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:50.890908  5692 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:50.891903  5692 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:50.893572  5692 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.899080  5710 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:50.900238  5710 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:50.782522 30550 rpcz_store.cc:276] 0709 13:16:49.153529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.156100 (+  2571us) service_pool.cc:224] Handling call
0709 13:16:49.157006 (+   906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:49.157532 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:49.157563 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:49.186909 (+ 29346us) log.cc:844] Serialized 26 byte log entry
0709 13:16:49.309017 (+122108us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:49.309085 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:49.309176 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:50.580258 (+1271082us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:50.780848 (+200590us) raft_consensus.cc:1685] finished
0709 13:16:50.780920 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:50.781469 (+   549us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4310,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9584,"spinlock_wait_cycles":28341504,"thread_start_us":121595,"threads_started":1,"wal-append.queue_time_us":264762}
I20260709 13:16:50.882956  5543 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:50.904148  5543 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:50.906950  5675 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:50.908210 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:50.911527 30552 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:50.915858 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.919409 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:50.928728  5682 raft_consensus.cc:3060] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:16:50.935042 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 149165us
I20260709 13:16:50.953248  4979 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:50.954082 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 284668us
I20260709 13:16:50.898365 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:50.953696 30552 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:16:50.958500  5407 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:51.002880 30522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.009s	sys 0.009s
W20260709 13:16:51.003670 30522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.009s	sys 0.009s
W20260709 13:16:51.004319 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 856304us
W20260709 13:16:51.018813 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 121009us
W20260709 13:16:51.019095 30596 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.834s (SENT) [suppressed 23 similar messages]
I20260709 13:16:51.022688  5675 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:16:51.026134 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.019368  5710 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:51.035247  5169 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.912s	user 0.017s	sys 0.012s
I20260709 13:16:51.036100  5546 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.036752  5169 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:51.039003  5042 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:16:51.040355 31367 raft_consensus.cc:1240] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: []
W20260709 13:16:51.022208 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.016842  5592 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.062372 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:50.067896 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:16:51.064586  5410 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.065721  5410 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:51.068419 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.068819  5410 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:51.069440 30703 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 18.
I20260709 13:16:51.069576  5571 consensus_queue.cc:1048] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:51.055516 30700 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1233) took 1452 ms. Trace:
I20260709 13:16:51.070698 30700 rpcz_store.cc:276] 0709 13:16:49.602945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.606671 (+  3726us) service_pool.cc:224] Handling call
0709 13:16:49.607068 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.378124 (+771056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.054707 (+676583us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b54003d8bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:51.055113 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561687296}
I20260709 13:16:51.072623 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:51.073751  5396 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.986s	user 0.017s	sys 0.009s
I20260709 13:16:51.077651  5042 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:16:51.089730 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75461.1us
I20260709 13:16:51.082207  5595 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.097494  5590 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:51.104025  5493 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.949s	user 0.017s	sys 0.041s
I20260709 13:16:51.094867 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:51.095230 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 133754us
I20260709 13:16:51.106835  5618 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:51.107995 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:51.112341  5726 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.112633  5568 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.112481  5407 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.113420  5317 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:51.116525  5440 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:51.120534 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 73) took 30439 ms (client timeout 15000 ms). Trace:
I20260709 13:16:51.121722  5590 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:51.092607 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80648.4us
W20260709 13:16:51.143487 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.157382 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.158897  5317 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.164811  5543 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.167485 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:51.167266  5407 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.168336 30556 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:51.166481  5317 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:51.174561 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.205514 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.206441  5549 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.172865  5669 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.174111 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1669) took 1091 ms. Trace:
I20260709 13:16:51.207353  5549 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:51.178920  5430 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.011s
I20260709 13:16:51.210624  5726 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.211338  5587 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.212062  5587 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:51.213544  5726 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.213922  5669 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.173678  5699 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.217908  5747 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:51.219878 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112042us
I20260709 13:16:51.214841  5587 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:51.234508 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1401) took 1280 ms. Trace:
I20260709 13:16:51.239269  5568 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.240376  5747 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:51.244637 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.146096 30378 rpcz_store.cc:269] 0709 13:16:20.795988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:20.801096 (+  5108us) service_pool.cc:224] Handling call
0709 13:16:51.120205 (+30319109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:43.177724 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:43.178757 (+  1033us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:43.178838 (+    81us) write_op.cc:435] Acquired schema lock
0709 13:16:43.178866 (+    28us) tablet.cc:662] Decoding operations
0709 13:16:43.181702 (+  2836us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:43.181855 (+   153us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:43.181900 (+    45us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:16:43.185747 (+  3847us) tablet.cc:701] Row locks acquired
0709 13:16:43.185820 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:16:43.186112 (+   292us) write_op.cc:270] Start()
0709 13:16:43.186303 (+   191us) write_op.cc:276] Timestamp: P: 1783603003186006 usec, L: 0
0709 13:16:43.186354 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 13:16:43.187658 (+  1304us) log.cc:844] Serialized 9770 byte log entry
0709 13:16:43.735716 (+548058us) op_driver.cc:464] REPLICATION: finished
0709 13:16:43.937689 (+201973us) write_op.cc:301] APPLY: starting
0709 13:16:43.946413 (+  8724us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:44.585086 (+638673us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:44.585238 (+   152us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:45.734409 (+1149171us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:45.884370 (+149961us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:16:45.912872 (+ 28502us) write_op.cc:312] APPLY: finished
0709 13:16:45.952111 (+ 39239us) log.cc:844] Serialized 219 byte log entry
0709 13:16:45.952459 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:45.952780 (+   321us) write_op.cc:454] Released schema lock
0709 13:16:45.952887 (+   107us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:45.997284 (+ 44397us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:45.998646 (+  1362us) write_op.cc:454] Released schema lock
0709 13:16:46.795609 (+796963us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b18003387a0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df0522 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
Metrics: {"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":172952,"apply.run_cpu_time_us":49780,"apply.run_wall_time_us":2044854,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108877,"num_ops":25,"prepare.queue_time_us":270991,"prepare.run_cpu_time_us":70482,"prepare.run_wall_time_us":219965,"raft.queue_time_us":174689,"raft.run_cpu_time_us":16588,"raft.run_wall_time_us":173677,"replication_time_us":549152,"spinlock_wait_cycles":24735488,"thread_start_us":555419,"threads_started":4,"wal-append.queue_time_us":1110036,"wal-append.run_cpu_time_us":35002,"wal-append.run_wall_time_us":2228638}]]}
W20260709 13:16:51.226534 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 131811us
I20260709 13:16:51.227675 31292 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:51.230511  5440 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:51.232550 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.183765 31367 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50014 (request call id 2372) took 1049 ms. Trace:
I20260709 13:16:51.229031  5669 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.208184 30628 rpcz_store.cc:276] 0709 13:16:50.082896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.644198 (+561302us) service_pool.cc:224] Handling call
0709 13:16:50.794585 (+150387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.796278 (+  1693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.173829 (+377551us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b54005f3cd0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:51.174025 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":344355584}
I20260709 13:16:51.184466 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.255887 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:51.155311  5571 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:51.259860  5595 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.276521 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.276949  5669 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.277555 30553 raft_consensus.cc:2393] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:51.277138  5595 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.260228 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1400) took 1332 ms. Trace:
I20260709 13:16:51.262238 31364 raft_consensus.cc:1240] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:16:51.278831  5587 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:16:51.279822 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.263465  5747 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.263903  4979 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.281327  5384 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:51.281490 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:51.266391  5699 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.263236 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:51.282563 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 120633us
I20260709 13:16:51.283022 31361 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:51.269650  5396 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:51.285362  5699 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.285563  5396 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.285976  5396 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:51.270256 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:51.270605 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.270856 31378 raft_consensus.cc:1240] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 13:16:51.271574 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.273543  5737 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.185935 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:51.275374 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.251360 31367 rpcz_store.cc:276] 0709 13:16:50.133986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.134422 (+   436us) service_pool.cc:224] Handling call
0709 13:16:50.134752 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:51.041147 (+906395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.183430 (+142283us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400523fd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:51.183667 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081269632}
I20260709 13:16:51.275743  5618 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.288079 31396 raft_consensus.cc:2468] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:51.279984  5595 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.281261  4979 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:51.265257 31389 raft_consensus.cc:1240] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:51.268932 30597 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:51.242383  5568 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.210283  5549 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.183655 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:51.170485  5407 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:51.286103  5699 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.305699 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.313984 30555 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:51.315151 30626 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1396) took 1521 ms. Trace:
I20260709 13:16:51.315663 30626 rpcz_store.cc:276] 0709 13:16:49.794070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:49.821743 (+ 27673us) service_pool.cc:224] Handling call
0709 13:16:51.314872 (+1493129us) spinlock_profiling.cc:243] Waited 929 ms on lock 0x7b54004a83d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:51.315067 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137089920}
I20260709 13:16:51.319482  5608 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:51.308092  5737 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.322794 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:51.323611 30558 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:51.326094 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.314180 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.235433 30631 rpcz_store.cc:276] 0709 13:16:49.954064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.633235 (+679171us) service_pool.cc:224] Handling call
0709 13:16:50.633689 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.791966 (+158277us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:16:50.845674 (+ 53708us) log.cc:844] Serialized 14 byte log entry
0709 13:16:51.220835 (+375161us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b18004a1700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b
0709 13:16:51.221663 (+   828us) spinlock_profiling.cc:243] Waited 9.01 ms on lock 0x7b18004a1700. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:16:51.222118 (+   455us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:51.222192 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:51.222318 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.233885 (+ 11567us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54004aa6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:51.233971 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.234230 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15414,"prepare.queue_time_us":96583,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":10398,"raft.queue_time_us":907,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":194,"spinlock_wait_cycles":384613248,"thread_start_us":160468,"threads_started":2,"wal-append.queue_time_us":216708}
I20260709 13:16:51.309968 31388 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:51.313287 30707 raft_consensus.cc:3060] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:16:51.312357 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.342650  4979 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:51.328074  5407 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:51.307775  5568 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.338883 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.327809  5431 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:51.346051 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.287607 30633 rpcz_store.cc:276] 0709 13:16:49.927593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.624678 (+697085us) service_pool.cc:224] Handling call
0709 13:16:50.793822 (+169144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:50.794454 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:16:50.847733 (+ 53279us) log.cc:844] Serialized 14 byte log entry
0709 13:16:51.254269 (+406536us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18002e2be0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:16:51.257675 (+  3406us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:51.257757 (+    82us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:51.259761 (+  2004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.259859 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.260028 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":153770,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":3468,"raft.queue_time_us":2964,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":2065,"spinlock_wait_cycles":3535616,"thread_start_us":154268,"threads_started":2,"wal-append.queue_time_us":279504}
I20260709 13:16:51.357949  5549 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:51.358732 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:51.362355 31364 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 29.
I20260709 13:16:51.305179  5618 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.373950 31389 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:16:51.347149  5407 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.362769  5568 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:51.365728  5726 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:51.390960 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.391762  5682 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:51.406225 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.396328 31396 raft_consensus.cc:1240] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:51.376988  5549 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:51.382248  5407 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:51.401376  5699 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:51.428308 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 242673us
I20260709 13:16:51.388057  4979 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.420320  5549 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:51.449116 31367 raft_consensus.cc:1275] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:16:51.449419 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.451035 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:51.452405  5742 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:16:51.452623 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:51.453509 30699 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:51.442631 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.454936 30552 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:51.456434  5727 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.457624  5727 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.459275 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.463011  5727 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:51.459280  5737 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.456869 31389 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:51.470597 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:51.482393 30707 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:51.492328 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.492331 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:51.474259  5752 log.cc:927] Time spent T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
W20260709 13:16:51.496512 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 201016us
I20260709 13:16:51.494561 30699 raft_consensus.cc:2468] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
I20260709 13:16:51.448475  5549 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:51.498769 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:51.422578  5682 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:51.512271 30523 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.515550  5753 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:16:51.515825  5737 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.516597  5737 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:51.521445 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.510217 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:51.493654  5711 consensus_meta.cc:287] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.619s	user 0.005s	sys 0.015s
I20260709 13:16:51.422875  5431 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.530702 30703 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: [23.13-24.14]
I20260709 13:16:51.504674  5213 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.536115 30523 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:51.534997 31396 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.539007 31396 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:51.540330 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.528959  5726 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:51.497442  4979 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.547827 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:51.550421 30633 raft_consensus.cc:1275] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:16:51.545127  5696 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.554091 30698 raft_consensus.cc:1275] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:16:51.556262 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.554929  5696 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.559701  5737 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.565588 31388 raft_consensus.cc:1240] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:51.569335 30522 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:51.575420 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:51.581720  5754 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 13:16:51.581954 30626 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:51.561831  5727 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.588877 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:51.584007 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:51.544060  5213 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.606878 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.607298 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1376) took 1487 ms. Trace:
I20260709 13:16:51.599129 30597 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:51.612609 30597 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:51.614199  5590 raft_consensus.cc:2804] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:51.615111  5590 raft_consensus.cc:697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:51.616147  5726 raft_consensus.cc:2804] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:51.616897  5726 raft_consensus.cc:697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:51.616631  5590 consensus_queue.cc:237] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.618175  5726 consensus_queue.cc:237] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:51.618818  5723 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:51.608047 30557 rpcz_store.cc:276] 0709 13:16:50.080340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.080479 (+   139us) service_pool.cc:224] Handling call
0709 13:16:50.080894 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.081326 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:50.081361 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:50.081863 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 13:16:50.961585 (+879722us) spinlock_profiling.cc:243] Waited 46.8 ms on lock 0x7b18000d8100. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:51.242837 (+281252us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:51.242940 (+   103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.243097 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:51.473347 (+230250us) spinlock_profiling.cc:243] Waited 7.16 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:51.561139 (+ 87792us) raft_consensus.cc:1685] finished
0709 13:16:51.561166 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.567740 (+  6574us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2398,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"spinlock_wait_cycles":124222080,"thread_start_us":131270,"threads_started":1,"wal-append.queue_time_us":1251930}
I20260709 13:16:51.621627 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:51.598084  5384 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 13:16:51.593318  5696 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.639230 31389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.640206 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:51.643213 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 172893us
I20260709 13:16:51.596927  5727 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.659080 30624 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:51.589730  5587 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:16:51.671516 30632 raft_consensus.cc:1240] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:16:51.641527  5384 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.667522 31389 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
W20260709 13:16:51.676810 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 88164us
I20260709 13:16:51.665117 30671 leader_election.cc:400] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20.
I20260709 13:16:51.687675 30671 leader_election.cc:403] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:16:51.690346 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.648440  5595 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:51.689556 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 171244us
I20260709 13:16:51.701159 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:51.707587 30624 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 14. (term mismatch)
I20260709 13:16:51.708359 30624 pending_rounds.cc:85] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 13
I20260709 13:16:51.708717 30624 pending_rounds.cc:107] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 26.14
I20260709 13:16:51.709045 30624 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:51.664393  5723 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.672315  5568 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:51.715184 30556 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.546s	user 0.017s	sys 0.002s
W20260709 13:16:51.717582 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.686757  5384 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 29 FOLLOWER]: Advancing to term 30
W20260709 13:16:51.726248 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.728317 30699 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:51.724749 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1"
I20260709 13:16:51.723922 30556 raft_consensus.cc:2468] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:51.692651  5696 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.697960  5618 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.747112  5440 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277) with higher term. Message: Invalid argument: T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20.
I20260709 13:16:51.747910 30552 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:51.745574 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:51.717034  5699 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:16:51.723551  5694 consensus_meta.cc:287] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.288s	user 0.025s	sys 0.037s
I20260709 13:16:51.727862  5682 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:51.770059 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.723300  5737 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:16:51.789192 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.772876 31383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2147) took 1019 ms. Trace:
I20260709 13:16:51.768597  5747 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:51.798738  5753 log.cc:927] Time spent T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:16:51.816691 30632 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:51.737967  5595 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.802350  5737 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:51.831665 30673 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:51.735646  5723 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.837185 30557 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:16:51.807046  5694 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:51.837067 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:51.805188  5618 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:16:51.804720 31383 rpcz_store.cc:276] 0709 13:16:50.582590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:50.582797 (+   207us) service_pool.cc:224] Handling call
0709 13:16:50.583182 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:50.583640 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:16:50.919188 (+335548us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b18001f8ac0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803ded455 00007f180a8d53c7 00007f1806ee79a7 00007f1806ef70bf 00007f1806ef7038
0709 13:16:50.929616 (+ 10428us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b18001f8ac0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803d6ac0a 00007f180d4aa219 00007f180adb158f 00007f1803df0571 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd
0709 13:16:51.028810 (+ 99194us) log.cc:844] Serialized 14 byte log entry
0709 13:16:51.311407 (+282597us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:51.428054 (+116647us) log.cc:844] Serialized 26 byte log entry
0709 13:16:51.428254 (+   200us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:16:51.428291 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:51.449101 (+ 20810us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:51.583943 (+134842us) raft_consensus.cc:1685] finished
0709 13:16:51.583976 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:51.602208 (+ 18232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17559,"prepare.queue_time_us":55978,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":1663,"raft.queue_time_us":11151,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":42245,"spinlock_wait_cycles":19950080,"thread_start_us":271071,"threads_started":2,"wal-append.queue_time_us":284462}
I20260709 13:16:51.825508  5675 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:51.835943  5737 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:51.849748 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 157754us
W20260709 13:16:51.875411 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 286205us
W20260709 13:16:51.887107 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 142438us
I20260709 13:16:51.887143 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:51.888743 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:51.891372  5757 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.294s	user 0.002s	sys 0.000s
I20260709 13:16:51.826050  5696 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:51.840395  5431 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.898945 30705 raft_consensus.cc:3060] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:51.879979  5760 log.cc:927] Time spent T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.280s	user 0.000s	sys 0.002s
I20260709 13:16:51.880149  5711 raft_consensus.cc:2804] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:16:51.927106 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46455.5us
I20260709 13:16:51.876235  4979 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:51.883826  5587 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:51.849118  5747 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:51.928699 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:51.939574 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.884410  5675 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:51.952309  5692 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:51.981750  5317 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:51.999505 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 70912.3us
W20260709 13:16:52.000912 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:51.995607 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:51.940857  5686 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:51.952051  5711 raft_consensus.cc:697] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:51.969532  5587 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.001729 30551 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
W20260709 13:16:52.015748 30558 consensus_meta.cc:287] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.691s	user 0.017s	sys 0.001s
I20260709 13:16:52.016512 30558 raft_consensus.cc:2468] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:51.941640  5431 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:16:52.005561 30671 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:51.984177  5213 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:51.772627 30373 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 19 to 21, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:16:52.070281 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 181710us
W20260709 13:16:52.072253  5767 log.cc:927] Time spent T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:16:52.002415  5692 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:52.079363 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53804.7us
I20260709 13:16:52.088338 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:52.025277  5778 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.076017  5213 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:52.116170 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 110711us
W20260709 13:16:52.098308  5756 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.093s	user 0.001s	sys 0.008s
I20260709 13:16:52.084450  5618 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:52.117617 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.103466  5775 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.027915  5317 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.097465  4979 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:52.024590  5686 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:52.140851 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.141223  5675 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.140884  5695 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:52.141683  5763 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:16:52.141804  5595 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.145156  5697 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:52.146143 30523 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:52.149288 30599 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.150523  5776 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:52.151142  5590 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:52.152697 30626 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.562s	user 0.003s	sys 0.010s
I20260709 13:16:52.154680 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.154908 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.156150  5384 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.156746  4979 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.158372 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:52.158782  5761 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 13:16:52.159268 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:52.193472 30626 raft_consensus.cc:2468] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
W20260709 13:16:52.163340  5768 log.cc:927] Time spent T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:16:52.163651  5766 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:16:52.164881  5549 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.007s
I20260709 13:16:52.165270  5317 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.131580  5775 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.167749 30373 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 8 to 15, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:16:52.170027  5747 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:52.170647  5711 consensus_queue.cc:237] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.173745 31388 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:16:52.174028 31364 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:52.176994 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.181162 31361 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.181835  5696 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:52.182770 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66704.4us
I20260709 13:16:52.183605  5783 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:52.187493 30702 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:52.203557  5783 raft_consensus.cc:697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:52.188563 31383 raft_consensus.cc:1275] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 13:16:52.204762 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.204778  5783 consensus_queue.cc:237] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.206641  5675 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:52.207314 31361 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 25: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:16:52.178460 30705 raft_consensus.cc:2468] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
I20260709 13:16:52.121824  5778 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.210487  5783 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.192823 30556 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:52.192770 30557 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 19.
I20260709 13:16:52.212196  5778 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.217468  5696 consensus_queue.cc:1048] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.213997  5549 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:52.220825 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:52.224797  5695 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:52.225823 30698 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: []
W20260709 13:16:52.231175 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.214169  5776 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:52.245414 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.221787  5775 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:52.221798  5384 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 30 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.218551  4979 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:52.241050  5396 consensus_meta.cc:287] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.954s	user 0.010s	sys 0.008s
I20260709 13:16:52.223924  5608 consensus_queue.cc:1048] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 13:16:52.255832  5776 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:52.257082 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.223505  5431 consensus_queue.cc:1059] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
W20260709 13:16:52.263525 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.230454  5695 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.267773  5686 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:52.277441 30699 raft_consensus.cc:1275] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:16:52.279500 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.280306 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:52.250815  5692 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.288998 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:52.257644  5549 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.290428  5431 raft_consensus.cc:3055] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:52.291481  5042 raft_consensus.cc:3055] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:52.291812  5440 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.290242 30700 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:16:52.220511  5696 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:16:52.289270 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.284715 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:52.270416  5723 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.299418  5042 raft_consensus.cc:740] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:52.225670 30373 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:52.300009  5440 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.300849  5431 raft_consensus.cc:740] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:52.277261  5396 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.319855  5775 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.320041  5747 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:52.320521  5775 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.244365  5590 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.321789  5778 consensus_queue.cc:1048] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:52.305081 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.324771  5431 consensus_queue.cc:260] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.326983  5568 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.327667  5568 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.330221  5568 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.332154  5618 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.332952 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.304097  5737 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.333521  5440 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.284255  4979 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.306061 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:52.309784 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.333676 30551 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:52.319468 30627 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:16:52.323168 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 66214.4us
I20260709 13:16:52.323621  5590 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.323807  5782 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:52.323892  5775 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.332813  5618 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.334319 30552 raft_consensus.cc:1240] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 22. Current term is 23. Ops: [22.11-22.11]
I20260709 13:16:52.293156  5695 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:52.304991 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:52.336654 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.333640  5694 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.350306 30558 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:52.346278 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:52.344367  5782 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:52.352605  5782 raft_consensus.cc:3060] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:52.306742  5384 consensus_queue.cc:1059] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:16:52.326542  5431 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:52.356417 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 51536.2us
I20260709 13:16:52.353781 31294 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.359459 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:52.321851  5396 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.355099  5618 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.366319  5778 consensus_queue.cc:1059] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:16:52.379653  5696 consensus_queue.cc:1059] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.380160  5778 raft_consensus.cc:3055] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:52.380685  5778 raft_consensus.cc:740] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:52.344626  4979 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.363848  5694 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.358734  5711 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:16:52.356657  5608 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:52.389706 31364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:52.391383  5780 log.cc:927] Time spent T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.241s	user 0.003s	sys 0.000s
I20260709 13:16:52.355932  5692 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:52.392457  5711 raft_consensus.cc:3055] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:52.393008  5711 raft_consensus.cc:740] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:52.392729  5692 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.391407  5785 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.394960 31391 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.395413  5692 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.391453  5727 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.396052 30705 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2547) took 1146 ms. Trace:
I20260709 13:16:52.395390  5785 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.396657 30705 rpcz_store.cc:276] 0709 13:16:51.249273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.249475 (+   202us) service_pool.cc:224] Handling call
0709 13:16:52.395838 (+1146363us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b5400222ed4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:52.395960 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464290048}
I20260709 13:16:52.370848  5549 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:52.398826  5384 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:16:52.395910  5430 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:16:52.407294 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.408097 30703 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:16:52.408299 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.409060  5692 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.409664  5682 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:16:52.409847 30597 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:52.384850  5775 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:52.409596  5692 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:52.399820 31391 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:52.399266  5549 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:52.388437  5737 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.389032  5608 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.419837  5396 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.421630 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.423466  5568 raft_consensus.cc:2804] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:52.425172  5042 consensus_queue.cc:260] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.428776 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:52.443595 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76478.1us
I20260709 13:16:52.450525 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:52.432096  5407 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.987s	user 0.018s	sys 0.000s
I20260709 13:16:52.458184 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:52.459749 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 113913us
I20260709 13:16:52.431614  5549 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:52.459182 31383 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
W20260709 13:16:52.475057 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 107393us
I20260709 13:16:52.435499  5608 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:52.476967 31394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.478119 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.458284  5682 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:52.460215  5568 raft_consensus.cc:697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:52.482687 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.489540 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
W20260709 13:16:52.499912 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 99411us
I20260709 13:16:52.354677 30373 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 21, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:52.503821 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.489419  5737 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.512913 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.513410  5737 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.464062  5407 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:52.515026  5587 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.516072  5737 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.490046  5682 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.521974 30631 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 19: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:52.505215 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:16:52.414744  5711 consensus_queue.cc:260] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.490293  5785 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.507490 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:52.439905  5778 consensus_queue.cc:260] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.528600 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.532502 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2243) took 1036 ms. Trace:
I20260709 13:16:52.533090 30631 rpcz_store.cc:276] 0709 13:16:51.507554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.527133 (+ 19579us) service_pool.cc:224] Handling call
0709 13:16:52.532274 (+1005141us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54007aa654. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:52.532424 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222476672}
I20260709 13:16:52.534158  4979 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:52.535703  5590 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:52.536309  5590 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:52.540778  4979 consensus_queue.cc:1059] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:52.542872 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:52.544066  5699 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:52.545081  5781 log.cc:927] Time spent T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.378s	user 0.005s	sys 0.000s
I20260709 13:16:52.544770  5699 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:52.544667  5770 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.447s	user 0.001s	sys 0.001s
I20260709 13:16:52.551725 30698 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:52.557632 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.558506 30556 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:52.559782 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.525687  5682 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:52.566989 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.568085 30700 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
W20260709 13:16:52.568279  5792 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 13:16:52.567039 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:52.563679 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.528108  5801 raft_consensus.cc:3055] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:52.525272  5543 consensus_queue.cc:1048] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:16:52.562907 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:52.569761 30629 raft_consensus.cc:2376] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 29. Re-sending same reply.
I20260709 13:16:52.582259 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:52.587395 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.588660 30556 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:52.553695  5697 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 13:16:52.598541 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.599386 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.523288  5796 consensus_queue.cc:1048] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 13:16:52.599267 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:52.619241 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.619539 30626 raft_consensus.cc:2376] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26. Re-sending same reply.
W20260709 13:16:52.596418 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72331.8us
I20260709 13:16:52.632055 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:52.597668  5723 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:52.656399  5723 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:16:52.656847 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 59374.3us
I20260709 13:16:52.658638  5618 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:52.658998  5694 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.659405 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:16:52.661267 21526 tablet_server.cc:179] TabletServer@127.21.5.129:41021 shutting down...
I20260709 13:16:52.662415  5430 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:52.662302 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.663017  5430 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.663456  5430 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:52.664003  5384 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:52.575593  5801 raft_consensus.cc:740] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:52.600782 30700 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 17.
W20260709 13:16:52.644148 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.672595 30523 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:52.601591 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.637660 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.584529  5407 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:52.602218 30556 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:52.646517 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:52.630779  5697 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.637185  5699 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:52.649529  5797 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:16:52.650436 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.650931 30630 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:16:52.651068  5787 consensus_queue.cc:1048] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:16:52.600689  5783 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:16:52.651676  5794 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:52.651937 31383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.654767  5772 consensus_queue.cc:1059] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.102s
I20260709 13:16:52.655388 31388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.597128  5571 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:52.657177  5723 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:52.634075 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.646474 30671 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:52.680413 30629 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 21: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:52.680734 30598 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.673830 31378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "015ca80eba674552b34856a4f20271f1" is_pre_election: true
I20260709 13:16:52.687085 30558 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:52.659050  5618 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:52.753473 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:52.776918 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.774961 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:52.754441 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.784751 31378 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
W20260709 13:16:52.784344 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 137994us
I20260709 13:16:52.782819 31388 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:16:52.679231  5384 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.746718  5794 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.797709 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.805371 30630 raft_consensus.cc:1275] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:16:52.701125  5787 consensus_queue.cc:1243] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [LEADER]: Peer 015ca80eba674552b34856a4f20271f1 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:16:52.619285  5568 consensus_queue.cc:237] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.823855 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.796543  5407 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.824842 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:52.833611 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57860.6us
I20260709 13:16:52.778489  5692 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.799336  5618 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:52.834296 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:52.809916  5431 raft_consensus.cc:2804] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:52.813517  5794 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:52.874651 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 202060us
I20260709 13:16:52.874728 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:52.881323 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:52.882934 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.839915  5775 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:52.881743 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 57983.4us
I20260709 13:16:52.894161 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:16:52.788695  5571 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:52.863149  5802 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:52.868110  5692 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:52.863778  4979 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:52.868521  5696 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:52.876425  5595 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:52.915875 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.845584  5407 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:52.878187  5431 raft_consensus.cc:697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:52.674360 30373 catalog_manager.cc:5697] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 12 to 15, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 15 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:16:52.900622  5592 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.934813 30631 raft_consensus.cc:2468] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:52.918923  5802 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:52.949326 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 441884us
I20260709 13:16:52.920504  5796 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:16:52.962646 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:52.956836 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:52.924682  5595 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.973603 30558 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
W20260709 13:16:52.946218  5695 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.580s	user 0.012s	sys 0.002s
I20260709 13:16:52.978461 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:52.908543  5775 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:52.979696 30559 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 16.
I20260709 13:16:52.991266  5805 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:52.994596 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:52.947520  5798 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:52.959144  5699 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:53.003110 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.963919  5796 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:52.955904  5802 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:53.010375 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:53.015393  5384 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:52.988667  5694 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:52.986531 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.018925  5805 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:52.980810  5595 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:53.002557  5772 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:52.929612  4979 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:53.072515 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 197905us
I20260709 13:16:53.024714  5407 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.025859 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:52.963204  5592 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:53.034478 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.035499 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.035737 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:53.036310 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.039129 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.069679 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.040194 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:53.050503 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 238924us
I20260709 13:16:53.104162 30626 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:53.104604 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:53.051949  5796 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:53.015676  5803 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:16:53.102494 30632 raft_consensus.cc:2468] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 27.
I20260709 13:16:53.051729  5696 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:53.107517 30596 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.103521 30625 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:53.105309 30631 raft_consensus.cc:2376] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20. Re-sending same reply.
I20260709 13:16:53.067317  5384 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:53.122999  5384 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:53.123221 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.123993 30624 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 29.
I20260709 13:16:52.998059  5682 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.135471 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:52.990208  5695 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.125386 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.121141 30701 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
I20260709 13:16:53.137075 30633 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:16:53.147084 30698 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:53.147152 30626 raft_consensus.cc:2410] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:16:53.152477 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.153334  5407 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:53.154450 30707 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer acf852f85f47483d99cbe4020baee627 for earlier term 23. Current term is 24. Ops: []
I20260709 13:16:53.109668  5737 raft_consensus.cc:3055] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:53.155901  5818 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:53.179615  5571 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:53.180154  5818 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.141072  5692 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:53.184455 30559 raft_consensus.cc:1275] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:53.185550  5695 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.181421  5697 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.213527 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:53.171700 30624 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 13:16:53.215019 30625 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:16:53.186681 30699 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:16:53.187500 30630 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 13:16:53.218327 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 337132us
I20260709 13:16:53.219185 30628 raft_consensus.cc:1275] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
W20260709 13:16:53.219715 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 112357us
I20260709 13:16:53.220350 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.222084 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:53.188897 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 257565us
I20260709 13:16:53.204550 30629 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: []
W20260709 13:16:53.225790 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 215546us
I20260709 13:16:53.222824 30631 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:53.219583 30628 pending_rounds.cc:85] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Aborting all ops after (but not including) 10
I20260709 13:16:53.228875 30628 pending_rounds.cc:107] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:16:53.229127 30628 raft_consensus.cc:2889] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:16:53.207653  5430 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.003s
I20260709 13:16:53.136487  5384 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:53.216949  5549 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.737s	user 0.013s	sys 0.003s
I20260709 13:16:53.219753 30523 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:53.239913 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.198596  5778 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:53.191520  5042 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:53.179169  5737 raft_consensus.cc:740] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:53.146111  5699 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.157912  5805 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.201258  5682 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:53.245088 30523 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:53.241053 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:53.197130  5711 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:53.231520  5726 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:16:53.261741  5396 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:53.262423  5396 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.264500  5805 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:53.265002  5805 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.265617  5805 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:53.267246  5737 consensus_queue.cc:260] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:53.269268  5808 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:16:53.271481  5737 raft_consensus.cc:3060] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:53.272394 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.273109 30705 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:53.282997  5699 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.284216  5592 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.195538  5801 consensus_queue.cc:260] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.285944  5801 raft_consensus.cc:3060] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:53.285640  5590 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.287128  5747 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.287226  5699 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:53.287799  5747 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:53.290088  5747 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:53.292182 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.292985 30624 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 26.
I20260709 13:16:53.191448 30373 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 12 to 16, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.297012 30373 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 22, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 22 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:16:53.254354 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 368866us
I20260709 13:16:53.200412  5818 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:53.299008 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:53.300606 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 55567.2us
I20260709 13:16:53.308179 30625 raft_consensus.cc:2468] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 30.
I20260709 13:16:53.301880 30633 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:53.298569 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.317332 30630 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:53.317145 30704 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:53.264926  5396 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.256327  5675 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:53.143977  5431 consensus_queue.cc:237] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.248004  5549 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:53.354228 30523 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Timed out: RequestConsensusVote RPC to 127.21.5.131:45277 timed out after 1.759s (SENT)
I20260709 13:16:53.245862  5695 consensus_queue.cc:1059] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 13:16:53.239425  5430 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:53.346601 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 512301us
I20260709 13:16:53.375257 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.366731 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:53.355531  5799 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.413s	user 0.002s	sys 0.000s
I20260709 13:16:53.377319 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.375182 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:53.348194  5723 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.516s	user 0.005s	sys 0.009s
I20260709 13:16:53.330468  5699 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:53.344745  5590 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:53.382939 30630 raft_consensus.cc:2393] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 24: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:53.383242 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.382125 30702 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
W20260709 13:16:53.383496 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 119499us
I20260709 13:16:53.385138 30552 raft_consensus.cc:1275] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:53.367444  5675 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.320328  5618 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:53.387183 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 128409us
I20260709 13:16:53.385144  5317 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:16:53.402395 30596 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.807s (SENT)
I20260709 13:16:53.406260 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.389241  5592 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:53.397984 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:53.392601  5782 consensus_meta.cc:287] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.039s	user 0.017s	sys 0.000s
I20260709 13:16:53.406818 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:53.406468 30705 raft_consensus.cc:2468] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:16:53.400043 30522 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:53.395180  5794 consensus_meta.cc:287] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.515s	user 0.012s	sys 0.001s
I20260709 13:16:53.413203 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.415400 30704 raft_consensus.cc:1240] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 24. Current term is 25. Ops: [23.13-24.14]
I20260709 13:16:53.412498 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.410708 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:53.420035 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:53.402549 31294 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:53.401269  5675 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:53.406713  5317 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:53.413573  5608 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.923s	user 0.012s	sys 0.004s
I20260709 13:16:53.392197  5723 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.459208 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.407140  5682 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:16:53.453688 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 99559.5us
I20260709 13:16:53.452967 30597 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:53.413895  5787 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:16:53.475117 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 72757.1us
W20260709 13:16:53.476367 30596 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.834s (SENT)
I20260709 13:16:53.476999 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.446605  5695 raft_consensus.cc:3055] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:53.409672  5829 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:53.483114 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.474169  5809 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:16:53.400413  5699 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.446842  5775 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:53.480381 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.485967 31364 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2171) took 1081 ms. Trace:
I20260709 13:16:53.455065  5587 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:53.410447  5806 consensus_queue.cc:1048] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.161s
I20260709 13:16:53.459071  5317 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:53.458498  5407 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:16:53.489614 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.496488 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.477861  5787 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:53.493753 31364 rpcz_store.cc:276] 0709 13:16:52.404591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.404781 (+   190us) service_pool.cc:224] Handling call
0709 13:16:52.405367 (+   586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.405844 (+   477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.405886 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.406659 (+   773us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.498274 (+ 91615us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:16:52.498330 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.498413 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:53.485719 (+987306us) raft_consensus.cc:1685] finished
0709 13:16:53.485745 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.485889 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34067,"raft.queue_time_us":58027,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":6169,"thread_start_us":86385,"threads_started":1,"wal-append.queue_time_us":155907}
I20260709 13:16:53.458841  5396 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:53.458304  4979 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:53.484890  5807 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.441s	user 0.000s	sys 0.001s
I20260709 13:16:53.460338  5747 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:53.423316  5794 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.458276  5726 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:53.495733 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.492352 30558 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:53.475652  5686 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.470959  5682 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.419322  5782 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.486421  5695 raft_consensus.cc:740] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:53.476095  5608 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.491123  5829 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.492108  5592 raft_consensus.cc:2804] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:16:53.531165 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 104655us
I20260709 13:16:53.488910  5809 raft_consensus.cc:493] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:53.506309  5587 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.499182  5568 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:53.536155 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.536556 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.535952 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.544106 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.513453  5787 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:53.546903  5800 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:53.572746 30705 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:53.547358  5821 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:53.574790  5592 raft_consensus.cc:697] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:53.575141  5821 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:53.576035  5592 consensus_queue.cc:237] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:53.576249  5821 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:53.547912  5824 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:53.579190  5824 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:53.580309  5608 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:53.558252  5829 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:53.548933 31389 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2176) took 1108 ms. Trace:
I20260709 13:16:53.583107  5430 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.583122 31389 rpcz_store.cc:276] 0709 13:16:52.440566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.440740 (+   174us) service_pool.cc:224] Handling call
0709 13:16:52.462033 (+ 21293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.487570 (+ 25537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.487600 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.499721 (+ 12121us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.826467 (+326746us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:52.826530 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.826645 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:53.548649 (+722004us) raft_consensus.cc:1685] finished
0709 13:16:53.548675 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.548863 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2590,"raft.queue_time_us":52074,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":149,"thread_start_us":264894,"threads_started":1,"wal-append.queue_time_us":403583}
I20260709 13:16:53.591564 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.592921  5782 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:53.593511 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.550338 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.594702 30627 raft_consensus.cc:2393] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 20: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:53.595698 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.542769 30373 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 19, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 19 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:53.599406  5695 consensus_queue.cc:260] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.600771  5695 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:53.604135 30557 raft_consensus.cc:2468] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:53.493880  5802 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:53.606474 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:53.556974 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80637.6us
I20260709 13:16:53.563999 30703 raft_consensus.cc:1240] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:16:53.607511 30698 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 26: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:53.549827  5812 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.360s	user 0.000s	sys 0.003s
I20260709 13:16:53.526418  5726 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.591986  5549 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.518996  5396 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.550441 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.637704 30704 raft_consensus.cc:1275] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 13:16:53.618830 30596 leader_election.cc:336] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.838s (SENT)
I20260709 13:16:53.625478 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.597101 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.542590  5682 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:53.634733 30633 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:53.635001 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.573132  5587 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:53.557686  5213 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:53.565265  5809 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:53.648008 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:53.641222 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53938 (request call id 2019) took 1183 ms. Trace:
I20260709 13:16:53.648851 30553 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:53.643070 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:53.526738  5747 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.606528  5824 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:53.633359  5726 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:53.658219 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.649295 30703 rpcz_store.cc:276] 0709 13:16:52.457776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.463851 (+  6075us) service_pool.cc:224] Handling call
0709 13:16:52.493171 (+ 29320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.623625 (+1130454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.640926 (+ 17301us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003d22d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:53.641152 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2461946240}
I20260709 13:16:53.544952  5686 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.660135 30555 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:53.651131 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:53.689347 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.690534 30705 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:53.691803 31396 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2181) took 1193 ms. Trace:
W20260709 13:16:53.692109 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73339.5us
W20260709 13:16:53.696375 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 60654.9us
I20260709 13:16:53.633862  5772 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 13:16:53.701368 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.698040 30523 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:53.706689 30628 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 17.
I20260709 13:16:53.715636  5802 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.717101 30373 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 21, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.719836  5794 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:53.719425 30373 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:53.587805  5800 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:53.692348 31396 rpcz_store.cc:276] 0709 13:16:52.498771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.527375 (+ 28604us) service_pool.cc:224] Handling call
0709 13:16:52.555263 (+ 27888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.584170 (+ 28907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.584225 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.675695 (+ 91470us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.924660 (+248965us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:52.924716 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.924809 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:53.548975 (+624166us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:53.691364 (+142389us) raft_consensus.cc:1685] finished
0709 13:16:53.691399 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:53.691709 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":9212,"spinlock_wait_cycles":59677952,"thread_start_us":158625,"threads_started":1,"wal-append.queue_time_us":365715}
I20260709 13:16:53.691993  5549 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:53.744922 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 46968.9us
I20260709 13:16:53.745628 30702 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53938 (request call id 2025) took 1215 ms. Trace:
I20260709 13:16:53.746062 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:53.746766  5831 log.cc:927] Time spent T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:16:53.749274 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:53.721458 30373 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 19, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.657095  5213 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.750576 30626 raft_consensus.cc:1275] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 13:16:53.754554 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:53.771046 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.746084 30702 rpcz_store.cc:276] 0709 13:16:52.530065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.530245 (+   180us) service_pool.cc:224] Handling call
0709 13:16:53.745410 (+1215165us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b540037d2d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:53.745561 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888961920}
I20260709 13:16:53.773562 30523 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:53.785156 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:53.802255 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:53.808164  5723 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:53.809818  5595 consensus_meta.cc:287] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.701s	user 0.008s	sys 0.006s
I20260709 13:16:53.787297  5042 consensus_queue.cc:1048] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:16:53.810357  5595 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:53.801782 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:53.812528 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:53.788733  5772 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:16:53.813632  5772 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:53.814003  5772 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:16:53.814703  5694 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.005s
I20260709 13:16:53.815300  5694 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:16:53.824859 30596 leader_election.cc:336] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:53.826351  5809 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.760998  5805 raft_consensus.cc:515] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.831558  5723 consensus_queue.cc:1059] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:53.832486  5590 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:53.833519  5042 raft_consensus.cc:3055] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:16:53.834215  5042 raft_consensus.cc:740] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:53.760502  5686 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:53.790830  5384 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
I20260709 13:16:53.835405  5384 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:53.835359  5042 consensus_queue.cc:260] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.838119  5384 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:53.812717  5595 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:53.782393 31292 connection.cc:256] RPC call timeout handler was delayed by -1.1028410270000677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:16:53.754151  5824 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:53.841575 30551 raft_consensus.cc:1240] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 24. Current term is 25. Ops: []
I20260709 13:16:53.851900 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:53.842106 30596 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:53.863668 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:53.861322 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0335410269999556s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260709 13:16:53.864727 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0002410269999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 13:16:53.865320 31292 connection.cc:256] RPC call timeout handler was delayed by -1.0002410269999018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 13:16:53.866181 30706 raft_consensus.cc:2468] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 20.
I20260709 13:16:53.879993 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.881228 30705 raft_consensus.cc:2393] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:53.764477  5737 consensus_queue.cc:1059] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.095s
I20260709 13:16:53.885074 30702 raft_consensus.cc:1240] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: []
I20260709 13:16:53.885975 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.777523  5820 consensus_queue.cc:1048] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:16:53.903986 30704 raft_consensus.cc:2368] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:16:53.792506  5822 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 13:16:53.912469 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.838817  5805 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:53.838241  5809 leader_election.cc:290] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:53.844259  5776 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:53.862371  5543 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:53.757412  5829 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:53.914160 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 185248us
I20260709 13:16:53.920598 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.921584 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:53.922745 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80676.2us
W20260709 13:16:53.901479  5798 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.889s	user 0.012s	sys 0.003s
I20260709 13:16:53.886448  5747 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:53.932242 30702 raft_consensus.cc:1240] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 16. Current term is 17. Ops: []
W20260709 13:16:53.934475 30596 leader_election.cc:336] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:53.940514 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:53.912165  5821 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:53.940564 30596 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:53.936507 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 50348.3us
I20260709 13:16:53.934396 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.943451 30705 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:53.950589 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:53.951576 30706 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:53.955314 31292 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.002477 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.003998 30698 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:53.946183  5805 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:53.935427  5822 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:53.934319  5595 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:53.930627  5825 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:54.006729 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:54.007573 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 73169.9us
I20260709 13:16:53.944918  4979 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:53.948987  5798 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:53.756489 30373 catalog_manager.cc:5697] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 13 to 16, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:53.956470  5782 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:54.031960 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:53.957307  5839 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:54.052626 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 97382.8us
I20260709 13:16:54.009416  5806 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.002353  5809 consensus_queue.cc:1048] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:16:54.083930 31292 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:53.949033  5543 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:53.960460  5747 consensus_queue.cc:1048] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 13:16:54.094408 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:54.096043 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:53.960808  5775 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:16:53.953030  5821 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:54.093261 30596 leader_election.cc:336] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:54.127602 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.136287 30596 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.137310 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.138793  5856 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.139420 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:54.141153  5842 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:16:54.141547 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:54.139499  5856 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:54.128074  5841 log.cc:927] Time spent T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:16:54.116374  5801 consensus_meta.cc:287] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.830s	user 0.020s	sys 0.009s
I20260709 13:16:54.091895  4979 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.148517  5801 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:54.129935 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.091668  5806 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.151213 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:54.151669  5213 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.151921  5806 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.152652 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.153025  5822 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.153841 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.154848 30706 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:16:54.159057  5845 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:16:54.092433  5595 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.159680  5856 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.160154  5595 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.153894 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:54.164768 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71544.4us
I20260709 13:16:54.159147 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.168040 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:54.169696 30596 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.153820  5772 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:54.173799  5317 consensus_meta.cc:287] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.001s
W20260709 13:16:54.175329  5796 consensus_meta.cc:287] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.009s	user 0.013s	sys 0.000s
W20260709 13:16:54.166518  5682 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.000s
I20260709 13:16:54.167410  5839 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:54.195542  5802 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:54.196215  5802 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:54.197166  5726 consensus_meta.cc:287] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.004s
W20260709 13:16:54.105891 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.198392 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.087090  5798 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:16:54.193033 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 109318us
I20260709 13:16:54.230443  5802 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.209182  5726 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.233556 30625 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 25: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:54.214634  5785 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:54.247924 30373 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 19, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.252429  5821 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.254210 30597 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.255054  5686 consensus_queue.cc:1048] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:54.255648  5407 raft_consensus.cc:2804] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:54.256275  5407 raft_consensus.cc:697] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:54.257186  5407 consensus_queue.cc:237] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:54.257856  5819 log.cc:927] Time spent T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 1.069s	user 0.005s	sys 0.002s
I20260709 13:16:54.223063  5813 raft_consensus.cc:3055] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:54.308444 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.308940  5813 raft_consensus.cc:740] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:54.310235  5813 consensus_queue.cc:260] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.312206  5813 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:54.313963 31292 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.316581 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.316965  5859 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.317575  5859 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:54.317896  5587 consensus_meta.cc:287] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.657s	user 0.009s	sys 0.004s
I20260709 13:16:54.254509  5772 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.318821  5821 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:16:54.319690 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.320600  5802 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:54.321877 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.322280 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:54.323127  5802 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:54.323504  5802 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.323549 31361 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50024 (request call id 2169) took 1936 ms. Trace:
I20260709 13:16:54.323951  5802 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:54.324733  5675 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.843s	user 0.015s	sys 0.007s
I20260709 13:16:54.325372  5675 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.199822  5682 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.328722 30632 raft_consensus.cc:1240] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:16:54.329313  5682 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.315331 30373 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 12 to 20, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:16:54.332473 30373 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 18 to 21, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:16:54.336786  5859 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.338346 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:54.339397  5798 consensus_queue.cc:1059] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:54.340493 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.346850 30706 raft_consensus.cc:1275] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:16:54.351181 30704 raft_consensus.cc:2393] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 18: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:54.191888  5317 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:54.352984  5804 log.cc:927] Time spent T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.048s	user 0.005s	sys 0.000s
I20260709 13:16:54.354655 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2362) took 1904 ms. Trace:
I20260709 13:16:54.355121 30523 leader_election.cc:400] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:16:54.363267 30523 leader_election.cc:403] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:16:54.363760 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.364559 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:54.324005 31361 rpcz_store.cc:276] 0709 13:16:52.386976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.387118 (+   142us) service_pool.cc:224] Handling call
0709 13:16:52.390629 (+  3511us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.477418 (+ 86789us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:52.477457 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.504757 (+ 27300us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.983176 (+478419us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:52.983285 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.983607 (+   322us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.257123 (+1273516us) spinlock_profiling.cc:243] Waited 36 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:54.323198 (+ 66075us) raft_consensus.cc:1685] finished
0709 13:16:54.323225 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.323480 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7679,"raft.queue_time_us":48133,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":16095,"spinlock_wait_cycles":82706304,"thread_start_us":499962,"threads_started":2,"wal-append.queue_time_us":638474}
I20260709 13:16:54.356273  5775 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.340351  5798 raft_consensus.cc:2804] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:16:54.367192  5798 raft_consensus.cc:697] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.367304 30558 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2369) took 1147 ms. Trace:
I20260709 13:16:54.367691 30558 rpcz_store.cc:276] 0709 13:16:53.219862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.220034 (+   172us) service_pool.cc:224] Handling call
0709 13:16:54.367066 (+1147032us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54000208d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.367205 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623572352}
I20260709 13:16:54.368251  5798 consensus_queue.cc:237] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.369189 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.370522 30552 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 24: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:54.366014  5818 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:16:54.366911  5775 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.370379 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.357069 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.338181  5820 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.381904 30558 raft_consensus.cc:2410] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:16:54.199452  5796 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.380748  5818 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:54.390935 30700 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Rejecting Update request from peer 5a03a8214bba4eb1b0ff00895c565564 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 13:16:54.381338  5787 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.793s	user 0.009s	sys 0.005s
I20260709 13:16:54.398299 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1452d68baa440e39ae0dad239d033c8" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.351464  5821 raft_consensus.cc:3055] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:16:54.355077 30554 rpcz_store.cc:276] 0709 13:16:52.449894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.462395 (+ 12501us) service_pool.cc:224] Handling call
0709 13:16:52.480080 (+ 17685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:52.480566 (+   486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:52.480590 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:52.516907 (+ 36317us) log.cc:844] Serialized 26 byte log entry
0709 13:16:52.633329 (+116422us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:16:52.633390 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:52.633484 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.184444 (+1550960us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:54.354282 (+169838us) raft_consensus.cc:1685] finished
0709 13:16:54.354309 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.354557 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2409,"raft.queue_time_us":486,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8848,"spinlock_wait_cycles":157332864,"thread_start_us":65441,"threads_started":1,"wal-append.queue_time_us":592072}
W20260709 13:16:54.401376 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63118.1us
I20260709 13:16:54.407348 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.386638  5384 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:54.385156  5682 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:54.411352 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:54.412901 30559 raft_consensus.cc:2393] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 27: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:54.394359  5772 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:54.318574  5587 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.419499 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:54.434958  5726 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.437007  5775 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.437825  5775 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:54.438503 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.402973  5821 raft_consensus.cc:740] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.440516 30556 raft_consensus.cc:2368] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 25. Current term is 26.
I20260709 13:16:54.416051 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.438319  5775 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.453401 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.393924  5431 consensus_queue.cc:1059] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:16:54.414726  5384 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.390419  5820 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.458261 30559 raft_consensus.cc:1240] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
W20260709 13:16:54.446934  5592 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.866s	user 0.028s	sys 0.028s
I20260709 13:16:54.468784 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:54.406594  5787 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.469861 30704 raft_consensus.cc:3060] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:54.444829  5793 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.241s	user 0.024s	sys 0.016s
I20260709 13:16:54.437018  5587 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.438393  5801 consensus_queue.cc:1059] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 13:16:54.457167  5859 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:54.454375  5695 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:54.471920  5592 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.029s	sys 0.029s
I20260709 13:16:54.420150  5682 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.469054  5384 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:16:54.475406  5793 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.028s	sys 0.016s
I20260709 13:16:54.475756  5815 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.457541 30373 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 18 to 20, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 20 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:54.523422  5860 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:54.441830  5818 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.471724  5850 consensus_queue.cc:1048] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:16:54.506579  5821 consensus_queue.cc:260] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.543810 30552 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1393) took 1064 ms. Trace:
I20260709 13:16:54.514801  5431 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.547739 30552 rpcz_store.cc:276] 0709 13:16:53.479881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.480092 (+   211us) service_pool.cc:224] Handling call
0709 13:16:54.543567 (+1063475us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b54000a0354. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.543739 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2016359808}
I20260709 13:16:54.508069  5801 consensus_queue.cc:1059] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:16:54.504915  5695 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.566672  5317 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.555361  5818 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:54.528774  5815 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:54.575112 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.024s	sys 0.024s
W20260709 13:16:54.575596 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.024s	sys 0.024s
I20260709 13:16:54.577175  5815 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:54.578167  5843 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.462s	user 0.000s	sys 0.001s
I20260709 13:16:54.521521  5775 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.541329  5782 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:54.524207 30373 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 13 to 18, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:16:54.586966  5796 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:54.588680  5864 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:16:54.594269 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.596760 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.597083 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.591286 30550 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39184 (request call id 1394) took 1111 ms. Trace:
I20260709 13:16:54.540041  5042 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:54.602397 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.536834  5787 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.590152  5396 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.608202 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.620016 30523 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.604293  5821 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:54.621932 30557 raft_consensus.cc:2410] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:16:54.620801 30599 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.624744 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.562829  5431 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.539824  5820 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.623438  5697 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:54.639719 30704 raft_consensus.cc:2468] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
W20260709 13:16:54.639670 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 169329us
I20260709 13:16:54.621556  5595 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.626190  5813 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:54.627679  5787 raft_consensus.cc:3060] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:54.661049 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:54.662437 30629 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:54.657977 30599 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:54.666023 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:54.667207 30522 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.633610  5855 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:54.667765 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 214424us
W20260709 13:16:54.669134 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 206118us
I20260709 13:16:54.670956 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
W20260709 13:16:54.657732  5826 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 1.249s	user 0.004s	sys 0.000s
I20260709 13:16:54.656396  5697 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.678071 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:54.664115  5863 log.cc:927] Time spent T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:16:54.658794  5814 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:54.695042 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.678658  5595 raft_consensus.cc:2804] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:54.682160  5697 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:54.701053 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:54.697774 30597 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.701840 30626 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:54.705178  5800 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.688066  5850 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:16:54.707438  5868 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:16:54.707767  4979 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.714537  5860 raft_consensus.cc:3055] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:16:54.715243  5860 raft_consensus.cc:740] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.716261  5860 consensus_queue.cc:260] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.697358  5798 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:54.719188  5695 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.720036 30698 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48122 (request call id 1260) took 1073 ms. Trace:
I20260709 13:16:54.720490 30698 rpcz_store.cc:276] 0709 13:16:53.646607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.649425 (+  2818us) service_pool.cc:224] Handling call
0709 13:16:53.649784 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:54.015781 (+365997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.719727 (+703946us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54005a23d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:54.719962 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805710080}
I20260709 13:16:54.721329  5431 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:54.722510  5832 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.082s	user 0.007s	sys 0.004s
I20260709 13:16:54.724166 30631 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1685) took 1435 ms. Trace:
I20260709 13:16:54.724561 30631 rpcz_store.cc:276] 0709 13:16:53.288584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.288705 (+   121us) service_pool.cc:224] Handling call
0709 13:16:53.289077 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:53.289559 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:16:53.289600 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:53.290205 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 13:16:53.401110 (+110905us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:16:53.401204 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:53.401311 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:54.699172 (+1297861us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:54.723799 (+ 24627us) raft_consensus.cc:1685] finished
0709 13:16:54.723822 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.724067 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1572,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"spinlock_wait_cycles":26407808,"thread_start_us":110400,"threads_started":1,"wal-append.queue_time_us":251998}
I20260709 13:16:54.727442  5850 raft_consensus.cc:2804] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:54.726985  5802 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.728992  5543 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.729681  5802 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.730540 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2377) took 1074 ms. Trace:
I20260709 13:16:54.731359  5772 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 for earlier term 17. Current term is 18.
I20260709 13:16:54.731568 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.733076 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.734606 30626 raft_consensus.cc:2468] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 27.
I20260709 13:16:54.700026  5595 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:54.736019  5595 raft_consensus.cc:3060] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:54.737433  5699 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.741750 30558 raft_consensus.cc:2468] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 18.
I20260709 13:16:54.744030 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.744782 30559 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:54.727006 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 56852.6us
I20260709 13:16:54.748540 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.731011 30556 rpcz_store.cc:276] 0709 13:16:53.656167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.185994 (+529827us) service_pool.cc:224] Handling call
0709 13:16:54.730303 (+544309us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b540067b6d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.730467 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540848512}
I20260709 13:16:54.762957  5798 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:54.763604  5828 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 1.252s	user 0.000s	sys 0.004s
I20260709 13:16:54.766808 30551 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
W20260709 13:16:54.769189 30627 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36384 (request call id 1405) took 2226 ms (client timeout 1518 ms). Trace:
W20260709 13:16:54.769799 30627 rpcz_store.cc:269] 0709 13:16:52.542345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:52.542504 (+   159us) service_pool.cc:224] Handling call
0709 13:16:54.768770 (+2226266us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54000712d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.769025 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416598912}
I20260709 13:16:54.770666  5396 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:54.773111  5675 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:54.782809  5793 raft_consensus.cc:3055] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:54.783128 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.784369  5872 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:54.785009  5872 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.787355  5872 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.788182 30628 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2265) took 1305 ms. Trace:
I20260709 13:16:54.788810 30628 rpcz_store.cc:276] 0709 13:16:53.483006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.483135 (+   129us) service_pool.cc:224] Handling call
0709 13:16:54.787920 (+1304785us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54004a9cd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.788090 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411432704}
I20260709 13:16:54.791564 30557 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2812) took 1144 ms. Trace:
I20260709 13:16:54.792253 30557 rpcz_store.cc:276] 0709 13:16:53.646759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.363378 (+716619us) service_pool.cc:224] Handling call
0709 13:16:54.791198 (+427820us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400112954. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.791466 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591147264}
I20260709 13:16:54.794649  5747 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:54.783450  5793 raft_consensus.cc:740] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.801510 30559 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:54.807310 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.809576 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.812590 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.814957 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.816613  5694 raft_consensus.cc:3055] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:16:54.804724 31294 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:54.817102  5694 raft_consensus.cc:740] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:54.817142 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.818849  5396 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:54.819412  5396 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:54.820250  5787 raft_consensus.cc:2749] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:16:54.820390  5396 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:54.821287 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 72851.1us
I20260709 13:16:54.822809  5798 consensus_queue.cc:1048] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:54.823300  5595 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.816056  5793 consensus_queue.cc:260] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.826068  5793 raft_consensus.cc:3060] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:54.827251  5595 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:54.599325 30550 rpcz_store.cc:276] 0709 13:16:53.480215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.480332 (+   117us) service_pool.cc:224] Handling call
0709 13:16:53.486834 (+  6502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:53.487159 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:53.508628 (+ 21469us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.590694 (+1082066us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18000d1440. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1806a854bc 00007f1803dd348b 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d23a
0709 13:16:54.590923 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:54.590983 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:54.591034 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:54.591108 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:54.591225 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20302,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":7904,"spinlock_wait_cycles":61516928,"thread_start_us":15943,"threads_started":1,"wal-append.queue_time_us":21151}
I20260709 13:16:54.828565 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:54.830928  5859 raft_consensus.cc:2804] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:16:54.831362  5859 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.831683  5859 raft_consensus.cc:3060] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:16:54.835352 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.836604  5800 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:54.838158 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.838446 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:54.838970 30628 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:54.840080 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.840633 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.842873  5798 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:54.843334  5798 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.843670  5798 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:54.844920 30625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2264) took 1362 ms (client timeout 1730 ms). Trace:
W20260709 13:16:54.845533 30625 rpcz_store.cc:269] 0709 13:16:53.482640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.482804 (+   164us) service_pool.cc:224] Handling call
0709 13:16:54.844679 (+1361875us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54004aa1d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.844849 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714448768}
W20260709 13:16:54.847965 30553 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49250 (request call id 2811) took 1275 ms (client timeout 1531 ms). Trace:
W20260709 13:16:54.848493 30553 rpcz_store.cc:269] 0709 13:16:53.572967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:53.596821 (+ 23854us) service_pool.cc:224] Handling call
0709 13:16:54.847797 (+1250976us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b5400289bd4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.847903 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21801728}
I20260709 13:16:54.849262 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:54.690762  5814 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.865901 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:54.753840  5850 raft_consensus.cc:697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:54.850071 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.851138  5682 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.851627 30551 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:54.851177 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.860407 30523 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.860828 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:54.875902 30704 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
W20260709 13:16:54.863956 30525 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.864187 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:54.754551  5858 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.308s	user 0.000s	sys 0.004s
I20260709 13:16:54.881002 30553 pending_rounds.cc:85] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting all ops after (but not including) 13
I20260709 13:16:54.874956 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:54.881726 30553 pending_rounds.cc:107] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 13:16:54.882062 30553 raft_consensus.cc:2889] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:16:54.865971 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.882548 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:54.840670 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:54.867959  5618 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.869903 30698 raft_consensus.cc:3060] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:54.802451  5860 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:54.880956  5802 raft_consensus.cc:2804] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:16:54.890334  5802 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.890959  5802 raft_consensus.cc:3060] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:16:54.880977  5772 raft_consensus.cc:2804] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:16:54.896010  5772 raft_consensus.cc:697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:54.897382  5772 consensus_queue.cc:237] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:54.878554 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.818131  5694 consensus_queue.cc:260] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.873061  5595 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:16:54.871557  5800 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.919734 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:54.920904 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.923022 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:54.923816 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:54.924938 30702 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 24: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:54.908449 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:54.932523 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:54.933507 30701 raft_consensus.cc:3060] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:54.939949 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.905822  5384 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:54.959725  5873 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:54.960343  5778 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:54.960990  5873 consensus_queue.cc:1059] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:16:54.961141  5778 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:54.966389 30555 raft_consensus.cc:2410] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:16:54.933223  5800 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:54.973443 30555 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49258 (request call id 2375) took 1320 ms (client timeout 1576 ms). Trace:
I20260709 13:16:54.888530  4979 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 13:16:54.977921  4979 raft_consensus.cc:3055] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:54.908236  5618 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:54.978973  5618 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.979460  5618 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:54.984025  5798 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:54.987098 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.989451  5778 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:54.993674 30558 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 29. Current term is 30. Ops: []
I20260709 13:16:54.996459 30551 raft_consensus.cc:2393] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:54.997736 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8c076453f774c4eb79096017a386e6c" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 26 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:54.991592 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:54.974023 30555 rpcz_store.cc:269] 0709 13:16:53.652814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.167670 (+514856us) service_pool.cc:224] Handling call
0709 13:16:54.973075 (+805405us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400626454. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:54.973341 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813485440}
I20260709 13:16:54.914286  5694 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:54.958168  5384 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:55.032439 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:55.034454 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:55.032371 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:54.912578  5595 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:55.045921 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:54.978511  4979 raft_consensus.cc:740] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:55.050076 30705 raft_consensus.cc:1275] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:16:54.992332  5806 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:16:55.030786  5696 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.496s	user 0.037s	sys 0.033s
I20260709 13:16:54.997252  5587 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:55.033249  5317 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:54.995925  5800 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:54.998885  5543 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:55.084077 30673 leader_election.cc:400] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 25. Current term is 26.
I20260709 13:16:55.052603  5595 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:16:55.104550 30673 leader_election.cc:403] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
W20260709 13:16:55.081279  5696 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.554s	user 0.037s	sys 0.033s
I20260709 13:16:55.079835  5797 raft_consensus.cc:2764] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:16:55.111284 30673 leader_election.cc:304] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.101645  5317 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:54.966555  5850 consensus_queue.cc:237] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:55.078976  5806 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:55.136675 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 392001us
W20260709 13:16:55.137550 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 401481us
I20260709 13:16:55.144560 30698 raft_consensus.cc:2468] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:16:55.150501 30629 raft_consensus.cc:2468] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:55.112110  5543 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:55.126643  5317 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:55.157673 30671 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:55.082573  5870 consensus_queue.cc:1059] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 13:16:55.136929  5431 raft_consensus.cc:2804] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:16:55.204568  5817 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:55.205683  5817 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:55.207834  5874 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:16:55.209610  5814 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:55.217610 30597 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:55.222972  5879 raft_consensus.cc:2804] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:55.223948  5879 raft_consensus.cc:697] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:55.219133  5849 consensus_queue.cc:1048] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:55.238184  5829 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:55.239815  5817 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:55.156324  5806 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:16:55.251012 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 91569.3us
W20260709 13:16:55.175410 30522 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:55.254231 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 87969.6us
W20260709 13:16:55.255535 30522 leader_election.cc:336] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.257164 30522 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:55.225310  5879 consensus_queue.cc:237] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:55.297600 30522 leader_election.cc:336] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.180593  5796 raft_consensus.cc:2804] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:55.182085  5797 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.300669 30522 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:55.316831 30522 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.304581  5723 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:16:55.317965 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 430846us
I20260709 13:16:55.285284  5431 raft_consensus.cc:697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:55.332041 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 468987us
I20260709 13:16:55.335757 31292 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.339558 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:55.317097  5747 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.514s	user 0.013s	sys 0.000s
W20260709 13:16:55.319453 30522 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.353533 30522 consensus_peers.cc:597] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.353899 30633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.039s	sys 0.085s
W20260709 13:16:55.354424 30633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.039s	sys 0.086s
W20260709 13:16:55.354215 30522 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.347939 30599 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:55.349514 30556 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 26: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:55.355158 30522 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.143383  4979 consensus_queue.cc:260] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:55.321250  5813 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:55.326509  5796 raft_consensus.cc:697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:55.303766  5870 consensus_queue.cc:1243] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 30.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:16:55.380493  5747 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:55.371919 30701 raft_consensus.cc:2468] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 26.
W20260709 13:16:55.352614  5859 consensus_meta.cc:287] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.005s
W20260709 13:16:55.403573 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 49376.3us
W20260709 13:16:55.364462 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.670s (SENT)
I20260709 13:16:55.358883 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.349968  5801 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:55.414772 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 66956.4us
I20260709 13:16:55.389146  5772 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.439810 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 75429.7us
I20260709 13:16:55.397337  5798 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:55.408061  5778 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:16:55.452610 30522 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.487685 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 54267.6us
I20260709 13:16:55.503518 30632 raft_consensus.cc:1275] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 13:16:55.324162  5797 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:55.449957  5543 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:55.451191  5839 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:55.420840  5872 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f for earlier term 25. Current term is 26.
W20260709 13:16:55.496385  5396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.015s	sys 0.025s
I20260709 13:16:55.529122 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:55.435971  4979 raft_consensus.cc:3060] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:55.513631  5860 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:55.515370  5723 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.483009 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.573604 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 90707.9us
I20260709 13:16:55.477214  5772 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:55.577070  5772 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:55.540573  5699 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.547178  5396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.016s	sys 0.026s
W20260709 13:16:55.542145  5882 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 13:16:55.574452 30522 consensus_peers.cc:597] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.593999  5694 consensus_meta.cc:287] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.558s	user 0.005s	sys 0.009s
I20260709 13:16:55.608091  5772 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.608094  5797 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:55.472563  5859 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:55.610189  5618 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.630s	user 0.005s	sys 0.010s
W20260709 13:16:55.618040 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 506874us
W20260709 13:16:55.582082  5802 consensus_meta.cc:287] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.691s	user 0.015s	sys 0.000s
W20260709 13:16:55.632733 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:55.621528  5881 log.cc:927] Time spent T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.186s	user 0.004s	sys 0.000s
I20260709 13:16:55.623879  5838 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:55.521344  5431 consensus_queue.cc:237] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:55.582813  5723 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:55.653277  5697 consensus_meta.cc:287] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.876s	user 0.009s	sys 0.006s
I20260709 13:16:55.614742  5699 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:55.634167  5772 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:55.581561  5796 consensus_queue.cc:237] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:55.690696 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 62030.5us
I20260709 13:16:55.636234 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.641288  5618 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:55.642716  5802 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:55.706211 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:55.722080 30633 raft_consensus.cc:1275] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 13:16:55.737671 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 44415.9us
W20260709 13:16:55.733681 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 97518.4us
I20260709 13:16:55.658811  5838 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:55.742779 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.531s (SENT)
W20260709 13:16:55.739604 30522 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:55.749449 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 406002us
I20260709 13:16:55.685115  5697 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:55.766309 30673 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.765157  5870 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.753577 30670 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.778940 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.756524  5816 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:55.769392  5618 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:55.771327  5396 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:55.794126 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
W20260709 13:16:55.795024 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:55.796133 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.796573 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:55.796789 30596 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.553s (SENT)
I20260709 13:16:55.791913 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:55.808140 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:55.809271 30670 leader_election.cc:336] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.809687 30670 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:55.781641  5899 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:16:55.794742  5317 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.003s
I20260709 13:16:55.778142  5870 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:55.800091  5872 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:55.842434 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:55.844995 30624 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
I20260709 13:16:55.846938 30633 raft_consensus.cc:1275] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:16:55.847079  5618 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:16:55.847252 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55456.5us
W20260709 13:16:55.847764 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:55.847929  5618 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:16:55.848788 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:55.849893  5699 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:55.851329  5894 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:16:55.851423  5802 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:55.852571 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:55.853235  5890 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:16:55.853858 30551 raft_consensus.cc:2393] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 25: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
W20260709 13:16:55.852609  5897 log.cc:927] Time spent T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:16:55.856540  5838 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:55.862792 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:55.813836  5696 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:55.865867 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:55.870451  5898 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:16:55.824120  5384 consensus_queue.cc:1048] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:16:55.840138  5859 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:55.804375  5396 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:55.873296 30550 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
I20260709 13:16:55.881019 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:55.815635  5682 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:55.886698 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:55.890957 30701 raft_consensus.cc:2393] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 17: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:55.871206  5860 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:55.893824 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 204739us
W20260709 13:16:55.895524 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.840366  5317 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:55.901451 30596 leader_election.cc:336] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.134s (SENT)
I20260709 13:16:55.898814 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:55.902264 30596 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:55.913259  5895 log.cc:927] Time spent T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:16:55.917778 30700 raft_consensus.cc:1275] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 13:16:55.903857  5860 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:55.939244  5896 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 13:16:55.951315 30596 leader_election.cc:336] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 2.175s (SENT)
I20260709 13:16:55.892983  5682 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:55.953150 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.708s (SENT)
I20260709 13:16:55.953755 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:55.954986  5682 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:55.966722  5793 consensus_meta.cc:287] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 1.133s	user 0.008s	sys 0.008s
W20260709 13:16:55.967659 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 72173.7us
W20260709 13:16:55.974514 30670 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:55.975162 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.976073  5870 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:55.962544 30707 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:16:55.981683  5806 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.687s	user 0.013s	sys 0.000s
I20260709 13:16:55.985380  5908 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:16:55.958444  5900 consensus_queue.cc:1048] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 13:16:55.990892  5883 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
I20260709 13:16:55.982417  5806 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:55.992534 30554 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2381) took 1610 ms. Trace:
I20260709 13:16:55.993191 30554 rpcz_store.cc:276] 0709 13:16:54.381837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.468515 (+ 86678us) service_pool.cc:224] Handling call
0709 13:16:54.468930 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.469409 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:16:54.469452 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:54.475859 (+  6407us) log.cc:844] Serialized 26 byte log entry
0709 13:16:54.775880 (+300021us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b1800217d80. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dd2e07 00007f180cc2dab9 00007f1803def69b 00007f1803ded93f 00007f1803defc02 00007f1806a7720b 00007f1806a7cd23 00007f1806a7d0d1 00007f1806e98a0f
0709 13:16:55.115063 (+339183us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:55.115110 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.115184 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:55.990151 (+874967us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:55.992135 (+  1984us) raft_consensus.cc:1685] finished
0709 13:16:55.992176 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:55.992434 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495,"raft.queue_time_us":1793,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7499,"spinlock_wait_cycles":11516032,"thread_start_us":338930,"threads_started":1,"wal-append.queue_time_us":1238954}
I20260709 13:16:55.995265  5806 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:55.997215 30523 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:55.998216  5384 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:56.002746  5870 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.003710  5870 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.006258  5870 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:56.007436 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.008409 30707 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:56.008760  5772 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.009879  5817 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:16:56.010198  5821 raft_consensus.cc:3055] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 13:16:56.010663  5821 raft_consensus.cc:740] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:56.012531 30670 leader_election.cc:336] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.013103 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:56.015542 30704 raft_consensus.cc:1240] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:16:56.019176 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.021783 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.023123 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:56.024019 30704 raft_consensus.cc:2393] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:56.024561 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 71468.4us
I20260709 13:16:56.026300 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.027266 30706 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:56.028208 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:56.031855 30596 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Timed out: RequestConsensusVote RPC to 127.21.5.129:41021 timed out after 1.843s (SENT)
I20260709 13:16:56.152292 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.096709 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.056149 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:56.164896 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:56.059909 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:56.165685 30701 raft_consensus.cc:2393] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 26: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:56.025763  5838 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.160290 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:56.165781 30631 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:16:56.173377 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.033847 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.059468 30523 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:56.030432 31292 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:56.098917 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.167857 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.042236 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.030488 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:56.175474 30633 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:56.176359 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 148100us
W20260709 13:16:56.167778 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 135994us
I20260709 13:16:56.165822 30555 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:16:56.177371 30706 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 17.
I20260709 13:16:56.177425 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:56.175038 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 31 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:56.055059 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.202189 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 147225us
W20260709 13:16:56.205657 31294 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.174211 30556 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:56.053494  5908 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:56.245642 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 186342us
I20260709 13:16:56.063813  5695 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:16:56.203372 30670 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 77 similar messages]
W20260709 13:16:56.098498  5595 consensus_meta.cc:287] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.984s	user 0.016s	sys 0.000s
I20260709 13:16:56.148145  5384 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.257750 30522 leader_election.cc:336] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.250355 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:56.288316 30522 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:56.041167  5850 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:56.036679  5699 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:56.252904  5908 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.042730  5697 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:56.254849  5822 raft_consensus.cc:2804] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:16:56.240739 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 63417.9us
I20260709 13:16:56.030823  5694 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:16:56.291368 30633 raft_consensus.cc:2468] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:16:56.260138 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 56823us
I20260709 13:16:56.293634 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.059773  5870 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:56.248077  5793 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:56.247701  5590 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:56.238534  5806 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:16:56.224025  5682 consensus_queue.cc:1048] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:16:56.031335  5821 consensus_queue.cc:260] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:56.260089  5802 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:16:56.332134 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.239625  5838 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.292985  5595 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.353011  5806 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.325042  5699 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:56.377219 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.345796  5860 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:56.378034  5543 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:56.380139  5821 raft_consensus.cc:3060] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 13:16:56.373411 30704 raft_consensus.cc:2393] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 26: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:56.239473 30628 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 25: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:56.376678  5802 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:56.298295  5384 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.356081  5850 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:56.391413  5796 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.013s	sys 0.031s
W20260709 13:16:56.467888 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 139703us
I20260709 13:16:56.355824  5870 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.353377  5697 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:56.415055  5806 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:16:56.468595 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 300997us
I20260709 13:16:56.497816 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:56.425480  5860 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.483337  5396 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:56.493866  5796 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.013s	sys 0.032s
W20260709 13:16:56.507725 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 252219us
I20260709 13:16:56.469731  5317 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.486040  5900 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.513082 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 255364us
W20260709 13:16:56.519261 30599 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.014s
I20260709 13:16:56.499225  5543 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:56.527702 30596 leader_election.cc:336] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.538851 30596 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.540040 30596 leader_election.cc:336] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.531255 30599 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.006s	sys 0.015s
I20260709 13:16:56.531859 30707 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:16:56.544945 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.546022 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:56.533370 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.547820 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:56.535909 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 285682us
I20260709 13:16:56.542944 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.552645 30628 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:56.541383 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.556790 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.557053 30522 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.557585 30522 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:56.559727  5384 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.557250 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:56.561275 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.562049 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.563144 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.563695 30627 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:16:56.564076 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.565547 30633 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:56.566262 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:56.566977 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.566892  5802 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.567566  5902 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:16:56.567034 30550 raft_consensus.cc:3060] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:56.564826 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.550218 30523 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:56.572292 30630 raft_consensus.cc:1240] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24. Ops: [23.12-23.12]
I20260709 13:16:56.576385 30555 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
I20260709 13:16:56.569142 30627 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:56.579380 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.522140  5822 raft_consensus.cc:697] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:56.532502  5396 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:56.580453 30629 raft_consensus.cc:2393] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:56.582435 30522 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.617496 30522 leader_election.cc:336] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.618209 30522 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:56.587316 30558 raft_consensus.cc:1240] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 13:16:56.492303  5850 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.503002  5697 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:56.589030 30596 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.591395 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.621063 30627 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 26
I20260709 13:16:56.592149 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:56.586536  5902 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:56.593531  5590 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:16:56.621500 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.622138 30670 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.595072 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:56.595477 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.623494 30628 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
W20260709 13:16:56.624807 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.626375 30629 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:56.622725  5902 raft_consensus.cc:697] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:56.631793  5870 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.631223  5902 consensus_queue.cc:237] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.599720 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.633100  5908 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.634402  5595 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:56.640519  5697 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:56.642562  5699 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.645391  5850 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:56.651822 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.652207  5838 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:56.626780  5885 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.927s	user 0.004s	sys 0.013s
I20260709 13:16:56.656111 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:56.656998 30702 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 25: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:56.657215  5822 consensus_queue.cc:237] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.11, Last appended by leader: 11, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.541203  5900 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.664054  5900 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:56.586917  5925 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.665156  5925 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.667506  5925 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.667701 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:56.669062 30671 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:56.669471  5805 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:56.669942  5805 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:56.670370  5805 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:56.670846  5901 raft_consensus.cc:2804] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:16:56.671566  5901 raft_consensus.cc:697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:56.672515  5901 consensus_queue.cc:237] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.674975  5723 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:56.610842  5916 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.676136  5916 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.612324 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:16:56.679322 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 67101.1us
I20260709 13:16:56.679618  5849 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:56.680109  5849 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.680452  5849 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:56.616935  5543 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:56.598035  5839 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.684242  5839 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.685648 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:56.688907 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.689800 30558 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:56.577347  5772 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:56.690980  5772 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.598840 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.692334 30551 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 25: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:56.692344 30559 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 16.
W20260709 13:16:56.622342 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.693787 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:56.625775 30670 leader_election.cc:336] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.694116 30626 raft_consensus.cc:2468] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 20.
I20260709 13:16:56.523413  5860 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:56.692631 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:56.694880 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69140.8us
W20260709 13:16:56.696478 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.697045 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:56.703890 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:56.704216 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 81917.5us
I20260709 13:16:56.704836 30706 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:16:56.705101 30596 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.628341 30522 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.709115 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 80817.2us
W20260709 13:16:56.710003 30522 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:56.717234 31291 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.720476 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.721283 30698 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
W20260709 13:16:56.723752 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 226062us
I20260709 13:16:56.724195 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.725692 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:56.722302 30673 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:56.725483 30630 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
W20260709 13:16:56.706266 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.736034 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:56.736884 30558 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
W20260709 13:16:56.722966 30522 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.733455 31292 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:56.743791 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:56.745304 30673 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:56.745738  5931 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.225s	user 0.005s	sys 0.005s
W20260709 13:16:56.739679  5890 log.cc:927] Time spent T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.084s	user 0.008s	sys 0.004s
I20260709 13:16:56.749094  5694 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:56.749577  5694 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.749935  5694 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:56.751176 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.752329 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.753768 31292 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:56.763140 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.764318 30706 raft_consensus.cc:2393] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:56.741858 30522 leader_election.cc:336] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.765007 30522 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:56.729609 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.737627  5912 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.297s	user 0.000s	sys 0.010s
W20260709 13:16:56.734876  5914 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.291s	user 0.004s	sys 0.002s
W20260709 13:16:56.735565  5909 log.cc:927] Time spent T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.290s	user 0.000s	sys 0.011s
W20260709 13:16:56.737226  5915 log.cc:927] Time spent T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: Append to log took a long time: real 0.108s	user 0.003s	sys 0.001s
W20260709 13:16:56.775651 30596 leader_election.cc:336] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.703696  5839 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:56.778767 30522 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.799975 31292 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:56.740806  5923 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.299s	user 0.008s	sys 0.000s
W20260709 13:16:56.741466  5918 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.302s	user 0.004s	sys 0.004s
W20260709 13:16:56.768558 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.764940  5595 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 13:16:56.824040 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55569.1us
W20260709 13:16:56.734557  5910 log.cc:927] Time spent T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.297s	user 0.004s	sys 0.004s
I20260709 13:16:56.730250  5697 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:16:56.780817 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64662us
W20260709 13:16:56.742867  5885 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.087s	user 0.001s	sys 0.002s
I20260709 13:16:56.736452  5838 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:56.733476  5916 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:56.751888  5549 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:56.741358  5870 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:56.789939 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.801527 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:56.802577 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:56.800334 30522 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.853080 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 52812.5us
I20260709 13:16:56.853231 30704 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.804157 30550 raft_consensus.cc:2468] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 22.
I20260709 13:16:56.832532 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.859145 30626 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 19.
I20260709 13:16:56.839031 30627 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:56.772773  5925 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:16:56.778273  5396 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:16:56.843503 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.772189  5543 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:56.867359 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.849406 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "72181014679a4c538f96f2d305a5474f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:56.850975 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:56.840860 30670 leader_election.cc:336] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.867458 30698 raft_consensus.cc:2468] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:56.786273  5924 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.879776 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 55006.1us
I20260709 13:16:56.853116  5797 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:56.880234 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:56.881525 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
W20260709 13:16:56.846799 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:56.886724 30705 raft_consensus.cc:2393] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 19: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:56.881438 30670 leader_election.cc:336] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.880707  5797 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:56.888811 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:56.889496 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:56.890170  5797 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:56.893220 30625 raft_consensus.cc:1217] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 20.14->[21.15-21.15]   Dedup: 21.15->[]
I20260709 13:16:56.893898 30625 raft_consensus.cc:1240] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:16:56.840541  5317 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:56.898561 30556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.007s	sys 0.017s
I20260709 13:16:56.781550  5805 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:56.902980 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:56.905671 30556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.008s	sys 0.017s
I20260709 13:16:56.786994  5859 consensus_queue.cc:1059] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:16:56.866878  5925 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:16:56.896596  5778 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.010s	sys 0.015s
I20260709 13:16:56.877264  5543 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:56.867887  5870 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.928579 30707 raft_consensus.cc:1240] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 22. Current term is 23. Ops: []
I20260709 13:16:56.934811 30699 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53928 (request call id 2587) took 2075 ms. Trace:
W20260709 13:16:56.921033  5778 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.011s	sys 0.016s
W20260709 13:16:56.935190 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 311423us
I20260709 13:16:56.899569  5930 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:56.960229  5930 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.961480 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:56.963009  5930 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:56.935456 30699 rpcz_store.cc:276] 0709 13:16:54.859040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:54.865044 (+  6004us) service_pool.cc:224] Handling call
0709 13:16:54.867443 (+  2399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:54.867822 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:16:54.868140 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 13:16:54.956715 (+ 88575us) spinlock_profiling.cc:243] Waited 6.34 ms on lock 0x7b1800420a60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803dbe2a6 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1
0709 13:16:54.957721 (+  1006us) spinlock_profiling.cc:243] Waited 9.84 ms on lock 0x7b1800420a60. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:16:55.161208 (+203487us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:55.169193 (+  7985us) log.cc:844] Serialized 26 byte log entry
0709 13:16:55.169439 (+   246us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:55.169477 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:55.169555 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:56.381366 (+1211811us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b4400022b4c. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803d16c85 00007f1803d176a2 00007f1806a891db 00007f1806a84aaf 00007f1806a7886e 00007f1806a77bb6 00007f1806a777d8 00007f1806a81221 00007f1806a811d9 00007f1806a81169
0709 13:16:56.934460 (+553094us) raft_consensus.cc:1685] finished
0709 13:16:56.934486 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:56.934721 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1365,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":63,"raft.queue_time_us":4540,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":6844,"spinlock_wait_cycles":82394112,"thread_start_us":291225,"threads_started":1,"wal-append.queue_time_us":775684}
I20260709 13:16:56.913806  5925 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:56.913168  5396 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:56.920316  5543 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:56.917560  5870 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:56.895735  5924 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:57.021335 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:56.939517  5859 consensus_queue.cc:1243] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer 6b393e79ded4461ea5da7b6b6063305f log is divergent from this leader: its last log entry 24.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:16:56.908610  5317 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:56.971572  5793 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:57.003325  5900 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:16:57.020169  5883 log.cc:927] Time spent T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:16:57.086125 30633 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:56.990938  5929 consensus_queue.cc:1048] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.142s
I20260709 13:16:57.066200  5900 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.082345  5396 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.101742 30599 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:57.120875 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.080271  5935 raft_consensus.cc:3055] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:16:57.121551 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.126206 30552 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 21.
I20260709 13:16:57.128418 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.135702 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.138962 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.111981  5822 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:16:57.106804  5793 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:57.113361  5860 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.121065  5900 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:57.148586 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 282489us
I20260709 13:16:57.119518  5927 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:57.125706  5916 raft_consensus.cc:2804] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:57.126060  5935 raft_consensus.cc:740] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:57.130677  5549 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
W20260709 13:16:57.176913 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 375452us
I20260709 13:16:57.169013 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.179881 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:57.183164 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 221828us
I20260709 13:16:57.123248  5396 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.184032 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:57.186067 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:57.173290  5937 log.cc:927] Time spent T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:16:57.200028 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.177163  5696 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.182941  5587 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:57.207926 31292 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:57.214681  5932 log.cc:927] Time spent T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 13:16:57.214092  5935 consensus_queue.cc:260] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.215817  5935 raft_consensus.cc:3060] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:57.216912  5924 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.217602  5924 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:16:57.180735  5927 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.220677  5943 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:57.220973  5927 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.221292  5943 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.221769  5943 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:57.222980  5939 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.223651  5939 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.225939  5396 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.193934  5879 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:16:57.209774 30522 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.209801  5587 raft_consensus.cc:493] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.226261  5939 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.164959  5793 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.186434  5916 raft_consensus.cc:697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:16:57.229198 30522 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:57.261117 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51379.2us
I20260709 13:16:57.261358  5860 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:16:57.236905 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:57.237891 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:57.236378  5587 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:57.248968  5949 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.249420 30558 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:16:57.206982  5805 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:57.240072  5940 log.cc:927] Time spent T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:16:57.253491  5821 consensus_meta.cc:287] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.006s
I20260709 13:16:57.258879  5939 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.210588  5859 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:16:57.288332  5549 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.202540  5950 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.301689 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 63947.9us
I20260709 13:16:57.303144 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.269912  5927 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:16:57.229022  5696 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.313935  5936 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 13:16:57.286300  5949 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.261137  5916 consensus_queue.cc:237] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.313860  5939 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.330068  5939 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.335043 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.345418 30556 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2398) took 1086 ms. Trace:
I20260709 13:16:57.348642 31292 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:57.348038 30556 rpcz_store.cc:276] 0709 13:16:56.258858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.259013 (+   155us) service_pool.cc:224] Handling call
0709 13:16:56.259382 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.259701 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:16:56.909252 (+649551us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.343652 (+434400us) log.cc:844] Serialized 14 byte log entry
0709 13:16:57.344759 (+  1107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:57.344852 (+    93us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:57.344939 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.345074 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.345298 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":187571,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":261157,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":6266,"thread_start_us":1003201,"threads_started":2}
I20260709 13:16:57.313701  5821 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:16:57.354550 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.358546 30699 raft_consensus.cc:1275] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:16:57.321970  5317 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.350500 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.346830  5723 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:57.353515  5924 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 13:16:57.373116 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.359689  5821 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.356000  5860 raft_consensus.cc:2804] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:57.321630  5859 consensus_queue.cc:1059] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 13:16:57.363019  5793 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:57.373116 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 174044us
I20260709 13:16:57.368009 31292 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.378865  5952 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.308784  5950 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.368432  5947 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:57.376879  5821 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:57.376787  5860 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:57.391004  5955 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:57.428356 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.400281  5935 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.436484 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:57.436642 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 174701us
I20260709 13:16:57.442303 30632 raft_consensus.cc:1275] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:16:57.418123  5793 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:57.450547 30550 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 19.
I20260709 13:16:57.467411 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:57.443516  5814 consensus_queue.cc:1048] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:16:57.420392  5860 raft_consensus.cc:3060] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:16:57.446904 30522 consensus_peers.cc:597] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:57.493167 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51346.1us
W20260709 13:16:57.498899 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:57.505234 30522 leader_election.cc:336] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.507517 30522 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.439271  5949 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.392400  5952 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.508795  5793 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
W20260709 13:16:57.523167  5806 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 1.015s	user 0.014s	sys 0.002s
I20260709 13:16:57.469244  5935 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:57.545671 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 112339us
I20260709 13:16:57.549974 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.526895  5859 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.532002  5949 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:57.443427  5955 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.537329  5947 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.556573 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.538087  5396 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.538439  5936 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:57.525316  5925 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.573926 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:57.574990 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69822.1us
I20260709 13:16:57.582477 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.571272  5802 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.548332  5806 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:57.612218  5958 log.cc:927] Time spent T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:16:57.613150  5772 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.616704  5955 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:57.617558  5950 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:57.565743  5859 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.618894  5935 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.619144  5975 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:57.579689  5947 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.574661  5396 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.620585  5793 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.581651  5936 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.618921  5976 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.628548  5900 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.656352 31292 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.600374  5696 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:16:57.662235  5838 consensus_meta.cc:287] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.793s	user 0.016s	sys 0.002s
I20260709 13:16:57.592087  5963 consensus_queue.cc:1048] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 13:16:57.664167  5772 raft_consensus.cc:493] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:57.674098  5975 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.682545  5947 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.684432  5859 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.687664  5950 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.688336  5396 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.689836  5924 raft_consensus.cc:3055] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:16:57.729708  5924 raft_consensus.cc:740] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:57.730693  5924 consensus_queue.cc:260] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:57.731282  5983 log.cc:927] Time spent T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:16:57.731834  5924 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:57.732312 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:57.641989  5802 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.736466  5802 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.690544  5806 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.693291  5860 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:16:57.700594 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44349.6us
W20260709 13:16:57.687701  5694 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.937s	user 0.016s	sys 0.001s
I20260709 13:16:57.702567  5976 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.712916  5772 raft_consensus.cc:515] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.720446 30703 raft_consensus.cc:2393] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 21: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:57.701958  5801 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:16:57.725994  5849 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: Time spent flushing consensus metadata: real 1.045s	user 0.015s	sys 0.003s
I20260709 13:16:57.710397  5838 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:57.726517  5925 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:57.751698  5543 consensus_meta.cc:287] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.729s	user 0.015s	sys 0.000s
I20260709 13:16:57.702975  5821 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.717665  5952 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.728600  5950 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.746384 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.757107  5772 leader_election.cc:290] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.727349  5900 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.771449 31292 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:57.752118  5973 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.765272  5957 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:57.774273 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:57.783119 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.784224 30698 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
W20260709 13:16:57.786983 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.783330 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:57.789127 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.789131 30707 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:57.790724  5802 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:57.793006  5950 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.788888  5957 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.795094  5821 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.797267  5957 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:57.772116  5943 consensus_meta.cc:287] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.550s	user 0.011s	sys 0.005s
I20260709 13:16:57.798218  5943 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.800668  5943 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.773413  5801 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:57.803898  5975 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.804574  5925 consensus_queue.cc:1059] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:16:57.778797  5793 raft_consensus.cc:2804] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:57.806226  5793 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.806669  5793 raft_consensus.cc:3060] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:57.807430  5976 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:57.811741  5860 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.812665 30624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46168 (request call id 1711) took 1248 ms. Trace:
I20260709 13:16:57.769528  5543 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.815064  5543 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.817030 30633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46186 (request call id 2307) took 1067 ms. Trace:
I20260709 13:16:57.752725  5694 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.817482 30633 rpcz_store.cc:276] 0709 13:16:56.749522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.764775 (+ 15253us) service_pool.cc:224] Handling call
0709 13:16:57.816855 (+1052080us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400094554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:57.816965 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536125696}
I20260709 13:16:57.817909 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.819386  5694 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:57.764485  5849 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:57.820107 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:16:57.820582 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 291077us
W20260709 13:16:57.821723 30596 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.821810 30552 raft_consensus.cc:2410] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:16:57.821914  5849 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.789636  5973 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:57.805840  5925 raft_consensus.cc:3055] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:16:57.829125 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:57.830844 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.813082 30624 rpcz_store.cc:276] 0709 13:16:56.563982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.635486 (+ 71504us) service_pool.cc:224] Handling call
0709 13:16:56.635864 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:56.689383 (+ 53519us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:16:56.689750 (+   367us) log.cc:844] Serialized 14 byte log entry
0709 13:16:56.973512 (+283762us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:16:56.973611 (+    99us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:16:56.973701 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:57.812423 (+838722us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b54007b0f54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:57.812470 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:57.812607 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13838,"prepare.queue_time_us":1191,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":40940,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":14568,"spinlock_wait_cycles":122464512,"thread_start_us":248688,"threads_started":1,"wal-append.queue_time_us":529439}
W20260709 13:16:57.835464 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.841732 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.845012 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:57.845755 30557 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:57.837952  5975 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:57.848033  5975 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:57.848726 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.848910  5796 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:57.850991  5796 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:57.843436 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:57.854724 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.862231 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.865248 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.867264 30707 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
W20260709 13:16:57.871141 30596 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:16:57.872215 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.872597 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:57.874656 30596 leader_election.cc:336] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.875130 30596 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.875496 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:57.878201 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:57.845527  5587 consensus_meta.cc:287] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.558s	user 0.005s	sys 0.009s
I20260709 13:16:57.883834 30627 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
W20260709 13:16:57.852052  5927 consensus_meta.cc:287] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.004s
I20260709 13:16:57.882418 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.885934 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.886270 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
W20260709 13:16:57.880802 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:57.886924 30633 raft_consensus.cc:3060] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:16:57.887430 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:57.837968  5925 raft_consensus.cc:740] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:57.890390 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.844184  5860 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:57.891852 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "490460505b2d424286323b79fb9fb2bc" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.892107 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:57.892640 30599 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:57.891583 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.894039 30627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.896626 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.897723  5317 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:57.879781  5849 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:57.896616 30596 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.874081  5694 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:57.903995 30596 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.904596 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:57.879540  5930 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:57.906980 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:57.900444 30599 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.901647 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.884321  5850 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:57.917701 30632 raft_consensus.cc:2393] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
W20260709 13:16:57.909838 30596 leader_election.cc:336] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:57.918421 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.898165  5317 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:57.895519  5927 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:57.894585  5860 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:57.899502  5879 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:57.926496 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:57.935041 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.920948 30627 raft_consensus.cc:1240] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 21. Current term is 22. Ops: []
I20260709 13:16:57.929046 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:57.935824 30698 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
I20260709 13:16:57.921247 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.908252  5407 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:57.892625  5587 raft_consensus.cc:515] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:57.959619 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.920480  5967 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:16:57.963977 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.933771  5317 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:57.936407  5860 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:57.946448  5870 consensus_meta.cc:287] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.919s	user 0.016s	sys 0.002s
I20260709 13:16:57.933127  5699 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:57.910233  5694 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:57.973037 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:57.973913 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:57.978155 30698 raft_consensus.cc:1275] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 24 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 13:16:57.980409 30553 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6b393e79ded4461ea5da7b6b6063305f in current term 20: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:57.979626  5839 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:58.007207 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.057219 30557 raft_consensus.cc:2468] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:58.067519 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.068390 30707 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:58.067709 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:58.027195  5996 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:16:58.071491 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.081084 30633 raft_consensus.cc:2468] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:58.082381 30551 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:16:58.084271 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 75260us
W20260709 13:16:58.086606  5988 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:16:58.086949  5943 consensus_queue.cc:1048] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:16:58.083009  5922 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:16:58.088181  5922 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.088524  5922 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:58.091387 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.094383  5936 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:58.095003  5994 log.cc:927] Time spent T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:16:57.981665  5870 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.101698  5870 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:57.970779  5925 consensus_queue.cc:260] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:58.104171 30633 raft_consensus.cc:1275] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:16:58.107136  5925 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:58.111326  5587 leader_election.cc:290] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:58.115602  5796 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.114071  5793 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.125633 30551 raft_consensus.cc:2468] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 21.
I20260709 13:16:58.128319  5793 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.130975 30707 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:58.126021  5860 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.146684  5973 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:58.149345  5974 log.cc:927] Time spent T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.544s	user 0.003s	sys 0.000s
I20260709 13:16:58.153403  5943 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:16:58.158066  5953 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:16:58.173417 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 88371.9us
I20260709 13:16:58.169853  5943 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:58.178879 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.181555 30553 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 22: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:58.180088 30523 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:58.183494 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.191594 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a55a2b63593d4202b9b70bac4a5ef28b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.192756 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.193776 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.194187 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8d07c4d131b54934aaedbe589e95d746" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.194478 30628 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 25.
I20260709 13:16:58.195444 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:58.197504 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01dd2254fe30409b8a08179436dd1e10" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.170028  5953 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:58.197583 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:58.199211 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:58.199515 30632 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:58.200055 30630 raft_consensus.cc:2393] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 20: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:58.203804 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dc6f2eea5d3844fab97d019c3af6cf2f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.189278  5928 raft_consensus.cc:2749] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:58.212146 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.214396 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:58.198947  5793 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.218761 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:58.223340 31293 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:16:58.207670  5925 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.247640 30523 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.257205 30523 leader_election.cc:304] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.251817  5925 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:58.264603  5925 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:58.259588  5813 raft_consensus.cc:2749] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:58.268785 30625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36384 (request call id 1433) took 1714 ms. Trace:
I20260709 13:16:58.269374 30625 rpcz_store.cc:276] 0709 13:16:56.554116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.581688 (+ 27572us) service_pool.cc:224] Handling call
0709 13:16:56.893057 (+311369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:56.896131 (+  3074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:58.268542 (+1372411us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400055ad0. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edc8ad 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:58.268708 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":714831616}
I20260709 13:16:58.274044  5694 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.259550  5980 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:58.249691  5928 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:16:58.227039  5793 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.288972  5953 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:58.254243  5943 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:58.291958 30626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.294648 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.294893 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.295692 30701 raft_consensus.cc:2368] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24.
I20260709 13:16:58.296272 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.312809 30624 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:16:58.313038 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.313179 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.313690 30702 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:58.315122 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.310571  5925 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.318816  5925 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.321126  5925 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.300272 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.324141  5925 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:58.326025 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.326989  5925 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:58.311700  5595 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:58.329823  5595 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.289016  5793 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.336030 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:58.339257 30523 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.299014 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.340919 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.341665 30557 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 24.
I20260709 13:16:58.339023 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.344426 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.356436 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.361030 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.363888 30523 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:58.350909  6000 log.cc:927] Time spent T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:16:58.281675  5928 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.360967 30673 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:58.364611 30702 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:58.365082  5980 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:58.366408 30523 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:16:58.410529 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 44201.3us
I20260709 13:16:58.414204  5943 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:16:58.414914  5922 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.362959  5925 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:16:58.364143  5860 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.360103  5793 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:58.366982  5595 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.442432 30703 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53928 (request call id 2624) took 1014 ms. Trace:
I20260709 13:16:58.443161 30703 rpcz_store.cc:276] 0709 13:16:57.427935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:57.428087 (+   152us) service_pool.cc:224] Handling call
0709 13:16:58.442148 (+1014061us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b5400050d54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:58.442348 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642701056}
I20260709 13:16:58.439431 30671 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:58.432996 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 66028.6us
I20260709 13:16:58.412089  5948 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:58.455737 30625 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 26. Current term is 27. Ops: []
I20260709 13:16:58.456034 30556 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:58.367942  5723 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:58.434595 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:58.436096 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 75264.4us
I20260709 13:16:58.419894  5943 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:58.459477 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.466356 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.462666 30673 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.470707 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.471501 30550 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 25.
I20260709 13:16:58.362881  5778 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:58.351184  5317 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:58.471278 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.413756  5928 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:58.451459  5901 raft_consensus.cc:2804] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:58.454789  5960 raft_consensus.cc:2804] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:58.476734 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.479562 30550 raft_consensus.cc:1240] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 26. Current term is 27. Ops: []
I20260709 13:16:58.478576 31292 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:58.491993 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.482012 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:58.497905 30550 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 22: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:16:58.489679 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:58.495153 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:58.499936 30628 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:16:58.499645 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.501561 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.502460 30554 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:16:58.467445  5694 consensus_queue.cc:1048] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 13:16:58.510391  5317 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:58.486963  5901 raft_consensus.cc:697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:58.516781 30628 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:16:58.517338  5901 consensus_queue.cc:237] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.524914 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.481415  5805 raft_consensus.cc:2804] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:16:58.464799  5948 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.466986  5793 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
W20260709 13:16:58.538965 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 47579.6us
W20260709 13:16:58.532899  5916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.199s	user 0.025s	sys 0.024s
I20260709 13:16:58.586736 31292 leader_election.cc:400] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24.
I20260709 13:16:58.587383 31292 leader_election.cc:403] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:16:58.580757 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.555474 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.481176  5778 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:58.588963 30552 raft_consensus.cc:2368] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24.
I20260709 13:16:58.482162  5925 raft_consensus.cc:3060] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:16:58.537906  5682 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.025s	sys 0.026s
I20260709 13:16:58.501073  5924 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:16:58.538678  5902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.418s	user 0.028s	sys 0.035s
I20260709 13:16:58.597841 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3410292026774f0ca95a2c77eab3a8b4" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.519712  5590 raft_consensus.cc:3060] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:58.609095 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.544561 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c7844be2162490091882b0a0344414c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.624639 30699 raft_consensus.cc:2368] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
W20260709 13:16:58.586844  5916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.258s	user 0.025s	sys 0.025s
I20260709 13:16:58.554118  5870 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:58.621850 30556 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:16:58.618403  5682 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.287s	user 0.031s	sys 0.034s
I20260709 13:16:58.530437  5960 raft_consensus.cc:697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:58.539350  5961 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:58.637800 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.554239  5922 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:58.529855  5975 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:58.644600 30550 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de1dd97d9f342288201946ead3a9445" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.653000 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:58.653749 30558 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:16:58.623586  5902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.038s	sys 0.044s
I20260709 13:16:58.634539  5595 raft_consensus.cc:3060] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:58.636722  5924 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:58.591851  5805 raft_consensus.cc:697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:58.650537 30556 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.620232  5860 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:58.654825 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 73196.5us
W20260709 13:16:58.714283 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 92452.2us
I20260709 13:16:58.649089  5961 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:58.723207 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.587095  5943 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:16:58.740561 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 258669us
W20260709 13:16:58.740178 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 250595us
I20260709 13:16:58.756424 30556 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:58.757874 31292 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:58.766713 30597 leader_election.cc:400] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
I20260709 13:16:58.767381 30597 leader_election.cc:403] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:16:58.772631 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:58.776721  6004 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:16:58.759513  5975 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:16:58.781637  5975 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:58.784224  5975 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:58.787093  5922 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:58.787638  5778 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:58.787753  5922 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:58.786141  5924 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.773486  5793 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:16:58.792932  5793 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:58.793942  5793 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:58.799497  5916 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 for earlier term 21. Current term is 22.
W20260709 13:16:58.801249 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:58.801827 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:58.790330  5922 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:58.802793 30555 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39184 (request call id 1427) took 2210 ms. Trace:
W20260709 13:16:58.803083 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:58.803450  5948 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:58.807116  5850 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:58.806800  5960 consensus_queue.cc:237] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:58.803217 30555 rpcz_store.cc:276] 0709 13:16:56.592236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:56.595198 (+  2962us) service_pool.cc:224] Handling call
0709 13:16:58.802527 (+2207329us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540052b554. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806ee12fa 00007f180bea2b17 00007f180541d6d4 00007f180541d662 00007f180541d591 00007f180541d50c 00007f180541c7c2 00007f180521848c 00007f18052178c6 00007f180521726f
0709 13:16:58.802701 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4822010240}
W20260709 13:16:58.803750 30596 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:58.812069 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:58.732338  5990 log.cc:927] Time spent T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.670s	user 0.001s	sys 0.002s
I20260709 13:16:58.757710  5961 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:58.819927 30554 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 18.
I20260709 13:16:58.713418  5870 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:58.820043 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 47470us
I20260709 13:16:58.820873 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29980abbc08249a490fcf7b22cf63bb5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.824031 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:58.818539 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 334822us
I20260709 13:16:58.826191 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "177c3663558a41cba8e75193425ead61" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.828454 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:58.829514 30701 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:16:58.832258 31294 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.833396 30558 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 26.
I20260709 13:16:58.843081 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.847438 30523 leader_election.cc:304] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.848554 31294 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:58.849299 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 45565.6us
W20260709 13:16:58.850356 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:58.850884 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:16:58.854151 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 82198.4us
I20260709 13:16:58.870585 30700 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 13:16:58.871578 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:58.872879 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.873946 30523 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:58.874815 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.877833 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54da309ba151423ab2739966b9caebd0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:58.876128 30625 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:16:58.818027  5860 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:16:58.849433  5975 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:58.890604 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "731ce9bb8d6143d4860cfae0d68c8024" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:58.826722  6008 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:16:58.842916  5870 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:58.898591 30705 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:58.888773 30701 raft_consensus.cc:2468] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:58.858897  5696 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:58.869642  5839 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:16:58.907205 30523 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:58.886067  5922 raft_consensus.cc:2749] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:58.885418  5927 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:16:58.897593  5850 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:58.891175  5797 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.884203  6010 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:58.905896  6009 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:58.966725  5975 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:58.966725 21526 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 31427)
I20260709 13:16:58.901175  5590 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:58.928079  5980 raft_consensus.cc:2804] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:16:59.017315 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.013370 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.021378 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.022680 30625 raft_consensus.cc:2393] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 22: Already voted for candidate acf852f85f47483d99cbe4020baee627 in this term.
I20260709 13:16:59.018391 30630 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 20.
I20260709 13:16:58.912045  5694 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 13:16:58.894248  5805 consensus_queue.cc:237] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:59.112701  6010 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.113577  6010 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.116725  6010 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.117515  6010 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.118963 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 68954.2us
W20260709 13:16:59.017619  5996 log.cc:927] Time spent T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
W20260709 13:16:59.139257  5925 consensus_meta.cc:287] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.513s	user 0.005s	sys 0.009s
I20260709 13:16:59.118235  5870 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:58.982748  5797 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.136751  5595 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425) with higher term. Message: Invalid argument: T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24.
I20260709 13:16:59.118201  6010 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:59.152402  5980 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:16:59.109992  5975 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
W20260709 13:16:59.179214 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 306750us
I20260709 13:16:58.993485  6009 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.167222  5925 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:16:59.194432 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:59.198065 30596 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.218331 30596 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:59.245301 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 361383us
I20260709 13:16:59.230507  5813 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:59.267174  5813 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:59.267683  5813 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:59.268287  5925 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:59.268987  5925 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.271979  5797 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:59.242718  5839 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:16:59.281615 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 53833.6us
I20260709 13:16:59.284426 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.291819 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:16:59.292213 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 443717us
I20260709 13:16:59.292788 30633 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:59.304376  5860 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 13:16:59.311138  6010 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:59.310969  5860 raft_consensus.cc:3055] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:16:59.311764  5860 raft_consensus.cc:740] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:59.312808  5860 consensus_queue.cc:260] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:59.300025  5595 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.297788  5870 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.319206  5595 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.320223  5870 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:16:59.313349  5928 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: Time spent flushing consensus metadata: real 0.807s	user 0.015s	sys 0.000s
I20260709 13:16:59.300971  5793 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:16:59.334509 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 421422us
I20260709 13:16:59.304311  5925 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:16:59.356673 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 59546.1us
I20260709 13:16:59.358883 31292 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.322602  5870 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:16:59.369066 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 73706.4us
I20260709 13:16:59.322398  5595 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.377727 31294 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:16:59.368630  5943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.014s	sys 0.024s
I20260709 13:16:59.375393 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:16:59.360198  5902 raft_consensus.cc:493] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:59.300617  5980 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:16:59.395387 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.396822  6012 log.cc:927] Time spent T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
W20260709 13:16:59.386416  5943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.015s	sys 0.025s
I20260709 13:16:59.385248 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.343468  5928 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:59.380846  5870 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:16:59.368350  6010 consensus_queue.cc:1059] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 13:16:59.345211  5793 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.394114  5925 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:16:59.413416  6008 raft_consensus.cc:697] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:59.415052 31294 leader_election.cc:400] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Vote denied by peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473) with higher term. Message: Invalid argument: T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 for earlier term 23. Current term is 24.
W20260709 13:16:59.425802 30699 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.009s	sys 0.011s
I20260709 13:16:59.429792 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.389251  5595 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:16:59.433211 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.433621 30551 raft_consensus.cc:1240] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Rejecting Update request from peer 6b393e79ded4461ea5da7b6b6063305f for earlier term 25. Current term is 26. Ops: []
I20260709 13:16:59.441468 30700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.444926 30625 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:16:59.437706 30699 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.009s	sys 0.011s
I20260709 13:16:59.437947 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:59.449745 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.454174 30698 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:16:59.454483 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.406241  5902 raft_consensus.cc:515] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:59.432487  6010 consensus_queue.cc:1243] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Peer 5a03a8214bba4eb1b0ff00895c565564 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 13:16:59.467406 31292 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:16:59.467790  6015 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:16:59.468461 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.427345  5943 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.451905  6019 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:16:59.488683 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.489343 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:16:59.463948  6018 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:59.486358  5924 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:16:59.508143  6014 log.cc:927] Time spent T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:16:59.504221  6019 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.505370  6009 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:59.473484  5943 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.539618  5928 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:16:59.544947  6013 log.cc:927] Time spent T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 13:16:59.545913  6016 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:16:59.549942 30522 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.558563  5902 leader_election.cc:290] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.556242 30522 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:59.562186 30522 leader_election.cc:336] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.567059  5793 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.562798  5902 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:59.563973  5961 consensus_meta.cc:287] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.720s	user 0.012s	sys 0.003s
I20260709 13:16:59.568737  5961 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.570458 30703 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:16:59.527146  6026 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:16:59.573225 30701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.576635 30551 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49258 (request call id 2421) took 1239 ms. Trace:
I20260709 13:16:59.499806  6008 consensus_queue.cc:237] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.577551 30551 rpcz_store.cc:276] 0709 13:16:58.337434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.339424 (+  1990us) service_pool.cc:224] Handling call
0709 13:16:58.352668 (+ 13244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:16:59.441021 (+1088353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.576131 (+135110us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400174b54. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e46571 00007f1806edde01 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2 00007f180521848c 00007f18052178c6
0709 13:16:59.576522 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2485459328}
I20260709 13:16:59.526506  6019 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:59.565800 30522 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.565150  5813 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:59.592895  5793 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:16:59.593657  5793 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.594180 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.596437 30700 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:16:59.597698 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:59.582695  5860 raft_consensus.cc:3060] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:16:59.613637 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 175877us
I20260709 13:16:59.614962 31294 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.616099  6026 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:16:59.616744 31294 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:59.619874 30553 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49250 (request call id 2858) took 1185 ms. Trace:
I20260709 13:16:59.620325 30553 rpcz_store.cc:276] 0709 13:16:58.434045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:58.434510 (+   465us) service_pool.cc:224] Handling call
0709 13:16:58.460557 (+ 26047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:16:58.460877 (+   320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:16:58.460916 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:16:59.287243 (+826327us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b2c00012b38. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f18057d6a11 00007f18057e6372 00007f1806f18a01 00007f1806edd43a 00007f1806edc83e 00007f180bea1419 00007f1805418bd4 00007f1805418b62 00007f1805418a91 00007f1805418a0c 00007f1805417cc2
0709 13:16:59.287873 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:16:59.525748 (+237875us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:16:59.525795 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:16:59.525880 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:16:59.619496 (+ 93616us) raft_consensus.cc:1685] finished
0709 13:16:59.619519 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:16:59.619798 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3801472,"thread_start_us":236412,"threads_started":1,"wal-append.queue_time_us":294742}
I20260709 13:16:59.621354 30698 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.622056  5778 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:16:59.623127  6031 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:16:59.624043 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 61900.9us
I20260709 13:16:59.624975  5961 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:59.631443  5870 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:16:59.632632  5870 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:16:59.634092  5793 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:59.635879 31294 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.639990  5813 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:16:59.640321  5870 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:16:59.643775 31294 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:59.645463  5805 consensus_queue.cc:1059] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:16:59.650535  5901 raft_consensus.cc:3055] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:16:59.651016  5901 raft_consensus.cc:740] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:59.651940  5901 consensus_queue.cc:260] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:59.653048  5901 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:16:59.660965 30625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3d2c26e9174e5c942f6f2e1c84cee7" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.661958 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.662796 30624 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:16:59.667528 30551 raft_consensus.cc:1275] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:16:59.656805 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:16:59.669438 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.667857 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.657219  6026 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:16:59.684008 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 195386us
I20260709 13:16:59.684338 30551 raft_consensus.cc:1275] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:16:59.691424 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.699198  5590 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:59.701102 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.701968 30554 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 20.
I20260709 13:16:59.696842  5839 raft_consensus.cc:493] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:16:59.705739  5839 raft_consensus.cc:515] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:16:59.708052  5839 leader_election.cc:290] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.708122  6019 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.713601  5806 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:16:59.715070 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 46389.3us
I20260709 13:16:59.681262 31294 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:59.716395 30703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:16:59.726167 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:16:59.727183 30702 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:16:59.731832  5943 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:16:59.736563 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:16:59.733830 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 70343.1us
I20260709 13:16:59.685027  5870 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:16:59.751168  5870 raft_consensus.cc:493] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.752390 30553 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:16:59.755101 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:16:59.756108 30554 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:59.751655  5870 raft_consensus.cc:3060] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:16:59.740175  6008 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:16:59.779142 30702 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
I20260709 13:16:59.782517  5694 consensus_queue.cc:1048] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:16:59.787139 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:59.775535  6009 consensus_queue.cc:1048] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:16:59.806577 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:59.808138 30630 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:59.808813 30630 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:59.810307 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.811558 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0e3e983dd2fc4a56857ff13e699d9517" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:16:59.813304 30632 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:16:59.815760 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.823107 31292 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:16:59.823256 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 197590us
W20260709 13:16:59.824206 30522 leader_election.cc:336] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.824698 30522 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:16:59.824967  5793 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:16:59.825724  5980 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:16:59.826021  6037 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.160s	user 0.001s	sys 0.003s
W20260709 13:16:59.826169 30522 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.829209 30551 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.832597 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.833437 30559 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:16:59.834492 30551 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 13:16:59.831092  6039 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Append to log took a long time: real 0.128s	user 0.002s	sys 0.000s
I20260709 13:16:59.835474 30552 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:16:59.834532 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:16:59.836889 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:16:59.824725 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:16:59.837791 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.838528  5793 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:16:59.841346 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.838539  5903 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:16:59.841990 31292 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:59.710539  6019 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:16:59.842267 30522 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:16:59.841611 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 55158.8us
I20260709 13:16:59.844913  5903 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:16:59.849885  5903 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:16:59.850068  5936 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:16:59.853320 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5315ab4cdcff4579984ba8de8f9d4d82" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:16:59.854703 30554 raft_consensus.cc:2468] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 25.
I20260709 13:16:59.862450 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:16:59.863547 30631 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 24
W20260709 13:16:59.867425 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 192464us
I20260709 13:16:59.873061 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.861441 30673 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:16:59.881682 30599 leader_election.cc:304] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:16:59.878144 30522 leader_election.cc:336] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.845350  5793 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:16:59.885612 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.859094  5980 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:16:59.912279  5961 raft_consensus.cc:2804] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:16:59.912690 30523 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.913132  5961 raft_consensus.cc:697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:16:59.914213 30523 leader_election.cc:304] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:16:59.914283  5961 consensus_queue.cc:237] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:16:59.919894  5927 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 13:16:59.921461 30624 raft_consensus.cc:1275] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:16:59.935324 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62358.7us
I20260709 13:16:59.860503  5936 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:16:59.861946  5590 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
W20260709 13:16:59.946390 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 44152.3us
I20260709 13:16:59.960963  5590 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:16:59.952400  6018 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:16:59.866051  5805 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:16:59.898043  5793 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:16:59.899906  5407 raft_consensus.cc:2749] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:16:59.949959 30630 raft_consensus.cc:2468] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 21.
W20260709 13:16:59.988409 21526 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 31427)
I20260709 13:16:59.964110  5590 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:16:59.954491  5813 raft_consensus.cc:2749] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:16:59.957017  6019 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:16:59.990163  6021 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:17:00.000809  6028 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:17:00.001302  6029 log.cc:927] Time spent T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 13:17:00.020035 30522 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:16:59.984082  6018 raft_consensus.cc:493] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.036851 31294 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:17:00.068663  6042 log.cc:927] Time spent T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.097s	user 0.001s	sys 0.001s
I20260709 13:17:00.071810  5936 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:16:59.998930  6032 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:17:00.099292 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "233dbd206c1f419697b1d1b21a74c633" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.050915  6018 raft_consensus.cc:515] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.103475  6018 leader_election.cc:290] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:00.108930 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 72230.8us
I20260709 13:17:00.110893  6008 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:00.110306 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
W20260709 13:17:00.134557 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.135429 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:00.120368  5860 consensus_meta.cc:287] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
W20260709 13:17:00.142351 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 100155us
W20260709 13:17:00.143813 30522 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.152458 30629 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:17:00.152451 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:00.152485 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:00.031731  6019 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:00.178897 30522 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:00.183197 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 73025.8us
I20260709 13:17:00.190762 30522 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:17:00.194206 30522 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.179562 30631 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 24.
W20260709 13:17:00.196412 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.198354 30522 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:00.199195 31294 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:17:00.150964  6020 log.cc:927] Time spent T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.251s	user 0.005s	sys 0.000s
I20260709 13:17:00.145191  6026 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.177186  5839 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:00.212813 30670 leader_election.cc:336] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:00.206121  5901 consensus_meta.cc:287] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.004s
I20260709 13:17:00.220151 30670 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.198012  5902 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:17:00.220289 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 224026us
I20260709 13:17:00.197376  5860 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:00.208535  6018 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:17:00.254323 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 392952us
I20260709 13:17:00.208844  5870 raft_consensus.cc:515] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.225389  6032 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:00.207788  5975 consensus_queue.cc:1059] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:17:00.240245  5805 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:00.284636 30523 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:17:00.289738 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 90637.5us
I20260709 13:17:00.297806 30702 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:17:00.298705 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:00.214265  6026 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:00.302099 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 122454us
I20260709 13:17:00.284030  5943 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:00.289641  5860 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:00.247450  6008 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 13:17:00.257933  6033 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 13:17:00.347107 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 62576.8us
I20260709 13:17:00.351722  5959 consensus_queue.cc:1048] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:00.360507  6065 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:00.274049  6018 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:00.314407  5778 raft_consensus.cc:2804] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:17:00.380067  5870 leader_election.cc:290] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:00.384269  6047 log.cc:927] Time spent T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 13:17:00.376987  5903 consensus_meta.cc:287] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.003s
I20260709 13:17:00.387004  5903 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.389073  6018 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:00.389390  5903 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:00.390079 30701 raft_consensus.cc:1240] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
W20260709 13:17:00.413102 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 64882.4us
I20260709 13:17:00.398722  5778 raft_consensus.cc:697] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:17:00.430542 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 133872us
I20260709 13:17:00.432158 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdaa99b885784af3b7e73563f113185a" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.432730 30557 raft_consensus.cc:1240] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 19. Current term is 20. Ops: []
I20260709 13:17:00.413537  6019 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:00.441938 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:00.419436  6074 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:00.449049 30702 raft_consensus.cc:1275] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:17:00.464182  6052 log.cc:927] Time spent T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: Append to log took a long time: real 0.225s	user 0.000s	sys 0.003s
I20260709 13:17:00.487010 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:00.488466 30633 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:00.495597 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:00.496263  6074 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.497893 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:00.500021 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:00.501228 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:00.501024  6026 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:00.517194  6073 consensus_queue.cc:1059] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:00.518354  6073 raft_consensus.cc:3055] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:17:00.495616  5778 consensus_queue.cc:237] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.518755  6073 raft_consensus.cc:740] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:00.519716  6073 consensus_queue.cc:260] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.521092  6073 raft_consensus.cc:3060] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:17:00.522066  6061 consensus_queue.cc:1048] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:17:00.522574  6063 raft_consensus.cc:3055] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:17:00.523137 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:00.523139  6063 raft_consensus.cc:740] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:00.524398 21526 raft_consensus.cc:2243] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:00.547251 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:00.566655 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b15e4d74180b49daad15cc459dde3181" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:00.567654 30557 raft_consensus.cc:2393] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 25: Already voted for candidate 6b393e79ded4461ea5da7b6b6063305f in this term.
I20260709 13:17:00.572765 30553 raft_consensus.cc:1240] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Rejecting Update request from peer 015ca80eba674552b34856a4f20271f1 for earlier term 20. Current term is 21. Ops: []
I20260709 13:17:00.581511  6074 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:00.572794 30702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:17:00.587184  5793 consensus_meta.cc:287] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1: Time spent flushing consensus metadata: real 0.578s	user 0.014s	sys 0.000s
I20260709 13:17:00.587965  5793 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.524451  6063 consensus_queue.cc:260] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.567203 21526 pending_rounds.cc:70] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:00.593221 31292 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.594350 30553 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.599443 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.595376 21526 pending_rounds.cc:77] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 23 index: 13 } timestamp: 7305637892695474176 op_type: NO_OP noop_request { }
I20260709 13:17:00.601310 21526 raft_consensus.cc:2889] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:00.601774 21526 raft_consensus.cc:2272] T 72181014679a4c538f96f2d305a5474f P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:00.598482  6074 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:00.599880 31294 leader_election.cc:304] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.608260 31294 leader_election.cc:304] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:00.611150  6074 raft_consensus.cc:2749] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:00.623153  6064 raft_consensus.cc:2749] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:17:00.606494  6063 raft_consensus.cc:3060] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:17:00.639968 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.640477 30670 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.642756  5902 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:00.607033  6026 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:00.596634  6019 consensus_queue.cc:1059] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:17:00.654187  6041 log.cc:927] Time spent T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:17:00.675473  5793 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:00.676040 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 237533us
I20260709 13:17:00.686793  5696 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.687494  5696 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.689703  5696 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:00.694042 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 56464.7us
W20260709 13:17:00.694446 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54529.8us
W20260709 13:17:00.697165 30670 leader_election.cc:336] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.697605  6054 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:17:00.700033 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 96646.2us
I20260709 13:17:00.702154 30555 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.703495 30558 raft_consensus.cc:1275] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:17:00.697651 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.707166  5696 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:00.708343 30706 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 13:17:00.711499 30702 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:17:00.716146 30670 leader_election.cc:336] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.716818 31292 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.719719  6076 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:00.722525  6076 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:17:00.721626 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.726150 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:00.728560  6070 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:00.728686  5924 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:17:00.733176 30670 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.733974 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:00.735028  6070 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:00.736402 30670 leader_election.cc:336] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:00.745795 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:00.748749  5860 raft_consensus.cc:3055] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:17:00.749356  5860 raft_consensus.cc:740] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:00.757905  6054 consensus_queue.cc:1048] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:00.750430  5860 consensus_queue.cc:260] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.764724  5860 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:17:00.677670 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:00.784919 30552 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:00.793877 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f029f5d5b1ee4391a2b10e12ec33b844" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:00.796834 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 143408us
I20260709 13:17:00.799723 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:00.800019 30629 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:17:00.800544 30631 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:17:00.815140 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76768.7us
I20260709 13:17:00.815554 30631 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
W20260709 13:17:00.816181 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:00.821998 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:00.823673 21526 raft_consensus.cc:2243] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:00.827850 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.828387 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:00.833266 30670 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.833693 30670 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:17:00.835677 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:00.836704 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:00.836839 21526 raft_consensus.cc:2272] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:00.837304 30670 leader_election.cc:336] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:00.852370 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:00.863045 31292 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:00.884110 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:00.841741  5694 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:00.886390 31292 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:00.891762  5924 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:00.909583 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57249.3us
I20260709 13:17:00.851464  6022 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:00.919903  6073 raft_consensus.cc:493] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:00.910552  6071 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 13:17:00.934587 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45168.9us
I20260709 13:17:00.912221  5924 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:17:00.941532  6019 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.953083  6019 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.934707  6073 raft_consensus.cc:515] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:00.954851  5924 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:00.955586  6019 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:00.955829  5924 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:00.938903  6018 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.958597  6018 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:00.960852  6018 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:00.951717  5697 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:00.939196  6033 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:00.966048 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:00.967653 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.868463  6063 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.073s
I20260709 13:17:00.975440 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:00.976754 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:00.977692 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:00.981178 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2b685fd119d941538e6fee8dc8f8c930" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:00.961781  5697 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:17:00.988956  6081 log.cc:927] Time spent T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:17:00.989907  5697 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:00.982504  6026 raft_consensus.cc:493] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.007148  6026 raft_consensus.cc:515] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.009670  6026 leader_election.cc:290] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:00.982205  6033 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:00.984625 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:01.012055 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:00.955591  6073 leader_election.cc:290] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:01.019718 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:01.024482 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.025424 30559 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 26: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:17:01.020282 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:01.026883 30552 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
I20260709 13:17:01.009713 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4b674451b4d5465d82d8ef4f14fb52f5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:17:01.030992 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 48617.9us
I20260709 13:17:01.021464 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.019990 30699 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:17:01.032800 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "125a8386fffd41c0b18a3e6e0690283b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.037096 31292 leader_election.cc:304] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:01.039129 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:01.043239  5850 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:17:01.043311  6073 raft_consensus.cc:2749] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:17:01.043855  5850 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:01.044503  5850 raft_consensus.cc:3060] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:17:01.038925  6074 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:01.045428  6033 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:01.034747  6070 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:01.047366 31292 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.048600 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:01.050045  6026 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:01.051540 31292 leader_election.cc:304] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.053243  6076 raft_consensus.cc:2749] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:01.071074 30624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "536e99f894744c35a0490416a32a875e" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.073976  5850 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:01.079491 31292 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.081231  6076 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:17:01.091953 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:01.087028  5850 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:01.107191 30706 raft_consensus.cc:2468] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 26.
I20260709 13:17:01.150441 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.158388 30597 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:01.170493 21526 raft_consensus.cc:2243] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:17:01.170243 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.179289  5860 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.180755  5407 raft_consensus.cc:2804] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:17:01.210542  5860 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.212556  5407 raft_consensus.cc:697] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
W20260709 13:17:01.222800 30522 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 139 similar messages]
I20260709 13:17:01.225939  6008 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:01.227212  6008 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:01.231873 30382 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 6 to 17, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 17 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:01.238960 30382 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 24. New cstate: current_term: 24 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.239657  5407 consensus_queue.cc:237] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:01.243140 21526 raft_consensus.cc:2272] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.242394 30382 catalog_manager.cc:5697] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 19 to 23, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:01.246017  5860 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:01.336812 30558 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bae0ee744c54691b48d52b137264f92" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:01.245679 30382 catalog_manager.cc:5697] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 19, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
W20260709 13:17:01.270001 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 47482us
I20260709 13:17:01.338572 30558 raft_consensus.cc:2393] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate acf852f85f47483d99cbe4020baee627 in current term 26: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:17:01.278487 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:17:01.280897 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:01.288911 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:01.347106 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 314534us
I20260709 13:17:01.246464  6008 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:01.306864 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.350488 21526 raft_consensus.cc:2243] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.311590 30629 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26f2e1d0fad840b5b8718f931fc92316" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.352874 21526 pending_rounds.cc:70] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:01.353297 21526 pending_rounds.cc:77] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 20 index: 11 } timestamp: 7305637924415102976 op_type: NO_OP noop_request { }
I20260709 13:17:01.353968 21526 raft_consensus.cc:2889] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:01.354643 21526 raft_consensus.cc:2272] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.313616 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "907e974a334846208c70f0b813199df0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.358767 30631 raft_consensus.cc:1275] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:17:01.364032 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.372413 21526 raft_consensus.cc:2243] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.379268 21526 pending_rounds.cc:70] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:01.380321 21526 pending_rounds.cc:77] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305637929343574016 op_type: NO_OP noop_request { }
I20260709 13:17:01.394331 30631 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:17:01.397195 30522 consensus_peers.cc:597] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:01.398703 30522 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.404021 21526 raft_consensus.cc:2889] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:01.407209 21526 raft_consensus.cc:2272] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.414464 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.416543 21526 raft_consensus.cc:2243] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:01.426007 30382 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 20, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 20 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.422580  6008 raft_consensus.cc:493] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.458907  6008 raft_consensus.cc:515] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:01.463536 30382 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:01.469201 30382 catalog_manager.cc:5697] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 25. New cstate: current_term: 25 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:01.462343  6008 leader_election.cc:290] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:01.484200 21526 raft_consensus.cc:2272] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:01.497723 30522 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.507973  5902 consensus_queue.cc:1048] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 13:17:01.518512 31292 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.519752  5860 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:01.521258 31292 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 45339us
I20260709 13:17:01.522207 31292 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.523169  5860 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:01.535097 30382 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:01.540863 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.546774  5860 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:01.550638 30382 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:17:01.545315 30522 consensus_peers.cc:597] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.560320 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.579668 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
W20260709 13:17:01.591193 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.600188 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.601642 30557 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:17:01.613812 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.615010 30522 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.615964 30522 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:01.616693 30522 consensus_peers.cc:597] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.598871 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9657733894c94f86ac3b047814fb8fd9" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.623581 21526 raft_consensus.cc:2243] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.625455 21526 raft_consensus.cc:2272] T 5de1dd97d9f342288201946ead3a9445 P 015ca80eba674552b34856a4f20271f1 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.625882  6008 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:01.628096  6008 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.628810  6008 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.631904  6008 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:01.645642 30382 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 20 to 23, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:01.647943 30382 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:17:01.649657 30373 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 138) took 35405 ms (client timeout 15000 ms). Trace:
I20260709 13:17:01.662073 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:01.674664 31292 leader_election.cc:304] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.679232  5860 raft_consensus.cc:2749] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:01.680938 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07fae2111d284c78bf054bd74aee6b94" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.688941 31292 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:01.692056  6008 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:17:01.691295 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:01.698249 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.700131 30559 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:17:01.702423  5407 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:17:01.701375 21526 raft_consensus.cc:2243] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:01.699090 30382 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 22 to 23, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 23 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:01.728744  6008 raft_consensus.cc:493] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
W20260709 13:17:01.660401 30373 rpcz_store.cc:269] 0709 13:16:26.243946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:26.244335 (+   389us) service_pool.cc:224] Handling call
0709 13:17:01.649329 (+35404994us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:16:55.666808 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:16:55.889408 (+222600us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:16:55.898321 (+  8913us) write_op.cc:435] Acquired schema lock
0709 13:16:55.901856 (+  3535us) tablet.cc:662] Decoding operations
0709 13:16:55.970814 (+ 68958us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:16:55.970952 (+   138us) write_op.cc:641] Partition lock acquired for write op
0709 13:16:55.970992 (+    40us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:16:55.972587 (+  1595us) tablet.cc:701] Row locks acquired
0709 13:16:55.972660 (+    73us) write_op.cc:260] PREPARE: finished
0709 13:16:55.972912 (+   252us) write_op.cc:270] Start()
0709 13:16:55.973095 (+   183us) write_op.cc:276] Timestamp: P: 1783603015972810 usec, L: 0
0709 13:16:55.973139 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 13:16:55.974133 (+   994us) log.cc:844] Serialized 6698 byte log entry
0709 13:16:57.266067 (+1291934us) op_driver.cc:464] REPLICATION: finished
0709 13:16:57.602701 (+336634us) write_op.cc:301] APPLY: starting
0709 13:16:57.648392 (+ 45691us) tablet.cc:1367] starting BulkCheckPresence
0709 13:16:58.335931 (+687539us) tablet.cc:1370] finished BulkCheckPresence
0709 13:16:58.336107 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:16:59.568627 (+1232520us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:16:59.569158 (+   531us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:16:59.569254 (+    96us) write_op.cc:312] APPLY: finished
0709 13:16:59.569969 (+   715us) log.cc:844] Serialized 155 byte log entry
0709 13:16:59.773841 (+203872us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:59.829775 (+ 55934us) write_op.cc:454] Released schema lock
0709 13:16:59.829880 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 13:16:59.830419 (+   539us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:16:59.830454 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":738887,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":278022,"apply.run_cpu_time_us":65610,"apply.run_wall_time_us":2251329,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75357,"num_ops":17,"prepare.queue_time_us":721453,"prepare.run_cpu_time_us":23628,"prepare.run_wall_time_us":820773,"raft.queue_time_us":292136,"raft.run_cpu_time_us":22274,"raft.run_wall_time_us":321460,"replication_time_us":1230303,"thread_start_us":1372784,"threads_started":5,"wal-append.queue_time_us":756868,"wal-append.run_cpu_time_us":18182,"wal-append.run_wall_time_us":2628338}]]}
W20260709 13:17:01.772024 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:01.794449 30382 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:01.772804  6008 raft_consensus.cc:515] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:01.814661  6008 leader_election.cc:290] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:01.815958 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:01.820094 21526 raft_consensus.cc:2272] T 233dbd206c1f419697b1d1b21a74c633 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:01.825511 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb3b5ffa2c364cacbd6f5fd9a69f1a79" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.821573  6008 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:01.900496 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 76174.6us
I20260709 13:17:01.906941  6008 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.908285  6031 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.909053  6031 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.911924 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:01.912127 31294 leader_election.cc:304] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:01.913870 30559 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:17:01.925177  5860 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.926016  5860 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:01.928267 30707 raft_consensus.cc:1275] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:17:01.928793  5860 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:01.929432  5860 raft_consensus.cc:2749] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:01.957425  5682 consensus_queue.cc:1048] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 13:17:01.966264  6063 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:01.967140  6063 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:01.970011  6063 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:01.970741  6019 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.971498  6019 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:01.973945  6031 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:01.974903 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:01.987169  6008 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:01.990811  5860 raft_consensus.cc:493] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:01.991770  5860 raft_consensus.cc:515] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:01.994765 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:01.996694 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.002799  6008 raft_consensus.cc:493] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.003820  6008 raft_consensus.cc:515] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:01.994858  5860 leader_election.cc:290] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
W20260709 13:17:02.024693 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.032420 21526 raft_consensus.cc:2243] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.037297 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.010924 30382 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 20, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:17:02.067997 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 156092us
I20260709 13:17:02.036437  6008 leader_election.cc:290] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:02.078358 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.087530 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97d1629ffbcd48d1b72c843365f37329" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.089532 30382 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:02.090502  6019 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.093500 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.092047 30382 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 13 to 18, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 18 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:02.097127 30382 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 19, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:02.098865 21526 pending_rounds.cc:70] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Trying to abort 2 pending ops.
I20260709 13:17:02.104337  5860 raft_consensus.cc:493] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:02.105211  5860 raft_consensus.cc:515] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:02.105764 21526 pending_rounds.cc:77] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 19 index: 11 } timestamp: 7305637968203034624 op_type: NO_OP noop_request { }
I20260709 13:17:02.106359 21526 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.106458  6106 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.106739 21526 pending_rounds.cc:77] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 18 index: 10 } timestamp: 7305637924889886720 op_type: NO_OP noop_request { }
I20260709 13:17:02.107051  6019 raft_consensus.cc:493] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.107365  6106 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.107903  5860 leader_election.cc:290] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:02.107743  6019 raft_consensus.cc:515] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.108220 30382 catalog_manager.cc:5697] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:02.107226 21526 raft_consensus.cc:2889] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.109886 21526 raft_consensus.cc:2272] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.111573  6019 leader_election.cc:290] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.113176  6106 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.114711 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.115444 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.116153 21526 raft_consensus.cc:2243] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.116994 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.117167 21526 raft_consensus.cc:2272] T 26f2e1d0fad840b5b8718f931fc92316 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.117874 30699 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 21.
I20260709 13:17:02.121891 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.123298 31292 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:17:02.134105  6106 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:17:02.134825  6106 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.124678 31294 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:17:02.135370  6106 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:17:02.136677 31294 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.123564 21526 raft_consensus.cc:2243] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:02.123952 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 124348us
I20260709 13:17:02.137614  5860 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:02.135855  6019 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:17:02.138756  6019 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.139216  6019 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:02.140386 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.142089 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "496aec2d89db4dbe892cb6d96b075225" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.131698 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 19 to 24, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 24 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.150484 30382 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:02.152156 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.162837 31294 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.163146 30632 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
W20260709 13:17:02.163635 31294 raft_consensus.cc:2686] T 496aec2d89db4dbe892cb6d96b075225 P 015ca80eba674552b34856a4f20271f1: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:17:02.154600 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.150746  5860 raft_consensus.cc:493] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:02.154704 21526 raft_consensus.cc:2272] T f029f5d5b1ee4391a2b10e12ec33b844 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.165105  5860 raft_consensus.cc:515] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.168046  5860 leader_election.cc:290] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:02.170703 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 86369.6us
I20260709 13:17:02.170325 30382 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:02.174176 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.181914 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b916ed19e94ddb8e8a2bca582ed0c6" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.187731 30382 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:02.189162 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f217397202e476a967c268887497cf5" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.194094 31294 leader_election.cc:304] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.205358 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4bc9ab09182445eaeceb5ea60a2ebfa" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.205005 30382 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 19 to 22, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:02.206926  6106 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.208035 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1265e789b7c41d7ba82825a0c126517" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.209271  6106 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.210008  6106 raft_consensus.cc:2749] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:02.210995 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dc8bff1e9b745c7a95ecd1b6306719d" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.199141 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.203871 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1d0922dea2d4414ca94fe5f85f65ca8b" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:02.213819 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.212309 30706 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:02.213876 30631 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:17:02.215163 30706 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:17:02.202308  6019 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.215306 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 12 to 15, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 15 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:02.217500 30554 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:02.218246 30554 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:17:02.201781 30522 consensus_peers.cc:597] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.232002 30382 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:02.243698 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 16 to 18, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 18 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:02.239876 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:17:02.245750 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.246488  5813 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:17:02.247080  5813 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:02.247854 30559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:02.247891  5813 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:02.240434  6019 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.230186 30554 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:17:02.244320  5860 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.252128  5860 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.253432 31292 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.254973  5860 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.255626  5860 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:02.196780 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.257148 31292 leader_election.cc:304] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.257388 21526 raft_consensus.cc:2243] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.258483  6106 raft_consensus.cc:2749] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:02.258805 31294 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564; no voters: 
I20260709 13:17:02.259563 21526 raft_consensus.cc:2272] T f9117b9966b14416ba3ee00bd7424f65 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.259826  6106 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:17:02.257942 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.259950 31292 leader_election.cc:304] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.260645 30382 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 reported cstate change: term changed from 7 to 21, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 21 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.264492 31292 leader_election.cc:304] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.263028  6106 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:02.266893  6019 raft_consensus.cc:2749] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:02.267755  6063 raft_consensus.cc:2749] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:17:02.266731  6106 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.270659 31292 leader_election.cc:304] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.271688  6063 raft_consensus.cc:2749] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:02.282922 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:02.283699 30632 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:02.285593 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "742a437d7cdc48f58f38b6c6a39ce5ae" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:02.300809 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.301815 31292 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.303002  6063 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:02.315441 30706 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 22.
I20260709 13:17:02.332114  5813 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.333675 30559 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 015ca80eba674552b34856a4f20271f1 in current term 23: Already voted for candidate 5a03a8214bba4eb1b0ff00895c565564 in this term.
I20260709 13:17:02.334913 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.336175 21526 raft_consensus.cc:2243] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.347152 21526 raft_consensus.cc:2272] T 6f217397202e476a967c268887497cf5 P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.351804 30632 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 015ca80eba674552b34856a4f20271f1 in term 23.
W20260709 13:17:02.348697 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.362169 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.363665 31292 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:02.365882  6019 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:17:02.366489  6019 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Becoming Leader. State: Replica: 015ca80eba674552b34856a4f20271f1, State: Running, Role: LEADER
I20260709 13:17:02.368088  6019 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.388314  5813 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.389846 21526 raft_consensus.cc:2243] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:02.393186 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.395085 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:02.396260 30628 raft_consensus.cc:2393] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 5a03a8214bba4eb1b0ff00895c565564 in current term 23: Already voted for candidate 015ca80eba674552b34856a4f20271f1 in this term.
I20260709 13:17:02.398408 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:02.399755 21526 raft_consensus.cc:2272] T 54da309ba151423ab2739966b9caebd0 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.400568  5813 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:17:02.401265 30522 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.433168 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.443176 21526 raft_consensus.cc:2243] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.444219 21526 raft_consensus.cc:2272] T 22a37ee3d67e45c394c7f5963e5767ed P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.447094 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.448756 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.450347 21526 raft_consensus.cc:2243] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:17:02.459813 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:02.474102 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.477707  5407 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.484298  5407 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.485285 21526 pending_rounds.cc:70] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:02.485747 21526 pending_rounds.cc:77] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 23 index: 11 } timestamp: 7305637979629142016 op_type: NO_OP noop_request { }
I20260709 13:17:02.486091 21526 raft_consensus.cc:2889] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.487227  5407 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.488516 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:17:02.499449 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.500029 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:02.502305 21526 raft_consensus.cc:2272] T 9fd45f50776e4101a900943b8df9353c P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.502818  5407 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:02.509325 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.515009 21526 raft_consensus.cc:2243] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.516108 21526 raft_consensus.cc:2272] T 536e99f894744c35a0490416a32a875e P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.523193  6019 raft_consensus.cc:493] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:02.524006  6019 raft_consensus.cc:515] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.527580 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.528519  6019 leader_election.cc:290] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.529680 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf65c2e5cf5b41a2956231a0868f2a1f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.531172 31294 leader_election.cc:304] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.532508  6019 raft_consensus.cc:2749] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:02.534703 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.536106 21526 raft_consensus.cc:2243] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.537135 21526 raft_consensus.cc:2272] T d4bc9ab09182445eaeceb5ea60a2ebfa P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.584200 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.581210 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.585820 21526 raft_consensus.cc:2243] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.587155 21526 raft_consensus.cc:2272] T 0dc8bff1e9b745c7a95ecd1b6306719d P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.591981 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.598456 21526 raft_consensus.cc:2243] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.600037 21526 raft_consensus.cc:2272] T b15e4d74180b49daad15cc459dde3181 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.604280 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.605394 21526 raft_consensus.cc:2243] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.606344 21526 raft_consensus.cc:2272] T a1265e789b7c41d7ba82825a0c126517 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.618846 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:02.629522 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.638150 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.639500 21526 raft_consensus.cc:2243] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.640630 21526 pending_rounds.cc:70] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:02.641009 21526 pending_rounds.cc:77] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 20 index: 11 } timestamp: 7305637903481356288 op_type: NO_OP noop_request { }
I20260709 13:17:02.641515 21526 raft_consensus.cc:2889] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.641954 21526 raft_consensus.cc:2272] T 125a8386fffd41c0b18a3e6e0690283b P 015ca80eba674552b34856a4f20271f1 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.647428 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:02.657166 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.672938  6092 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.673797  6092 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.677572 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:02.678025 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.678453 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:17:02.679559 21526 raft_consensus.cc:2243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.680923 21526 raft_consensus.cc:2272] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.679490  6092 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.683346 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
W20260709 13:17:02.687420 30522 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.688390 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.689828 21526 raft_consensus.cc:2243] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.691224 21526 raft_consensus.cc:2272] T b85a1f7a06b642de885b159b7f7bba1e P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.695268 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.696507 21526 raft_consensus.cc:2243] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.697616 21526 raft_consensus.cc:2272] T 3410292026774f0ca95a2c77eab3a8b4 P 015ca80eba674552b34856a4f20271f1 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.701833 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.701880 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.702539 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.703572 21526 raft_consensus.cc:2243] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.706600 21526 raft_consensus.cc:2272] T c4358103a0de4d8d88d2fa7c86875470 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.709184 30699 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:17:02.711736  6019 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:17:02.713677 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.717625  6092 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:17:02.719776  6092 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.720201  6092 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:02.715058 21526 raft_consensus.cc:2243] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.731629  6019 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.732427  6019 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.735561  6019 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.736148 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.736987 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.731076 21526 pending_rounds.cc:70] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:02.738952 21526 pending_rounds.cc:77] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 29 index: 15 } timestamp: 7305637946956042240 op_type: NO_OP noop_request { }
I20260709 13:17:02.739367 21526 raft_consensus.cc:2889] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.739176 31292 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.739869 21526 raft_consensus.cc:2272] T 07fae2111d284c78bf054bd74aee6b94 P 015ca80eba674552b34856a4f20271f1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.741967  6019 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:02.743901  6019 raft_consensus.cc:493] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.745935  5806 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.746670  5806 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.749395  5806 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.750753 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.755566 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:02.755959 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.756592 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:02.756938 21526 raft_consensus.cc:2243] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.744634  6019 raft_consensus.cc:515] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.759002  6070 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:17:02.760885  6019 leader_election.cc:290] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.761685 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.762550 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "37a97221d4ba4f889ce430d3dadc5754" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.765089 31292 leader_election.cc:304] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.767139  6106 raft_consensus.cc:2749] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:02.769675 21526 raft_consensus.cc:2272] T 471d4059785144ffa1ac2459a1f183fb P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.769997  6092 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.773949 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d847606257cc4750be11b056651106a6" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:02.774658 30632 raft_consensus.cc:3060] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:17:02.776611 30522 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.778594 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.779685 21526 raft_consensus.cc:2243] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.783584 30632 raft_consensus.cc:2468] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:17:02.785246 30523 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:02.786567  5943 raft_consensus.cc:2804] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:17:02.787408  5943 raft_consensus.cc:697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:17:02.788762  5943 consensus_queue.cc:237] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:02.794411 21526 pending_rounds.cc:70] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Trying to abort 2 pending ops.
I20260709 13:17:02.795025 21526 pending_rounds.cc:77] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 26 index: 16 } timestamp: 7305637947723083776 op_type: NO_OP noop_request { }
I20260709 13:17:02.795560 21526 raft_consensus.cc:2889] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.795960 21526 pending_rounds.cc:77] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 24 index: 15 } timestamp: 7305637898204524544 op_type: NO_OP noop_request { }
I20260709 13:17:02.796442 21526 raft_consensus.cc:2889] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.796846 21526 raft_consensus.cc:2272] T 907e974a334846208c70f0b813199df0 P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.800742 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.802539  5813 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.811007  5813 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.813724  5813 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:02.816956 30522 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:02.821015 21526 raft_consensus.cc:2243] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:02.805009  6092 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:02.822950 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.823951 30699 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 18.
I20260709 13:17:02.828814 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:02.822057 21526 pending_rounds.cc:70] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:02.830986 21526 pending_rounds.cc:77] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 22 index: 11 } timestamp: 7305637931369779200 op_type: NO_OP noop_request { }
I20260709 13:17:02.831441 21526 raft_consensus.cc:2889] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:02.831831 21526 raft_consensus.cc:2272] T 37a97221d4ba4f889ce430d3dadc5754 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:02.833853  5813 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:17:02.834398  5813 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:02.834825  5813 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:02.835093  6106 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:02.835695  6106 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.843940 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:02.845563 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c351eef49bfd49568fd63877d05eab98" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 27 candidate_status { last_received { term: 24 index: 11 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:02.853595 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:02.855182  5813 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.858397 31294 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:02.857373  6106 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:02.874012  6019 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:02.893905  5813 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:02.898470 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87824e2fdc9d4d33885d41c010f7bec4" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:02.899399 30699 raft_consensus.cc:3060] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:17:02.917212 30699 raft_consensus.cc:2468] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 19.
W20260709 13:17:02.910780 30522 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:02.919332 31294 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 61110.3us
I20260709 13:17:02.921548 30557 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Refusing update from remote peer 015ca80eba674552b34856a4f20271f1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:17:02.924592 30523 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:02.926236  5813 raft_consensus.cc:2804] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:17:02.926910  5813 raft_consensus.cc:697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:17:02.927924  5813 consensus_queue.cc:237] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:02.957609 21526 raft_consensus.cc:2243] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:02.958703  6120 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:17:02.977779 30382 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:35874 (request call id 74) took 18279 ms (client timeout 15000 ms). Trace:
I20260709 13:17:02.941548 30374 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 10 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 16 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:02.983139 21526 raft_consensus.cc:2272] T ef183e787ad14ee6bef37037a6ba59c0 P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:02.991572 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:02.978518 30382 rpcz_store.cc:269] 0709 13:16:44.698391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:44.698730 (+   339us) service_pool.cc:224] Handling call
0709 13:17:02.977638 (+18278908us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:02.389690 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:02.395061 (+  5371us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:02.395123 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:17:02.395147 (+    24us) tablet.cc:662] Decoding operations
0709 13:17:02.404724 (+  9577us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:02.404870 (+   146us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:02.404905 (+    35us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:17:02.409448 (+  4543us) tablet.cc:701] Row locks acquired
0709 13:17:02.409505 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:17:02.409735 (+   230us) write_op.cc:270] Start()
0709 13:17:02.409934 (+   199us) write_op.cc:276] Timestamp: P: 1783603022409656 usec, L: 0
0709 13:17:02.409976 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:17:02.411816 (+  1840us) log.cc:844] Serialized 10544 byte log entry
0709 13:17:02.547306 (+135490us) op_driver.cc:464] REPLICATION: finished
0709 13:17:02.607890 (+ 60584us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b1800271960. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180ccb8de1 00007f180ccc55b1 00007f1803df03f3 00007f1803df27c1 00007f1803df2779 00007f1803df2709 00007f1803df26d1 00007f1803df19cd 000000000038f354 000000000038f199 00007f1803dd383a
0709 13:17:02.608094 (+   204us) write_op.cc:301] APPLY: starting
0709 13:17:02.608205 (+   111us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:02.704057 (+ 95852us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:02.704113 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:02.768090 (+ 63977us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:02.768633 (+   543us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:17:02.768750 (+   117us) write_op.cc:312] APPLY: finished
0709 13:17:02.790689 (+ 21939us) log.cc:844] Serialized 235 byte log entry
0709 13:17:02.793675 (+  2986us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:02.793977 (+   302us) write_op.cc:454] Released schema lock
0709 13:17:02.794071 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:02.807127 (+ 13056us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:02.807162 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1582048,"reported_tablets":45,"child_traces":[["op",{"apply.queue_time_us":27603,"apply.run_cpu_time_us":41969,"apply.run_wall_time_us":198986,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117257,"num_ops":27,"prepare.queue_time_us":1704,"prepare.run_cpu_time_us":70647,"prepare.run_wall_time_us":94744,"raft.queue_time_us":55666,"raft.run_cpu_time_us":2612,"raft.run_wall_time_us":27397,"replication_time_us":137126,"spinlock_wait_cycles":72688896,"thread_start_us":146117,"threads_started":4,"wal-append.queue_time_us":72759}]]}
W20260709 13:17:03.000638 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:02.981590 30374 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 14 to 17, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 17 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.027833  6120 raft_consensus.cc:493] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:03.028599  6120 raft_consensus.cc:515] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:03.030510 30374 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 10 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.032294 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.046973 21526 raft_consensus.cc:2243] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.048007 21526 pending_rounds.cc:70] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:03.048271 21526 pending_rounds.cc:77] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 24 index: 11 } timestamp: 7305637901135515648 op_type: NO_OP noop_request { }
I20260709 13:17:03.048628 21526 raft_consensus.cc:2889] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.049005 21526 raft_consensus.cc:2272] T c351eef49bfd49568fd63877d05eab98 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.062278  6120 leader_election.cc:290] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:03.067026 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:03.071419 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3511f2706d78410aa91eb654ab345803" candidate_uuid: "015ca80eba674552b34856a4f20271f1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:03.081523 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.082940 21526 raft_consensus.cc:2243] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.087877 30374 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 9 to 14, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 14 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.093988 31292 leader_election.cc:304] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 015ca80eba674552b34856a4f20271f1; no voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627
W20260709 13:17:03.105674 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:03.124317 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:03.123059  6120 raft_consensus.cc:2749] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:03.131597 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:03.139261 21526 raft_consensus.cc:2272] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.158159 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.159408 21526 raft_consensus.cc:2243] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.160264 21526 pending_rounds.cc:70] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:03.170373 21526 pending_rounds.cc:77] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 23 index: 15 } timestamp: 7305637929478803456 op_type: NO_OP noop_request { }
I20260709 13:17:03.174283 21526 raft_consensus.cc:2889] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.174860 21526 raft_consensus.cc:2272] T 4b674451b4d5465d82d8ef4f14fb52f5 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.195971 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.197245 21526 raft_consensus.cc:2243] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.198105 21526 raft_consensus.cc:2272] T bf65c2e5cf5b41a2956231a0868f2a1f P 015ca80eba674552b34856a4f20271f1 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.202090 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.203153 21526 raft_consensus.cc:2243] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.203946 21526 raft_consensus.cc:2272] T 3511f2706d78410aa91eb654ab345803 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.245669 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.251754 21526 raft_consensus.cc:2243] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.252700 21526 raft_consensus.cc:2272] T 97d1629ffbcd48d1b72c843365f37329 P 015ca80eba674552b34856a4f20271f1 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.275640 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 50917.9us
I20260709 13:17:03.297796 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.300508 21526 raft_consensus.cc:2243] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.302359 21526 raft_consensus.cc:2272] T bdaa99b885784af3b7e73563f113185a P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.304378 30522 consensus_peers.cc:597] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.343870 30632 raft_consensus.cc:1275] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:17:03.352677 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69820.6us
W20260709 13:17:03.354101 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 58261.1us
I20260709 13:17:03.357977  5813 consensus_queue.cc:1048] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
W20260709 13:17:03.373920 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 69620.1us
I20260709 13:17:03.384588 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.385938 21526 raft_consensus.cc:2243] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.390051 21526 raft_consensus.cc:2272] T 2b685fd119d941538e6fee8dc8f8c930 P 015ca80eba674552b34856a4f20271f1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.393910 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.426555 21526 raft_consensus.cc:2243] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:03.429071 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 46164.5us
W20260709 13:17:03.438838 30522 consensus_peers.cc:597] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:03.439178 21526 pending_rounds.cc:70] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:03.439744 21526 pending_rounds.cc:77] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 23 index: 12 } timestamp: 7305637929469673472 op_type: NO_OP noop_request { }
I20260709 13:17:03.440102 21526 raft_consensus.cc:2889] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.440459 21526 raft_consensus.cc:2272] T 9657733894c94f86ac3b047814fb8fd9 P 015ca80eba674552b34856a4f20271f1 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.464509 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.467741 21526 raft_consensus.cc:2243] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.468717 21526 pending_rounds.cc:70] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:03.469040 21526 pending_rounds.cc:77] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 27 index: 14 } timestamp: 7305637931907108864 op_type: NO_OP noop_request { }
I20260709 13:17:03.469379 21526 raft_consensus.cc:2889] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.469408 30707 raft_consensus.cc:1275] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:17:03.469893 21526 raft_consensus.cc:2272] T 1d0922dea2d4414ca94fe5f85f65ca8b P 015ca80eba674552b34856a4f20271f1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.471169  5943 consensus_queue.cc:1048] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:17:03.485559 30374 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 160) took 39264 ms (client timeout 15000 ms). Trace:
W20260709 13:17:03.495752 30374 rpcz_store.cc:269] 0709 13:16:24.221114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:24.221425 (+   311us) service_pool.cc:224] Handling call
0709 13:17:03.485426 (+39264001us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:03.184809 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:03.185501 (+   692us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:03.185546 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:17:03.185571 (+    25us) tablet.cc:662] Decoding operations
0709 13:17:03.186283 (+   712us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:03.186381 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:03.186415 (+    34us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:17:03.186932 (+   517us) tablet.cc:701] Row locks acquired
0709 13:17:03.186954 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:17:03.187142 (+   188us) write_op.cc:270] Start()
0709 13:17:03.187284 (+   142us) write_op.cc:276] Timestamp: P: 1783603023187083 usec, L: 0
0709 13:17:03.187309 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:03.188182 (+   873us) log.cc:844] Serialized 1693 byte log entry
0709 13:17:03.274548 (+ 86366us) op_driver.cc:464] REPLICATION: finished
0709 13:17:03.275117 (+   569us) write_op.cc:301] APPLY: starting
0709 13:17:03.275212 (+    95us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:03.277736 (+  2524us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:03.277836 (+   100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:03.289852 (+ 12016us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:03.290149 (+   297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:17:03.290220 (+    71us) write_op.cc:312] APPLY: finished
0709 13:17:03.290590 (+   370us) log.cc:844] Serialized 48 byte log entry
0709 13:17:03.308822 (+ 18232us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:03.308987 (+   165us) write_op.cc:454] Released schema lock
0709 13:17:03.309067 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:03.314691 (+  5624us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:03.314731 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":271,"apply.run_cpu_time_us":7950,"apply.run_wall_time_us":39453,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16760,"num_ops":4,"prepare.queue_time_us":11452,"prepare.run_cpu_time_us":3758,"prepare.run_wall_time_us":3760,"replication_time_us":86705,"thread_start_us":8185,"threads_started":1}]]}
W20260709 13:17:03.525202 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:03.494647 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.542448 21526 raft_consensus.cc:2243] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.554471 21526 pending_rounds.cc:70] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Trying to abort 1 pending ops.
I20260709 13:17:03.558214 21526 pending_rounds.cc:77] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 25 index: 13 } timestamp: 7305637918363054080 op_type: NO_OP noop_request { }
I20260709 13:17:03.558889 21526 raft_consensus.cc:2889] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.559471 21526 raft_consensus.cc:2272] T 742a437d7cdc48f58f38b6c6a39ce5ae P 015ca80eba674552b34856a4f20271f1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.580646 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.581940 21526 raft_consensus.cc:2243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.611258 21526 pending_rounds.cc:70] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Trying to abort 2 pending ops.
I20260709 13:17:03.614880 21526 pending_rounds.cc:77] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 26 index: 14 } timestamp: 7305637928701538304 op_type: NO_OP noop_request { }
I20260709 13:17:03.615566 21526 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.616017 21526 pending_rounds.cc:77] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1: Aborting op as it isn't in flight: id { term: 24 index: 13 } timestamp: 7305637898150191104 op_type: NO_OP noop_request { }
I20260709 13:17:03.616489 21526 raft_consensus.cc:2889] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:03.616943 21526 raft_consensus.cc:2272] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 015ca80eba674552b34856a4f20271f1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.628952 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.638201 21526 raft_consensus.cc:2243] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:03.639303 21526 raft_consensus.cc:2272] T 2cbf727a8c7b4599bb21be645db6983f P 015ca80eba674552b34856a4f20271f1 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:03.643574 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:03.659825 21526 raft_consensus.cc:2243] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 23 LEADER]: Raft consensus shutting down.
W20260709 13:17:03.671615 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:03.635349 30522 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:03.686892 30522 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 51623.9us
I20260709 13:17:03.687551 21526 raft_consensus.cc:2272] T fd1749966f254b6a8c54ddfd73cb8d14 P 015ca80eba674552b34856a4f20271f1 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:03.708679 30375 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 159) took 65392 ms (client timeout 15000 ms). Trace:
I20260709 13:17:03.720417  5943 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.721202  5943 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:03.725142 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
W20260709 13:17:03.721167 30375 rpcz_store.cc:269] 0709 13:15:58.316140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:58.317135 (+   995us) service_pool.cc:224] Handling call
0709 13:17:03.708552 (+65391417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11894,"reported_tablets":44}
I20260709 13:17:03.727522  5943 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:03.728992 30522 leader_election.cc:336] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.729661 30522 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:03.732362  5813 raft_consensus.cc:2749] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:03.795396  5407 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.796051  5407 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:03.803655 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:03.805291 30376 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 146) took 69301 ms (client timeout 15000 ms). Trace:
W20260709 13:17:03.805991 30376 rpcz_store.cc:269] 0709 13:15:54.504270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:54.512569 (+  8299us) service_pool.cc:224] Handling call
0709 13:17:03.805148 (+69292579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":816961,"reported_tablets":44}
I20260709 13:17:03.808830 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:03.809706 30557 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 23.
I20260709 13:17:03.811129 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:03.811044 30377 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.812902  5919 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:17:03.813393  5919 raft_consensus.cc:493] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.813877  5919 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:17:03.813184 30377 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.818692  5407 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:03.819442 30596 leader_election.cc:336] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.831660 30377 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.834143 30377 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 16 to 18, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 18 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.835320  5919 raft_consensus.cc:515] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:03.835803 30377 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:03.837589 30377 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 17 to 20, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.852093  5919 leader_election.cc:290] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:03.852933  5919 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.853322 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9117b9966b14416ba3ee00bd7424f65" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564"
I20260709 13:17:03.853427  5919 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:03.854270 30557 raft_consensus.cc:3060] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:17:03.855983  5919 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:03.859982 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:03.860437 30377 catalog_manager.cc:5697] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 14 to 21, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:17:03.861730 30596 leader_election.cc:336] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.862215 30377 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:03.864485 30377 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 20 to 21, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 21 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:03.875418 30596 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.876128 30596 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:03.877938  6124 raft_consensus.cc:2749] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:03.880206  5806 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.881067  5806 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:03.882129 30522 consensus_peers.cc:597] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:03.888334 30377 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 16 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:03.892416 30377 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 15 to 20. New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.894773 30377 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:03.889106 30557 raft_consensus.cc:2468] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:17:03.899190 30599 leader_election.cc:304] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
W20260709 13:17:03.900523 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:03.900676  5919 raft_consensus.cc:2804] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:17:03.901613  5919 raft_consensus.cc:697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:17:03.903002  5919 consensus_queue.cc:237] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:03.906427 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:03.907339  5806 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:03.908913 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:03.911630  6131 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:03.938686 30522 consensus_peers.cc:597] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:03.972776  6070 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:03.973592  6070 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:03.979225  6070 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:03.982337 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:03.997462 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.002561 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:04.017177 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:04.018675  5806 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:04.029067 30522 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.041843  5943 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.042713  5943 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:04.053252  5943 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.066545 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:04.067703 30699 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 26.
I20260709 13:17:04.071601 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:17:04.073660  5813 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:17:04.074254  5813 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.074801  5813 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:17:04.079185  6124 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.080310  6124 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:04.088457 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:17:04.089421  6124 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:04.093940 30522 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:17:04.114017 30522 consensus_peers.cc:597] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:04.119920 30597 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.132:35425, user_credentials={real_user=slave}} blocked reactor thread for 68039.3us
I20260709 13:17:04.123107 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:04.123950 30699 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 24.
I20260709 13:17:04.129232  6131 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.129983  6131 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:04.132139 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:04.138890  5407 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:17:04.139374  5407 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.139722  5407 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:17:04.140322  6131 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.144573 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:04.145401 30628 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 26.
W20260709 13:17:04.148941 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.150041 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:04.156780  5806 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:17:04.181329  5407 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:04.160873 30670 connection.cc:573] client connection to 127.21.5.129:41021 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:17:04.185313 30670 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:17:04.178601  5806 raft_consensus.cc:493] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.189550  5806 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:17:04.195092 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "471d4059785144ffa1ac2459a1f183fb" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:04.195914 30699 raft_consensus.cc:3060] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:17:04.191669  5813 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:04.199848  5813 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.201638 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4358103a0de4d8d88d2fa7c86875470" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:04.202343 30707 raft_consensus.cc:3060] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:17:04.204430  5407 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:04.205715 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 44101.8us
I20260709 13:17:04.218250  5806 raft_consensus.cc:515] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:04.232481 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b85a1f7a06b642de885b159b7f7bba1e" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:04.233470 30628 raft_consensus.cc:3060] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:17:04.237426  5806 leader_election.cc:290] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 election: Requested vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.253424 30707 raft_consensus.cc:2468] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 27.
I20260709 13:17:04.259315 30699 raft_consensus.cc:2468] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf852f85f47483d99cbe4020baee627 in term 25.
I20260709 13:17:04.266126 30628 raft_consensus.cc:2468] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 27.
I20260709 13:17:04.265336 30597 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:04.267416 30523 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:17:04.271953  6124 raft_consensus.cc:2804] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:17:04.272714  6124 raft_consensus.cc:697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Becoming Leader. State: Replica: acf852f85f47483d99cbe4020baee627, State: Running, Role: LEADER
I20260709 13:17:04.281551 30671 leader_election.cc:304] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:04.283784  5806 raft_consensus.cc:2804] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:17:04.284456  5806 raft_consensus.cc:697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 27 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:17:04.280817  5943 raft_consensus.cc:2804] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:17:04.290870  6124 consensus_queue.cc:237] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:04.280714  6129 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.299405  6129 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:04.301790  6129 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:04.304495 30557 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
W20260709 13:17:04.308955 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.310717  5806 consensus_queue.cc:237] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:04.312359 30557 raft_consensus.cc:1275] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:17:04.329474 30596 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.358856  5407 consensus_queue.cc:1048] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:17:04.350404  5943 raft_consensus.cc:697] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:17:04.403304 30670 leader_election.cc:336] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:04.408241 30522 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:04.418566 30522 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:04.433169 30522 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:04.434162 30377 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 148) took 24528 ms (client timeout 15000 ms). Trace:
W20260709 13:17:04.434842 30377 rpcz_store.cc:269] 0709 13:16:39.887247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:39.887700 (+   453us) service_pool.cc:224] Handling call
0709 13:17:04.415540 (+24527840us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:03.975007 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:03.975624 (+   617us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:03.975667 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:17:03.975685 (+    18us) tablet.cc:662] Decoding operations
0709 13:17:03.979800 (+  4115us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:03.979908 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:03.979938 (+    30us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:17:03.981197 (+  1259us) tablet.cc:701] Row locks acquired
0709 13:17:03.981218 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:17:03.981392 (+   174us) write_op.cc:270] Start()
0709 13:17:03.981536 (+   144us) write_op.cc:276] Timestamp: P: 1783603023981334 usec, L: 0
0709 13:17:03.981568 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:17:04.015018 (+ 33450us) log.cc:844] Serialized 4773 byte log entry
0709 13:17:04.030201 (+ 15183us) op_driver.cc:464] REPLICATION: finished
0709 13:17:04.068650 (+ 38449us) write_op.cc:301] APPLY: starting
0709 13:17:04.068763 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:04.184347 (+115584us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:04.184559 (+   212us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:04.260216 (+ 75657us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:04.260671 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=24,mrs_lookups=0
0709 13:17:04.260746 (+    75us) write_op.cc:312] APPLY: finished
0709 13:17:04.261305 (+   559us) log.cc:844] Serialized 112 byte log entry
0709 13:17:04.269092 (+  7787us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.271009 (+  1917us) write_op.cc:454] Released schema lock
0709 13:17:04.271091 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:04.271605 (+   514us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.271644 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":223806,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":38116,"apply.run_cpu_time_us":23026,"apply.run_wall_time_us":202926,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54407,"num_ops":12,"prepare.queue_time_us":55710,"prepare.run_cpu_time_us":5148,"prepare.run_wall_time_us":44032,"replication_time_us":48430,"thread_start_us":86907,"threads_started":2}]]}
I20260709 13:17:04.468102  5813 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:17:04.468716 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 64150.7us
I20260709 13:17:04.468853  5813 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:04.471609  5813 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.473580 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:04.474435 30707 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 22.
I20260709 13:17:04.475740 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
W20260709 13:17:04.473745 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.476660 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:04.477671  6140 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:17:04.478209  5806 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:04.478238  6140 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.479135  6140 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:17:04.484254 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:04.488195 30379 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 147) took 42072 ms (client timeout 15000 ms). Trace:
W20260709 13:17:04.488766 30379 rpcz_store.cc:269] 0709 13:16:22.415695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:22.415995 (+   300us) service_pool.cc:224] Handling call
0709 13:17:04.487939 (+42071944us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":44}
I20260709 13:17:04.493167  6140 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:04.494879 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:04.495756  5943 consensus_queue.cc:237] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:04.503998 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef183e787ad14ee6bef37037a6ba59c0" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f"
I20260709 13:17:04.504801 30699 raft_consensus.cc:3060] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:04.505168  6140 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Requested vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:04.523077 30699 raft_consensus.cc:2468] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
W20260709 13:17:04.533866 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.540946 30523 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f; no voters: 
I20260709 13:17:04.542174  5813 raft_consensus.cc:2804] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:17:04.542990  5813 raft_consensus.cc:697] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
W20260709 13:17:04.545439 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.545214  5813 consensus_queue.cc:237] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:04.547385 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:04.548110 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.554914  6145 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:04.555665  6145 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:04.558188 30522 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.571079  6145 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:04.580763 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:04.585306 30380 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 158) took 87366 ms (client timeout 15000 ms). Trace:
W20260709 13:17:04.609620 30380 rpcz_store.cc:269] 0709 13:15:37.219173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:37.219436 (+   263us) service_pool.cc:224] Handling call
0709 13:15:37.899445 (+680009us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54000f32d4. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1806e47090 00007f1806ee5f3a 00007f180cc9c1cc 00007f180cd9a37e 00007f180b25b2a4 00007f180b25b232 00007f180b25b161 00007f180b25b0dc 00007f180b25a392 00007f180521848c 00007f18052178c6
0709 13:17:04.585062 (+86685617us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18449,"reported_tablets":44,"spinlock_wait_cycles":1134793984}
W20260709 13:17:04.615430 30523 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 55930.7us
I20260709 13:17:04.619167 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:04.620003 30628 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:17:04.624603 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:04.625818  6145 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:17:04.626151 30522 consensus_peers.cc:597] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.626314  6145 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:04.627133  6145 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:17:04.685096  6139 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:04.685967  6139 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:04.675900  6145 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:04.692236  6145 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:04.693492 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fd45f50776e4101a900943b8df9353c" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:04.694337 30628 raft_consensus.cc:3060] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:17:04.695294 30378 catalog_manager.cc:5697] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 25, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 25 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:04.697491 30522 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.698714 30378 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.701152 30378 catalog_manager.cc:5697] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 19 to 22, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:04.707340 30699 raft_consensus.cc:1275] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Refusing update from remote peer acf852f85f47483d99cbe4020baee627: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 13:17:04.714895 30378 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 18 to 22, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 22 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:04.718007  5407 consensus_queue.cc:1048] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:17:04.719877 30381 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 157) took 107487 ms (client timeout 15000 ms). Trace:
W20260709 13:17:04.720693 30381 rpcz_store.cc:269] 0709 13:15:17.222874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:15:17.224518 (+  1644us) service_pool.cc:224] Handling call
0709 13:17:04.710097 (+107485579us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11806,"reported_tablets":44}
I20260709 13:17:04.717718 30378 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 21 to 22, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:17:04.725524 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:04.723796 30378 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 14 to 16, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.730866 30378 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:04.733197  6139 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:04.732652 30378 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 14 to 16, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 16 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:04.735388 30552 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5a03a8214bba4eb1b0ff00895c565564" is_pre_election: true
I20260709 13:17:04.739142 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:04.746270  6139 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:04.789350 30628 raft_consensus.cc:2468] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:17:04.791714 30523 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:04.793529  6145 raft_consensus.cc:2804] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:17:04.794190  6145 raft_consensus.cc:697] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:17:04.797053  6145 consensus_queue.cc:237] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:04.797681 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:04.840916 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:04.862007 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:04.900883 30707 raft_consensus.cc:1275] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 13:17:04.907619  6154 consensus_queue.cc:1048] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:17:04.915850 30378 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 139) took 16018 ms (client timeout 15000 ms). Trace:
I20260709 13:17:04.916785 30373 catalog_manager.cc:5697] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 20 to 21, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:04.919502 30373 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 19 to 23, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 23 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.921689 30373 catalog_manager.cc:5697] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:04.939440 30631 raft_consensus.cc:1275] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:17:04.945274  6070 consensus_queue.cc:1048] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:17:04.922844 30378 rpcz_store.cc:269] 0709 13:16:48.897510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:16:51.316224 (+2418714us) service_pool.cc:224] Handling call
0709 13:17:04.915707 (+13599483us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:04.813323 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:04.813950 (+   627us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:04.813994 (+    44us) write_op.cc:435] Acquired schema lock
0709 13:17:04.814014 (+    20us) tablet.cc:662] Decoding operations
0709 13:17:04.814985 (+   971us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:04.815090 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:04.815129 (+    39us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:17:04.815827 (+   698us) tablet.cc:701] Row locks acquired
0709 13:17:04.815847 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:17:04.816032 (+   185us) write_op.cc:270] Start()
0709 13:17:04.816168 (+   136us) write_op.cc:276] Timestamp: P: 1783603024815976 usec, L: 0
0709 13:17:04.816194 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:17:04.824405 (+  8211us) log.cc:844] Serialized 3229 byte log entry
0709 13:17:04.832931 (+  8526us) op_driver.cc:464] REPLICATION: finished
0709 13:17:04.842180 (+  9249us) write_op.cc:301] APPLY: starting
0709 13:17:04.842298 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:04.867285 (+ 24987us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:04.867421 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:04.873437 (+  6016us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:04.873719 (+   282us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:17:04.873776 (+    57us) write_op.cc:312] APPLY: finished
0709 13:17:04.874155 (+   379us) log.cc:844] Serialized 80 byte log entry
0709 13:17:04.874397 (+   242us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.874549 (+   152us) write_op.cc:454] Released schema lock
0709 13:17:04.874636 (+    87us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:04.875097 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:04.875127 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19564,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":8943,"apply.run_cpu_time_us":13639,"apply.run_wall_time_us":32906,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37647,"num_ops":8,"prepare.queue_time_us":40823,"prepare.run_cpu_time_us":4037,"prepare.run_wall_time_us":11438,"replication_time_us":16534,"thread_start_us":26549,"threads_started":2}]]}
I20260709 13:17:04.957724 30373 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 18 to 20, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:04.961639 30373 catalog_manager.cc:5697] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 21 to 22, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:04.970181 30373 catalog_manager.cc:5697] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 20 to 21, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 21 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.979425 30373 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 17 to 23, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 23 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:04.992864 30373 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 23 to 27, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 27 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:05.079726 30373 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 161) took 15943 ms (client timeout 15000 ms). Trace:
W20260709 13:17:05.080945 30373 rpcz_store.cc:269] 0709 13:16:49.135936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:01.954803 (+12818867us) service_pool.cc:224] Handling call
0709 13:17:05.079394 (+3124591us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:05.011017 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:05.011803 (+   786us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:05.011867 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:17:05.011894 (+    27us) tablet.cc:662] Decoding operations
0709 13:17:05.013022 (+  1128us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:05.013093 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:05.013128 (+    35us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:17:05.013889 (+   761us) tablet.cc:701] Row locks acquired
0709 13:17:05.013910 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:17:05.014087 (+   177us) write_op.cc:270] Start()
0709 13:17:05.014237 (+   150us) write_op.cc:276] Timestamp: P: 1783603025014034 usec, L: 0
0709 13:17:05.014261 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.015296 (+  1035us) log.cc:844] Serialized 3233 byte log entry
0709 13:17:05.017980 (+  2684us) op_driver.cc:464] REPLICATION: finished
0709 13:17:05.018514 (+   534us) write_op.cc:301] APPLY: starting
0709 13:17:05.018643 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:05.023726 (+  5083us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:05.023847 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:05.029377 (+  5530us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:05.029630 (+   253us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 13:17:05.029695 (+    65us) write_op.cc:312] APPLY: finished
0709 13:17:05.030033 (+   338us) log.cc:844] Serialized 80 byte log entry
0709 13:17:05.034948 (+  4915us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.035130 (+   182us) write_op.cc:454] Released schema lock
0709 13:17:05.035196 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:05.042579 (+  7383us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.042658 (+    79us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":10485,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":307,"apply.run_cpu_time_us":12471,"apply.run_wall_time_us":17124,"cfile_cache_hit":32,"cfile_cache_hit_bytes":33520,"num_ops":8,"prepare.queue_time_us":11200,"prepare.run_cpu_time_us":4704,"prepare.run_wall_time_us":6878,"replication_time_us":3549}]]}
I20260709 13:17:05.084834 30707 raft_consensus.cc:1275] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:17:05.085083 30382 catalog_manager.cc:5697] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.087965  5813 consensus_queue.cc:1048] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:17:05.089269 30382 catalog_manager.cc:5697] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.091127 30382 catalog_manager.cc:5697] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 27 to 30, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 30 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:05.093830 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.094754 30382 catalog_manager.cc:5697] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.097209 30382 catalog_manager.cc:5697] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 18 to 26. New cstate: current_term: 26 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.099335 30382 catalog_manager.cc:5697] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:05.094811 30522 consensus_peers.cc:597] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:05.102600 30382 catalog_manager.cc:5697] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 24 to 27, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 27 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.104816 30382 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 18 to 19, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 19 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.107606 30382 catalog_manager.cc:5697] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 20 to 22, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.112047 30382 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 15 to 16, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 16 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.114050 30382 catalog_manager.cc:5697] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 22 to 23, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.115674 30382 catalog_manager.cc:5697] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 22 to 23, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.117170 30382 catalog_manager.cc:5697] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.119232 30382 catalog_manager.cc:5697] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 20 to 22, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 22 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.121392 30382 catalog_manager.cc:5697] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.124575 30382 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 19 to 22, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 22 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:05.125676 30382 catalog_manager.cc:5697] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
W20260709 13:17:05.129182 30522 consensus_peers.cc:597] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:05.134007 30522 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.135183 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.140218 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.144838  6145 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
W20260709 13:17:05.145423 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.145776  6145 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:05.148562  6145 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:05.151895 30628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:05.152652 30628 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 23.
I20260709 13:17:05.154012 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:05.155130  5813 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:17:05.155545  5813 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 015ca80eba674552b34856a4f20271f1)
I20260709 13:17:05.156026  5813 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:17:05.157002 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.200989 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 31423)
W20260709 13:17:05.221388 30522 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:05.222903 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.226140  5813 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:05.231127  5813 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Requested vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:05.232391 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.237545 30632 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd1749966f254b6a8c54ddfd73cb8d14" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:05.238380 30632 raft_consensus.cc:3060] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:17:05.241000 30522 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:05.241376 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.250324 30632 raft_consensus.cc:2468] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a03a8214bba4eb1b0ff00895c565564 in term 24.
I20260709 13:17:05.252851 30523 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1
I20260709 13:17:05.256565  6154 raft_consensus.cc:2804] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 13:17:05.259799 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.263319  6154 raft_consensus.cc:697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Becoming Leader. State: Replica: 5a03a8214bba4eb1b0ff00895c565564, State: Running, Role: LEADER
I20260709 13:17:05.264580  6154 consensus_queue.cc:237] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:05.271579 30632 raft_consensus.cc:1275] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 24 index: 11. (index mismatch)
I20260709 13:17:05.274663  6169 consensus_queue.cc:1048] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:17:05.295441 30522 consensus_peers.cc:597] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:05.366325 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.387742 30376 catalog_manager.cc:5697] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 26. New cstate: current_term: 26 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.390060 30376 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } }
I20260709 13:17:05.395980 30376 catalog_manager.cc:5697] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 19, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.398159 30376 catalog_manager.cc:5697] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 22 to 23, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.401087 30376 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 24 to 26, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 26 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.394747 30382 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58828 (request call id 149) took 8898 ms. Trace:
I20260709 13:17:05.415095 30382 rpcz_store.cc:276] 0709 13:16:56.534590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.005166 (+6470576us) service_pool.cc:224] Handling call
0709 13:17:05.394669 (+2389503us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:05.137010 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:05.137438 (+   428us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:05.137462 (+    24us) write_op.cc:435] Acquired schema lock
0709 13:17:05.137474 (+    12us) tablet.cc:662] Decoding operations
0709 13:17:05.138870 (+  1396us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:05.138952 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:05.138980 (+    28us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:17:05.140330 (+  1350us) tablet.cc:701] Row locks acquired
0709 13:17:05.140359 (+    29us) write_op.cc:260] PREPARE: finished
0709 13:17:05.140564 (+   205us) write_op.cc:270] Start()
0709 13:17:05.140700 (+   136us) write_op.cc:276] Timestamp: P: 1783603025140497 usec, L: 0
0709 13:17:05.140727 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.141788 (+  1061us) log.cc:844] Serialized 6694 byte log entry
0709 13:17:05.156836 (+ 15048us) op_driver.cc:464] REPLICATION: finished
0709 13:17:05.159978 (+  3142us) write_op.cc:301] APPLY: starting
0709 13:17:05.160088 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:05.226561 (+ 66473us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:05.226711 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:05.311301 (+ 84590us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:05.311607 (+   306us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:17:05.311659 (+    52us) write_op.cc:312] APPLY: finished
0709 13:17:05.312084 (+   425us) log.cc:844] Serialized 155 byte log entry
0709 13:17:05.326861 (+ 14777us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.327113 (+   252us) write_op.cc:454] Released schema lock
0709 13:17:05.327194 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:05.351707 (+ 24513us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.351745 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":2841,"apply.run_cpu_time_us":26275,"apply.run_wall_time_us":191694,"cfile_cache_hit":68,"cfile_cache_hit_bytes":71230,"num_ops":17,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":5056,"prepare.run_wall_time_us":19594,"replication_time_us":15876}]]}
I20260709 13:17:05.420224 30376 catalog_manager.cc:5697] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.422559 30376 catalog_manager.cc:5697] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.425809 30376 catalog_manager.cc:5697] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.427850 30376 catalog_manager.cc:5697] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.429765 30376 catalog_manager.cc:5697] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 20, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 20 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.432091 30376 catalog_manager.cc:5697] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 19 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 20 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.443811 30376 catalog_manager.cc:5697] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.445797 30376 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.447544 30376 catalog_manager.cc:5697] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 26, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 26 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.449415 30376 catalog_manager.cc:5697] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.451553 30376 catalog_manager.cc:5697] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.453646 30376 catalog_manager.cc:5697] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.455271 30376 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.457178 30376 catalog_manager.cc:5697] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 18 to 19, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 19 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.460093 30376 catalog_manager.cc:5697] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 20, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 20 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.461925 30376 catalog_manager.cc:5697] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 24, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 24 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.464203 30376 catalog_manager.cc:5697] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 25, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 25 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.466256 30376 catalog_manager.cc:5697] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.468142 30376 catalog_manager.cc:5697] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 23 to 26, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 26 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.470083 30376 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 20 to 24, leader changed from 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132) to 015ca80eba674552b34856a4f20271f1 (127.21.5.129). New cstate: current_term: 24 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
I20260709 13:17:05.471834 30376 catalog_manager.cc:5697] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 16 to 23, leader changed from acf852f85f47483d99cbe4020baee627 (127.21.5.131) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:05.493160 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.583762 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.601805 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:05.645592 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:05.651708 30376 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58838 (request call id 140) took 1678 ms. Trace:
W20260709 13:17:05.662885 30522 consensus_peers.cc:597] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:05.652305 30376 rpcz_store.cc:276] 0709 13:17:03.973736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:03.973903 (+   167us) service_pool.cc:224] Handling call
0709 13:17:05.651600 (+1677697us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:05.482399 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:05.483016 (+   617us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:05.483059 (+    43us) write_op.cc:435] Acquired schema lock
0709 13:17:05.483072 (+    13us) tablet.cc:662] Decoding operations
0709 13:17:05.484909 (+  1837us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:05.484969 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:05.484998 (+    29us) tablet.cc:685] Acquiring locks for 26 operations
0709 13:17:05.487151 (+  2153us) tablet.cc:701] Row locks acquired
0709 13:17:05.487171 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:17:05.487362 (+   191us) write_op.cc:270] Start()
0709 13:17:05.487509 (+   147us) write_op.cc:276] Timestamp: P: 1783603025487306 usec, L: 0
0709 13:17:05.487529 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.488452 (+   923us) log.cc:844] Serialized 10159 byte log entry
0709 13:17:05.492051 (+  3599us) op_driver.cc:464] REPLICATION: finished
0709 13:17:05.492586 (+   535us) write_op.cc:301] APPLY: starting
0709 13:17:05.492693 (+   107us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:05.506496 (+ 13803us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:05.506961 (+   465us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:05.585370 (+ 78409us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:05.585713 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=52,key_file_lookups=52,delta_file_lookups=52,mrs_lookups=0
0709 13:17:05.585758 (+    45us) write_op.cc:312] APPLY: finished
0709 13:17:05.586744 (+   986us) log.cc:844] Serialized 227 byte log entry
0709 13:17:05.586947 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.587215 (+   268us) write_op.cc:454] Released schema lock
0709 13:17:05.587295 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:05.587709 (+   414us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.587740 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20827,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":311,"apply.run_cpu_time_us":33201,"apply.run_wall_time_us":95048,"cfile_cache_hit":104,"cfile_cache_hit_bytes":108940,"num_ops":26,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":6454,"prepare.run_wall_time_us":24409,"replication_time_us":4334}]]}
I20260709 13:17:05.661767 30375 catalog_manager.cc:5697] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 20 to 22, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 22 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.685856 30375 catalog_manager.cc:5697] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 21 to 24, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 24 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:05.688024 30375 catalog_manager.cc:5697] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 20 to 21, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 21 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:05.692250 30375 catalog_manager.cc:5697] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 22 to 23, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 23 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
I20260709 13:17:05.695472 30375 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "015ca80eba674552b34856a4f20271f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } }
W20260709 13:17:05.782672 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:05.788203 30631 raft_consensus.cc:1275] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Refusing update from remote peer 5a03a8214bba4eb1b0ff00895c565564: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:17:05.790819  6154 consensus_queue.cc:1048] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:17:05.816133 30376 catalog_manager.cc:5697] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 26 to 27, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 27 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:05.818122 30376 catalog_manager.cc:5697] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 22 to 23, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 23 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.819514 30376 catalog_manager.cc:5697] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 26 to 27, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 27 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } }
I20260709 13:17:05.820508 30376 catalog_manager.cc:5697] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 24 to 25, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to acf852f85f47483d99cbe4020baee627 (127.21.5.131). New cstate: current_term: 25 leader_uuid: "acf852f85f47483d99cbe4020baee627" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } }
W20260709 13:17:06.001713 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:06.004438 30375 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58834 (request call id 162) took 1753 ms. Trace:
I20260709 13:17:06.006489 30382 catalog_manager.cc:5697] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:06.011660 30382 catalog_manager.cc:5697] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 reported cstate change: term changed from 23 to 24, leader changed from 015ca80eba674552b34856a4f20271f1 (127.21.5.129) to 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130). New cstate: current_term: 24 leader_uuid: "5a03a8214bba4eb1b0ff00895c565564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:06.005074 30375 rpcz_store.cc:276] 0709 13:17:04.251230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:04.251525 (+   295us) service_pool.cc:224] Handling call
0709 13:17:06.004341 (+1752816us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:05.700835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:17:05.701421 (+   586us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:17:05.701466 (+    45us) write_op.cc:435] Acquired schema lock
0709 13:17:05.701483 (+    17us) tablet.cc:662] Decoding operations
0709 13:17:05.702187 (+   704us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:17:05.702248 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:17:05.702282 (+    34us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:17:05.702851 (+   569us) tablet.cc:701] Row locks acquired
0709 13:17:05.702874 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:17:05.703039 (+   165us) write_op.cc:270] Start()
0709 13:17:05.703178 (+   139us) write_op.cc:276] Timestamp: P: 1783603025702986 usec, L: 0
0709 13:17:05.703198 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:17:05.703904 (+   706us) log.cc:844] Serialized 2078 byte log entry
0709 13:17:05.707533 (+  3629us) op_driver.cc:464] REPLICATION: finished
0709 13:17:05.708061 (+   528us) write_op.cc:301] APPLY: starting
0709 13:17:05.708165 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 13:17:05.711375 (+  3210us) tablet.cc:1370] finished BulkCheckPresence
0709 13:17:05.711506 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:17:05.773722 (+ 62216us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:17:05.773939 (+   217us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:17:05.773983 (+    44us) write_op.cc:312] APPLY: finished
0709 13:17:05.774264 (+   281us) log.cc:844] Serialized 56 byte log entry
0709 13:17:05.774458 (+   194us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.774578 (+   120us) write_op.cc:454] Released schema lock
0709 13:17:05.796513 (+ 21935us) write_op.cc:341] FINISH: Updating metrics
0709 13:17:05.801052 (+  4539us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:17:05.801094 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":39646,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":245,"apply.run_cpu_time_us":8308,"apply.run_wall_time_us":92970,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20950,"num_ops":5,"prepare.queue_time_us":131,"prepare.run_cpu_time_us":3390,"prepare.run_wall_time_us":7168,"replication_time_us":4200}]]}
W20260709 13:17:06.125414 30522 consensus_peers.cc:597] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:06.126061 30522 consensus_peers.cc:597] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:06.139231 30522 consensus_peers.cc:597] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:06.155001 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 76258.9us
W20260709 13:17:06.201642 21526 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 31423)
W20260709 13:17:06.240522 30670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111) [suppressed 419 similar messages]
W20260709 13:17:06.262576 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.340641 30522 consensus_peers.cc:597] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:06.372104 30522 consensus_peers.cc:597] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.417514 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:06.681743 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.771281 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.772828 30522 consensus_peers.cc:597] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.783269 30522 consensus_peers.cc:597] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.783776 30522 consensus_peers.cc:597] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.784276 30522 consensus_peers.cc:597] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.784622 30522 consensus_peers.cc:597] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.935657 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:06.977118 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.044338 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:07.063010 21526 tablet_server.cc:196] TabletServer@127.21.5.129:41021 shutdown complete.
W20260709 13:17:07.113181 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.180359 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.225579 30522 consensus_peers.cc:597] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.226476 30522 consensus_peers.cc:597] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.228245 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.275765 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.350798 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.514761 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:07.519186 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
W20260709 13:17:07.536144 30522 consensus_peers.cc:597] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.564944 30673 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.594301 30599 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.593217 30673 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.606099 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.609215 30599 consensus_peers.cc:597] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.610731 30522 consensus_peers.cc:597] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.616755 30522 consensus_peers.cc:597] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.621457 30599 consensus_peers.cc:597] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.668041 30599 consensus_peers.cc:597] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.669925 30522 consensus_peers.cc:597] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.672740 30673 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.681402 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.710382 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.733450 30522 consensus_peers.cc:597] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.760002 30522 consensus_peers.cc:597] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:07.761189 30673 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.764599 30599 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.776348 30599 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.768270 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.798668 30599 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.799535 30673 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.799633 30599 consensus_peers.cc:597] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.811302 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:07.848918 30673 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.857857 30599 consensus_peers.cc:597] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.861868 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:17:07.882859 30673 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.924597 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:07.925755 21526 tablet_replica.cc:333] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:07.950194 21526 raft_consensus.cc:2243] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:07.951234 21526 raft_consensus.cc:2272] T 01dd2254fe30409b8a08179436dd1e10 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:07.960085 30673 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.967517 30673 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:07.970182 30673 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:07.969450 21526 tablet_replica.cc:333] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:07.994179 21526 raft_consensus.cc:2243] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:07.996949 30599 consensus_peers.cc:597] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:08.013049 30673 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:08.013653 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:08.042380 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 45505.3us
I20260709 13:17:08.056855 21526 raft_consensus.cc:2272] T 8d07c4d131b54934aaedbe589e95d746 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:08.097702 30673 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:08.122776 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:08.125577 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:08.095001 30599 consensus_peers.cc:597] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:08.100733 21526 tablet_replica.cc:333] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.154882 21526 raft_consensus.cc:2243] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.160521 21526 raft_consensus.cc:2272] T 0dc8bff1e9b745c7a95ecd1b6306719d P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:08.172797 30673 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:08.217116 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:08.233556 21526 tablet_replica.cc:333] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.234851 21526 raft_consensus.cc:2243] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.264873 21526 raft_consensus.cc:2272] T 731ce9bb8d6143d4860cfae0d68c8024 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.291477 21526 tablet_replica.cc:333] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.292491 21526 raft_consensus.cc:2243] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.293277 21526 raft_consensus.cc:2272] T 37a97221d4ba4f889ce430d3dadc5754 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.296931 21526 tablet_replica.cc:333] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.299891 21526 raft_consensus.cc:2243] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.301522 21526 raft_consensus.cc:2272] T 2b685fd119d941538e6fee8dc8f8c930 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.305521 21526 tablet_replica.cc:333] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.306720 21526 raft_consensus.cc:2243] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.307708 21526 raft_consensus.cc:2272] T 177c3663558a41cba8e75193425ead61 P 5a03a8214bba4eb1b0ff00895c565564 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.311590 21526 tablet_replica.cc:333] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.312822 21526 raft_consensus.cc:2243] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.315898 21526 raft_consensus.cc:2272] T 125a8386fffd41c0b18a3e6e0690283b P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.322602 21526 tablet_replica.cc:333] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.329769 21526 raft_consensus.cc:2243] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.331074 21526 raft_consensus.cc:2272] T 9fd45f50776e4101a900943b8df9353c P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.334821 21526 tablet_replica.cc:333] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.335898 21526 raft_consensus.cc:2243] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.341678 21526 raft_consensus.cc:2272] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.345845 21526 tablet_replica.cc:333] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.347424 21526 raft_consensus.cc:2243] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.348464 21526 raft_consensus.cc:2272] T bdaa99b885784af3b7e73563f113185a P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.352411 21526 tablet_replica.cc:333] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.353536 21526 raft_consensus.cc:2243] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.355123 21526 raft_consensus.cc:2272] T 3410292026774f0ca95a2c77eab3a8b4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.366907 21526 tablet_replica.cc:333] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.368072 21526 raft_consensus.cc:2243] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.369530 21526 raft_consensus.cc:2272] T 742a437d7cdc48f58f38b6c6a39ce5ae P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.373624 21526 tablet_replica.cc:333] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.374889 21526 raft_consensus.cc:2243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.375774 21526 raft_consensus.cc:2272] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.379439 21526 tablet_replica.cc:333] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.380573 21526 raft_consensus.cc:2243] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:17:08.391829 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:08.399369 21526 raft_consensus.cc:2272] T 496aec2d89db4dbe892cb6d96b075225 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.403581 21526 tablet_replica.cc:333] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.404675 21526 raft_consensus.cc:2243] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.406087 21526 raft_consensus.cc:2272] T 233dbd206c1f419697b1d1b21a74c633 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:08.410519 30522 consensus_peers.cc:597] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:08.415771 21526 tablet_replica.cc:333] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.417079 21526 raft_consensus.cc:2243] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.420781 21526 raft_consensus.cc:2272] T 5de1dd97d9f342288201946ead3a9445 P 5a03a8214bba4eb1b0ff00895c565564 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.425053 21526 tablet_replica.cc:333] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.426195 21526 raft_consensus.cc:2243] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.427232 21526 raft_consensus.cc:2272] T e1452d68baa440e39ae0dad239d033c8 P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.434041 21526 tablet_replica.cc:333] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
W20260709 13:17:08.434032 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:08.440676 21526 raft_consensus.cc:2243] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.442759 21526 raft_consensus.cc:2272] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.446764 21526 tablet_replica.cc:333] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.447799 21526 raft_consensus.cc:2243] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.449214 21526 raft_consensus.cc:2272] T 87824e2fdc9d4d33885d41c010f7bec4 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.452967 21526 tablet_replica.cc:333] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.454138 21526 raft_consensus.cc:2243] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.454994 21526 raft_consensus.cc:2272] T 490460505b2d424286323b79fb9fb2bc P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.458979 21526 tablet_replica.cc:333] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.459918 21526 raft_consensus.cc:2243] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.462589 21526 raft_consensus.cc:2272] T d5048dabcc194abca9a02315086e0eee P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.467375 21526 tablet_replica.cc:333] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.468346 21526 raft_consensus.cc:2243] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.469874 21526 raft_consensus.cc:2272] T 29980abbc08249a490fcf7b22cf63bb5 P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.473790 21526 tablet_replica.cc:333] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.474807 21526 raft_consensus.cc:2243] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.475813 21526 raft_consensus.cc:2272] T a55a2b63593d4202b9b70bac4a5ef28b P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.482748 21526 tablet_replica.cc:333] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.484207 21526 raft_consensus.cc:2243] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.485234 21526 raft_consensus.cc:2272] T b15e4d74180b49daad15cc459dde3181 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.490535 21526 tablet_replica.cc:333] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.492121  6183 raft_consensus.cc:493] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:08.492931  6183 raft_consensus.cc:515] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:08.496587  6183 leader_election.cc:290] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:08.498085 21526 raft_consensus.cc:2243] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:08.500236 30522 leader_election.cc:336] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:08.501706 21526 raft_consensus.cc:2272] T f8c076453f774c4eb79096017a386e6c P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.505640 21526 tablet_replica.cc:333] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.506848 21526 raft_consensus.cc:2243] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.507686 21526 raft_consensus.cc:2272] T 5315ab4cdcff4579984ba8de8f9d4d82 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.511373 21526 tablet_replica.cc:333] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.512454 21526 raft_consensus.cc:2243] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.513259 21526 raft_consensus.cc:2272] T f9117b9966b14416ba3ee00bd7424f65 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.520705 21526 tablet_replica.cc:333] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.521932 21526 raft_consensus.cc:2243] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:17:08.522745 30524 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:08.523393 21526 raft_consensus.cc:2272] T 6f217397202e476a967c268887497cf5 P 5a03a8214bba4eb1b0ff00895c565564 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.527074 21526 tablet_replica.cc:333] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.530126 21526 raft_consensus.cc:2243] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.531526 21526 raft_consensus.cc:2272] T 907e974a334846208c70f0b813199df0 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:08.541606 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:08.542344 30705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf727a8c7b4599bb21be645db6983f" candidate_uuid: "5a03a8214bba4eb1b0ff00895c565564" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:08.544247 21526 tablet_replica.cc:333] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.545380 21526 raft_consensus.cc:2243] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.546248 21526 raft_consensus.cc:2272] T 0e3e983dd2fc4a56857ff13e699d9517 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.555776 30523 leader_election.cc:304] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5a03a8214bba4eb1b0ff00895c565564; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:08.557029 21526 tablet_replica.cc:333] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.557318  6174 raft_consensus.cc:2749] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:08.558202 21526 raft_consensus.cc:2243] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 LEADER]: Raft consensus shutting down.
W20260709 13:17:08.559217 30522 consensus_peers.cc:597] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:08.559650 21526 raft_consensus.cc:2272] T f029f5d5b1ee4391a2b10e12ec33b844 P 5a03a8214bba4eb1b0ff00895c565564 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.563264 21526 tablet_replica.cc:333] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.564251 21526 raft_consensus.cc:2243] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.566143 21526 raft_consensus.cc:2272] T 6c7844be2162490091882b0a0344414c P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.571839 21526 tablet_replica.cc:333] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.572978 21526 raft_consensus.cc:2243] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.574566 21526 raft_consensus.cc:2272] T fd1749966f254b6a8c54ddfd73cb8d14 P 5a03a8214bba4eb1b0ff00895c565564 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.577914 21526 tablet_replica.cc:333] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.578785 21526 raft_consensus.cc:2243] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.579615 21526 raft_consensus.cc:2272] T 2cbf727a8c7b4599bb21be645db6983f P 5a03a8214bba4eb1b0ff00895c565564 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.584646 21526 tablet_replica.cc:333] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.585644 21526 raft_consensus.cc:2243] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.587834 21526 raft_consensus.cc:2272] T c351eef49bfd49568fd63877d05eab98 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.591455 21526 tablet_replica.cc:333] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.592505 21526 raft_consensus.cc:2243] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.594439 21526 raft_consensus.cc:2272] T ef183e787ad14ee6bef37037a6ba59c0 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.597977 21526 tablet_replica.cc:333] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.598949 21526 raft_consensus.cc:2243] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.600551 21526 raft_consensus.cc:2272] T 07fae2111d284c78bf054bd74aee6b94 P 5a03a8214bba4eb1b0ff00895c565564 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.603955 21526 tablet_replica.cc:333] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.604934 21526 raft_consensus.cc:2243] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.606415 21526 raft_consensus.cc:2272] T c4358103a0de4d8d88d2fa7c86875470 P 5a03a8214bba4eb1b0ff00895c565564 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.609539 21526 tablet_replica.cc:333] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.610337 21526 raft_consensus.cc:2243] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.611164 21526 raft_consensus.cc:2272] T dc6f2eea5d3844fab97d019c3af6cf2f P 5a03a8214bba4eb1b0ff00895c565564 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.617134 21526 tablet_replica.cc:333] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.619473 21526 raft_consensus.cc:2243] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.625190 21526 raft_consensus.cc:2272] T d847606257cc4750be11b056651106a6 P 5a03a8214bba4eb1b0ff00895c565564 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.631634 21526 tablet_replica.cc:333] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.632660 21526 raft_consensus.cc:2243] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:17:08.634069 21526 raft_consensus.cc:2272] T 97d1629ffbcd48d1b72c843365f37329 P 5a03a8214bba4eb1b0ff00895c565564 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.657188 21526 tablet_replica.cc:333] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.658296 21526 raft_consensus.cc:2243] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.659824 21526 raft_consensus.cc:2272] T 6bae0ee744c54691b48d52b137264f92 P 5a03a8214bba4eb1b0ff00895c565564 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:08.673054 21526 tablet_replica.cc:333] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564: stopping tablet replica
I20260709 13:17:08.675006 21526 raft_consensus.cc:2243] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:08.675799 21526 raft_consensus.cc:2272] T bf65c2e5cf5b41a2956231a0868f2a1f P 5a03a8214bba4eb1b0ff00895c565564 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:08.905886 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:09.110131 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:09.351145  6124 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.351840  6124 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.354027  6124 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:09.360298  6189 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.361176  6189 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:09.384603 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:09.391453 30599 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.394574  6189 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:09.420327  6189 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.421120  6189 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.424708  6189 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.397406 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:09.451155 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 53812us
W20260709 13:17:09.453277 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:09.454257 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.454886 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.457430  6124 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:09.463148 30599 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.469405 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:09.474511 30599 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.500008  6124 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.500833  6124 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:09.505330 30707 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:09.503455  6124 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:09.515537 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:09.524051 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 57234.4us
I20260709 13:17:09.524849  6124 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:09.531308 30599 leader_election.cc:336] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.539892 30599 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:09.540992  6124 raft_consensus.cc:2749] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:09.554471 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:09.557271 30596 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.557771 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.558719  6189 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:09.558969 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:09.566028  6185 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.566943  6185 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:09.584491 30670 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.587756  6185 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.602396 30673 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.603216 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.604215  6185 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:09.660300  6185 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.660934  6185 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:09.668926  6185 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.680645 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:09.682798 30673 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.683439 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.684398  6185 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:09.692565 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:09.696390  6185 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.697204  6185 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:09.699896  6185 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.706183 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.709297  6185 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.709991  6185 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:17:09.711001 30673 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.711571 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.712558  6164 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:09.713618  6185 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:09.713899 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627" is_pre_election: true
I20260709 13:17:09.714725 30631 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 22.
I20260709 13:17:09.716331 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:09.717749  6185 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:17:09.718185  6185 raft_consensus.cc:493] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.718556  6185 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:09.730772  6164 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.731746  6164 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:09.738883  6164 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.747118 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:09.751051 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.767838  6124 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.768586  6124 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.769558  6189 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.770164  6189 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.770923  6124 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:09.771677  6164 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.772831  6189 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:09.779376  6164 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:09.785948  6164 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.790930 30599 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.797307 30599 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.800966 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:09.796103 30673 leader_election.cc:336] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.804342 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:09.794695 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.805521  6189 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:09.807955 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.808465 30596 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:09.811767 30673 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.813161 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.818642  6164 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:09.813203  6185 raft_consensus.cc:515] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:09.814575  6189 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:17:09.837123 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:09.836961 30631 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c05f1e9839ab444dbb4bcf0fa071cfc5" candidate_uuid: "6b393e79ded4461ea5da7b6b6063305f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "acf852f85f47483d99cbe4020baee627"
I20260709 13:17:09.839195 30631 raft_consensus.cc:3060] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:17:09.839821  6189 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.840421  6189 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.852334  6189 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.859623 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 30580)
I20260709 13:17:09.866019 30631 raft_consensus.cc:2468] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b393e79ded4461ea5da7b6b6063305f in term 23.
W20260709 13:17:09.857463 30599 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.866899  6185 leader_election.cc:290] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Requested vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:09.855345 30673 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:09.870834 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.872241 30671 leader_election.cc:304] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f, acf852f85f47483d99cbe4020baee627; no voters: 
I20260709 13:17:09.872938  6185 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:09.873767  6191 raft_consensus.cc:2804] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:17:09.874402  6191 raft_consensus.cc:697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Becoming Leader. State: Replica: 6b393e79ded4461ea5da7b6b6063305f, State: Running, Role: LEADER
I20260709 13:17:09.875402  6191 consensus_queue.cc:237] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:17:09.879781 30673 leader_election.cc:336] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.887554 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.888345 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.891886  6189 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:09.903992  6189 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.912210  6124 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.912916  6124 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:09.913691 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:09.914351 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:09.916249  6124 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:09.904640  6189 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.919862  6189 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:09.922286 30375 catalog_manager.cc:5697] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f reported cstate change: term changed from 22 to 23, leader changed from 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130) to 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132). New cstate: current_term: 23 leader_uuid: "6b393e79ded4461ea5da7b6b6063305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:09.937636 30599 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.939723 30599 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.954352 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:09.958161  6189 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:09.959107  6189 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:09.963069  6189 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:09.976320 30599 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:09.987444 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:09.991956 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:09.992985  6189 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:09.994647 30596 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.002100 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:10.007428  6189 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:17:10.004792 30599 consensus_peers.cc:597] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:10.013164  6189 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:10.013911  6189 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:10.017027  6189 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:10.027302 30673 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.035274 30599 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:10.037112  6191 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:10.044062  6191 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:10.045914  6187 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:10.046561  6187 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:10.047079  6191 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:10.048513  6191 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:10.048807  6187 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:10.049012  6191 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:10.041059  6189 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:10.050868  6189 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:10.053083  6189 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:10.054061  6191 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:10.060381 30673 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:10.063982 30673 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:10.064843 30670 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.066219 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:10.071614  6185 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:10.073318 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.073841 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:10.074683 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:10.078009  6164 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:17:10.079229 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.079712 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:10.084338  6191 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:10.087913 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.106732 30596 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:10.112802  6189 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:10.115782 30673 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.119441 30596 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.126070 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:10.128964  6189 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:10.120467 30599 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:10.130240 30599 consensus_peers.cc:597] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.143803 30673 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.150383 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:10.153061 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:10.158226 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.158867 30673 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:10.163482  6189 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:10.168303 30673 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.160966 30599 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.179836 30673 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.200011 30599 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.214851 30599 consensus_peers.cc:597] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.220265 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.224221 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.237703 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.261876 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.280418 30673 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:10.285946 30599 consensus_peers.cc:597] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.293066 30599 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.315920 30673 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.322156 30599 consensus_peers.cc:597] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.361819 30673 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.363930 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.368712 30599 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.392122 30673 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.398784 30599 consensus_peers.cc:597] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:10.448045 30631 raft_consensus.cc:1275] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Refusing update from remote peer 6b393e79ded4461ea5da7b6b6063305f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:17:10.450114  6164 consensus_queue.cc:1048] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:17:10.528048 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.528590 30599 consensus_peers.cc:597] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.532693 30673 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.554561 30673 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.570659 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.588780 30673 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.669221 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:10.718987 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:10.860244 21526 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:10.877183 30673 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:10.954502 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:11.202013  6124 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.202726  6124 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:11.213483  6191 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.214288  6191 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:11.218289  6191 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.228646 30673 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.229735 30670 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.230373 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.231695  6202 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:11.236289 30599 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.238222  6124 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.243592 30670 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111) [suppressed 560 similar messages]
W20260709 13:17:11.245630 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.248739 30596 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.251459  6124 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:11.278738  6189 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.279462  6189 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:11.287437 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:11.289379  6189 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.293735 30599 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.318181 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:11.320917 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.321837 30596 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.322834  6189 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:11.401090  6164 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.403522  6164 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:11.437412  6164 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.442795 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.463692  6164 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.464725  6164 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:11.483377 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.485783  6164 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.495715 30673 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.496497 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.500814  6164 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:11.527351  6208 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.528445  6208 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:11.537801  6208 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.542461 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:11.546504 30599 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.547524 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.550822  6208 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:11.560686  6124 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.561687  6124 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:11.571174 30673 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.572011 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.574100  6164 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:11.574348 30596 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.576982  6124 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:11.593206  6206 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.594415  6206 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:11.593394 30599 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.599954 30599 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.599434  6206 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:11.601249  6202 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.602200  6202 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:11.605062  6202 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:11.608333  6124 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 13:17:11.610236 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:11.619287 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:11.622948 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:11.623533 30673 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.626741 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.628005  6191 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:11.629400 30673 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.632407 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.634744  6191 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:17:11.643508 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:11.665855  6208 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.667121  6208 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:11.681210 30599 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.681767 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:11.694511  6208 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:11.696386  6164 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.697404  6164 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:11.708526 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.710002  6164 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.711925 30596 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.712658 30596 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.714648  6208 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:11.718180 30673 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.720419 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.725306  6164 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:11.809495  6208 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.810333  6208 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:11.812795  6206 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.813098  6208 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:11.813763  6206 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:11.816392  6206 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.822770 30599 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.824941 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.826067 30596 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.826970  6208 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:11.827677 30673 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.830673 30670 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.831285 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.834815  6206 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:11.836176  6208 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.838650  6208 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:11.846812 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.848500  6208 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.850598 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:11.861123 21526 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:11.868922 30599 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:11.869583 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.871201  6124 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:11.953681  6124 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.954496  6124 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:11.957108  6124 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:11.963941 30599 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:11.965054 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:11.965593 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:11.966516  6124 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:11.985221  6208 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:11.986132  6208 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:11.988955  6208 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:11.996896 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:12.001613 30599 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:12.002274 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:12.005336  6208 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:17:12.014989  6124 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:12.015743  6124 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:12.018139  6124 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:12.019775  6124 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:12.022269  6124 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:12.024765  6124 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:12.020668 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.030855 30599 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:12.033872 30599 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:12.038036 30596 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:12.038689 30596 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:12.039884 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.041141 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:12.041679 30596 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:12.040470  6124 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:17:12.043356  6208 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:12.061054 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:17:12.067456  6206 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:12.068498  6206 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:12.089802 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:12.095830  6206 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:12.099310 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:12.102763 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:12.105873  6202 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:12.123100 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.170266 30599 consensus_peers.cc:597] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.232270 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.390789 30673 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.468931 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.505504 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.569823 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.635129 30599 consensus_peers.cc:597] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.645947 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.662012 30599 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.671063 30673 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.679702 30599 consensus_peers.cc:597] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.690781 30599 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.683210 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:12.698343 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:12.718694 30673 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.722198 30673 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.756074 30673 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:12.784727 30599 consensus_peers.cc:597] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.812611 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.821379 30673 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.839607 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.859002 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.861752 21526 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:12.868736 30673 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.882758 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.908509 30673 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.925812 30599 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.935634 30673 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.958776 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:12.975553 30599 consensus_peers.cc:597] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:12.989377 30673 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.002290 30673 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.064357 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:13.122396 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:13.156828 30673 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.160732 30599 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.198351 30599 consensus_peers.cc:597] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.208124 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:13.248857 30599 consensus_peers.cc:597] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:13.311034  6208 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:13.312085  6208 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:13.314702  6208 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:13.348892 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:13.359649 30673 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:13.378469 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:13.363176 30599 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:13.387524 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:13.403245  6209 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:17:13.418874 30673 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:13.745481  6202 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:13.746236  6202 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:13.765849 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:13.780851 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:13.783171  6202 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:13.809631 30673 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:13.810364 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:13.816080  6202 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:13.820961  6209 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:13.821785  6209 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:13.831189 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:13.834479  6209 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:13.840488 30599 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:13.841334 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:13.842900  6209 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:13.862365 21526 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:13.862905 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:13.971452  6202 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:13.972210  6202 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:13.984194  6202 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:13.987025 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:13.994302 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:13.995623 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:13.999037  6202 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:14.004009  6209 raft_consensus.cc:493] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.004760  6209 raft_consensus.cc:515] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:14.015753 30599 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.019436  6209 leader_election.cc:290] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.020152 30596 leader_election.cc:336] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.020835 30596 leader_election.cc:304] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.022387  6209 raft_consensus.cc:2749] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 13:17:14.096923  6208 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.097658  6208 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:14.100185  6208 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:14.106534  6209 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.109687  6209 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:14.111770 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:14.112775 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.128520  6209 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:14.132648  6202 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.141871  6202 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:14.143157 30599 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.143734 30599 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.145205  6209 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:14.161815 30673 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.165117  6202 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.169602 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.175457 30670 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.177987  6202 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:14.140949 30596 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:14.195482 30599 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.196715 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.197749  6209 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:14.195473 30596 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 54554.1us
I20260709 13:17:14.224373  6209 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.225303  6209 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:14.233022  6209 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.245039 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:14.282194 30599 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.289283 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.294104  6208 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:14.328819 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:14.371646  6208 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.372304  6208 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:14.375063  6208 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.379319 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:14.389492 30599 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:14.390228 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:14.390919 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.395103  6208 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:17:14.397210  6208 raft_consensus.cc:493] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.397965  6208 raft_consensus.cc:515] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:14.402159  6208 leader_election.cc:290] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.416337 30596 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.427076  6208 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.428453  6208 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:14.433038 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:14.468281 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:14.469085 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.449764  6208 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.504103 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:14.524488  6164 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.525391  6164 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:14.529948  6164 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.540688 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:14.542037 30673 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:14.565021 30599 leader_election.cc:336] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.572472 30599 leader_election.cc:304] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.566354  6164 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.578146  6164 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:14.585325  6213 raft_consensus.cc:2749] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:14.579430  6208 raft_consensus.cc:493] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.589370  6208 raft_consensus.cc:515] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:14.620954 30596 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.603572  6164 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:14.654091  6208 leader_election.cc:290] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.660812 30599 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.665131 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.666668  6208 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:14.670908 30673 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:14.672479 30670 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.677318 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.678887  6164 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:14.694110  6214 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.694972  6214 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:14.697875  6214 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.751794 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57997.9us
W20260709 13:17:14.759262 30673 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:14.789484 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.794934 30670 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.828115  6214 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:14.818604 30599 consensus_peers.cc:597] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:14.829644 30599 leader_election.cc:336] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.845403 30599 leader_election.cc:304] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.854235  6208 raft_consensus.cc:2749] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:14.858724 30673 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:14.863021 21526 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:14.859459 30670 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.866006 30670 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:14.869313  6214 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:14.870275  6208 raft_consensus.cc:493] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.871055  6208 raft_consensus.cc:515] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
W20260709 13:17:14.880769 30596 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:14.885118  6208 leader_election.cc:290] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:14.894970  6214 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:14.897121  6214 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:14.901336  6214 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:14.973794 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:14.979206 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 52382.7us
W20260709 13:17:14.982540 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.014099 30599 leader_election.cc:336] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:14.996066  6164 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:15.042414  6164 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:15.045193  6164 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:15.024775 30673 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:15.046319 30599 leader_election.cc:304] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:15.047411 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:15.061295 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:15.068593 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:15.068734  6164 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:15.069200 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:15.071655 30599 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 57616.2us
W20260709 13:17:15.072683 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:15.076176  6208 raft_consensus.cc:2749] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:15.102548 30673 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.116639 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.124133 30599 consensus_peers.cc:597] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.170567 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.172497 30673 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:15.173094 30673 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:15.173971  6164 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:17:15.184914 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:15.185441 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.187074 30599 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.201265 30673 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.205473 30673 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.224511 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.271176 30599 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.273945 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:15.293144 30673 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:15.320768 30599 consensus_peers.cc:597] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.321489 30599 consensus_peers.cc:597] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.327849 30673 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.387635 30673 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.394428 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.428812 30673 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.433879 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:15.457441 30599 consensus_peers.cc:597] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.488008 30673 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.495163 30596 consensus_peers.cc:597] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.499720 30599 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.517776 30599 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.561618 30673 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.571135 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.612900 30673 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.640007 30599 consensus_peers.cc:597] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.661531 30673 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.687180 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.703330 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.818049 30599 consensus_peers.cc:597] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.840764 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:17:15.841663 30673 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:15.863626 21526 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 30580)
I20260709 13:17:15.876519  6209 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:15.877329  6209 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:15.889830  6209 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:15.905777 30599 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:15.906960 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:15.907696 30596 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:15.917249  6209 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:15.918304  6213 raft_consensus.cc:493] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:15.919010  6213 raft_consensus.cc:515] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:15.922971  6213 leader_election.cc:290] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:15.926007 30596 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:15.935065 30599 leader_election.cc:336] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:15.935681 30599 leader_election.cc:304] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:15.937913  6213 raft_consensus.cc:2749] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:15.962424 30673 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:16.244871 30596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111) [suppressed 576 similar messages]
W20260709 13:17:16.278470 30596 consensus_peers.cc:597] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:16.375535  6214 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:16.376304  6214 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:16.380774  6214 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:16.387962 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:16.394317 30673 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:16.395321 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:16.396344  6214 raft_consensus.cc:2749] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:16.404500  6209 raft_consensus.cc:493] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:16.405292  6209 raft_consensus.cc:515] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:16.407718  6209 leader_election.cc:290] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:16.414811 30599 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:16.420372 30596 leader_election.cc:336] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:16.420914 30596 leader_election.cc:304] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:16.422552  6209 raft_consensus.cc:2749] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:17:16.464788 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:16.552423  6214 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:16.553110  6214 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:16.563261  6214 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:16.566525 30673 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:16.568534 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:16.572086 30670 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:16.576161  6214 raft_consensus.cc:2749] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:16.741668 30596 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:16.761444  6218 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:16.762195  6218 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:16.772549 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:16.778234  6218 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:16.779217 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:16.780509 30670 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:16.784358  6218 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:17:16.842672 30596 consensus_peers.cc:597] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:16.843526 30596 consensus_peers.cc:597] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:16.864288 21526 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 30580)
W20260709 13:17:16.985018 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:16.997289 30596 consensus_peers.cc:597] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:17.102033 30673 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:17.107643 30596 consensus_peers.cc:597] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:17.138692 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
I20260709 13:17:17.184803  6217 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:17.185779  6217 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:17.192140 30599 consensus_peers.cc:597] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:17.200215 30673 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:17.219468  6217 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:17.225096 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:17.230592 30670 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:17.232493  6217 raft_consensus.cc:2749] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:17.275686 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:17:17.358235 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
W20260709 13:17:17.380950 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:17.429766 30673 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:17.438448 30671 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:17.448796 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:17.488780 30671 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:17.494964 30596 consensus_peers.cc:597] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:17.494273 30599 consensus_peers.cc:597] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:17.507444 30671 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:17.513451 30671 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:17.545059 30671 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:17.557143 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:17.576257 21526 tablet_replica.cc:333] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:17.577965 21526 raft_consensus.cc:2243] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.580968 21526 raft_consensus.cc:2272] T 4b674451b4d5465d82d8ef4f14fb52f5 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.585115 21526 tablet_replica.cc:333] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:17.589836 21526 raft_consensus.cc:2243] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:17.592999 30671 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:17.586076 30596 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:17.600152 21526 raft_consensus.cc:2272] T b85a1f7a06b642de885b159b7f7bba1e P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.604408 21526 tablet_replica.cc:333] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:17.610935 21526 raft_consensus.cc:2243] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:17.623476 21526 raft_consensus.cc:2272] T 07fae2111d284c78bf054bd74aee6b94 P acf852f85f47483d99cbe4020baee627 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:17.617938 30671 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:17.628029 21526 tablet_replica.cc:333] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:17.629346 21526 raft_consensus.cc:2243] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:17.631054 21526 raft_consensus.cc:2272] T 536e99f894744c35a0490416a32a875e P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.635251 21526 tablet_replica.cc:333] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
W20260709 13:17:17.636775 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:17.640377 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:17.641861 21526 raft_consensus.cc:2243] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:17:17.643532 30671 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:17.643613 21526 raft_consensus.cc:2272] T 8d07c4d131b54934aaedbe589e95d746 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:17.663874  6217 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:17.664675  6217 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:17.672227  6217 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:17.683363 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:17.691147 30673 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.055801  6220 raft_consensus.cc:493] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.056661  6220 raft_consensus.cc:515] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:18.059934  6220 leader_election.cc:290] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:18.108250  6221 raft_consensus.cc:493] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.109064  6221 raft_consensus.cc:515] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:18.111052  6220 raft_consensus.cc:493] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.111806  6220 raft_consensus.cc:515] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:18.113266  6221 leader_election.cc:290] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:18.114050  6220 leader_election.cc:290] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:18.115291 30670 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:18.115799 30670 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41021, user_credentials={real_user=slave}} blocked reactor thread for 422085us
I20260709 13:17:18.150740 21526 tablet_replica.cc:333] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.152120 21526 raft_consensus.cc:2243] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.155166 21526 raft_consensus.cc:2272] T 72181014679a4c538f96f2d305a5474f P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.156818 30596 consensus_peers.cc:597] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:17:18.158351 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:18.159895 30673 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:39473, user_credentials={real_user=slave}} blocked reactor thread for 474321us
I20260709 13:17:18.161967  6214 raft_consensus.cc:2749] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:18.175585 21526 tablet_replica.cc:333] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.179134 21526 raft_consensus.cc:2243] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.180281 21526 raft_consensus.cc:2272] T 6bae0ee744c54691b48d52b137264f92 P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.191977 30596 consensus_peers.cc:597] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:18.193240 30596 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:18.198300 30596 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:18.199131 30596 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.206809  6223 raft_consensus.cc:493] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.207729  6223 raft_consensus.cc:515] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:18.197765 21526 tablet_replica.cc:333] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.223065 21526 raft_consensus.cc:2243] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.210541  6223 leader_election.cc:290] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:18.235162  6209 raft_consensus.cc:493] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.238185  6209 raft_consensus.cc:515] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } }
I20260709 13:17:18.240792  6209 leader_election.cc:290] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:18.235721 21526 raft_consensus.cc:2272] T bdaa99b885784af3b7e73563f113185a P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.262856 30596 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:18.263592 30670 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:18.264706 30671 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:45277, user_credentials={real_user=slave}} blocked reactor thread for 73322.8us
I20260709 13:17:18.271739 21526 tablet_replica.cc:333] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.272897 21526 raft_consensus.cc:2243] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:18.279178 30599 consensus_peers.cc:597] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.279784 30599 leader_election.cc:336] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.280192 30599 leader_election.cc:304] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:18.281136 30673 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.281847 30673 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:17:18.282347 30673 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.284330 30673 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.284864 30673 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.285389 30670 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:18.286935 30670 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:18.287490 30670 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:18.288316 30670 consensus_peers.cc:597] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:18.273809 21526 raft_consensus.cc:2272] T 01dd2254fe30409b8a08179436dd1e10 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.290932  6209 raft_consensus.cc:2749] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:17:18.291218 30671 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.295245 21526 tablet_replica.cc:333] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.296334 21526 raft_consensus.cc:2243] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.297334 21526 raft_consensus.cc:2272] T a1265e789b7c41d7ba82825a0c126517 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.298486 30673 consensus_peers.cc:597] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.300256 30599 consensus_peers.cc:597] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.307730 30671 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:18.308049 30599 consensus_peers.cc:597] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.308670 30599 leader_election.cc:336] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.309252 30599 leader_election.cc:304] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.301093 21526 tablet_replica.cc:333] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
W20260709 13:17:18.309541 30673 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.310274 30673 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:18.310768 21526 raft_consensus.cc:2243] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.316212  6209 raft_consensus.cc:2749] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:18.317350 21526 raft_consensus.cc:2272] T 26f2e1d0fad840b5b8718f931fc92316 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.321069 21526 tablet_replica.cc:333] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.323285 21526 raft_consensus.cc:2243] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:17:18.311226 30670 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:18.323712 30671 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:18.324093 30670 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:17:18.321990 30599 leader_election.cc:336] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:18.324997 30671 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.325027 30599 leader_election.cc:304] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.326321 21526 raft_consensus.cc:2272] T 54da309ba151423ab2739966b9caebd0 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.326956  6209 raft_consensus.cc:2749] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:18.326331 30671 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:18.331403 30671 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:18.333509 30671 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:18.331341 21526 tablet_replica.cc:333] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.335647 21526 raft_consensus.cc:2243] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.336323  6209 raft_consensus.cc:493] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.337116  6209 raft_consensus.cc:515] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:18.337893 30599 consensus_peers.cc:597] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.338593 30599 leader_election.cc:336] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.339046 30599 leader_election.cc:304] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.339586  6209 leader_election.cc:290] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:18.340059  6219 raft_consensus.cc:2749] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:18.331545 30673 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.341938 30673 leader_election.cc:336] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.342460 30673 leader_election.cc:304] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.344352  6223 raft_consensus.cc:2749] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:17:18.345672 30596 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.343660 21526 raft_consensus.cc:2272] T 5315ab4cdcff4579984ba8de8f9d4d82 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.348227 30599 leader_election.cc:336] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.348796 30599 leader_election.cc:304] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.349692  6219 raft_consensus.cc:2749] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:17:18.350389 21526 tablet_replica.cc:333] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.352908 21526 raft_consensus.cc:2243] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.353881 21526 raft_consensus.cc:2272] T 6f217397202e476a967c268887497cf5 P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.357602 21526 tablet_replica.cc:333] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.359362 21526 raft_consensus.cc:2243] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.361256 21526 raft_consensus.cc:2272] T f9117b9966b14416ba3ee00bd7424f65 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.364917 21526 tablet_replica.cc:333] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.365969 21526 raft_consensus.cc:2243] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.366942 21526 raft_consensus.cc:2272] T 3511f2706d78410aa91eb654ab345803 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.370798 21526 tablet_replica.cc:333] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.371778 21526 raft_consensus.cc:2243] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.372511 21526 raft_consensus.cc:2272] T e1452d68baa440e39ae0dad239d033c8 P acf852f85f47483d99cbe4020baee627 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.375916 21526 tablet_replica.cc:333] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.376899 21526 raft_consensus.cc:2243] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.378521 21526 raft_consensus.cc:2272] T 125a8386fffd41c0b18a3e6e0690283b P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.380808  6223 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.381491  6223 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:18.390194 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.390872  6223 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:18.394850 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.395874 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.397512  6223 raft_consensus.cc:2749] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:18.393283 21526 tablet_replica.cc:333] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.399444 21526 raft_consensus.cc:2243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.401175 21526 raft_consensus.cc:2272] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.405640 21526 tablet_replica.cc:333] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.406831 21526 raft_consensus.cc:2243] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.408357 21526 raft_consensus.cc:2272] T 490460505b2d424286323b79fb9fb2bc P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.412128 21526 tablet_replica.cc:333] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.413285 21526 raft_consensus.cc:2243] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.414882 21526 raft_consensus.cc:2272] T f8c076453f774c4eb79096017a386e6c P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.418192  6219 raft_consensus.cc:493] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:18.418586 21526 tablet_replica.cc:333] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.418900  6219 raft_consensus.cc:515] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:18.419884 21526 raft_consensus.cc:2243] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.420928 21526 raft_consensus.cc:2272] T 742a437d7cdc48f58f38b6c6a39ce5ae P acf852f85f47483d99cbe4020baee627 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.424503 21526 tablet_replica.cc:333] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.425470 21526 raft_consensus.cc:2243] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.426213 21526 raft_consensus.cc:2272] T 1d0922dea2d4414ca94fe5f85f65ca8b P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.426939 30599 leader_election.cc:336] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.427026 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5048dabcc194abca9a02315086e0eee" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:18.429128  6209 raft_consensus.cc:493] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:18.430054 21526 tablet_replica.cc:333] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.429697  6209 raft_consensus.cc:515] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:18.431283 30597 leader_election.cc:304] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 5a03a8214bba4eb1b0ff00895c565564, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:18.431923  6209 leader_election.cc:290] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:18.432482  6209 raft_consensus.cc:2749] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:17:18.433816 21526 raft_consensus.cc:2243] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.435211 21526 raft_consensus.cc:2272] T dc6f2eea5d3844fab97d019c3af6cf2f P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.439177 21526 tablet_replica.cc:333] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.440179 21526 raft_consensus.cc:2243] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.441612 21526 raft_consensus.cc:2272] T 177c3663558a41cba8e75193425ead61 P acf852f85f47483d99cbe4020baee627 [term 15 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.442819 30596 leader_election.cc:336] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.444446 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e734f70ec47c410d8a7468e2bf91608d" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:18.445333 21526 tablet_replica.cc:333] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.446528 21526 raft_consensus.cc:2243] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.446977 30597 leader_election.cc:304] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:18.447775  6219 leader_election.cc:290] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425)
I20260709 13:17:18.447333 21526 raft_consensus.cc:2272] T 9657733894c94f86ac3b047814fb8fd9 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.456079  6223 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:18.461174  6219 raft_consensus.cc:2749] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:17:18.475386  6223 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:18.478154 21526 tablet_replica.cc:333] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.479382 21526 raft_consensus.cc:2243] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.480130 21526 raft_consensus.cc:2272] T d847606257cc4750be11b056651106a6 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.483722 21526 tablet_replica.cc:333] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.484728 21526 raft_consensus.cc:2243] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.485513 21526 raft_consensus.cc:2272] T d5048dabcc194abca9a02315086e0eee P acf852f85f47483d99cbe4020baee627 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.489080 21526 tablet_replica.cc:333] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.490141 21526 raft_consensus.cc:2243] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.491291 21526 raft_consensus.cc:2272] T 29980abbc08249a490fcf7b22cf63bb5 P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.494799  6223 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:18.494768 21526 tablet_replica.cc:333] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627: stopping tablet replica
W20260709 13:17:18.495361 30673 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.496665 21526 raft_consensus.cc:2243] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.498171 21526 raft_consensus.cc:2272] T a55a2b63593d4202b9b70bac4a5ef28b P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.499591 30670 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.500274 30670 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:18.501222  6223 raft_consensus.cc:2749] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:17:18.503656 21526 tablet_replica.cc:333] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.504936 21526 raft_consensus.cc:2243] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.506495 21526 raft_consensus.cc:2272] T 471d4059785144ffa1ac2459a1f183fb P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.510334 21526 tablet_replica.cc:333] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.511364  6219 raft_consensus.cc:493] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6b393e79ded4461ea5da7b6b6063305f)
I20260709 13:17:18.511668 21526 raft_consensus.cc:2243] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.512110  6219 raft_consensus.cc:515] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:18.512601 21526 raft_consensus.cc:2272] T 907e974a334846208c70f0b813199df0 P acf852f85f47483d99cbe4020baee627 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.517823 21526 tablet_replica.cc:333] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.518841 30699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22a37ee3d67e45c394c7f5963e5767ed" candidate_uuid: "acf852f85f47483d99cbe4020baee627" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6b393e79ded4461ea5da7b6b6063305f" is_pre_election: true
I20260709 13:17:18.519115 21526 raft_consensus.cc:2243] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.520206 21526 raft_consensus.cc:2272] T fd1749966f254b6a8c54ddfd73cb8d14 P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.520951 30596 leader_election.cc:336] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.522110  6219 leader_election.cc:290] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 6b393e79ded4461ea5da7b6b6063305f (127.21.5.132:35425), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:18.522014 30597 leader_election.cc:304] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: acf852f85f47483d99cbe4020baee627; no voters: 015ca80eba674552b34856a4f20271f1, 6b393e79ded4461ea5da7b6b6063305f
I20260709 13:17:18.523101  6219 raft_consensus.cc:2749] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:17:18.530529 21526 tablet_replica.cc:333] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.531728 21526 raft_consensus.cc:2243] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.537042 21526 raft_consensus.cc:2272] T 9fd45f50776e4101a900943b8df9353c P acf852f85f47483d99cbe4020baee627 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.540931 21526 tablet_replica.cc:333] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.541893 21526 raft_consensus.cc:2243] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.542806 21526 raft_consensus.cc:2272] T 22a37ee3d67e45c394c7f5963e5767ed P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.549332 21526 tablet_replica.cc:333] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.550261 21526 raft_consensus.cc:2243] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.551085 21526 raft_consensus.cc:2272] T 0e3e983dd2fc4a56857ff13e699d9517 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.555119 21526 tablet_replica.cc:333] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.556125 21526 raft_consensus.cc:2243] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.557408 21526 raft_consensus.cc:2272] T f029f5d5b1ee4391a2b10e12ec33b844 P acf852f85f47483d99cbe4020baee627 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.560999 21526 tablet_replica.cc:333] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.561969 21526 raft_consensus.cc:2243] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.562932 21526 raft_consensus.cc:2272] T 6c7844be2162490091882b0a0344414c P acf852f85f47483d99cbe4020baee627 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.566435 21526 tablet_replica.cc:333] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.567333 21526 raft_consensus.cc:2243] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.568066 21526 raft_consensus.cc:2272] T e734f70ec47c410d8a7468e2bf91608d P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.571826 21526 tablet_replica.cc:333] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.572762 21526 raft_consensus.cc:2243] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.573833 21526 raft_consensus.cc:2272] T c05f1e9839ab444dbb4bcf0fa071cfc5 P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.577224 21526 tablet_replica.cc:333] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.578176 21526 raft_consensus.cc:2243] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.579056 21526 raft_consensus.cc:2272] T 5de1dd97d9f342288201946ead3a9445 P acf852f85f47483d99cbe4020baee627 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.582749 21526 tablet_replica.cc:333] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.583665 21526 raft_consensus.cc:2243] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.584756 21526 raft_consensus.cc:2272] T 233dbd206c1f419697b1d1b21a74c633 P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.588457 21526 tablet_replica.cc:333] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.589396 21526 raft_consensus.cc:2243] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:18.590725 21526 raft_consensus.cc:2272] T 0dc8bff1e9b745c7a95ecd1b6306719d P acf852f85f47483d99cbe4020baee627 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:18.594452 21526 tablet_replica.cc:333] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627: stopping tablet replica
I20260709 13:17:18.595371 21526 raft_consensus.cc:2243] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:18.596307 21526 raft_consensus.cc:2272] T d4bc9ab09182445eaeceb5ea60a2ebfa P acf852f85f47483d99cbe4020baee627 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:18.613950 30673 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.667775 30673 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:17:18.687520  6226 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:18.688328  6226 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:18.692651  6226 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:18.699998 30671 connection.cc:769] client connection to 127.21.5.131:45277 send error: Network error: sendmsg error: Broken pipe (error 32)
W20260709 13:17:18.712198 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08000854e0 after lost signal to thread 21529
W20260709 13:17:18.713996 30413 debug-util.cc:398] Leaking SignalData structure 0x7b0800114e80 after lost signal to thread 30416
W20260709 13:17:18.714671 30413 debug-util.cc:398] Leaking SignalData structure 0x7b080038b5e0 after lost signal to thread 30657
W20260709 13:17:18.716146 30413 debug-util.cc:398] Leaking SignalData structure 0x7b08000485a0 after lost signal to thread 30731
W20260709 13:17:18.721954 30670 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:18.728964 30671 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:18.729488 30671 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:18.730300  6226 raft_consensus.cc:2749] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:17:18.748620 30673 consensus_peers.cc:597] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Couldn't send request to peer 5a03a8214bba4eb1b0ff00895c565564. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:17:18.749728 30670 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:18.912587  6226 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:18.913341  6226 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:18.915992  6226 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:18.921047 30671 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
W20260709 13:17:18.923516 30670 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:18.924072 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:18.924935  6226 raft_consensus.cc:2749] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:18.966607  6226 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:18.967530  6226 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:18.973805  6226 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:18.976432 30671 leader_election.cc:336] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
W20260709 13:17:18.977437 30673 leader_election.cc:336] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:18.977902 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:18.978757  6226 raft_consensus.cc:2749] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:17:19.078684 30670 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:17:19.534221 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08001c45a0 after lost signal to thread 21529
W20260709 13:17:19.535893 30654 debug-util.cc:398] Leaking SignalData structure 0x7b08002b40e0 after lost signal to thread 30416
W20260709 13:17:19.536685 30654 debug-util.cc:398] Leaking SignalData structure 0x7b0800320020 after lost signal to thread 30657
W20260709 13:17:19.538267 30654 debug-util.cc:398] Leaking SignalData structure 0x7b080023a540 after lost signal to thread 30731
I20260709 13:17:19.663264  6226 raft_consensus.cc:493] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.664160  6218 raft_consensus.cc:493] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.664821  6226 raft_consensus.cc:515] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:19.664863  6218 raft_consensus.cc:515] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:19.667937  6218 leader_election.cc:290] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:19.669878  6226 leader_election.cc:290] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:19.682349 30671 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.685868  6226 raft_consensus.cc:493] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.686810  6226 raft_consensus.cc:515] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:19.689437  6226 leader_election.cc:290] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:19.693442 30671 leader_election.cc:336] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
W20260709 13:17:19.699957 30670 leader_election.cc:336] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:19.701139  6226 raft_consensus.cc:493] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.700637 30670 leader_election.cc:304] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.701887  6226 raft_consensus.cc:515] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:19.702551  6218 raft_consensus.cc:2749] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:19.703948 30670 leader_election.cc:336] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:19.704417  6226 leader_election.cc:290] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:19.704571 30670 leader_election.cc:304] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.705488  6218 raft_consensus.cc:493] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.706867  6218 raft_consensus.cc:515] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:19.709424  6218 leader_election.cc:290] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:19.711831  6218 raft_consensus.cc:2749] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:19.718272 21526 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 30654)
I20260709 13:17:19.720429  6226 raft_consensus.cc:493] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.721133  6226 raft_consensus.cc:515] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:19.723930  6226 leader_election.cc:290] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:19.724586  6226 raft_consensus.cc:493] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.725148  6226 raft_consensus.cc:515] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:19.720319  6218 raft_consensus.cc:493] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:17:19.728243 30673 leader_election.cc:336] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:19.728497  6218 raft_consensus.cc:515] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
W20260709 13:17:19.729085 30673 leader_election.cc:336] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:19.731138  6218 leader_election.cc:290] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:19.731581  6226 leader_election.cc:290] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
W20260709 13:17:19.732518 30671 leader_election.cc:336] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.734153 30671 leader_election.cc:304] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.737514  6226 raft_consensus.cc:2749] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:17:19.737982  6218 raft_consensus.cc:493] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:19.738741  6218 raft_consensus.cc:515] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
W20260709 13:17:19.741636 30673 leader_election.cc:336] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:19.742413 30673 leader_election.cc:336] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:19.733088  6232 raft_consensus.cc:493] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:19.743587  6232 raft_consensus.cc:515] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:19.745978 30671 leader_election.cc:336] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.749028  6226 raft_consensus.cc:493] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:17:19.752808 30671 leader_election.cc:336] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.753568 30671 leader_election.cc:304] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.749886  6226 raft_consensus.cc:515] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:19.757191  6226 leader_election.cc:290] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:19.757687  6234 raft_consensus.cc:2749] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:19.761945 30670 leader_election.cc:336] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
I20260709 13:17:19.763811 30670 leader_election.cc:304] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.746342  6232 leader_election.cc:290] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:19.741298  6218 leader_election.cc:290] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
I20260709 13:17:19.765823  6234 raft_consensus.cc:2749] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:19.771121 30670 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:19.771121 30673 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
W20260709 13:17:19.772921 30670 leader_election.cc:336] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:19.774726 30671 leader_election.cc:336] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.775305 30671 leader_election.cc:304] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.777642  6234 raft_consensus.cc:2749] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:17:19.765897  6235 raft_consensus.cc:493] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
W20260709 13:17:19.774230 30670 leader_election.cc:336] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:19.779034 30671 leader_election.cc:336] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.778936  6235 raft_consensus.cc:515] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:19.779098 30670 leader_election.cc:304] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
I20260709 13:17:19.780274  6234 raft_consensus.cc:2749] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:17:19.779744 30671 leader_election.cc:304] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.781431  6235 leader_election.cc:290] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:19.781412  6234 raft_consensus.cc:2749] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:17:19.783630 30671 leader_election.cc:336] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.784134 30671 leader_election.cc:304] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.785022  6234 raft_consensus.cc:2749] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:17:19.786468 30671 leader_election.cc:336] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:19.787590 30671 leader_election.cc:304] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
I20260709 13:17:19.788501  6234 raft_consensus.cc:2749] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:17:19.807991 30671 leader_election.cc:336] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
W20260709 13:17:19.826707 30670 consensus_peers.cc:597] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:19.830987 30671 leader_election.cc:336] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
W20260709 13:17:19.919643 30670 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:19.948814 30670 consensus_peers.cc:597] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:20.003543 30671 consensus_peers.cc:597] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.078972 30670 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:20.117633 30670 consensus_peers.cc:597] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:17:20.159224 30670 consensus_peers.cc:597] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f -> Peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Couldn't send request to peer 015ca80eba674552b34856a4f20271f1. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:17:20.249125  6235 raft_consensus.cc:493] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.249769  6235 raft_consensus.cc:515] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:20.252342  6235 leader_election.cc:290] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:20.256062 30670 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:20.267160 30671 consensus_peers.cc:597] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:20.269712  6235 raft_consensus.cc:493] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.270738  6235 raft_consensus.cc:515] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:20.277333  6235 leader_election.cc:290] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:20.281194 30670 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Network error: Client connection negotiation failed: client connection to 127.21.5.129:41021: connect: Connection refused (error 111)
W20260709 13:17:20.282303 30671 consensus_peers.cc:597] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.286793 30671 consensus_peers.cc:597] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.371616 30671 consensus_peers.cc:597] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.378108 30671 consensus_peers.cc:597] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.392148 30671 consensus_peers.cc:597] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:20.473124  6234 raft_consensus.cc:493] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.473829  6234 raft_consensus.cc:515] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:20.476147  6234 leader_election.cc:290] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:20.531715 30671 consensus_peers.cc:597] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.535161 30671 consensus_peers.cc:597] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.539887 30671 consensus_peers.cc:597] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:20.545749  6234 raft_consensus.cc:493] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.546515  6234 raft_consensus.cc:515] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:20.548920  6234 leader_election.cc:290] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
I20260709 13:17:20.593479  6234 raft_consensus.cc:493] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.594172  6234 raft_consensus.cc:515] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
I20260709 13:17:20.596731  6234 leader_election.cc:290] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:20.618021 30671 consensus_peers.cc:597] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:20.625437  6235 raft_consensus.cc:493] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:20.626312  6235 raft_consensus.cc:515] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } } peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } }
I20260709 13:17:20.629711  6235 leader_election.cc:290] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473), acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277)
I20260709 13:17:20.635151  6235 raft_consensus.cc:493] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 5a03a8214bba4eb1b0ff00895c565564)
I20260709 13:17:20.635777  6235 raft_consensus.cc:515] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } } peers { permanent_uuid: "5a03a8214bba4eb1b0ff00895c565564" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 39473 } }
W20260709 13:17:20.636204 30671 leader_election.cc:336] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:20.638935  6235 leader_election.cc:290] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021), 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473)
W20260709 13:17:20.676282 30671 consensus_peers.cc:597] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.762727 30671 consensus_peers.cc:597] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.811880 30673 leader_election.cc:336] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:20.812549 30673 leader_election.cc:304] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
I20260709 13:17:20.817543  6235 raft_consensus.cc:2749] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:17:20.825721 30671 consensus_peers.cc:597] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:20.833770 30671 consensus_peers.cc:597] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f -> Peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Couldn't send request to peer acf852f85f47483d99cbe4020baee627. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:20.924101  6235 raft_consensus.cc:493] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:20.924713  6235 raft_consensus.cc:515] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:20.927402  6235 leader_election.cc:290] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:20.937781 30671 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:20.996891  6235 raft_consensus.cc:493] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election (detected failure of leader acf852f85f47483d99cbe4020baee627)
I20260709 13:17:20.997699  6235 raft_consensus.cc:515] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf852f85f47483d99cbe4020baee627" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45277 } } peers { permanent_uuid: "6b393e79ded4461ea5da7b6b6063305f" member_type: VOTER last_known_addr { host: "127.21.5.132" port: 35425 } } peers { permanent_uuid: "015ca80eba674552b34856a4f20271f1" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41021 } }
I20260709 13:17:21.000510  6235 leader_election.cc:290] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277), 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021)
W20260709 13:17:21.005353 30671 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer acf852f85f47483d99cbe4020baee627 (127.21.5.131:45277): Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111)
I20260709 13:17:21.024078 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:17:21.125896 21526 tablet_server.cc:179] TabletServer@127.21.5.132:0 shutting down...
I20260709 13:17:21.197258 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:21.198490 21526 tablet_replica.cc:333] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.199868 21526 raft_consensus.cc:2243] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.200810 21526 raft_consensus.cc:2272] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.204118 21526 tablet_replica.cc:333] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.205154 21526 raft_consensus.cc:2243] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.205974 21526 raft_consensus.cc:2272] T 54da309ba151423ab2739966b9caebd0 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.209461 21526 tablet_replica.cc:333] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.210196 21526 raft_consensus.cc:2243] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.211149 21526 raft_consensus.cc:2272] T 5315ab4cdcff4579984ba8de8f9d4d82 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:21.211182 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.213985 21526 tablet_replica.cc:333] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.214821 21526 raft_consensus.cc:2243] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.215736 21526 raft_consensus.cc:2272] T b85a1f7a06b642de885b159b7f7bba1e P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.219321 21526 tablet_replica.cc:333] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.220296 21526 raft_consensus.cc:2243] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.221040 21526 raft_consensus.cc:2272] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.223917 21526 tablet_replica.cc:333] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.224587 21526 raft_consensus.cc:2243] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.225261 21526 raft_consensus.cc:2272] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.227967 21526 tablet_replica.cc:333] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.228701 21526 raft_consensus.cc:2243] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.229481 21526 raft_consensus.cc:2272] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.232199 21526 tablet_replica.cc:333] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.232851 21526 raft_consensus.cc:2243] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.233366 21526 raft_consensus.cc:2272] T 72181014679a4c538f96f2d305a5474f P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.235917 21526 tablet_replica.cc:333] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.236541 21526 raft_consensus.cc:2243] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.237047 21526 raft_consensus.cc:2272] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.239755 21526 tablet_replica.cc:333] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.240720 21526 raft_consensus.cc:2243] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.241444 21526 raft_consensus.cc:2272] T 37a97221d4ba4f889ce430d3dadc5754 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.244014 21526 tablet_replica.cc:333] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.244673 21526 raft_consensus.cc:2243] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.245716 21526 raft_consensus.cc:2272] T cb3b5ffa2c364cacbd6f5fd9a69f1a79 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:21.248528 30671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.131:45277: connect: Connection refused (error 111) [suppressed 485 similar messages]
I20260709 13:17:21.248962 21526 tablet_replica.cc:333] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.249830 21526 raft_consensus.cc:2243] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.250778 21526 raft_consensus.cc:2272] T 01dd2254fe30409b8a08179436dd1e10 P 6b393e79ded4461ea5da7b6b6063305f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.253720 21526 tablet_replica.cc:333] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.254406 21526 raft_consensus.cc:2243] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.255295 21526 raft_consensus.cc:2272] T a1265e789b7c41d7ba82825a0c126517 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.257784 21526 tablet_replica.cc:333] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.258450 21526 raft_consensus.cc:2243] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.259089 21526 raft_consensus.cc:2272] T 26f2e1d0fad840b5b8718f931fc92316 P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.261847 21526 tablet_replica.cc:333] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.262497 21526 raft_consensus.cc:2243] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.263355 21526 raft_consensus.cc:2272] T d5048dabcc194abca9a02315086e0eee P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.265874 21526 tablet_replica.cc:333] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.266505 21526 raft_consensus.cc:2243] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.267434 21526 raft_consensus.cc:2272] T 29980abbc08249a490fcf7b22cf63bb5 P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.270044 21526 tablet_replica.cc:333] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.270823 21526 raft_consensus.cc:2243] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.271431 21526 raft_consensus.cc:2272] T a55a2b63593d4202b9b70bac4a5ef28b P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.274022 21526 tablet_replica.cc:333] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.274789 21526 raft_consensus.cc:2243] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.275481 21526 raft_consensus.cc:2272] T b15e4d74180b49daad15cc459dde3181 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.278172 21526 tablet_replica.cc:333] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.278870 21526 raft_consensus.cc:2243] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.279440 21526 raft_consensus.cc:2272] T f8c076453f774c4eb79096017a386e6c P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.283123 21526 tablet_replica.cc:333] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.284183 21526 raft_consensus.cc:2243] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.285472 21526 raft_consensus.cc:2272] T 1d0922dea2d4414ca94fe5f85f65ca8b P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.288496 21526 tablet_replica.cc:333] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.289291 21526 raft_consensus.cc:2243] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.290024 21526 raft_consensus.cc:2272] T a6b916ed19e94ddb8e8a2bca582ed0c6 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.293227 21526 tablet_replica.cc:333] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.294046 21526 raft_consensus.cc:2243] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.294690 21526 raft_consensus.cc:2272] T 3f3d2c26e9174e5c942f6f2e1c84cee7 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.297741 21526 tablet_replica.cc:333] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.298319 21526 raft_consensus.cc:2243] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.298902 21526 raft_consensus.cc:2272] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.301203 21526 tablet_replica.cc:333] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.301882 21526 raft_consensus.cc:2243] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.302750 21526 raft_consensus.cc:2272] T dc6f2eea5d3844fab97d019c3af6cf2f P 6b393e79ded4461ea5da7b6b6063305f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.305231 21526 tablet_replica.cc:333] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.305853 21526 raft_consensus.cc:2243] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.306488 21526 raft_consensus.cc:2272] T 2b685fd119d941538e6fee8dc8f8c930 P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.309159 21526 tablet_replica.cc:333] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.310009 21526 raft_consensus.cc:2243] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.310755 21526 raft_consensus.cc:2272] T 177c3663558a41cba8e75193425ead61 P 6b393e79ded4461ea5da7b6b6063305f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.313987 21526 tablet_replica.cc:333] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.314745 21526 raft_consensus.cc:2243] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.315586 21526 raft_consensus.cc:2272] T 9657733894c94f86ac3b047814fb8fd9 P 6b393e79ded4461ea5da7b6b6063305f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.318197 21526 tablet_replica.cc:333] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.318943 21526 raft_consensus.cc:2243] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.319479 21526 raft_consensus.cc:2272] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.322206 21526 tablet_replica.cc:333] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.323061 21526 raft_consensus.cc:2243] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.324083 21526 raft_consensus.cc:2272] T e1452d68baa440e39ae0dad239d033c8 P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.327306 21526 tablet_replica.cc:333] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.328047 21526 raft_consensus.cc:2243] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.328974 21526 raft_consensus.cc:2272] T e734f70ec47c410d8a7468e2bf91608d P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.331662 21526 tablet_replica.cc:333] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.332340 21526 raft_consensus.cc:2243] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.333364 21526 raft_consensus.cc:2272] T c05f1e9839ab444dbb4bcf0fa071cfc5 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:21.336238 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.336344 21526 tablet_replica.cc:333] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.337350 21526 raft_consensus.cc:2243] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.337924 21526 raft_consensus.cc:2272] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.340416 21526 tablet_replica.cc:333] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.341092 21526 raft_consensus.cc:2243] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.341951 21526 raft_consensus.cc:2272] T 3511f2706d78410aa91eb654ab345803 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.344480 21526 tablet_replica.cc:333] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.345080 21526 raft_consensus.cc:2243] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.345659 21526 raft_consensus.cc:2272] T 97d1629ffbcd48d1b72c843365f37329 P 6b393e79ded4461ea5da7b6b6063305f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.347970 21526 tablet_replica.cc:333] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.348567 21526 raft_consensus.cc:2243] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.349131 21526 raft_consensus.cc:2272] T 6bae0ee744c54691b48d52b137264f92 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.351552 21526 tablet_replica.cc:333] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.352102 21526 raft_consensus.cc:2243] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.352694 21526 raft_consensus.cc:2272] T bf65c2e5cf5b41a2956231a0868f2a1f P 6b393e79ded4461ea5da7b6b6063305f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.355187 21526 tablet_replica.cc:333] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.355787 21526 raft_consensus.cc:2243] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.356469 21526 raft_consensus.cc:2272] T c4358103a0de4d8d88d2fa7c86875470 P 6b393e79ded4461ea5da7b6b6063305f [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.359176 21526 tablet_replica.cc:333] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.359738 21526 raft_consensus.cc:2243] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.360291 21526 raft_consensus.cc:2272] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.362874 21526 tablet_replica.cc:333] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.363445 21526 raft_consensus.cc:2243] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.364149 21526 raft_consensus.cc:2272] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:21.364185 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:17:21.367290 21526 tablet_replica.cc:333] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.368045 21526 raft_consensus.cc:2243] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.368844 21526 raft_consensus.cc:2272] T 2cbf727a8c7b4599bb21be645db6983f P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.371476 21526 tablet_replica.cc:333] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.372042 21526 raft_consensus.cc:2243] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.372905 21526 raft_consensus.cc:2272] T 22a37ee3d67e45c394c7f5963e5767ed P 6b393e79ded4461ea5da7b6b6063305f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.375522 21526 tablet_replica.cc:333] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.376173 21526 raft_consensus.cc:2243] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.376986 21526 raft_consensus.cc:2272] T 0e3e983dd2fc4a56857ff13e699d9517 P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.379434 21526 tablet_replica.cc:333] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.379973 21526 raft_consensus.cc:2243] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:17:21.380837 21526 raft_consensus.cc:2272] T d4bc9ab09182445eaeceb5ea60a2ebfa P 6b393e79ded4461ea5da7b6b6063305f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:21.383349 21526 tablet_replica.cc:333] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f: stopping tablet replica
I20260709 13:17:21.383878 21526 raft_consensus.cc:2243] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:21.384382 21526 raft_consensus.cc:2272] T 471d4059785144ffa1ac2459a1f183fb P 6b393e79ded4461ea5da7b6b6063305f [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:21.400728 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.408586 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.413780 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.415719 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.422725 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.426043 30673 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.431712 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.473987 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.487326 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.489202 30670 leader_election.cc:336] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:17:21.489976 30670 leader_election.cc:304] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:17:21.490926 30670 raft_consensus.cc:2686] T 536e99f894744c35a0490416a32a875e P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.493186 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.497078 30670 leader_election.cc:336] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
I20260709 13:17:21.497779 30670 leader_election.cc:304] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, acf852f85f47483d99cbe4020baee627
W20260709 13:17:21.498472 30670 raft_consensus.cc:2686] T 4b674451b4d5465d82d8ef4f14fb52f5 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.512063 30670 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
W20260709 13:17:21.514175 30670 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
W20260709 13:17:21.516230 30670 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
W20260709 13:17:21.519457 30670 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 015ca80eba674552b34856a4f20271f1 (127.21.5.129:41021): Service unavailable: Unable to start connection negotiation thread: The pool has been shut down.
W20260709 13:17:21.548220 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.559849 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.577520 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.612042 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.637482 30671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.701113 30672 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.827713 30670 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:17:21.833892 30673 leader_election.cc:336] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.835124 30673 leader_election.cc:304] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:17:21.836553 30673 raft_consensus.cc:2686] T 490460505b2d424286323b79fb9fb2bc P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.853183 30673 leader_election.cc:336] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.853962 30673 leader_election.cc:304] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.854866 30673 raft_consensus.cc:2686] T 731ce9bb8d6143d4860cfae0d68c8024 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.859093 30673 leader_election.cc:336] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.859810 30673 leader_election.cc:304] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.860594 30673 raft_consensus.cc:2686] T 3410292026774f0ca95a2c77eab3a8b4 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.864691 30673 leader_election.cc:336] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.865233 30673 leader_election.cc:304] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.865967 30673 raft_consensus.cc:2686] T 496aec2d89db4dbe892cb6d96b075225 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.869673 30673 leader_election.cc:336] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.870178 30673 leader_election.cc:304] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.870920 30673 raft_consensus.cc:2686] T ef183e787ad14ee6bef37037a6ba59c0 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.874811 30673 leader_election.cc:336] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.875331 30673 leader_election.cc:304] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.876114 30673 raft_consensus.cc:2686] T 87824e2fdc9d4d33885d41c010f7bec4 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.879881 30673 leader_election.cc:336] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.880422 30673 leader_election.cc:304] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 5a03a8214bba4eb1b0ff00895c565564, acf852f85f47483d99cbe4020baee627
W20260709 13:17:21.881242 30673 raft_consensus.cc:2686] T 8d07c4d131b54934aaedbe589e95d746 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:17:21.884970 30673 leader_election.cc:336] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 5a03a8214bba4eb1b0ff00895c565564 (127.21.5.130:39473): Network error: Client connection negotiation failed: client connection to 127.21.5.130:39473: connect: Connection refused (error 111)
I20260709 13:17:21.885461 30673 leader_election.cc:304] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6b393e79ded4461ea5da7b6b6063305f; no voters: 015ca80eba674552b34856a4f20271f1, 5a03a8214bba4eb1b0ff00895c565564
W20260709 13:17:21.886193 30673 raft_consensus.cc:2686] T c351eef49bfd49568fd63877d05eab98 P 6b393e79ded4461ea5da7b6b6063305f: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 13:17:22.456367 21526 tablet_server.cc:196] TabletServer@127.21.5.132:0 shutdown complete.
I20260709 13:17:22.577358 21526 master.cc:562] Master@127.21.5.190:45121 shutting down...
I20260709 13:17:22.618846 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:22.619797 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:22.620141 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9e2d0e882b2940d8b150869e4b2b2e79: stopping tablet replica
I20260709 13:17:22.760448 21526 master.cc:584] Master@127.21.5.190:45121 shutdown complete.
I20260709 13:17:22.814309 21526 test_util.cc:182] -----------------------------------------------
I20260709 13:17:22.814581 21526 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602506348444-21526-0
[  FAILED  ] LeaderRebalancerTest.RestartTserver (251221 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:17:22.819049 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:34941
I20260709 13:17:22.820312 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:22.827596  6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.827685  6238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:22.829394  6240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:22.830971 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:22.834296 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:22.834527 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:22.834793 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603042834749 us; error 0 us; skew 500 ppm
I20260709 13:17:22.835386 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:22.838145 21526 webserver.cc:533] Webserver started at http://127.21.5.190:33803/ using document root <none> and password file <none>
I20260709 13:17:22.838814 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:22.839048 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:22.839359 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:22.840803 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "aa44b1260e0342eba1882cfae98eaea6"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-h0rz"
I20260709 13:17:22.845994 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 13:17:22.850077  6245 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:22.851023 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:22.851369 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "aa44b1260e0342eba1882cfae98eaea6"
format_stamp: "Formatted at 2026-07-09 13:17:22 on dist-test-slave-h0rz"
I20260709 13:17:22.851709 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:22.873608 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:22.875254 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:22.921551 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:34941
I20260709 13:17:22.921664  6296 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:34941 every 8 connection(s)
I20260709 13:17:22.926849  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:22.941736  6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6: Bootstrap starting.
I20260709 13:17:22.947844  6297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:22.953580  6297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6: No bootstrap required, opened a new log
I20260709 13:17:22.956423  6297 raft_consensus.cc:359] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER }
I20260709 13:17:22.956964  6297 raft_consensus.cc:385] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:22.957274  6297 raft_consensus.cc:740] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa44b1260e0342eba1882cfae98eaea6, State: Initialized, Role: FOLLOWER
I20260709 13:17:22.958047  6297 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER }
I20260709 13:17:22.958570  6297 raft_consensus.cc:399] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:22.958882  6297 raft_consensus.cc:493] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:22.959198  6297 raft_consensus.cc:3060] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:22.965231  6297 raft_consensus.cc:515] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER }
I20260709 13:17:22.965919  6297 leader_election.cc:304] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa44b1260e0342eba1882cfae98eaea6; no voters: 
I20260709 13:17:22.967260  6297 leader_election.cc:290] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:22.967796  6300 raft_consensus.cc:2804] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:22.969528  6300 raft_consensus.cc:697] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 1 LEADER]: Becoming Leader. State: Replica: aa44b1260e0342eba1882cfae98eaea6, State: Running, Role: LEADER
I20260709 13:17:22.970257  6300 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER }
I20260709 13:17:22.970736  6297 sys_catalog.cc:565] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:22.978233  6302 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa44b1260e0342eba1882cfae98eaea6. Latest consensus state: current_term: 1 leader_uuid: "aa44b1260e0342eba1882cfae98eaea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER } }
I20260709 13:17:22.978727  6301 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa44b1260e0342eba1882cfae98eaea6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa44b1260e0342eba1882cfae98eaea6" member_type: VOTER } }
I20260709 13:17:22.979089  6302 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.979558  6301 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:22.981600  6309 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:22.988299  6309 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:22.989668 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:22.999063  6309 catalog_manager.cc:1383] Generated new cluster ID: f3b25fe044784bba82bd0a019632680d
I20260709 13:17:22.999440  6309 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:23.020393  6309 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:23.022097  6309 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:23.040159  6309 catalog_manager.cc:6070] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6: Generated new TSK 0
I20260709 13:17:23.040956  6309 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:23.057039 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.064994  6318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.066422  6319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.069051  6321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.069622 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:23.073155 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.073452 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.073665 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603043073641 us; error 0 us; skew 500 ppm
I20260709 13:17:23.074301 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.077258 21526 webserver.cc:533] Webserver started at http://127.21.5.129:37967/ using document root <none> and password file <none>
I20260709 13:17:23.077870 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.078114 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.078433 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.080050 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "3191e72fbc4a4687be69a048a0db8c1e"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.085820 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:23.090224  6326 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.091295 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:23.091794 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "3191e72fbc4a4687be69a048a0db8c1e"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.092301 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.104869 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.106431 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.108300 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.111166 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.111378 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.111728 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.111920 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.161571 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:43103
I20260709 13:17:23.161748  6388 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:43103 every 8 connection(s)
I20260709 13:17:23.167953 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.177520  6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.178751  6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.183370  6396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.183653 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:23.184552  6389 heartbeater.cc:344] Connected to a master server at 127.21.5.190:34941
I20260709 13:17:23.185003  6389 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.185830  6389 heartbeater.cc:507] Master 127.21.5.190:34941 requested a full tablet report, sending...
I20260709 13:17:23.187402 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.187727 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.187919 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603043187898 us; error 0 us; skew 500 ppm
I20260709 13:17:23.188544  6262 ts_manager.cc:194] Registered new tserver with Master: 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103)
I20260709 13:17:23.188653 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.191088  6262 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42104
I20260709 13:17:23.191807 21526 webserver.cc:533] Webserver started at http://127.21.5.130:40131/ using document root <none> and password file <none>
I20260709 13:17:23.192518 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.192773 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.193125 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.194716 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "30d9641628464c388fe9162931b2e50a"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.199698 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:17:23.203909  6401 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.204873 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:23.205209 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "30d9641628464c388fe9162931b2e50a"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.205574 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.216360 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.217795 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.219631 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.222256 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.222507 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.222878 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.223053 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.278890 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:36685
I20260709 13:17:23.279026  6463 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:36685 every 8 connection(s)
I20260709 13:17:23.284901 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:23.293576  6467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.296510  6468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:23.298173  6470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:23.301941  6464 heartbeater.cc:344] Connected to a master server at 127.21.5.190:34941
I20260709 13:17:23.302717  6464 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.304597  6464 heartbeater.cc:507] Master 127.21.5.190:34941 requested a full tablet report, sending...
I20260709 13:17:23.308439  6262 ts_manager.cc:194] Registered new tserver with Master: 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:23.309850 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:23.310582  6262 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42118
I20260709 13:17:23.311342 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:23.311694 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:23.311935 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603043311908 us; error 0 us; skew 500 ppm
I20260709 13:17:23.312819 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:23.316219 21526 webserver.cc:533] Webserver started at http://127.21.5.131:43241/ using document root <none> and password file <none>
I20260709 13:17:23.316939 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:23.317179 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:23.317529 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:23.319180 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "1547e62e8b0b4fb0875bb48ec8029011"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.324468 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 13:17:23.328547  6475 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.329523 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:17:23.329896 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "1547e62e8b0b4fb0875bb48ec8029011"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:23.330246 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:23.342828 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:23.344295 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:23.346056 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:23.348825 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:23.349120 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.349409 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:23.349602 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:23.398679 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:37307
I20260709 13:17:23.398756  6537 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:37307 every 8 connection(s)
I20260709 13:17:23.415706  6538 heartbeater.cc:344] Connected to a master server at 127.21.5.190:34941
I20260709 13:17:23.416262  6538 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:23.417296  6538 heartbeater.cc:507] Master 127.21.5.190:34941 requested a full tablet report, sending...
I20260709 13:17:23.420049  6262 ts_manager.cc:194] Registered new tserver with Master: 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:23.420220 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017969911s
I20260709 13:17:23.421805  6262 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42132
I20260709 13:17:23.451154  6262 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42140:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:23.454839  6262 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:23.539593  6503 tablet_service.cc:1511] Processing CreateTablet for tablet bfc55c1f1361406498184cefd741f4b6 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:23.541484  6503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc55c1f1361406498184cefd741f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.550266  6502 tablet_service.cc:1511] Processing CreateTablet for tablet c2f26317d8b64de88dee1aea602b36c5 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:23.552362  6502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f26317d8b64de88dee1aea602b36c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.556365  6501 tablet_service.cc:1511] Processing CreateTablet for tablet 663726b99cef48a4a549cac3bcfd2967 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:23.557135  6429 tablet_service.cc:1511] Processing CreateTablet for tablet bfc55c1f1361406498184cefd741f4b6 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:23.558149  6501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663726b99cef48a4a549cac3bcfd2967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.558749  6429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc55c1f1361406498184cefd741f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.562369  6428 tablet_service.cc:1511] Processing CreateTablet for tablet c2f26317d8b64de88dee1aea602b36c5 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:23.564194  6428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f26317d8b64de88dee1aea602b36c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.564567  6500 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7d9accc9634a3497956966206488eb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:23.566221  6500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7d9accc9634a3497956966206488eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.568692  6427 tablet_service.cc:1511] Processing CreateTablet for tablet 663726b99cef48a4a549cac3bcfd2967 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:23.573376  6426 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7d9accc9634a3497956966206488eb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:23.575846  6425 tablet_service.cc:1511] Processing CreateTablet for tablet 33cb1674e38245feae34aaa325730adb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:23.575680  6499 tablet_service.cc:1511] Processing CreateTablet for tablet 33cb1674e38245feae34aaa325730adb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:23.578154  6499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cb1674e38245feae34aaa325730adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.580649  6424 tablet_service.cc:1511] Processing CreateTablet for tablet a5bc9b7472ec4638970ca1dd0fb43450 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:23.570554  6427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663726b99cef48a4a549cac3bcfd2967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.585328  6498 tablet_service.cc:1511] Processing CreateTablet for tablet a5bc9b7472ec4638970ca1dd0fb43450 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:23.587719  6425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cb1674e38245feae34aaa325730adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.587951  6498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5bc9b7472ec4638970ca1dd0fb43450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.589421  6424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5bc9b7472ec4638970ca1dd0fb43450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.591063  6426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7d9accc9634a3497956966206488eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.595242  6423 tablet_service.cc:1511] Processing CreateTablet for tablet a23d2e8677e44bd6a64cea6c2fb04f86 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:23.597602  6423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a23d2e8677e44bd6a64cea6c2fb04f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.615828  6497 tablet_service.cc:1511] Processing CreateTablet for tablet a23d2e8677e44bd6a64cea6c2fb04f86 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:23.617785  6497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a23d2e8677e44bd6a64cea6c2fb04f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.620183  6422 tablet_service.cc:1511] Processing CreateTablet for tablet 513d24a089564be8b2c88697d453b336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:23.621978  6422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513d24a089564be8b2c88697d453b336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.628400  6421 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab7a4874aa479b8ffab687976b4003 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:23.630360  6421 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab7a4874aa479b8ffab687976b4003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.636128  6495 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab7a4874aa479b8ffab687976b4003 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:23.637926  6495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab7a4874aa479b8ffab687976b4003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.633882  6496 tablet_service.cc:1511] Processing CreateTablet for tablet 513d24a089564be8b2c88697d453b336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:23.640038  6496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513d24a089564be8b2c88697d453b336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.673560  6354 tablet_service.cc:1511] Processing CreateTablet for tablet bfc55c1f1361406498184cefd741f4b6 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:17:23.675412  6353 tablet_service.cc:1511] Processing CreateTablet for tablet c2f26317d8b64de88dee1aea602b36c5 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:17:23.677107  6353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2f26317d8b64de88dee1aea602b36c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.679541  6420 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9afc925d7e40098fa97ce4453ed336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:23.681296  6420 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9afc925d7e40098fa97ce4453ed336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.683673  6348 tablet_service.cc:1511] Processing CreateTablet for tablet a23d2e8677e44bd6a64cea6c2fb04f86 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:17:23.684491  6350 tablet_service.cc:1511] Processing CreateTablet for tablet 33cb1674e38245feae34aaa325730adb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:17:23.685633  6348 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a23d2e8677e44bd6a64cea6c2fb04f86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.685271  6346 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab7a4874aa479b8ffab687976b4003 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:17:23.684525  6494 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9afc925d7e40098fa97ce4453ed336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:23.688279  6346 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab7a4874aa479b8ffab687976b4003. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.688704  6494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9afc925d7e40098fa97ce4453ed336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.683673  6352 tablet_service.cc:1511] Processing CreateTablet for tablet 663726b99cef48a4a549cac3bcfd2967 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:17:23.692355  6352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 663726b99cef48a4a549cac3bcfd2967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.694468  6350 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33cb1674e38245feae34aaa325730adb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.699331  6553 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:23.686684  6345 tablet_service.cc:1511] Processing CreateTablet for tablet 9d9afc925d7e40098fa97ce4453ed336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:17:23.683758  6347 tablet_service.cc:1511] Processing CreateTablet for tablet 513d24a089564be8b2c88697d453b336 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:17:23.686558  6349 tablet_service.cc:1511] Processing CreateTablet for tablet a5bc9b7472ec4638970ca1dd0fb43450 (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:17:23.713214  6345 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d9afc925d7e40098fa97ce4453ed336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.715432  6351 tablet_service.cc:1511] Processing CreateTablet for tablet 9f7d9accc9634a3497956966206488eb (DEFAULT_TABLE table=test-workload [id=a2708cd15e0f4cd592eda5d75e97908c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:17:23.717097  6351 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f7d9accc9634a3497956966206488eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.719022  6354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc55c1f1361406498184cefd741f4b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.723119  6349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5bc9b7472ec4638970ca1dd0fb43450. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.727285  6347 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 513d24a089564be8b2c88697d453b336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:23.764856  6553 tablet_bootstrap.cc:654] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.888445  6554 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:23.905833  6555 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:23.907907  6554 tablet_bootstrap.cc:654] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.908689  6553 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:23.913022  6553 ts_tablet_manager.cc:1403] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.214s	user 0.039s	sys 0.061s
I20260709 13:17:23.918679  6553 raft_consensus.cc:359] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:23.919951  6553 raft_consensus.cc:385] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.920336  6553 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.923995  6554 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:23.924685  6554 ts_tablet_manager.cc:1403] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.037s	user 0.006s	sys 0.018s
I20260709 13:17:23.924971  6555 tablet_bootstrap.cc:654] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.927204  6553 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:23.927762  6554 raft_consensus.cc:359] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:23.928596  6554 raft_consensus.cc:385] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.929114  6554 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.929961  6538 heartbeater.cc:499] Master 127.21.5.190:34941 was elected leader, sending a full tablet report...
I20260709 13:17:23.930585  6555 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:23.930182  6554 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:23.930588  6553 ts_tablet_manager.cc:1434] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 13:17:23.931258  6555 ts_tablet_manager.cc:1403] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.005s
I20260709 13:17:23.932508  6553 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:23.935158  6555 raft_consensus.cc:359] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:23.936108  6555 raft_consensus.cc:385] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.936487  6555 raft_consensus.cc:740] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.937551  6555 consensus_queue.cc:260] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:23.940582  6464 heartbeater.cc:499] Master 127.21.5.190:34941 was elected leader, sending a full tablet report...
I20260709 13:17:23.944041  6554 ts_tablet_manager.cc:1434] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 13:17:23.944446  6555 ts_tablet_manager.cc:1434] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:17:23.945354  6389 heartbeater.cc:499] Master 127.21.5.190:34941 was elected leader, sending a full tablet report...
I20260709 13:17:23.945683  6554 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:23.949183  6553 tablet_bootstrap.cc:654] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.954922  6554 tablet_bootstrap.cc:654] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.956516  6555 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:23.969381  6555 tablet_bootstrap.cc:654] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:23.978590  6554 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:23.979310  6554 ts_tablet_manager.cc:1403] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.010s
I20260709 13:17:23.981765  6555 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:23.982695  6555 ts_tablet_manager.cc:1403] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.001s
I20260709 13:17:23.982944  6554 raft_consensus.cc:359] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
W20260709 13:17:23.983516  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:23.984141  6554 raft_consensus.cc:385] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.984550  6554 raft_consensus.cc:740] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.985469  6554 consensus_queue.cc:260] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:23.988000  6554 ts_tablet_manager.cc:1434] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:23.987504  6555 raft_consensus.cc:359] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:23.988546  6555 raft_consensus.cc:385] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:23.988983  6555 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:23.989316  6554 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:23.990307  6559 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:23.990126  6555 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:23.991153  6559 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:23.993521  6555 ts_tablet_manager.cc:1434] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:17:23.996397  6555 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:23.998303  6554 tablet_bootstrap.cc:654] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.007258  6559 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.021548  6555 tablet_bootstrap.cc:654] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.022356  6553 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
W20260709 13:17:24.034047  6465 tablet.cc:2406] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:24.036886  6553 ts_tablet_manager.cc:1403] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.105s	user 0.024s	sys 0.054s
I20260709 13:17:24.041754  6553 raft_consensus.cc:359] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.045125  6553 raft_consensus.cc:385] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.045710  6553 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.047106  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
W20260709 13:17:24.049238  6476 leader_election.cc:343] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.047008  6553 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.050716  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.051662  6439 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
I20260709 13:17:24.053151  6554 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.053885  6554 ts_tablet_manager.cc:1403] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.065s	user 0.028s	sys 0.032s
I20260709 13:17:24.053694  6478 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.054003  6553 ts_tablet_manager.cc:1434] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.003s
I20260709 13:17:24.055274  6553 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.055908  6559 raft_consensus.cc:2804] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.056417  6559 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.056772  6559 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.057756  6554 raft_consensus.cc:359] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.058656  6554 raft_consensus.cc:385] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.059010  6554 raft_consensus.cc:740] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.059921  6554 consensus_queue.cc:260] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.062372  6554 ts_tablet_manager.cc:1434] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:24.064599  6554 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.065253  6555 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.065920  6555 ts_tablet_manager.cc:1403] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.000s
I20260709 13:17:24.069752  6555 raft_consensus.cc:359] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.063028  6553 tablet_bootstrap.cc:654] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.070667  6555 raft_consensus.cc:385] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.071079  6555 raft_consensus.cc:740] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.071980  6555 consensus_queue.cc:260] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.074268  6554 tablet_bootstrap.cc:654] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.078128  6555 ts_tablet_manager.cc:1434] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 13:17:24.079643  6555 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.087095  6555 tablet_bootstrap.cc:654] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.091756  6559 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:24.095368  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
W20260709 13:17:24.097226  6476 leader_election.cc:343] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.099903  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.100622  6439 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.101373  6553 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.102046  6553 ts_tablet_manager.cc:1403] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.020s
I20260709 13:17:24.105911  6553 raft_consensus.cc:359] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.106779  6553 raft_consensus.cc:385] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.107141  6553 raft_consensus.cc:740] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.108346  6553 consensus_queue.cc:260] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.110818  6559 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.126734  6439 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.129247  6478 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.131435  6559 raft_consensus.cc:2804] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.132440  6559 raft_consensus.cc:697] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:24.133873  6559 consensus_queue.cc:237] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:24.140435  6560 raft_consensus.cc:493] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.141196  6560 raft_consensus.cc:515] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.142323  6553 ts_tablet_manager.cc:1434] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.040s	user 0.014s	sys 0.026s
I20260709 13:17:24.143734  6553 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.146387  6560 leader_election.cc:290] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.147214  6555 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.148053  6555 ts_tablet_manager.cc:1403] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.069s	user 0.018s	sys 0.033s
I20260709 13:17:24.149101  6260 catalog_manager.cc:5697] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.151633  6555 raft_consensus.cc:359] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.152853  6555 raft_consensus.cc:385] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.153249  6555 raft_consensus.cc:740] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.152331  6554 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.154508  6555 consensus_queue.cc:260] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.155423  6554 ts_tablet_manager.cc:1403] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.091s	user 0.026s	sys 0.047s
I20260709 13:17:24.158041  6555 ts_tablet_manager.cc:1434] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:17:24.159596  6555 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.162175  6554 raft_consensus.cc:359] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.165756  6554 raft_consensus.cc:385] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.166685  6554 raft_consensus.cc:740] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.167675  6555 tablet_bootstrap.cc:654] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.170226  6554 consensus_queue.cc:260] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.173202  6554 ts_tablet_manager.cc:1434] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:17:24.174466  6554 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.184098  6554 tablet_bootstrap.cc:654] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.184597  6561 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.185863  6553 tablet_bootstrap.cc:654] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.185484  6561 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.196739  6561 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.199465  6579 raft_consensus.cc:493] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.205476  6579 raft_consensus.cc:515] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.209746  6553 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.218595  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.219231  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.219817  6555 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.220511  6555 ts_tablet_manager.cc:1403] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.061s	user 0.016s	sys 0.000s
W20260709 13:17:24.220925  6402 leader_election.cc:343] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.210521  6553 ts_tablet_manager.cc:1403] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.002s
W20260709 13:17:24.224080  6402 leader_election.cc:343] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.224678  6402 leader_election.cc:304] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d9641628464c388fe9162931b2e50a; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.223615  6579 leader_election.cc:290] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.226133  6560 raft_consensus.cc:2749] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.233781  6555 raft_consensus.cc:359] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.234714  6555 raft_consensus.cc:385] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.238899  6553 raft_consensus.cc:359] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.242506  6554 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.235064  6555 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.249794  6553 raft_consensus.cc:385] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.250061  6554 ts_tablet_manager.cc:1403] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.076s	user 0.027s	sys 0.023s
I20260709 13:17:24.250330  6553 raft_consensus.cc:740] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.250589  6555 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.253274  6555 ts_tablet_manager.cc:1434] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 13:17:24.247284  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.248162  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.254114  6554 raft_consensus.cc:359] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.251578  6553 consensus_queue.cc:260] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.255084  6555 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.255206  6554 raft_consensus.cc:385] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.255717  6554 raft_consensus.cc:740] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
W20260709 13:17:24.256285  6329 leader_election.cc:343] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.256652  6554 consensus_queue.cc:260] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.254509  6439 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:24.258708  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.259235  6554 ts_tablet_manager.cc:1434] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:17:24.259459  6513 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:24.260430  6554 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.261667  6329 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.262027  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.262872  6579 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.263294  6579 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.263710  6579 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.271088  6561 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.271790  6561 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.272435  6554 tablet_bootstrap.cc:654] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.272756  6555 tablet_bootstrap.cc:654] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.274506  6561 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.284984  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.286011  6559 raft_consensus.cc:493] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.287689  6559 raft_consensus.cc:515] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.291652  6553 ts_tablet_manager.cc:1434] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 13:17:24.290277  6559 leader_election.cc:290] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.291960  6439 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:24.295406  6329 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.297137  6561 raft_consensus.cc:2804] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.297725  6561 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.298199  6561 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.299165  6553 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.302377  6554 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.303228  6554 ts_tablet_manager.cc:1403] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.043s	user 0.027s	sys 0.009s
I20260709 13:17:24.306125  6579 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.286217  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.312458  6555 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.313162  6555 ts_tablet_manager.cc:1403] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.058s	user 0.026s	sys 0.016s
I20260709 13:17:24.313024  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.313587  6579 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Requested vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.314265  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.315062  6439 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:24.315169  6478 leader_election.cc:343] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.315812  6559 raft_consensus.cc:493] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.316478  6559 raft_consensus.cc:515] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.317087  6555 raft_consensus.cc:359] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.313575  6554 raft_consensus.cc:359] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.317937  6555 raft_consensus.cc:385] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.318152  6554 raft_consensus.cc:385] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.318344  6555 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.319350  6555 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.320276  6559 leader_election.cc:290] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
W20260709 13:17:24.287802  6327 leader_election.cc:343] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.323108  6327 leader_election.cc:304] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.324014  6579 raft_consensus.cc:2749] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.313059  6553 tablet_bootstrap.cc:654] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.325933  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:24.326198  6439 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:24.322108  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.326736  6513 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.318639  6554 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.328059  6554 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.329360  6438 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
I20260709 13:17:24.330163  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.332453  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.333173  6364 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
W20260709 13:17:24.334273  6327 leader_election.cc:343] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.335232  6329 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
W20260709 13:17:24.336277  6476 leader_election.cc:343] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.336401  6579 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.336912  6476 leader_election.cc:304] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011; no voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.336941  6561 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.337668  6513 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:24.322394  6555 ts_tablet_manager.cc:1434] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 13:17:24.340301  6476 leader_election.cc:304] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.341466  6572 raft_consensus.cc:2749] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.341742  6555 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.342233  6572 raft_consensus.cc:2804] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.342796  6572 raft_consensus.cc:493] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.343122  6572 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.346997  6561 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Requested vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.334348  6554 ts_tablet_manager.cc:1434] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.031s	user 0.001s	sys 0.007s
I20260709 13:17:24.347994  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.348822  6438 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.349457  6554 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.352479  6555 tablet_bootstrap.cc:654] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.348001  6561 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.359512  6561 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.361768  6554 tablet_bootstrap.cc:654] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.362270  6561 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.363220  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.364145  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
W20260709 13:17:24.365231  6329 leader_election.cc:343] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.366999  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.367461  6553 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.371120  6553 ts_tablet_manager.cc:1403] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.072s	user 0.007s	sys 0.010s
I20260709 13:17:24.367141  6572 raft_consensus.cc:515] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.376024  6579 raft_consensus.cc:697] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Becoming Leader. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Running, Role: LEADER
W20260709 13:17:24.374675  6327 leader_election.cc:343] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.379468  6579 consensus_queue.cc:237] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.381203  6555 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.381837  6555 ts_tablet_manager.cc:1403] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.006s
I20260709 13:17:24.382193  6438 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:24.384088  6572 leader_election.cc:290] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.385337  6329 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.385493  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.386284  6438 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.386562  6591 raft_consensus.cc:2804] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.386193  6555 raft_consensus.cc:359] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.383832  6327 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.387437  6555 raft_consensus.cc:385] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.387822  6555 raft_consensus.cc:740] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.387522  6591 raft_consensus.cc:697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Becoming Leader. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Running, Role: LEADER
I20260709 13:17:24.388092  6561 raft_consensus.cc:2749] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.388715  6555 consensus_queue.cc:260] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.390266  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:24.379412  6553 raft_consensus.cc:359] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.390964  6364 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.391320  6553 raft_consensus.cc:385] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.391724  6553 raft_consensus.cc:740] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.392858  6553 consensus_queue.cc:260] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
W20260709 13:17:24.391957  6327 leader_election.cc:343] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.396486  6553 ts_tablet_manager.cc:1434] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 13:17:24.397979  6553 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.398983  6572 raft_consensus.cc:493] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.399741  6572 raft_consensus.cc:515] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.402827  6591 consensus_queue.cc:237] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.404220  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
W20260709 13:17:24.406548  6478 leader_election.cc:343] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.408118  6561 raft_consensus.cc:493] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.408792  6561 raft_consensus.cc:515] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.406661  6364 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.412870  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.413736  6476 leader_election.cc:304] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.414786  6559 raft_consensus.cc:2804] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.407646  6572 leader_election.cc:290] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
W20260709 13:17:24.414743  6329 leader_election.cc:343] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.418712  6555 ts_tablet_manager.cc:1434] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.004s
I20260709 13:17:24.418888  6553 tablet_bootstrap.cc:654] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.421329  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.423794  6364 raft_consensus.cc:2468] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
I20260709 13:17:24.424777  6555 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.425243  6476 leader_election.cc:304] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.425444  6561 leader_election.cc:290] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.426369  6572 raft_consensus.cc:2804] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.426960  6572 raft_consensus.cc:493] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.427440  6572 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.431968  6260 catalog_manager.cc:5697] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129). New cstate: current_term: 1 leader_uuid: "3191e72fbc4a4687be69a048a0db8c1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.433624  6260 catalog_manager.cc:5697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129). New cstate: current_term: 1 leader_uuid: "3191e72fbc4a4687be69a048a0db8c1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.432684  6553 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.434715  6559 raft_consensus.cc:697] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:24.434981  6553 ts_tablet_manager.cc:1403] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.003s
I20260709 13:17:24.436168  6559 consensus_queue.cc:237] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.438776  6553 raft_consensus.cc:359] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.439642  6553 raft_consensus.cc:385] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.440016  6553 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.441051  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.441087  6553 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
W20260709 13:17:24.442755  6327 leader_election.cc:343] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.443301  6327 leader_election.cc:304] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.444164  6561 raft_consensus.cc:2749] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.444747  6553 ts_tablet_manager.cc:1434] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:24.445336  6438 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.446027  6553 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.456625  6553 tablet_bootstrap.cc:654] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.457393  6259 catalog_manager.cc:5697] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:24.460737  6555 tablet_bootstrap.cc:654] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.461992  6572 raft_consensus.cc:515] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.462023  6554 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.468201  6572 leader_election.cc:290] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.468451  6554 ts_tablet_manager.cc:1403] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.119s	user 0.022s	sys 0.019s
I20260709 13:17:24.469796  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.471257  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
W20260709 13:17:24.472142  6478 leader_election.cc:343] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.472306  6364 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.488407  6554 raft_consensus.cc:359] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.489756  6554 raft_consensus.cc:385] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.490191  6554 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.494779  6554 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.497591  6555 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.498500  6555 ts_tablet_manager.cc:1403] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.074s	user 0.010s	sys 0.027s
I20260709 13:17:24.502717  6555 raft_consensus.cc:359] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:24.503713  6555 raft_consensus.cc:385] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.504062  6555 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.504283  6554 ts_tablet_manager.cc:1434] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.003s
I20260709 13:17:24.497298  6364 raft_consensus.cc:2468] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.505105  6555 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:24.506286  6554 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.506088  6476 leader_election.cc:304] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.507622  6572 raft_consensus.cc:2804] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.508136  6572 raft_consensus.cc:697] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:24.509652  6553 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.510077  6555 ts_tablet_manager.cc:1434] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:17:24.510277  6553 ts_tablet_manager.cc:1403] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.065s	user 0.027s	sys 0.031s
I20260709 13:17:24.510007  6572 consensus_queue.cc:237] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.511482  6555 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.514434  6553 raft_consensus.cc:359] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.515357  6553 raft_consensus.cc:385] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.515725  6553 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.516744  6553 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.517346  6554 tablet_bootstrap.cc:654] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.519114  6555 tablet_bootstrap.cc:654] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.519740  6553 ts_tablet_manager.cc:1434] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:17:24.521173  6553 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.525878  6560 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.526893  6560 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.532680  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.536239  6554 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
W20260709 13:17:24.536051  6402 leader_election.cc:343] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.537565  6560 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.539695  6554 ts_tablet_manager.cc:1403] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.003s
I20260709 13:17:24.540117  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.542972  6554 raft_consensus.cc:359] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.543846  6554 raft_consensus.cc:385] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.544132  6554 raft_consensus.cc:740] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.544685  6261 catalog_manager.cc:5697] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:24.546254  6402 leader_election.cc:343] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.546870  6402 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d9641628464c388fe9162931b2e50a; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.547832  6560 raft_consensus.cc:2749] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.546566  6554 consensus_queue.cc:260] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.550981  6554 ts_tablet_manager.cc:1434] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 13:17:24.552263  6554 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.554641  6553 tablet_bootstrap.cc:654] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.556928  6560 raft_consensus.cc:493] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.557487  6560 raft_consensus.cc:515] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.558821  6554 tablet_bootstrap.cc:654] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.559953  6560 leader_election.cc:290] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.558854  6555 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.561766  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.562453  6555 ts_tablet_manager.cc:1403] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.051s	user 0.024s	sys 0.012s
I20260709 13:17:24.562654  6364 raft_consensus.cc:2468] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 0.
I20260709 13:17:24.564972  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.565666  6513 raft_consensus.cc:2468] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 0.
I20260709 13:17:24.565804  6555 raft_consensus.cc:359] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.566641  6555 raft_consensus.cc:385] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.566931  6555 raft_consensus.cc:740] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.567791  6555 consensus_queue.cc:260] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.571904  6555 ts_tablet_manager.cc:1434] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:24.572422  6402 leader_election.cc:304] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:24.574080  6560 raft_consensus.cc:2804] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.574138  6555 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:24.574524  6560 raft_consensus.cc:493] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.574965  6560 raft_consensus.cc:3060] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.584136  6555 tablet_bootstrap.cc:654] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.572974  6553 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.587339  6553 ts_tablet_manager.cc:1403] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.002s
I20260709 13:17:24.588925  6606 raft_consensus.cc:493] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.589529  6606 raft_consensus.cc:515] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.591918  6606 leader_election.cc:290] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.593741  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.594573  6364 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 0.
I20260709 13:17:24.597213  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.597944  6513 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 0.
I20260709 13:17:24.598873  6402 leader_election.cc:304] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.600430  6606 raft_consensus.cc:2804] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.600808  6606 raft_consensus.cc:493] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.601168  6606 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.601660  6572 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.605199  6561 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.606014  6561 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.609824  6553 raft_consensus.cc:359] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.610744  6553 raft_consensus.cc:385] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.611040  6553 raft_consensus.cc:740] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.611294  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
W20260709 13:17:24.613404  6327 leader_election.cc:343] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.614172  6561 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.615310  6553 consensus_queue.cc:260] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.616660  6554 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.596448  6560 raft_consensus.cc:515] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.617358  6554 ts_tablet_manager.cc:1403] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.029s
I20260709 13:17:24.617789  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.618847  6439 raft_consensus.cc:2468] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:24.619061  6560 leader_election.cc:290] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.620950  6554 raft_consensus.cc:359] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.621765  6554 raft_consensus.cc:385] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.622058  6554 raft_consensus.cc:740] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.621925  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:24.623237  6364 raft_consensus.cc:3060] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.623152  6554 consensus_queue.cc:260] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.625670  6554 ts_tablet_manager.cc:1434] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.001s
I20260709 13:17:24.631652  6553 ts_tablet_manager.cc:1434] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 13:17:24.633690  6554 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: Bootstrap starting.
I20260709 13:17:24.634042  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f7d9accc9634a3497956966206488eb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:24.635099  6513 raft_consensus.cc:3060] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.640848  6553 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.641626  6554 tablet_bootstrap.cc:654] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.652935  6553 tablet_bootstrap.cc:654] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.663959  6329 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:24.666270  6364 raft_consensus.cc:2468] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 1.
I20260709 13:17:24.667685  6561 raft_consensus.cc:2804] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.669039  6561 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.669427  6561 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.671030  6554 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: No bootstrap required, opened a new log
I20260709 13:17:24.671715  6554 ts_tablet_manager.cc:1403] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.019s
I20260709 13:17:24.673014  6402 leader_election.cc:304] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.674100  6560 raft_consensus.cc:2804] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.674603  6560 raft_consensus.cc:697] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:24.675613  6560 consensus_queue.cc:237] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.683233  6606 raft_consensus.cc:515] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.689779  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:24.690630  6364 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.690826  6555 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: No bootstrap required, opened a new log
I20260709 13:17:24.690693  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:24.691679  6555 ts_tablet_manager.cc:1403] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.118s	user 0.058s	sys 0.033s
I20260709 13:17:24.691902  6512 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.699342  6555 raft_consensus.cc:359] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.700393  6555 raft_consensus.cc:385] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.700757  6555 raft_consensus.cc:740] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.703013  6555 consensus_queue.cc:260] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.705616  6554 raft_consensus.cc:359] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.706754  6554 raft_consensus.cc:385] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.707121  6554 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.713429  6363 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.688158  6513 raft_consensus.cc:2468] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 1.
I20260709 13:17:24.697584  6606 leader_election.cc:290] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.726830  6554 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.729621  6555 ts_tablet_manager.cc:1434] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.001s
I20260709 13:17:24.733506  6554 ts_tablet_manager.cc:1434] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.002s
I20260709 13:17:24.733596  6512 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 1.
I20260709 13:17:24.737635  6402 leader_election.cc:304] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:24.738583  6608 raft_consensus.cc:2804] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.739154  6608 raft_consensus.cc:697] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:24.740067  6608 consensus_queue.cc:237] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.746878  6561 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.751065  6561 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Requested vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:24.752200  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.753010  6438 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.753118  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
W20260709 13:17:24.756072  6327 leader_election.cc:343] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:24.756130  6259 catalog_manager.cc:5697] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 1 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.761152  6364 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 1.
I20260709 13:17:24.764633  6438 raft_consensus.cc:2468] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:24.767175  6329 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:24.768360  6591 raft_consensus.cc:2804] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.768888  6591 raft_consensus.cc:697] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Becoming Leader. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Running, Role: LEADER
I20260709 13:17:24.769802  6591 consensus_queue.cc:237] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.773456  6553 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.774199  6553 ts_tablet_manager.cc:1403] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.134s	user 0.016s	sys 0.011s
I20260709 13:17:24.774361  6572 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.777755  6561 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.778426  6561 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:24.778605  6553 raft_consensus.cc:359] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.779675  6553 raft_consensus.cc:385] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.780109  6553 raft_consensus.cc:740] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.781047  6553 consensus_queue.cc:260] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.782006  6572 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.782508  6572 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.783815  6553 ts_tablet_manager.cc:1434] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:24.785261  6553 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: Bootstrap starting.
I20260709 13:17:24.786037  6572 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.787870  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.788036  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:24.788844  6438 raft_consensus.cc:2393] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1547e62e8b0b4fb0875bb48ec8029011 in current term 1: Already voted for candidate 3191e72fbc4a4687be69a048a0db8c1e in this term.
I20260709 13:17:24.789829  6561 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.790458  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.792027  6329 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3191e72fbc4a4687be69a048a0db8c1e; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a
I20260709 13:17:24.792052  6363 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.796869  6561 raft_consensus.cc:2749] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:24.797987  6553 tablet_bootstrap.cc:654] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:24.802592  6476 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011; no voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:24.803683  6572 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.805541  6259 catalog_manager.cc:5697] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129). New cstate: current_term: 1 leader_uuid: "3191e72fbc4a4687be69a048a0db8c1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.806494  6553 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: No bootstrap required, opened a new log
I20260709 13:17:24.807250  6553 ts_tablet_manager.cc:1403] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.000s
I20260709 13:17:24.810521  6553 raft_consensus.cc:359] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.811323  6553 raft_consensus.cc:385] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:24.811619  6553 raft_consensus.cc:740] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Initialized, Role: FOLLOWER
I20260709 13:17:24.812501  6553 consensus_queue.cc:260] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.814407  6572 raft_consensus.cc:2749] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:24.817289  6553 ts_tablet_manager.cc:1434] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:17:24.818567  6261 catalog_manager.cc:5697] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 0 to 1, leader changed from <none> to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 1 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:24.867337  6561 consensus_queue.cc:1048] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:24.880111  6561 consensus_queue.cc:1048] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.886508  6579 consensus_queue.cc:1048] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.901368  6512 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.910408  6561 consensus_queue.cc:1048] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:24.926319  6572 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:24.927042  6572 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.929502  6572 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.930356  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e" is_pre_election: true
I20260709 13:17:24.930725  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:24.931303  6364 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
I20260709 13:17:24.931464  6438 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 0.
I20260709 13:17:24.932534  6476 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.933398  6572 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:24.933835  6572 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:24.934135  6572 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.940326  6572 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.942567  6572 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:24.943568  6364 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:24.943661  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:24.944656  6364 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.944715  6438 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:24.951073  6364 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.951073  6438 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:24.952657  6478 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:24.953562  6572 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:24.954025  6572 raft_consensus.cc:697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:24.954807  6572 consensus_queue.cc:237] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:24.962818  6259 catalog_manager.cc:5697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
W20260709 13:17:24.990442  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:24.994536  6572 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.008356  6596 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.028491  6559 consensus_queue.cc:1048] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.038772  6438 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.050004  6596 consensus_queue.cc:1048] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.080893  6608 consensus_queue.cc:1048] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.093380  6606 consensus_queue.cc:1048] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.140280  6561 raft_consensus.cc:493] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:25.141013  6561 raft_consensus.cc:515] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:25.143342  6561 leader_election.cc:290] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:25.144304  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:25.144443  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:25.145036  6438 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:25.145135  6512 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 0.
I20260709 13:17:25.146247  6329 leader_election.cc:304] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:25.147101  6561 raft_consensus.cc:2804] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:25.147423  6561 raft_consensus.cc:493] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:25.147701  6561 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.154068  6561 raft_consensus.cc:515] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:25.155999  6561 leader_election.cc:290] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Requested vote from peers 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:25.156970  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:25.157119  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "3191e72fbc4a4687be69a048a0db8c1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:25.157687  6438 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.157744  6512 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:25.161849  6561 consensus_queue.cc:1048] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.165899  6512 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:25.166697  6438 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3191e72fbc4a4687be69a048a0db8c1e in term 1.
I20260709 13:17:25.167950  6327 leader_election.cc:304] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:25.169226  6561 raft_consensus.cc:2804] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:25.169824  6561 raft_consensus.cc:697] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Becoming Leader. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Running, Role: LEADER
I20260709 13:17:25.170866  6561 consensus_queue.cc:237] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:25.180598  6591 consensus_queue.cc:1048] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.181846  6259 catalog_manager.cc:5697] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129). New cstate: current_term: 1 leader_uuid: "3191e72fbc4a4687be69a048a0db8c1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:25.226485  6606 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:25.239292  6606 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:25.367954 21526 ts_manager.cc:295] Set tserver state for 3191e72fbc4a4687be69a048a0db8c1e to MAINTENANCE_MODE
I20260709 13:17:25.368851 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
I20260709 13:17:25.395740 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:25.396620 21526 tablet_replica.cc:333] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.397388 21526 raft_consensus.cc:2243] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:25.398447 21526 raft_consensus.cc:2272] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.401392 21526 tablet_replica.cc:333] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.402100 21526 raft_consensus.cc:2243] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:25.402872 21526 pending_rounds.cc:70] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Trying to abort 1 pending ops.
I20260709 13:17:25.403085 21526 pending_rounds.cc:77] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305638073031192576 op_type: NO_OP noop_request { }
I20260709 13:17:25.403429 21526 raft_consensus.cc:2889] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:25.403728 21526 raft_consensus.cc:2272] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.406723 21526 tablet_replica.cc:333] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.407534 21526 raft_consensus.cc:2243] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:25.408125 21526 raft_consensus.cc:2272] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.411616 21526 tablet_replica.cc:333] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.412388 21526 raft_consensus.cc:2243] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:25.412616  6476 consensus_peers.cc:597] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:25.412948 21526 raft_consensus.cc:2272] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.415957 21526 tablet_replica.cc:333] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.416917 21526 raft_consensus.cc:2243] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:25.417872 21526 raft_consensus.cc:2272] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.420378 21526 tablet_replica.cc:333] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.420996 21526 raft_consensus.cc:2243] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:25.421433 21526 raft_consensus.cc:2272] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.423858 21526 tablet_replica.cc:333] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.424629 21526 raft_consensus.cc:2243] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:25.425189 21526 raft_consensus.cc:2272] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.429018 21526 tablet_replica.cc:333] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.429837 21526 raft_consensus.cc:2243] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:25.430454 21526 raft_consensus.cc:2272] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.434552 21526 tablet_replica.cc:333] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.435382 21526 raft_consensus.cc:2243] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:25.435976 21526 raft_consensus.cc:2272] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.439168 21526 tablet_replica.cc:333] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: stopping tablet replica
I20260709 13:17:25.439704 21526 raft_consensus.cc:2243] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:25.440577 21526 raft_consensus.cc:2272] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:25.473060 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
I20260709 13:17:25.497364  6572 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:17:25.502514  6476 consensus_peers.cc:597] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.580292  6476 consensus_peers.cc:597] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.648624  6402 consensus_peers.cc:597] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.707963  6402 consensus_peers.cc:597] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.715831  6476 consensus_peers.cc:597] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:25.991662  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:26.404354  6606 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.405123  6606 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.407119  6606 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
W20260709 13:17:26.408769  6402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111) [suppressed 35 similar messages]
I20260709 13:17:26.409142  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
W20260709 13:17:26.411862  6402 leader_election.cc:336] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.412338  6402 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d9641628464c388fe9162931b2e50a; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.413091  6606 raft_consensus.cc:2749] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.677170  6606 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.677778  6606 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.679870  6606 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:26.681782  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
W20260709 13:17:26.684268  6402 leader_election.cc:336] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.684729  6402 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d9641628464c388fe9162931b2e50a; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.684984  6559 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.685442  6606 raft_consensus.cc:2749] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.685652  6559 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.687860  6559 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.689116  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:26.689884  6439 raft_consensus.cc:2468] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:26.691309  6478 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
W20260709 13:17:26.691936  6476 leader_election.cc:336] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.692071  6559 raft_consensus.cc:2804] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.692589  6559 raft_consensus.cc:493] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.692910  6559 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.699572  6559 raft_consensus.cc:515] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.701494  6559 leader_election.cc:290] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.702952  6439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d9afc925d7e40098fa97ce4453ed336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:26.703733  6439 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:26.706075  6476 leader_election.cc:336] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.711336  6439 raft_consensus.cc:2468] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 2.
I20260709 13:17:26.712865  6478 leader_election.cc:304] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.713618  6559 raft_consensus.cc:2804] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:26.714564  6559 raft_consensus.cc:697] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:26.715545  6559 consensus_queue.cc:237] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.723686  6262 catalog_manager.cc:5697] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 1 to 2, leader changed from 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.763049  6641 raft_consensus.cc:493] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:26.763605  6641 raft_consensus.cc:515] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.765506  6641 leader_election.cc:290] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.766851  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:26.767639  6438 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:26.768682  6478 leader_election.cc:304] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:26.769577  6641 raft_consensus.cc:2804] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.769929  6641 raft_consensus.cc:493] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:26.769937  6476 leader_election.cc:336] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.770347  6641 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.777195  6641 raft_consensus.cc:515] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.779209  6641 leader_election.cc:290] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.780457  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "513d24a089564be8b2c88697d453b336" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:26.781251  6438 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:26.783601  6476 leader_election.cc:336] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.788335  6438 raft_consensus.cc:2468] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 2.
I20260709 13:17:26.789670  6478 leader_election.cc:304] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.790588  6641 raft_consensus.cc:2804] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:26.791119  6641 raft_consensus.cc:697] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:26.791880  6641 consensus_queue.cc:237] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.800906  6262 catalog_manager.cc:5697] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 1 to 2, leader changed from 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.826397  6641 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.827054  6641 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.829078  6641 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.830451  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:26.831301  6438 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:26.832526  6478 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
W20260709 13:17:26.833364  6476 leader_election.cc:336] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.833469  6641 raft_consensus.cc:2804] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.834009  6641 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.834390  6641 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.841907  6641 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.843879  6641 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.845191  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:26.845942  6438 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:17:26.847946  6476 leader_election.cc:336] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.852415  6438 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 2.
I20260709 13:17:26.853792  6478 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.854729  6641 raft_consensus.cc:2804] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:26.855293  6641 raft_consensus.cc:697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:26.856266  6641 consensus_queue.cc:237] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.864563  6262 catalog_manager.cc:5697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 1 to 2, leader changed from 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:26.927613  6606 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.928275  6606 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.929768  6641 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.930459  6641 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.930410  6606 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:26.932806  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:26.933609  6512 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 1.
I20260709 13:17:26.933950  6641 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.934346  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a" is_pre_election: true
I20260709 13:17:26.935547  6402 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:26.935739  6438 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 1.
I20260709 13:17:26.936313  6606 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.936728  6606 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.937140  6606 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.937250  6478 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
W20260709 13:17:26.938546  6402 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
W20260709 13:17:26.938736  6476 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.939781  6641 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:26.940176  6641 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3191e72fbc4a4687be69a048a0db8c1e)
I20260709 13:17:26.940497  6641 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:26.944386  6606 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.946564  6606 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:26.946756  6641 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:26.948447  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:26.949304  6512 raft_consensus.cc:2393] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 30d9641628464c388fe9162931b2e50a in current term 2: Already voted for candidate 1547e62e8b0b4fb0875bb48ec8029011 in this term.
I20260709 13:17:26.949944  6641 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:26.951694  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "30d9641628464c388fe9162931b2e50a"
W20260709 13:17:26.952209  6402 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.952646  6438 raft_consensus.cc:2393] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1547e62e8b0b4fb0875bb48ec8029011 in current term 2: Already voted for candidate 30d9641628464c388fe9162931b2e50a in this term.
I20260709 13:17:26.952714  6402 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30d9641628464c388fe9162931b2e50a; no voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e
W20260709 13:17:26.954015  6476 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:26.954385  6606 raft_consensus.cc:2749] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:26.954643  6476 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011; no voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:26.955468  6641 raft_consensus.cc:2749] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:17:26.992991  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:27.203449  6476 consensus_peers.cc:597] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:27.260699  6438 raft_consensus.cc:1275] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:27.262709  6641 consensus_queue.cc:1048] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:27.283963  6438 raft_consensus.cc:1275] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:17:27.285233  6641 consensus_queue.cc:1048] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:17:27.301054  6476 consensus_peers.cc:597] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:27.310767  6476 consensus_peers.cc:597] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:27.461623  6438 raft_consensus.cc:1275] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:27.463356  6559 consensus_queue.cc:1048] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:27.836123  6476 consensus_peers.cc:597] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:27.914827  6476 consensus_peers.cc:597] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:27.979154  6402 consensus_peers.cc:597] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:27.994228  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:28.131995  6402 consensus_peers.cc:597] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.221756  6476 consensus_peers.cc:597] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:28.370404  6476 consensus_peers.cc:597] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:17:28.549795  6606 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:28.550447  6606 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:28.552572  6606 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:28.554400  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011" is_pre_election: true
I20260709 13:17:28.555298  6512 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:28.556660  6402 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:28.557570  6606 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:17:28.557650  6402 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:28.558023  6606 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:28.558395  6606 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:28.565341  6606 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:28.567381  6606 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:28.569132  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:28.569900  6512 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:28.571504  6402 leader_election.cc:336] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111)
I20260709 13:17:28.577056  6512 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 3.
I20260709 13:17:28.578424  6402 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 3191e72fbc4a4687be69a048a0db8c1e
I20260709 13:17:28.579249  6606 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:28.580219  6606 raft_consensus.cc:697] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:28.581187  6606 consensus_queue.cc:237] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:28.589027  6259 catalog_manager.cc:5697] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 1 to 3, leader changed from 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 3 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:28.995471  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:29.031083  6512 raft_consensus.cc:1275] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:29.032769  6606 consensus_queue.cc:1048] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:17:29.052277  6402 consensus_peers.cc:597] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:29.612253  6476 consensus_peers.cc:597] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:29.665445  6476 consensus_peers.cc:597] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:29.980528  6476 consensus_peers.cc:597] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:17:29.996711  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:17:30.190950  6476 consensus_peers.cc:597] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:17:30.320559  6476 consensus_peers.cc:597] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:30.498152 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:30.506395  6667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:30.507157  6668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:30.508021  6670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:30.512259 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:30.513535 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:30.513911 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:30.514174 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603050514155 us; error 0 us; skew 500 ppm
I20260709 13:17:30.515167 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:30.520401 21526 webserver.cc:533] Webserver started at http://127.21.5.129:37967/ using document root <none> and password file <none>
I20260709 13:17:30.521169 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:30.521451 21526 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:17:30.526526 21526 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.004s
W20260709 13:17:30.528693  6402 consensus_peers.cc:597] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:30.531404  6675 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:30.532544 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:30.532882 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "3191e72fbc4a4687be69a048a0db8c1e"
format_stamp: "Formatted at 2026-07-09 13:17:23 on dist-test-slave-h0rz"
I20260709 13:17:30.533215 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:17:30.556571  6402 consensus_peers.cc:597] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:30.556728 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:30.558245 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:30.560000 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:30.564124  6683 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:17:30.631254 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:17:30.631647 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.069s	user 0.002s	sys 0.000s
I20260709 13:17:30.632015 21526 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:17:30.638166  6683 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.655259  6683 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.656297  6683 tablet_bootstrap.cc:492] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.656968  6683 ts_tablet_manager.cc:1403] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:17:30.659471  6683 raft_consensus.cc:359] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.660092  6683 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.661069  6683 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.664007  6683 ts_tablet_manager.cc:1434] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:17:30.664871  6683 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.679940  6683 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.681046  6683 tablet_bootstrap.cc:492] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.681717  6683 ts_tablet_manager.cc:1403] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:17:30.684255  6683 raft_consensus.cc:359] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.684871  6683 raft_consensus.cc:740] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.685643  6683 consensus_queue.cc:260] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.686034 21526 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:17:30.686491 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.054s	user 0.051s	sys 0.000s
I20260709 13:17:30.687557  6683 ts_tablet_manager.cc:1434] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:17:30.688498  6683 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
W20260709 13:17:30.705260  6476 consensus_peers.cc:597] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:17:30.709676  6683 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.711398  6683 tablet_bootstrap.cc:492] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.712441  6683 ts_tablet_manager.cc:1403] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.004s
I20260709 13:17:30.716478  6683 raft_consensus.cc:359] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.717501  6683 raft_consensus.cc:740] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.718528  6683 consensus_queue.cc:260] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.721136  6683 ts_tablet_manager.cc:1434] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:30.722327  6683 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.743533  6683 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.745019  6683 tablet_bootstrap.cc:492] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.746057  6683 ts_tablet_manager.cc:1403] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:17:30.749336  6683 raft_consensus.cc:359] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.750264  6683 raft_consensus.cc:740] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.751436  6683 consensus_queue.cc:260] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.751971 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:43103
I20260709 13:17:30.752112  6744 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:43103 every 8 connection(s)
I20260709 13:17:30.753906  6683 ts_tablet_manager.cc:1434] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
I20260709 13:17:30.755005  6683 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.757862 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:30.773048  6683 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.774395  6683 tablet_bootstrap.cc:492] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.775312  6683 ts_tablet_manager.cc:1403] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:17:30.778590  6683 raft_consensus.cc:359] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.779640  6683 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.780596  6683 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.780951  6745 heartbeater.cc:344] Connected to a master server at 127.21.5.190:34941
I20260709 13:17:30.781764  6745 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:30.783063  6745 heartbeater.cc:507] Master 127.21.5.190:34941 requested a full tablet report, sending...
I20260709 13:17:30.783327  6683 ts_tablet_manager.cc:1434] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:30.784471  6683 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.790542  6259 ts_manager.cc:194] Re-registered known tserver with Master: 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103)
I20260709 13:17:30.795567  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:30.796574  6513 raft_consensus.cc:606] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:30.799630  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:30.800226  6513 raft_consensus.cc:606] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:30.800606  6683 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.801793  6683 tablet_bootstrap.cc:492] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.802572  6683 ts_tablet_manager.cc:1403] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.000s
I20260709 13:17:30.803009  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:30.803839  6513 raft_consensus.cc:606] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:30.804373  6259 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59426
I20260709 13:17:30.805126 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:30.805465  6683 raft_consensus.cc:359] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.806154 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:30.806231  6683 raft_consensus.cc:740] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.807181 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:30.807322  6683 consensus_queue.cc:260] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.809182  6745 heartbeater.cc:499] Master 127.21.5.190:34941 was elected leader, sending a full tablet report...
I20260709 13:17:30.809414  6683 ts_tablet_manager.cc:1434] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:17:30.810344  6683 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.825074  6683 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:17:30.826331  6683 tablet_bootstrap.cc:492] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.827239  6683 ts_tablet_manager.cc:1403] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.008s
I20260709 13:17:30.829947  6683 raft_consensus.cc:359] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.830822  6683 raft_consensus.cc:740] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.831910  6666 raft_consensus.cc:993] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:30.831679  6683 consensus_queue.cc:260] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.832691  6661 raft_consensus.cc:1081] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:30.834031  6683 ts_tablet_manager.cc:1434] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:17:30.834831  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:30.835359  6683 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.835712  6438 raft_consensus.cc:493] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:30.836476  6438 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.847009  6438 raft_consensus.cc:515] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.852002  6438 leader_election.cc:290] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:30.853791  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:30.854786  6513 raft_consensus.cc:3055] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:30.855170  6513 raft_consensus.cc:740] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:30.856223  6513 consensus_queue.cc:260] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.857957  6513 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.860858  6683 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.862236  6683 tablet_bootstrap.cc:492] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.863057  6683 ts_tablet_manager.cc:1403] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.004s
I20260709 13:17:30.866034  6683 raft_consensus.cc:359] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:30.866798  6719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a23d2e8677e44bd6a64cea6c2fb04f86" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:30.866871  6683 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.867635  6719 raft_consensus.cc:3060] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.868058  6683 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:30.868659  6513 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 3.
I20260709 13:17:30.870074  6402 leader_election.cc:304] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:30.870594  6683 ts_tablet_manager.cc:1434] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:30.872417  6655 raft_consensus.cc:2804] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:30.872982  6683 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.873065  6655 raft_consensus.cc:697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:30.874188  6655 consensus_queue.cc:237] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.877512  6719 raft_consensus.cc:2468] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 3.
I20260709 13:17:30.888847  6262 catalog_manager.cc:5697] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 2 to 3, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 3 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:30.893628  6683 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.894557  6683 tablet_bootstrap.cc:492] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.895303  6683 ts_tablet_manager.cc:1403] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.012s
I20260709 13:17:30.898237  6683 raft_consensus.cc:359] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.899055  6683 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.900096  6683 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.902192  6683 ts_tablet_manager.cc:1434] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:17:30.903344  6683 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap starting.
I20260709 13:17:30.920050  6683 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:17:30.921005  6683 tablet_bootstrap.cc:492] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Bootstrap complete.
I20260709 13:17:30.921659  6683 ts_tablet_manager.cc:1403] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.000s
I20260709 13:17:30.922449  6719 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:30.924652  6683 raft_consensus.cc:359] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.925441  6683 raft_consensus.cc:740] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3191e72fbc4a4687be69a048a0db8c1e, State: Initialized, Role: FOLLOWER
I20260709 13:17:30.926220  6683 consensus_queue.cc:260] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:30.928503  6683 ts_tablet_manager.cc:1434] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
W20260709 13:17:30.998032  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:31.093730  6666 raft_consensus.cc:993] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:31.094455  6661 raft_consensus.cc:1081] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:31.096624  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:31.097280  6438 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:31.097642  6438 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.099002  6719 raft_consensus.cc:3060] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.103354  6438 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:31.105455  6438 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:31.107031  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:31.106855  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:31.107877  6513 raft_consensus.cc:3055] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:31.108125  6718 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.108326  6513 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:31.109452  6513 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:31.112037  6513 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.117964  6718 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:31.119442  6513 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:31.119486  6402 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:31.120623  6655 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:31.121140  6655 raft_consensus.cc:697] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:31.122097  6655 consensus_queue.cc:237] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:31.128450  6666 raft_consensus.cc:993] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:31.129083  6666 raft_consensus.cc:1081] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:31.130985  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:31.131604  6438 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:31.132010  6438 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.132495  6259 catalog_manager.cc:5697] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 1 to 2, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 2 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:31.139412  6438 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:31.141520  6438 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Requested vote from peers 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307), 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103)
I20260709 13:17:31.142781  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:31.143081  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:31.143651  6513 raft_consensus.cc:3055] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:31.144032  6513 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:31.144076  6718 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.145092  6513 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:31.146538  6513 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.153395  6718 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:31.155268  6513 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:31.156227  6402 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:31.157226  6680 raft_consensus.cc:2804] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:31.157776  6680 raft_consensus.cc:697] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:31.158762  6680 consensus_queue.cc:237] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:31.166374  6259 catalog_manager.cc:5697] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 1 to 2, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 2 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:31.183485  6718 raft_consensus.cc:3060] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:31.190260  6718 pending_rounds.cc:85] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Aborting all ops after (but not including) 0
I20260709 13:17:31.190591  6718 pending_rounds.cc:107] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:17:31.190958  6718 raft_consensus.cc:2889] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:31.314571  6513 raft_consensus.cc:1275] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:31.316259  6680 consensus_queue.cc:1048] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:31.326668  6718 raft_consensus.cc:1275] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:31.328778  6680 consensus_queue.cc:1048] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:31.624063  6718 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:31.626055  6655 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:31.637255  6718 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:31.640338  6655 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:31.640673  6513 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:31.642751  6655 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:31.660035  6512 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:31.663060  6680 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:31.817394 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:31.840524  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:31.841248  6438 raft_consensus.cc:606] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:31.844367  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:31.844973  6438 raft_consensus.cc:606] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:31.846239 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:31.846968 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:31.847860 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:31.999464  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:32.023782  6655 raft_consensus.cc:993] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:32.024508  6680 raft_consensus.cc:1081] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:32.026443  6513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:32.027045  6513 raft_consensus.cc:493] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:32.027334  6513 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.038143  6513 raft_consensus.cc:515] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.040526  6513 leader_election.cc:290] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 4 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:32.041930  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:32.041924  6719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2f26317d8b64de88dee1aea602b36c5" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:32.042786  6438 raft_consensus.cc:3055] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:32.042918  6719 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.043118  6438 raft_consensus.cc:740] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:32.044338  6438 consensus_queue.cc:260] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.045590  6438 raft_consensus.cc:3060] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:32.051862  6719 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 4.
I20260709 13:17:32.054574  6476 leader_election.cc:304] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:32.055162  6438 raft_consensus.cc:2468] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 4.
I20260709 13:17:32.055945  6777 raft_consensus.cc:2804] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:32.056622  6777 raft_consensus.cc:697] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:32.058727  6777 consensus_queue.cc:237] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.068949  6259 catalog_manager.cc:5697] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 3 to 4, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.375947  6680 raft_consensus.cc:993] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:32.376497  6655 raft_consensus.cc:1081] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:32.378325  6513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:32.378988  6513 raft_consensus.cc:493] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:32.379383  6513 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:32.387404  6513 raft_consensus.cc:515] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.389474  6513 leader_election.cc:290] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:32.390612  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:32.390637  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:32.391458  6438 raft_consensus.cc:3055] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:32.391561  6718 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:32.391821  6438 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:32.392762  6438 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.394528  6438 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:32.401001  6718 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 2.
I20260709 13:17:32.401536  6438 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 2.
I20260709 13:17:32.402563  6476 leader_election.cc:304] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:32.403570  6777 raft_consensus.cc:2804] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:32.404142  6777 raft_consensus.cc:697] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:32.405210  6777 consensus_queue.cc:237] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:32.413965  6259 catalog_manager.cc:5697] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 1 to 2, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:32.517660  6718 raft_consensus.cc:1275] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:32.519273  6777 consensus_queue.cc:1048] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:32.531702  6438 raft_consensus.cc:1275] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:32.533552  6661 consensus_queue.cc:1048] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:32.855036 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:32.863339  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33cb1674e38245feae34aaa325730adb"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:32.863994  6513 raft_consensus.cc:606] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:32.867292  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:32.867895  6513 raft_consensus.cc:606] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:32.870357  6718 raft_consensus.cc:1275] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:32.871047  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:32.871819  6438 raft_consensus.cc:606] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:32.872061  6792 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:32.873322 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:32.873994 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:32.874758 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:32.882860  6438 raft_consensus.cc:1275] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:32.885043  6661 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:17:33.000797  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:33.061691  6777 raft_consensus.cc:993] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:33.062322  6661 raft_consensus.cc:1081] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:33.064050  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:33.064585  6438 raft_consensus.cc:493] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.064899  6438 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.070847  6438 raft_consensus.cc:515] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.072755  6438 leader_election.cc:290] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:33.073923  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:33.074168  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "663726b99cef48a4a549cac3bcfd2967" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:33.074761  6718 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.074862  6513 raft_consensus.cc:3055] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:33.075364  6513 raft_consensus.cc:740] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:33.076233  6513 consensus_queue.cc:260] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.077618  6513 raft_consensus.cc:3060] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.083389  6718 raft_consensus.cc:2468] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:33.085892  6402 leader_election.cc:304] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:33.086910  6680 raft_consensus.cc:2804] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:33.087479  6680 raft_consensus.cc:697] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:33.088054  6513 raft_consensus.cc:2468] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:33.088747  6680 consensus_queue.cc:237] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.097416  6259 catalog_manager.cc:5697] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 1 to 2, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 2 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.152825  6777 raft_consensus.cc:993] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:33.153375  6661 raft_consensus.cc:1081] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:33.155339  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33cb1674e38245feae34aaa325730adb"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:33.155915  6438 raft_consensus.cc:493] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.156230  6438 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.162433  6438 raft_consensus.cc:515] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.164255  6438 leader_election.cc:290] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:33.165443  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:33.165581  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:33.166420  6718 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.166489  6513 raft_consensus.cc:3055] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:33.167032  6513 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:33.167994  6513 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.169194  6513 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:33.173985  6718 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:33.175643  6402 leader_election.cc:304] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:33.176666  6513 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 2.
I20260709 13:17:33.176913  6680 raft_consensus.cc:2804] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:33.177597  6680 raft_consensus.cc:697] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:33.178798  6680 consensus_queue.cc:237] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.187063  6259 catalog_manager.cc:5697] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 1 to 2, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 2 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:33.322046  6655 raft_consensus.cc:993] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:33.322716  6680 raft_consensus.cc:1081] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:33.324555  6513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:33.325150  6513 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.325479  6513 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.333098  6513 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.334921  6513 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:33.336110  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:33.336304  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:33.336885  6438 raft_consensus.cc:3055] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.336979  6718 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.337225  6438 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:33.338306  6438 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.339648  6438 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.344520  6718 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:33.346110  6476 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:33.347105  6661 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:33.347289  6438 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:33.347752  6661 raft_consensus.cc:697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:33.349581  6661 consensus_queue.cc:237] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.358498  6259 catalog_manager.cc:5697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 2 to 3, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:33.506711  6718 raft_consensus.cc:1275] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.508234  6680 consensus_queue.cc:1048] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.517705  6513 raft_consensus.cc:1275] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.520285  6655 consensus_queue.cc:1048] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.573215  6513 raft_consensus.cc:1275] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.574715  6655 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.588517  6718 raft_consensus.cc:1275] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:33.590919  6680 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:33.858287  6718 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.859912  6661 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:33.873466  6438 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:33.875228  6815 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:33.885115 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:33.894685  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33cb1674e38245feae34aaa325730adb"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:33.895332  6438 raft_consensus.cc:606] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:33.898260  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:33.898826  6438 raft_consensus.cc:606] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:33.901346  6513 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:33.901841  6513 raft_consensus.cc:606] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:33.903118 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:33.903743 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:33.904451 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:33.986174  6680 raft_consensus.cc:993] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:33.986876  6655 raft_consensus.cc:1081] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:33.988312  6513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33cb1674e38245feae34aaa325730adb"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:33.988824  6513 raft_consensus.cc:493] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:33.989122  6513 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.995159  6513 raft_consensus.cc:515] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:33.996820  6513 leader_election.cc:290] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:33.998087  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:33.998109  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33cb1674e38245feae34aaa325730adb" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:33.998833  6718 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:33.998946  6438 raft_consensus.cc:3055] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:33.999383  6438 raft_consensus.cc:740] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:34.000044  6438 consensus_queue.cc:260] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:34.001083  6438 raft_consensus.cc:3060] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:34.002091  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:34.006186  6718 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:34.006927  6438 raft_consensus.cc:2468] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:34.007822  6476 leader_election.cc:304] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:34.008821  6815 raft_consensus.cc:2804] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:34.009398  6815 raft_consensus.cc:697] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:34.010231  6815 consensus_queue.cc:237] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:34.017854  6261 catalog_manager.cc:5697] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 2 to 3, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:34.049736  6680 raft_consensus.cc:993] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:34.050587  6655 raft_consensus.cc:1081] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:34.052266  6512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:34.052842  6512 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:34.053177  6512 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:34.061023  6512 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:34.063184  6512 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:34.064137  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:34.064458  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:34.064924  6718 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:34.065173  6438 raft_consensus.cc:3055] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:34.065585  6438 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:34.066445  6438 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:34.067770  6438 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:34.072914  6718 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:34.074296  6476 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:34.074980  6438 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 3.
I20260709 13:17:34.075098  6815 raft_consensus.cc:2804] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:34.075650  6815 raft_consensus.cc:697] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:34.076690  6815 consensus_queue.cc:237] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:34.084890  6261 catalog_manager.cc:5697] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 2 to 3, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.494843  6438 raft_consensus.cc:1275] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:34.496134  6815 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:34.513027  6718 raft_consensus.cc:1275] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:34.514330  6815 raft_consensus.cc:993] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:34.515504  6823 raft_consensus.cc:1081] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:34.515586  6815 consensus_queue.cc:1048] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:34.518213  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:34.518914  6438 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:34.519724  6438 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.530112  6438 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:34.532073  6438 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 4 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:34.533334  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:34.533377  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:34.534341  6512 raft_consensus.cc:3055] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:34.534395  6718 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.534945  6512 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:34.535982  6512 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:34.537611  6512 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:34.542927  6718 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 4.
I20260709 13:17:34.544688  6402 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:34.545321  6512 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 4.
I20260709 13:17:34.545660  6655 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:34.546317  6655 raft_consensus.cc:697] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:34.547299  6655 consensus_queue.cc:237] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:34.554788  6259 catalog_manager.cc:5697] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 3 to 4, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 4 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:34.668957  6718 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:34.670517  6815 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:34.682128  6438 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:34.683825  6815 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:34.911863 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:34.919922  6512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:34.920742  6512 raft_consensus.cc:606] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:34.924157  6512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
mode: GRACEFUL
new_leader_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:55244
I20260709 13:17:34.924755  6512 raft_consensus.cc:606] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Received request to transfer leadership to TS 30d9641628464c388fe9162931b2e50a
I20260709 13:17:34.927772  6438 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
mode: GRACEFUL
new_leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:46956
I20260709 13:17:34.928351  6438 raft_consensus.cc:606] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Received request to transfer leadership to TS 1547e62e8b0b4fb0875bb48ec8029011
I20260709 13:17:34.929863 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:17:34.930522 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:34.931242 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:35.003381  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:35.025489  6512 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:35.027235  6655 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:35.034461  6807 raft_consensus.cc:993] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: : Instructing follower 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:35.035925  6807 raft_consensus.cc:1081] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Signalling peer 1547e62e8b0b4fb0875bb48ec8029011 to start an election
I20260709 13:17:35.038139  6513 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003"
dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
 from {username='slave'} at 127.0.0.1:38546
I20260709 13:17:35.038838  6513 raft_consensus.cc:493] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:35.039263  6513 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.041750  6718 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:35.043591  6832 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:35.049131  6513 raft_consensus.cc:515] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.053686  6438 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "30d9641628464c388fe9162931b2e50a"
I20260709 13:17:35.054350  6438 raft_consensus.cc:3055] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:35.054673  6438 raft_consensus.cc:740] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:35.055404  6438 consensus_queue.cc:260] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.056401  6438 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.059940  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab7a4874aa479b8ffab687976b4003" candidate_uuid: "1547e62e8b0b4fb0875bb48ec8029011" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:35.060741  6718 raft_consensus.cc:3060] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:35.065151  6438 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 5.
I20260709 13:17:35.066380  6513 leader_election.cc:290] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 5 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685)
I20260709 13:17:35.066926  6478 leader_election.cc:304] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1547e62e8b0b4fb0875bb48ec8029011, 30d9641628464c388fe9162931b2e50a; no voters: 
I20260709 13:17:35.067880  6815 raft_consensus.cc:2804] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:35.068393  6815 raft_consensus.cc:697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 5 LEADER]: Becoming Leader. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:35.069149  6815 consensus_queue.cc:237] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.069763  6718 raft_consensus.cc:2468] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1547e62e8b0b4fb0875bb48ec8029011 in term 5.
I20260709 13:17:35.076799  6261 catalog_manager.cc:5697] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 reported cstate change: term changed from 4 to 5, leader changed from 30d9641628464c388fe9162931b2e50a (127.21.5.130) to 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131). New cstate: current_term: 5 leader_uuid: "1547e62e8b0b4fb0875bb48ec8029011" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.255178  6822 raft_consensus.cc:993] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:35.255931  6826 raft_consensus.cc:1081] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:35.257740  6439 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:35.258358  6439 raft_consensus.cc:493] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:35.258785  6439 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:35.259163  6826 raft_consensus.cc:993] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: : Instructing follower 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:35.259771  6815 raft_consensus.cc:1081] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Signalling peer 30d9641628464c388fe9162931b2e50a to start an election
I20260709 13:17:35.261463  6438 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450"
dest_uuid: "30d9641628464c388fe9162931b2e50a"
 from {username='slave'} at 127.0.0.1:48416
I20260709 13:17:35.262001  6438 raft_consensus.cc:493] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:35.262290  6438 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:35.268021  6439 raft_consensus.cc:515] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:35.268962  6438 raft_consensus.cc:515] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.270232  6439 leader_election.cc:290] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 4 election: Requested vote from peers 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307), 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103)
I20260709 13:17:35.270706  6438 leader_election.cc:290] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Requested vote from peers 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103), 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131:37307)
I20260709 13:17:35.271428  6718 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:35.271898  6719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3191e72fbc4a4687be69a048a0db8c1e"
I20260709 13:17:35.272068  6718 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:35.271958  6513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc55c1f1361406498184cefd741f4b6" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:35.272787  6719 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:35.273099  6513 raft_consensus.cc:3055] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:35.273403  6513 raft_consensus.cc:740] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:35.274215  6513 consensus_queue.cc:260] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:35.275557  6513 raft_consensus.cc:3060] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:35.278255  6718 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 4.
I20260709 13:17:35.272055  6512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5bc9b7472ec4638970ca1dd0fb43450" candidate_uuid: "30d9641628464c388fe9162931b2e50a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1547e62e8b0b4fb0875bb48ec8029011"
I20260709 13:17:35.278921  6512 raft_consensus.cc:3055] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:35.279263  6512 raft_consensus.cc:740] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1547e62e8b0b4fb0875bb48ec8029011, State: Running, Role: LEADER
I20260709 13:17:35.279680  6402 leader_election.cc:304] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:35.280542  6832 raft_consensus.cc:2804] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:35.280478  6719 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 3.
I20260709 13:17:35.281323  6832 raft_consensus.cc:697] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:35.280161  6512 consensus_queue.cc:260] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.282229  6832 consensus_queue.cc:237] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } }
I20260709 13:17:35.283164  6513 raft_consensus.cc:2468] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 4.
I20260709 13:17:35.283584  6512 raft_consensus.cc:3060] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:35.284525  6402 leader_election.cc:304] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30d9641628464c388fe9162931b2e50a, 3191e72fbc4a4687be69a048a0db8c1e; no voters: 
I20260709 13:17:35.285256  6807 raft_consensus.cc:2804] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:35.285755  6807 raft_consensus.cc:697] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Becoming Leader. State: Replica: 30d9641628464c388fe9162931b2e50a, State: Running, Role: LEADER
I20260709 13:17:35.286458  6807 consensus_queue.cc:237] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } }
I20260709 13:17:35.291654  6261 catalog_manager.cc:5697] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 3 to 4, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 4 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:35.294799  6512 raft_consensus.cc:2468] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30d9641628464c388fe9162931b2e50a in term 3.
I20260709 13:17:35.304499  6261 catalog_manager.cc:5697] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a reported cstate change: term changed from 2 to 3, leader changed from 1547e62e8b0b4fb0875bb48ec8029011 (127.21.5.131) to 30d9641628464c388fe9162931b2e50a (127.21.5.130). New cstate: current_term: 3 leader_uuid: "30d9641628464c388fe9162931b2e50a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:35.486861  6718 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 5 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:35.488229  6815 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:35.496573  6439 raft_consensus.cc:1275] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 5 FOLLOWER]: Refusing update from remote peer 1547e62e8b0b4fb0875bb48ec8029011: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:35.498597  6826 consensus_queue.cc:1048] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30d9641628464c388fe9162931b2e50a" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 36685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:35.713513  6512 raft_consensus.cc:1275] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:35.715087  6807 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:35.725932  6718 raft_consensus.cc:1275] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:35.727876  6832 consensus_queue.cc:1048] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:35.833063  6718 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:35.834533  6832 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3191e72fbc4a4687be69a048a0db8c1e" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43103 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:35.844542  6512 raft_consensus.cc:1275] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Refusing update from remote peer 30d9641628464c388fe9162931b2e50a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:35.846822  6807 consensus_queue.cc:1048] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1547e62e8b0b4fb0875bb48ec8029011" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 37307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:35.939029 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:35.976604 21526 tablet_server.cc:179] TabletServer@127.21.5.129:43103 shutting down...
W20260709 13:17:36.004880  6310 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:17:36.008226 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.009270 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.010142 21526 raft_consensus.cc:2243] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.010686 21526 raft_consensus.cc:2272] T 9f7d9accc9634a3497956966206488eb P 3191e72fbc4a4687be69a048a0db8c1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.013516 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.014235 21526 raft_consensus.cc:2243] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.014814 21526 raft_consensus.cc:2272] T bfc55c1f1361406498184cefd741f4b6 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.018319 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.019052 21526 raft_consensus.cc:2243] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.019569 21526 raft_consensus.cc:2272] T 513d24a089564be8b2c88697d453b336 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.022380 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.023067 21526 raft_consensus.cc:2243] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.023510 21526 raft_consensus.cc:2272] T c2f26317d8b64de88dee1aea602b36c5 P 3191e72fbc4a4687be69a048a0db8c1e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.026077 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.026712 21526 raft_consensus.cc:2243] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.027163 21526 raft_consensus.cc:2272] T 663726b99cef48a4a549cac3bcfd2967 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.029590 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.030105 21526 raft_consensus.cc:2243] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.030637 21526 raft_consensus.cc:2272] T 9d9afc925d7e40098fa97ce4453ed336 P 3191e72fbc4a4687be69a048a0db8c1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.033332 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.033953 21526 raft_consensus.cc:2243] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.034529 21526 raft_consensus.cc:2272] T 33cb1674e38245feae34aaa325730adb P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.037043 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.037559 21526 raft_consensus.cc:2243] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.037961 21526 raft_consensus.cc:2272] T a5bc9b7472ec4638970ca1dd0fb43450 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.041368 21526 tablet_replica.cc:333] stopping tablet replica
I20260709 13:17:36.042060 21526 raft_consensus.cc:2243] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.042771 21526 raft_consensus.cc:2272] T e5ab7a4874aa479b8ffab687976b4003 P 3191e72fbc4a4687be69a048a0db8c1e [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:36.043519  6476 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 94 similar messages]
I20260709 13:17:36.046638 21526 tablet_replica.cc:333] stopping tablet replica
W20260709 13:17:36.047394  6476 consensus_peers.cc:597] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.047483 21526 raft_consensus.cc:2243] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.048231 21526 raft_consensus.cc:2272] T a23d2e8677e44bd6a64cea6c2fb04f86 P 3191e72fbc4a4687be69a048a0db8c1e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.077041 21526 tablet_server.cc:196] TabletServer@127.21.5.129:43103 shutdown complete.
I20260709 13:17:36.111253 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
W20260709 13:17:36.119259  6478 consensus_peers.cc:597] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Couldn't send request to peer 30d9641628464c388fe9162931b2e50a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:36.122939  6478 consensus_peers.cc:597] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 30d9641628464c388fe9162931b2e50a (127.21.5.130:36685): Couldn't send request to peer 30d9641628464c388fe9162931b2e50a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.147707 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.148633 21526 tablet_replica.cc:333] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.149451 21526 raft_consensus.cc:2243] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.150458 21526 raft_consensus.cc:2272] T 663726b99cef48a4a549cac3bcfd2967 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.153375 21526 tablet_replica.cc:333] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.154136 21526 raft_consensus.cc:2243] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.155185 21526 raft_consensus.cc:2272] T 9f7d9accc9634a3497956966206488eb P 30d9641628464c388fe9162931b2e50a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.157727 21526 tablet_replica.cc:333] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.158301 21526 raft_consensus.cc:2243] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.159335 21526 raft_consensus.cc:2272] T a5bc9b7472ec4638970ca1dd0fb43450 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.162581 21526 tablet_replica.cc:333] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.163308 21526 raft_consensus.cc:2243] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.163800 21526 raft_consensus.cc:2272] T 33cb1674e38245feae34aaa325730adb P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.166250 21526 tablet_replica.cc:333] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.166937 21526 raft_consensus.cc:2243] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.167484 21526 raft_consensus.cc:2272] T 9d9afc925d7e40098fa97ce4453ed336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.170203 21526 tablet_replica.cc:333] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.170949 21526 raft_consensus.cc:2243] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.171897 21526 raft_consensus.cc:2272] T bfc55c1f1361406498184cefd741f4b6 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.175339 21526 tablet_replica.cc:333] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.176014 21526 raft_consensus.cc:2243] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.176532 21526 raft_consensus.cc:2272] T 513d24a089564be8b2c88697d453b336 P 30d9641628464c388fe9162931b2e50a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.179260 21526 tablet_replica.cc:333] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.179850 21526 raft_consensus.cc:2243] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.180289 21526 raft_consensus.cc:2272] T c2f26317d8b64de88dee1aea602b36c5 P 30d9641628464c388fe9162931b2e50a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.183061 21526 tablet_replica.cc:333] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.183681 21526 raft_consensus.cc:2243] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.184221 21526 raft_consensus.cc:2272] T e5ab7a4874aa479b8ffab687976b4003 P 30d9641628464c388fe9162931b2e50a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.187475 21526 tablet_replica.cc:333] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a: stopping tablet replica
I20260709 13:17:36.188138 21526 raft_consensus.cc:2243] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.189181 21526 raft_consensus.cc:2272] T a23d2e8677e44bd6a64cea6c2fb04f86 P 30d9641628464c388fe9162931b2e50a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.226438 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
W20260709 13:17:36.239050  6476 consensus_peers.cc:597] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 -> Peer 3191e72fbc4a4687be69a048a0db8c1e (127.21.5.129:43103): Couldn't send request to peer 3191e72fbc4a4687be69a048a0db8c1e. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43103: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:36.262420 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
I20260709 13:17:36.295333 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:36.296201 21526 tablet_replica.cc:333] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.297035 21526 raft_consensus.cc:2243] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.298194 21526 raft_consensus.cc:2272] T 9d9afc925d7e40098fa97ce4453ed336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.301090 21526 tablet_replica.cc:333] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.301764 21526 raft_consensus.cc:2243] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.302409 21526 raft_consensus.cc:2272] T a23d2e8677e44bd6a64cea6c2fb04f86 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.305092 21526 tablet_replica.cc:333] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.305660 21526 raft_consensus.cc:2243] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.306548 21526 raft_consensus.cc:2272] T e5ab7a4874aa479b8ffab687976b4003 P 1547e62e8b0b4fb0875bb48ec8029011 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.309798 21526 tablet_replica.cc:333] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.310537 21526 raft_consensus.cc:2243] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.311128 21526 raft_consensus.cc:2272] T 663726b99cef48a4a549cac3bcfd2967 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.313895 21526 tablet_replica.cc:333] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.314673 21526 raft_consensus.cc:2243] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.315289 21526 raft_consensus.cc:2272] T 9f7d9accc9634a3497956966206488eb P 1547e62e8b0b4fb0875bb48ec8029011 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.318114 21526 tablet_replica.cc:333] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.318867 21526 raft_consensus.cc:2243] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.319386 21526 raft_consensus.cc:2272] T a5bc9b7472ec4638970ca1dd0fb43450 P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.322532 21526 tablet_replica.cc:333] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.323274 21526 raft_consensus.cc:2243] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.324258 21526 raft_consensus.cc:2272] T 33cb1674e38245feae34aaa325730adb P 1547e62e8b0b4fb0875bb48ec8029011 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.327106 21526 tablet_replica.cc:333] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.327725 21526 raft_consensus.cc:2243] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.328686 21526 raft_consensus.cc:2272] T c2f26317d8b64de88dee1aea602b36c5 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.331533 21526 tablet_replica.cc:333] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.332366 21526 raft_consensus.cc:2243] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.333304 21526 raft_consensus.cc:2272] T 513d24a089564be8b2c88697d453b336 P 1547e62e8b0b4fb0875bb48ec8029011 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.336227 21526 tablet_replica.cc:333] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011: stopping tablet replica
I20260709 13:17:36.336738 21526 raft_consensus.cc:2243] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:36.337175 21526 raft_consensus.cc:2272] T bfc55c1f1361406498184cefd741f4b6 P 1547e62e8b0b4fb0875bb48ec8029011 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.373347 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:17:36.405186 21526 master.cc:562] Master@127.21.5.190:34941 shutting down...
I20260709 13:17:36.440238 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:36.441035 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:36.441442 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P aa44b1260e0342eba1882cfae98eaea6: stopping tablet replica
I20260709 13:17:36.465772 21526 master.cc:584] Master@127.21.5.190:34941 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13705 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:17:36.524857 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:43845
I20260709 13:17:36.526237 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.533582  6848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.533591  6849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.533845  6851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.535032 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:36.536310 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.536527 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.536648 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603056536636 us; error 0 us; skew 500 ppm
I20260709 13:17:36.537163 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.539983 21526 webserver.cc:533] Webserver started at http://127.21.5.190:34089/ using document root <none> and password file <none>
I20260709 13:17:36.540534 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.540707 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.540956 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.542220 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "6575f107f8d5498a8416b8e5ec1f9019"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.548254 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 13:17:36.552325  6856 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.553333 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:36.553679 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "6575f107f8d5498a8416b8e5ec1f9019"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.554008 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.586108 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.587736 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.634526 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:43845
I20260709 13:17:36.634747  6907 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:43845 every 8 connection(s)
I20260709 13:17:36.639093  6908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:36.653743  6908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019: Bootstrap starting.
I20260709 13:17:36.659866  6908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:36.665446  6908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019: No bootstrap required, opened a new log
I20260709 13:17:36.668313  6908 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER }
I20260709 13:17:36.668818  6908 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:36.669107  6908 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6575f107f8d5498a8416b8e5ec1f9019, State: Initialized, Role: FOLLOWER
I20260709 13:17:36.669806  6908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER }
I20260709 13:17:36.670343  6908 raft_consensus.cc:399] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:36.670598  6908 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:36.670953  6908 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:36.677275  6908 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER }
I20260709 13:17:36.677956  6908 leader_election.cc:304] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6575f107f8d5498a8416b8e5ec1f9019; no voters: 
I20260709 13:17:36.679440  6908 leader_election.cc:290] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:36.679874  6911 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:36.681577  6911 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 1 LEADER]: Becoming Leader. State: Replica: 6575f107f8d5498a8416b8e5ec1f9019, State: Running, Role: LEADER
I20260709 13:17:36.682312  6911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER }
I20260709 13:17:36.683074  6908 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:36.687744  6912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6575f107f8d5498a8416b8e5ec1f9019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER } }
I20260709 13:17:36.688207  6913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6575f107f8d5498a8416b8e5ec1f9019. Latest consensus state: current_term: 1 leader_uuid: "6575f107f8d5498a8416b8e5ec1f9019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6575f107f8d5498a8416b8e5ec1f9019" member_type: VOTER } }
I20260709 13:17:36.689077  6913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:36.688542  6912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:36.698012  6919 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:36.706449  6919 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:36.707764 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:36.717371  6919 catalog_manager.cc:1383] Generated new cluster ID: b1a8a0a9d2924a84bcd5629007a6c13e
I20260709 13:17:36.717881  6919 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:36.754530  6919 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:36.756350  6919 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:36.773061  6919 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019: Generated new TSK 0
I20260709 13:17:36.773818  6919 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:36.839852 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.847429  6929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.848922  6930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.851791  6932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.851765 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:36.853415 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:36.853703 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.853915 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603056853891 us; error 0 us; skew 500 ppm
I20260709 13:17:36.854797 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.857865 21526 webserver.cc:533] Webserver started at http://127.21.5.129:44809/ using document root <none> and password file <none>
I20260709 13:17:36.858429 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.858662 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.858982 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.860558 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "0c7c85d3a7294ab8918a5450fc459255"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.866137 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:36.870373  6937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.871296 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:36.871654 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "0c7c85d3a7294ab8918a5450fc459255"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.871974 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:36.886756 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:36.888178 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:36.889919 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:36.892750 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:36.893031 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.893299 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:36.893519 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.941706 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:43831
I20260709 13:17:36.941831  6999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:43831 every 8 connection(s)
I20260709 13:17:36.947732 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:36.957669  7004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.958321  7005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:36.961165  7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:36.962594 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:36.964205  7000 heartbeater.cc:344] Connected to a master server at 127.21.5.190:43845
I20260709 13:17:36.964352 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:17:36.964718  7000 heartbeater.cc:461] Registering TS with master...
W20260709 13:17:36.964795 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:36.965375 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603056965338 us; error 0 us; skew 500 ppm
I20260709 13:17:36.965848  7000 heartbeater.cc:507] Master 127.21.5.190:43845 requested a full tablet report, sending...
I20260709 13:17:36.966313 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:36.968770  6873 ts_manager.cc:194] Registered new tserver with Master: 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831)
I20260709 13:17:36.969635 21526 webserver.cc:533] Webserver started at http://127.21.5.130:33075/ using document root <none> and password file <none>
I20260709 13:17:36.970310 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:36.970551 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:36.970916 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:36.971194  6873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32848
I20260709 13:17:36.972489 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "0210e512b5a0441896a1748d566d02ba"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.978219 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.004s
I20260709 13:17:36.982669  7012 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:36.983794 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:36.984136 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "0210e512b5a0441896a1748d566d02ba"
format_stamp: "Formatted at 2026-07-09 13:17:36 on dist-test-slave-h0rz"
I20260709 13:17:36.984428 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.010886 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.012257 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.013903 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.016534 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.016856 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.017179 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.017431 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.065528 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:32833
I20260709 13:17:37.065659  7074 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:32833 every 8 connection(s)
I20260709 13:17:37.071638 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:37.081202  7078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:37.082556  7079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.085563  7075 heartbeater.cc:344] Connected to a master server at 127.21.5.190:43845
I20260709 13:17:37.086130  7075 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.087106 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:37.087253  7075 heartbeater.cc:507] Master 127.21.5.190:43845 requested a full tablet report, sending...
W20260709 13:17:37.087565  7081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:37.088946 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:37.089207 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:37.089457 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603057089432 us; error 0 us; skew 500 ppm
I20260709 13:17:37.089967  6873 ts_manager.cc:194] Registered new tserver with Master: 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833)
I20260709 13:17:37.090114 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:37.092087  6873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32858
I20260709 13:17:37.093941 21526 webserver.cc:533] Webserver started at http://127.21.5.131:42779/ using document root <none> and password file <none>
I20260709 13:17:37.094552 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:37.094828 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:37.095181 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:37.096477 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-h0rz"
I20260709 13:17:37.101751 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:17:37.105899  7086 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.107023 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:37.107415 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
format_stamp: "Formatted at 2026-07-09 13:17:37 on dist-test-slave-h0rz"
I20260709 13:17:37.107755 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:37.134429 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:37.135912 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:37.137734 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:37.140484 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:37.140767 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.141057 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:37.141255 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:37.191606 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:45495
I20260709 13:17:37.191753  7148 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:45495 every 8 connection(s)
I20260709 13:17:37.208485  7149 heartbeater.cc:344] Connected to a master server at 127.21.5.190:43845
I20260709 13:17:37.209066  7149 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:37.210003  7149 heartbeater.cc:507] Master 127.21.5.190:43845 requested a full tablet report, sending...
I20260709 13:17:37.212692  6873 ts_manager.cc:194] Registered new tserver with Master: 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.212847 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017743828s
I20260709 13:17:37.214447  6873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32872
I20260709 13:17:37.243098  6873 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32880:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:37.245851  6873 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:37.306414  6965 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac2517fb431495f9dad4d81f38f51e3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.307695  6964 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddaee59e8a4402eb90b72287f1c3916 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.308216  6965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac2517fb431495f9dad4d81f38f51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.309233  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddaee59e8a4402eb90b72287f1c3916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.314767  7114 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac2517fb431495f9dad4d81f38f51e3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.314893  7113 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddaee59e8a4402eb90b72287f1c3916 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.316525  7114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac2517fb431495f9dad4d81f38f51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.317443  7113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddaee59e8a4402eb90b72287f1c3916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.320842  7040 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac2517fb431495f9dad4d81f38f51e3 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.322479  7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac2517fb431495f9dad4d81f38f51e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.326409  7039 tablet_service.cc:1511] Processing CreateTablet for tablet 1ddaee59e8a4402eb90b72287f1c3916 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=575754e9ff6a4946bc0ab1bd00929230]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.328063  7039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ddaee59e8a4402eb90b72287f1c3916. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.340826  7164 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: Bootstrap starting.
I20260709 13:17:37.350354  7165 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Bootstrap starting.
I20260709 13:17:37.362344  7165 tablet_bootstrap.cc:654] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.367619  7164 tablet_bootstrap.cc:654] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.369989  7166 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: Bootstrap starting.
I20260709 13:17:37.378221  7165 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: No bootstrap required, opened a new log
I20260709 13:17:37.379251  7165 ts_tablet_manager.cc:1403] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.015s
I20260709 13:17:37.382287  7164 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: No bootstrap required, opened a new log
I20260709 13:17:37.382781  7166 tablet_bootstrap.cc:654] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.382913  7164 ts_tablet_manager.cc:1403] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.010s
I20260709 13:17:37.383394  7165 raft_consensus.cc:359] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.384477  7165 raft_consensus.cc:385] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.384925  7165 raft_consensus.cc:740] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.386018  7165 consensus_queue.cc:260] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.386504  7164 raft_consensus.cc:359] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.387588  7164 raft_consensus.cc:385] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.388005  7164 raft_consensus.cc:740] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.389070  7164 consensus_queue.cc:260] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.389884  7149 heartbeater.cc:499] Master 127.21.5.190:43845 was elected leader, sending a full tablet report...
I20260709 13:17:37.390837  7165 ts_tablet_manager.cc:1434] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 13:17:37.392522  7165 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Bootstrap starting.
I20260709 13:17:37.396055  7166 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: No bootstrap required, opened a new log
I20260709 13:17:37.396903  7166 ts_tablet_manager.cc:1403] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.009s
I20260709 13:17:37.400655  7165 tablet_bootstrap.cc:654] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.401064  7166 raft_consensus.cc:359] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.402096  7166 raft_consensus.cc:385] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.402542  7166 raft_consensus.cc:740] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.403702  7166 consensus_queue.cc:260] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.412878  7075 heartbeater.cc:499] Master 127.21.5.190:43845 was elected leader, sending a full tablet report...
I20260709 13:17:37.414296  7165 tablet_bootstrap.cc:492] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: No bootstrap required, opened a new log
I20260709 13:17:37.415412  7165 ts_tablet_manager.cc:1403] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.003s
I20260709 13:17:37.414345  7166 ts_tablet_manager.cc:1434] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.007s
I20260709 13:17:37.418784  7000 heartbeater.cc:499] Master 127.21.5.190:43845 was elected leader, sending a full tablet report...
I20260709 13:17:37.419219  7164 ts_tablet_manager.cc:1434] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: Time spent starting tablet: real 0.036s	user 0.022s	sys 0.011s
I20260709 13:17:37.420379  7164 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: Bootstrap starting.
I20260709 13:17:37.420974  7165 raft_consensus.cc:359] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.421852  7165 raft_consensus.cc:385] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.422173  7165 raft_consensus.cc:740] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.423112  7165 consensus_queue.cc:260] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.425771  7165 ts_tablet_manager.cc:1434] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:17:37.426673  7166 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: Bootstrap starting.
I20260709 13:17:37.429055  7164 tablet_bootstrap.cc:654] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.435320  7166 tablet_bootstrap.cc:654] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.437884  7164 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: No bootstrap required, opened a new log
I20260709 13:17:37.438561  7164 ts_tablet_manager.cc:1403] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 13:17:37.442370  7166 tablet_bootstrap.cc:492] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: No bootstrap required, opened a new log
I20260709 13:17:37.442006  7164 raft_consensus.cc:359] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.443040  7164 raft_consensus.cc:385] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.443388  7166 ts_tablet_manager.cc:1403] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.001s
I20260709 13:17:37.443710  7171 raft_consensus.cc:493] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.444756  7171 raft_consensus.cc:515] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.443423  7164 raft_consensus.cc:740] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.447347  7166 raft_consensus.cc:359] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.448122  7171 leader_election.cc:290] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.448326  7166 raft_consensus.cc:385] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.448086  7164 consensus_queue.cc:260] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.448971  7166 raft_consensus.cc:740] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.450214  7166 consensus_queue.cc:260] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.460201  7166 ts_tablet_manager.cc:1434] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:17:37.450963  7164 ts_tablet_manager.cc:1434] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:17:37.467640  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0210e512b5a0441896a1748d566d02ba" is_pre_election: true
I20260709 13:17:37.468526  7050 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 0.
I20260709 13:17:37.470535  6938 leader_election.cc:304] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:37.471500  7171 raft_consensus.cc:2804] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:37.471926  7171 raft_consensus.cc:493] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.472342  7171 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.472735  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" is_pre_election: true
I20260709 13:17:37.473711  7124 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 0.
I20260709 13:17:37.479651  7171 raft_consensus.cc:515] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.481652  7171 leader_election.cc:290] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 1 election: Requested vote from peers 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.482945  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:37.483173  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:37.483924  7050 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.484009  7124 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.490960  7050 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 1.
I20260709 13:17:37.490962  7124 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 1.
I20260709 13:17:37.492450  6938 leader_election.cc:304] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7c85d3a7294ab8918a5450fc459255, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:37.493372  7171 raft_consensus.cc:2804] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.494836  7171 raft_consensus.cc:697] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 LEADER]: Becoming Leader. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:37.495980  7171 consensus_queue.cc:237] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.506415  6870 catalog_manager.cc:5697] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "0c7c85d3a7294ab8918a5450fc459255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.541743  7173 raft_consensus.cc:493] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.542455  7173 raft_consensus.cc:515] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.545259  7173 leader_election.cc:290] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.555615  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255" is_pre_election: true
I20260709 13:17:37.556597  6975 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 0.
I20260709 13:17:37.557749  7013 leader_election.cc:304] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:37.557876  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" is_pre_election: true
I20260709 13:17:37.558650  7173 raft_consensus.cc:2804] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:37.558713  7124 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 0.
I20260709 13:17:37.559165  7173 raft_consensus.cc:493] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.559571  7173 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.566583  7173 raft_consensus.cc:515] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.568660  7173 leader_election.cc:290] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 election: Requested vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.570093  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:37.569855  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
I20260709 13:17:37.570875  6975 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.570875  7124 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.580161  7124 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 1.
I20260709 13:17:37.580161  6975 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 1.
I20260709 13:17:37.581920  7013 leader_election.cc:304] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:37.582990  7173 raft_consensus.cc:2804] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.584336  7173 raft_consensus.cc:697] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Becoming Leader. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Running, Role: LEADER
I20260709 13:17:37.585237  7173 consensus_queue.cc:237] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:37.593901  6870 catalog_manager.cc:5697] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 0210e512b5a0441896a1748d566d02ba (127.21.5.130). New cstate: current_term: 1 leader_uuid: "0210e512b5a0441896a1748d566d02ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:37.671185  6870 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32884:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:37.673449  6870 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260709 13:17:37.696638  6921 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:37.700589  6965 tablet_service.cc:1511] Processing CreateTablet for tablet da8bd8f392a340bea4cc2d8f3565924c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.702248  6965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8bd8f392a340bea4cc2d8f3565924c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.702212  7040 tablet_service.cc:1511] Processing CreateTablet for tablet da8bd8f392a340bea4cc2d8f3565924c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.703860  7040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8bd8f392a340bea4cc2d8f3565924c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.704170  7113 tablet_service.cc:1511] Processing CreateTablet for tablet da8bd8f392a340bea4cc2d8f3565924c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:17:37.705581  7113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da8bd8f392a340bea4cc2d8f3565924c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.709004  7039 tablet_service.cc:1511] Processing CreateTablet for tablet a3d66de192604f0598b9a47002fb4714 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.709249  7114 tablet_service.cc:1511] Processing CreateTablet for tablet a3d66de192604f0598b9a47002fb4714 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.710760  7039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d66de192604f0598b9a47002fb4714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.711078  7114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d66de192604f0598b9a47002fb4714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.709010  6964 tablet_service.cc:1511] Processing CreateTablet for tablet a3d66de192604f0598b9a47002fb4714 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=b6ea8e523f15442096160652cf1e267d]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:17:37.713807  6964 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3d66de192604f0598b9a47002fb4714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:37.730470  7166 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: Bootstrap starting.
I20260709 13:17:37.730504  7165 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Bootstrap starting.
I20260709 13:17:37.739210  7166 tablet_bootstrap.cc:654] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.743046  7165 tablet_bootstrap.cc:654] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.752562  7166 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: No bootstrap required, opened a new log
I20260709 13:17:37.753154  7164 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: Bootstrap starting.
I20260709 13:17:37.753283  7166 ts_tablet_manager.cc:1403] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.008s
I20260709 13:17:37.757495  7166 raft_consensus.cc:359] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.758139  7165 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: No bootstrap required, opened a new log
I20260709 13:17:37.758478  7166 raft_consensus.cc:385] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.758879  7165 ts_tablet_manager.cc:1403] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.007s
I20260709 13:17:37.759302  7166 raft_consensus.cc:740] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.760840  7166 consensus_queue.cc:260] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.763270  7164 tablet_bootstrap.cc:654] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.762933  7165 raft_consensus.cc:359] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.763291  7166 ts_tablet_manager.cc:1434] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:17:37.763840  7165 raft_consensus.cc:385] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.764304  7165 raft_consensus.cc:740] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.765027  7166 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: Bootstrap starting.
I20260709 13:17:37.765561  7165 consensus_queue.cc:260] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.769529  7165 ts_tablet_manager.cc:1434] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:17:37.771590  7165 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Bootstrap starting.
I20260709 13:17:37.773874  7166 tablet_bootstrap.cc:654] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.782366  7165 tablet_bootstrap.cc:654] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.794482  7165 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: No bootstrap required, opened a new log
I20260709 13:17:37.794512  7164 tablet_bootstrap.cc:492] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: No bootstrap required, opened a new log
I20260709 13:17:37.794541  7166 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: No bootstrap required, opened a new log
I20260709 13:17:37.795225  7165 ts_tablet_manager.cc:1403] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.010s
I20260709 13:17:37.795495  7164 ts_tablet_manager.cc:1403] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: Time spent bootstrapping tablet: real 0.043s	user 0.020s	sys 0.019s
I20260709 13:17:37.795542  7166 ts_tablet_manager.cc:1403] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.010s
I20260709 13:17:37.798448  7164 raft_consensus.cc:359] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.799173  7164 raft_consensus.cc:385] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.799160  7165 raft_consensus.cc:359] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.799979  7164 raft_consensus.cc:740] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.800102  7165 raft_consensus.cc:385] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.800469  7165 raft_consensus.cc:740] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.800956  7164 consensus_queue.cc:260] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.801453  7165 consensus_queue.cc:260] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.803387  7164 ts_tablet_manager.cc:1434] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:37.799386  7166 raft_consensus.cc:359] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.804245  7166 raft_consensus.cc:385] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.804670  7166 raft_consensus.cc:740] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.804684  7164 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: Bootstrap starting.
I20260709 13:17:37.805569  7166 consensus_queue.cc:260] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.807029  7165 ts_tablet_manager.cc:1434] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:17:37.813454  7166 ts_tablet_manager.cc:1434] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:17:37.814695  7164 tablet_bootstrap.cc:654] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:37.821933  7164 tablet_bootstrap.cc:492] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: No bootstrap required, opened a new log
I20260709 13:17:37.822579  7164 ts_tablet_manager.cc:1403] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:17:37.825397  7170 raft_consensus.cc:493] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.825922  7164 raft_consensus.cc:359] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.826018  7170 raft_consensus.cc:515] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.826756  7164 raft_consensus.cc:385] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:37.827142  7164 raft_consensus.cc:740] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Initialized, Role: FOLLOWER
I20260709 13:17:37.827881  7164 consensus_queue.cc:260] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.828836  7170 leader_election.cc:290] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833)
I20260709 13:17:37.830363  7164 ts_tablet_manager.cc:1434] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:17:37.842190  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255" is_pre_election: true
I20260709 13:17:37.842886  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0210e512b5a0441896a1748d566d02ba" is_pre_election: true
I20260709 13:17:37.843086  6975 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 0.
I20260709 13:17:37.843765  7050 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 0.
I20260709 13:17:37.844530  7173 raft_consensus.cc:493] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:37.845109  7173 raft_consensus.cc:515] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.845119  7087 leader_election.cc:304] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7c85d3a7294ab8918a5450fc459255, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:37.847127  7170 raft_consensus.cc:2804] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:37.847545  7170 raft_consensus.cc:493] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.847728  7173 leader_election.cc:290] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.847925  7170 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.849016  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255" is_pre_election: true
I20260709 13:17:37.849255  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" is_pre_election: true
I20260709 13:17:37.849858  6975 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 0.
I20260709 13:17:37.849900  7124 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 0.
I20260709 13:17:37.851339  7013 leader_election.cc:304] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:37.852138  7173 raft_consensus.cc:2804] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:37.852556  7173 raft_consensus.cc:493] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:37.852903  7173 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.857050  7170 raft_consensus.cc:515] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.859721  7170 leader_election.cc:290] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 1 election: Requested vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833)
I20260709 13:17:37.860749  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
I20260709 13:17:37.860880  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:37.861517  6975 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.861600  7050 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.861856  7173 raft_consensus.cc:515] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.864354  7173 leader_election.cc:290] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 election: Requested vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:37.865723  7124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:37.865803  6974 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
I20260709 13:17:37.866655  7124 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.866796  6974 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:37.871065  7050 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 1.
I20260709 13:17:37.872660  7087 leader_election.cc:304] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:37.873165  6975 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 1.
I20260709 13:17:37.874197  7170 raft_consensus.cc:2804] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.876056  7170 raft_consensus.cc:697] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 LEADER]: Becoming Leader. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Running, Role: LEADER
I20260709 13:17:37.876820  6974 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 1.
I20260709 13:17:37.877110  7170 consensus_queue.cc:237] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.878296  7013 leader_election.cc:304] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:37.878559  7124 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 1.
I20260709 13:17:37.879395  7173 raft_consensus.cc:2804] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:37.879961  7173 raft_consensus.cc:697] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Becoming Leader. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Running, Role: LEADER
I20260709 13:17:37.881273  7173 consensus_queue.cc:237] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:37.887758  6872 catalog_manager.cc:5697] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131). New cstate: current_term: 1 leader_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:37.891700  6871 catalog_manager.cc:5697] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 0210e512b5a0441896a1748d566d02ba (127.21.5.130). New cstate: current_term: 1 leader_uuid: "0210e512b5a0441896a1748d566d02ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:37.919819  7050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3"
dest_uuid: "0210e512b5a0441896a1748d566d02ba"
mode: GRACEFUL
new_leader_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:53146
I20260709 13:17:37.920480  7050 raft_consensus.cc:606] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Received request to transfer leadership to TS 0c7c85d3a7294ab8918a5450fc459255
I20260709 13:17:37.934996  6974 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916"
dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
mode: GRACEFUL
new_leader_uuid: "0210e512b5a0441896a1748d566d02ba"
 from {username='slave'} at 127.0.0.1:47218
I20260709 13:17:37.935585  6974 raft_consensus.cc:606] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 LEADER]: Received request to transfer leadership to TS 0210e512b5a0441896a1748d566d02ba
I20260709 13:17:37.951392  7124 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c"
dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
mode: GRACEFUL
new_leader_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:56672
I20260709 13:17:37.952061  7124 raft_consensus.cc:606] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 LEADER]: Received request to transfer leadership to TS 0c7c85d3a7294ab8918a5450fc459255
I20260709 13:17:37.958290  7050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714"
dest_uuid: "0210e512b5a0441896a1748d566d02ba"
mode: GRACEFUL
new_leader_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:53146
I20260709 13:17:37.958992  7050 raft_consensus.cc:606] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Received request to transfer leadership to TS 0c7c85d3a7294ab8918a5450fc459255
I20260709 13:17:37.981770  7171 consensus_queue.cc:1048] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.000315  7173 consensus_queue.cc:1048] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.002889  7171 consensus_queue.cc:1048] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.017522  7177 raft_consensus.cc:993] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: : Instructing follower 0210e512b5a0441896a1748d566d02ba to start an election
I20260709 13:17:38.018064  7177 raft_consensus.cc:1081] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 LEADER]: Signalling peer 0210e512b5a0441896a1748d566d02ba to start an election
I20260709 13:17:38.020454  7050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916"
dest_uuid: "0210e512b5a0441896a1748d566d02ba"
 from {username='slave'} at 127.0.0.1:53124
I20260709 13:17:38.021173  7050 raft_consensus.cc:493] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.021570  7050 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.027724  7180 consensus_queue.cc:1048] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.036994  7050 raft_consensus.cc:515] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.039140  7173 raft_consensus.cc:993] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: : Instructing follower 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.039928  7173 raft_consensus.cc:1081] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Signalling peer 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.043241  6975 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3"
dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:47200
I20260709 13:17:38.043916  7050 leader_election.cc:290] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 2 election: Requested vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:38.044175  6975 raft_consensus.cc:493] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.044888  6975 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.048096  6973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
I20260709 13:17:38.048543  7123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ddaee59e8a4402eb90b72287f1c3916" candidate_uuid: "0210e512b5a0441896a1748d566d02ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:38.049183  6973 raft_consensus.cc:3055] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.049503  7123 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.049726  6973 raft_consensus.cc:740] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:38.051533  6973 consensus_queue.cc:260] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.056663  6973 raft_consensus.cc:3060] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.063228  7123 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 2.
I20260709 13:17:38.065007  7013 leader_election.cc:304] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:38.065876  7180 raft_consensus.cc:2804] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.065639  6975 raft_consensus.cc:515] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:38.066502  7180 raft_consensus.cc:697] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 2 LEADER]: Becoming Leader. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Running, Role: LEADER
I20260709 13:17:38.074534  7180 consensus_queue.cc:237] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.081247  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:38.081935  7050 raft_consensus.cc:3055] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.082293  7050 raft_consensus.cc:740] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Running, Role: LEADER
I20260709 13:17:38.083235  7050 consensus_queue.cc:260] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:38.084538  7050 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.084674  6872 catalog_manager.cc:5697] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba reported cstate change: term changed from 1 to 2, leader changed from 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129) to 0210e512b5a0441896a1748d566d02ba (127.21.5.130). New cstate: current_term: 2 leader_uuid: "0210e512b5a0441896a1748d566d02ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.088371  7123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac2517fb431495f9dad4d81f38f51e3" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:38.089107  7123 raft_consensus.cc:3060] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.090761  6975 leader_election.cc:290] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Requested vote from peers 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495), 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833)
I20260709 13:17:38.091882  6973 raft_consensus.cc:2468] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0210e512b5a0441896a1748d566d02ba in term 2.
I20260709 13:17:38.095734  7050 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.097266  6938 leader_election.cc:304] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:38.098306  7177 raft_consensus.cc:2804] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.098875  7177 raft_consensus.cc:697] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Becoming Leader. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:38.099552  7123 raft_consensus.cc:2468] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.099967  7177 consensus_queue.cc:237] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } }
I20260709 13:17:38.109139  6870 catalog_manager.cc:5697] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 reported cstate change: term changed from 1 to 2, leader changed from 0210e512b5a0441896a1748d566d02ba (127.21.5.130) to 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "0c7c85d3a7294ab8918a5450fc459255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.289786  7180 consensus_queue.cc:1048] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.309461  7173 consensus_queue.cc:1048] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.324412  7207 raft_consensus.cc:993] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: : Instructing follower 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.325017  7173 raft_consensus.cc:1081] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Signalling peer 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.327076  6973 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714"
dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:47200
I20260709 13:17:38.327805  6973 raft_consensus.cc:493] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.328265  6973 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.336596  6973 raft_consensus.cc:515] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.338717  6973 leader_election.cc:290] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Requested vote from peers 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:38.339474  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:38.339895  7123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:38.340317  7050 raft_consensus.cc:3055] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.340639  7123 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.340765  7050 raft_consensus.cc:740] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0210e512b5a0441896a1748d566d02ba, State: Running, Role: LEADER
I20260709 13:17:38.341791  7050 consensus_queue.cc:260] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.343147  7050 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.349813  7123 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.351259  6938 leader_election.cc:304] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c7c85d3a7294ab8918a5450fc459255, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:38.351595  7050 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.352244  7177 raft_consensus.cc:2804] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.352800  7177 raft_consensus.cc:697] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Becoming Leader. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:38.353798  7177 consensus_queue.cc:237] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.361186  6871 catalog_manager.cc:5697] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 reported cstate change: term changed from 1 to 2, leader changed from 0210e512b5a0441896a1748d566d02ba (127.21.5.130) to 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "0c7c85d3a7294ab8918a5450fc459255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.395941  7170 consensus_queue.cc:1048] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:38.404268  7210 raft_consensus.cc:993] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: : Instructing follower 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.405272  7170 raft_consensus.cc:1081] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 LEADER]: Signalling peer 0c7c85d3a7294ab8918a5450fc459255 to start an election
I20260709 13:17:38.408375  6975 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c"
dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
 from {username='slave'} at 127.0.0.1:47202
I20260709 13:17:38.409054  6975 raft_consensus.cc:493] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.409466  6975 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.411772  7211 consensus_queue.cc:1048] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:38.418684  6975 raft_consensus.cc:515] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.423027  6975 leader_election.cc:290] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Requested vote from peers 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833), 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131:45495)
I20260709 13:17:38.423940  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:38.424757  7050 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.425215  7123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da8bd8f392a340bea4cc2d8f3565924c" candidate_uuid: "0c7c85d3a7294ab8918a5450fc459255" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
I20260709 13:17:38.425951  7123 raft_consensus.cc:3055] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:38.426299  7123 raft_consensus.cc:740] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Running, Role: LEADER
I20260709 13:17:38.427245  7123 consensus_queue.cc:260] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.428771  7123 raft_consensus.cc:3060] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:38.434435  7050 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.436156  6938 leader_election.cc:304] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0c7c85d3a7294ab8918a5450fc459255; no voters: 
I20260709 13:17:38.437114  7177 raft_consensus.cc:2804] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:38.437635  7123 raft_consensus.cc:2468] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c7c85d3a7294ab8918a5450fc459255 in term 2.
I20260709 13:17:38.437691  7177 raft_consensus.cc:697] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Becoming Leader. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:38.438973  7177 consensus_queue.cc:237] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.446372  6871 catalog_manager.cc:5697] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 reported cstate change: term changed from 1 to 2, leader changed from 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131) to 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "0c7c85d3a7294ab8918a5450fc459255" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:38.491775  7123 raft_consensus.cc:1275] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.493357  7177 consensus_queue.cc:1048] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.504033  7050 raft_consensus.cc:1275] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.507159  7213 consensus_queue.cc:1048] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:38.508143 21526 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:17:38.510959 21526 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:17:38.511600 21526 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:17:38.530210  6975 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714"
dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
mode: GRACEFUL
new_leader_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
 from {username='slave'} at 127.0.0.1:47220
I20260709 13:17:38.530869  6975 raft_consensus.cc:606] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Received request to transfer leadership to TS 0cd48c4aa7bd4b838db0f6e3c4091c9d
I20260709 13:17:38.532128 21526 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:17:38.568589  6975 raft_consensus.cc:1275] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Refusing update from remote peer 0210e512b5a0441896a1748d566d02ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.570335  7173 consensus_queue.cc:1048] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.579963  7123 raft_consensus.cc:1275] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Refusing update from remote peer 0210e512b5a0441896a1748d566d02ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.581982  7180 consensus_queue.cc:1048] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.861032  7123 raft_consensus.cc:1275] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.862525  7177 consensus_queue.cc:1048] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.868227  7213 raft_consensus.cc:993] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: : Instructing follower 0cd48c4aa7bd4b838db0f6e3c4091c9d to start an election
I20260709 13:17:38.868798  7171 raft_consensus.cc:1081] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Signalling peer 0cd48c4aa7bd4b838db0f6e3c4091c9d to start an election
I20260709 13:17:38.870805  7124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714"
dest_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d"
 from {username='slave'} at 127.0.0.1:56654
I20260709 13:17:38.873109  7124 raft_consensus.cc:493] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:38.873338  7050 raft_consensus.cc:1275] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.873641  7124 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:38.876808  7213 consensus_queue.cc:1048] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:38.882506  7124 raft_consensus.cc:515] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.888115  7124 leader_election.cc:290] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 3 election: Requested vote from peers 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831), 0210e512b5a0441896a1748d566d02ba (127.21.5.130:32833)
I20260709 13:17:38.889550  7050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0210e512b5a0441896a1748d566d02ba"
I20260709 13:17:38.890115  6975 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3d66de192604f0598b9a47002fb4714" candidate_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0c7c85d3a7294ab8918a5450fc459255"
I20260709 13:17:38.890435  7050 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:38.890830  6975 raft_consensus.cc:3055] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:38.891176  6975 raft_consensus.cc:740] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0c7c85d3a7294ab8918a5450fc459255, State: Running, Role: LEADER
I20260709 13:17:38.892109  6975 consensus_queue.cc:260] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.893581  6975 raft_consensus.cc:3060] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:38.900799  7050 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 3.
I20260709 13:17:38.902793  6975 raft_consensus.cc:2468] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0cd48c4aa7bd4b838db0f6e3c4091c9d in term 3.
I20260709 13:17:38.904559  7087 leader_election.cc:304] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0210e512b5a0441896a1748d566d02ba, 0cd48c4aa7bd4b838db0f6e3c4091c9d; no voters: 
I20260709 13:17:38.906376  7211 raft_consensus.cc:2804] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:38.907183  7211 raft_consensus.cc:697] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 LEADER]: Becoming Leader. State: Replica: 0cd48c4aa7bd4b838db0f6e3c4091c9d, State: Running, Role: LEADER
I20260709 13:17:38.908136  7211 consensus_queue.cc:237] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } }
I20260709 13:17:38.915946  6872 catalog_manager.cc:5697] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d reported cstate change: term changed from 2 to 3, leader changed from 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129) to 0cd48c4aa7bd4b838db0f6e3c4091c9d (127.21.5.131). New cstate: current_term: 3 leader_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c7c85d3a7294ab8918a5450fc459255" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 43831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:38.997457  7050 raft_consensus.cc:1275] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:38.999248  7213 consensus_queue.cc:1048] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0210e512b5a0441896a1748d566d02ba" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 32833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.008376  7124 raft_consensus.cc:1275] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Refusing update from remote peer 0c7c85d3a7294ab8918a5450fc459255: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:39.011000  7177 consensus_queue.cc:1048] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cd48c4aa7bd4b838db0f6e3c4091c9d" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:39.176050 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
I20260709 13:17:39.212411 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:39.213245 21526 tablet_replica.cc:333] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255: stopping tablet replica
I20260709 13:17:39.214152 21526 raft_consensus.cc:2243] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:39.215293 21526 raft_consensus.cc:2272] T da8bd8f392a340bea4cc2d8f3565924c P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.218317 21526 tablet_replica.cc:333] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255: stopping tablet replica
I20260709 13:17:39.219034 21526 raft_consensus.cc:2243] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.219506 21526 raft_consensus.cc:2272] T a3d66de192604f0598b9a47002fb4714 P 0c7c85d3a7294ab8918a5450fc459255 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.222186 21526 tablet_replica.cc:333] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255: stopping tablet replica
I20260709 13:17:39.222858 21526 raft_consensus.cc:2243] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:39.223791 21526 raft_consensus.cc:2272] T 0ac2517fb431495f9dad4d81f38f51e3 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.226737 21526 tablet_replica.cc:333] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255: stopping tablet replica
I20260709 13:17:39.227270 21526 raft_consensus.cc:2243] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.227733 21526 raft_consensus.cc:2272] T 1ddaee59e8a4402eb90b72287f1c3916 P 0c7c85d3a7294ab8918a5450fc459255 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.254424 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
I20260709 13:17:39.271018 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
I20260709 13:17:39.297684 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:39.298404 21526 tablet_replica.cc:333] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba: stopping tablet replica
I20260709 13:17:39.299129 21526 raft_consensus.cc:2243] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.299696 21526 raft_consensus.cc:2272] T da8bd8f392a340bea4cc2d8f3565924c P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.302552 21526 tablet_replica.cc:333] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba: stopping tablet replica
I20260709 13:17:39.303387 21526 raft_consensus.cc:2243] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.303905 21526 raft_consensus.cc:2272] T a3d66de192604f0598b9a47002fb4714 P 0210e512b5a0441896a1748d566d02ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.308316 21526 tablet_replica.cc:333] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba: stopping tablet replica
I20260709 13:17:39.309163 21526 raft_consensus.cc:2243] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:17:39.309283  7087 consensus_peers.cc:597] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d -> Peer 0c7c85d3a7294ab8918a5450fc459255 (127.21.5.129:43831): Couldn't send request to peer 0c7c85d3a7294ab8918a5450fc459255. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:43831: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:39.309793 21526 raft_consensus.cc:2272] T 0ac2517fb431495f9dad4d81f38f51e3 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.312340 21526 tablet_replica.cc:333] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba: stopping tablet replica
I20260709 13:17:39.312897 21526 raft_consensus.cc:2243] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:39.313809 21526 raft_consensus.cc:2272] T 1ddaee59e8a4402eb90b72287f1c3916 P 0210e512b5a0441896a1748d566d02ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.339614 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
I20260709 13:17:39.355690 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
I20260709 13:17:39.381599 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:39.382350 21526 tablet_replica.cc:333] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d: stopping tablet replica
I20260709 13:17:39.383046 21526 raft_consensus.cc:2243] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.383652 21526 raft_consensus.cc:2272] T da8bd8f392a340bea4cc2d8f3565924c P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.386377 21526 tablet_replica.cc:333] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: stopping tablet replica
I20260709 13:17:39.387050 21526 raft_consensus.cc:2243] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:39.387916 21526 pending_rounds.cc:70] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Trying to abort 1 pending ops.
I20260709 13:17:39.388144 21526 pending_rounds.cc:77] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305638129299156992 op_type: NO_OP noop_request { }
I20260709 13:17:39.388489 21526 raft_consensus.cc:2889] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:17:39.388809 21526 raft_consensus.cc:2272] T a3d66de192604f0598b9a47002fb4714 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.391775 21526 tablet_replica.cc:333] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: stopping tablet replica
I20260709 13:17:39.392428 21526 raft_consensus.cc:2243] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.392963 21526 raft_consensus.cc:2272] T 1ddaee59e8a4402eb90b72287f1c3916 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.395848 21526 tablet_replica.cc:333] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d: stopping tablet replica
I20260709 13:17:39.396452 21526 raft_consensus.cc:2243] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:39.396989 21526 raft_consensus.cc:2272] T 0ac2517fb431495f9dad4d81f38f51e3 P 0cd48c4aa7bd4b838db0f6e3c4091c9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.423345 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:17:39.441505 21526 master.cc:562] Master@127.21.5.190:43845 shutting down...
I20260709 13:17:39.474727 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:39.475744 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:39.476279 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6575f107f8d5498a8416b8e5ec1f9019: stopping tablet replica
I20260709 13:17:39.501816 21526 master.cc:584] Master@127.21.5.190:43845 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (3011 ms)
[----------] 6 tests from LeaderRebalancerTest (552999 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:17:39.536052 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:36863
I20260709 13:17:39.537372 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:39.543303  7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.544477  7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.545264  7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:39.546823 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:39.547894 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:39.548137 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:39.548316 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603059548277 us; error 0 us; skew 500 ppm
I20260709 13:17:39.548908 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:39.551549 21526 webserver.cc:533] Webserver started at http://127.21.5.190:33007/ using document root <none> and password file <none>
I20260709 13:17:39.552119 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:39.552340 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:39.552636 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:39.553907 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "1439561648a74e079c3e5754923a4f27"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.559238 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 13:17:39.563259  7228 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.564172 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:17:39.564646 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "1439561648a74e079c3e5754923a4f27"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.565145 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:39.577261 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:39.578784 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:39.623298 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:36863
I20260709 13:17:39.623422  7279 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:36863 every 8 connection(s)
I20260709 13:17:39.627969  7280 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:39.643303  7280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: Bootstrap starting.
I20260709 13:17:39.648468  7280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:39.653174  7280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: No bootstrap required, opened a new log
I20260709 13:17:39.655720  7280 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER }
I20260709 13:17:39.656205  7280 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:39.656395  7280 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1439561648a74e079c3e5754923a4f27, State: Initialized, Role: FOLLOWER
I20260709 13:17:39.657063  7280 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER }
I20260709 13:17:39.657559  7280 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:39.657760  7280 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:39.657981  7280 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:39.664149  7280 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER }
I20260709 13:17:39.664747  7280 leader_election.cc:304] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1439561648a74e079c3e5754923a4f27; no voters: 
I20260709 13:17:39.665999  7280 leader_election.cc:290] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:39.666446  7283 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:39.668270  7283 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 1 LEADER]: Becoming Leader. State: Replica: 1439561648a74e079c3e5754923a4f27, State: Running, Role: LEADER
I20260709 13:17:39.668989  7283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER }
I20260709 13:17:39.669601  7280 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:39.672192  7285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1439561648a74e079c3e5754923a4f27. Latest consensus state: current_term: 1 leader_uuid: "1439561648a74e079c3e5754923a4f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER } }
I20260709 13:17:39.672178  7284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1439561648a74e079c3e5754923a4f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1439561648a74e079c3e5754923a4f27" member_type: VOTER } }
I20260709 13:17:39.673082  7285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:39.673161  7284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:39.680698  7288 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:39.686384  7288 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:39.687662 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:39.697422  7288 catalog_manager.cc:1383] Generated new cluster ID: 6cb2b75497c94a20800ee73251e8f7c9
I20260709 13:17:39.697804  7288 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:39.722929  7288 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:39.724476  7288 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:39.739812  7288 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: Generated new TSK 0
I20260709 13:17:39.740543  7288 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:39.755375 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:39.762372  7301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.764011  7302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.769505  7304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:39.769686 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:39.770857 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:39.771095 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:39.771289 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603059771267 us; error 0 us; skew 500 ppm
I20260709 13:17:39.771920 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:39.774569 21526 webserver.cc:533] Webserver started at http://127.21.5.129:42371/ using document root <none> and password file <none>
I20260709 13:17:39.775230 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:39.775453 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:39.775782 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:39.777115 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "9caae85dad6340fab81354bfb82e3c91"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.782316 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:39.786125  7309 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.786979 21526 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:17:39.787330 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "9caae85dad6340fab81354bfb82e3c91"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.787667 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:39.819231 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:39.820578 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:39.822219 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:39.824841 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:39.825088 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.825358 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:39.825546 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.869841 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:41907
I20260709 13:17:39.869956  7371 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:41907 every 8 connection(s)
I20260709 13:17:39.876082 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:39.884946  7376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.885846  7377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:39.892360 21526 server_base.cc:1061] running on GCE node
W20260709 13:17:39.892333  7379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:39.893596  7372 heartbeater.cc:344] Connected to a master server at 127.21.5.190:36863
I20260709 13:17:39.894105  7372 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:39.894104 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:39.894585 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:39.894830 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603059894805 us; error 0 us; skew 500 ppm
I20260709 13:17:39.895351  7372 heartbeater.cc:507] Master 127.21.5.190:36863 requested a full tablet report, sending...
I20260709 13:17:39.895725 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:39.898293  7245 ts_manager.cc:194] Registered new tserver with Master: 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907)
I20260709 13:17:39.899178 21526 webserver.cc:533] Webserver started at http://127.21.5.130:39267/ using document root <none> and password file <none>
I20260709 13:17:39.900010 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:39.900291 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:39.900643 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:39.901185  7245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47040
I20260709 13:17:39.902330 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "3e9e84d2e507455aa80bc7595a1ef075"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.908209 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:17:39.912513  7384 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.913457 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:17:39.913805 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "3e9e84d2e507455aa80bc7595a1ef075"
format_stamp: "Formatted at 2026-07-09 13:17:39 on dist-test-slave-h0rz"
I20260709 13:17:39.914147 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:39.930827 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:39.932091 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:39.933782 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:39.936393 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:39.936637 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.936906 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:39.937094 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:39.983160 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:34207
I20260709 13:17:39.983314  7446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:34207 every 8 connection(s)
I20260709 13:17:39.988835 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:39.997598  7450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:39.998751  7451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.002401 21526 server_base.cc:1061] running on GCE node
W20260709 13:17:40.003121  7454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:40.004081  7447 heartbeater.cc:344] Connected to a master server at 127.21.5.190:36863
I20260709 13:17:40.004626  7447 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:40.004627 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:40.005090 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:40.005321 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603060005300 us; error 0 us; skew 500 ppm
I20260709 13:17:40.005838  7447 heartbeater.cc:507] Master 127.21.5.190:36863 requested a full tablet report, sending...
I20260709 13:17:40.006385 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:40.009068  7245 ts_manager.cc:194] Registered new tserver with Master: 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.010296 21526 webserver.cc:533] Webserver started at http://127.21.5.131:33813/ using document root <none> and password file <none>
I20260709 13:17:40.011077 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:40.011113  7245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47042
I20260709 13:17:40.011304 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:40.011823 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:40.013371 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-h0rz"
I20260709 13:17:40.018934 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:40.022782  7458 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.023729 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:40.024089 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
format_stamp: "Formatted at 2026-07-09 13:17:40 on dist-test-slave-h0rz"
I20260709 13:17:40.024371 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:40.049437 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:40.050916 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:40.052675 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:40.055419 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:40.055701 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.055987 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:40.056208 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:40.104019 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:41951
I20260709 13:17:40.104166  7520 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:41951 every 8 connection(s)
I20260709 13:17:40.120468  7521 heartbeater.cc:344] Connected to a master server at 127.21.5.190:36863
I20260709 13:17:40.121058  7521 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:40.122002  7521 heartbeater.cc:507] Master 127.21.5.190:36863 requested a full tablet report, sending...
I20260709 13:17:40.124630  7245 ts_manager.cc:194] Registered new tserver with Master: c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.125224 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017517655s
I20260709 13:17:40.126559  7245 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47044
I20260709 13:17:40.153992  7245 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47050:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:40.157115  7245 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:40.244951  7337 tablet_service.cc:1511] Processing CreateTablet for tablet bbd27671e688455da30507409a3aa997 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:40.246505  7337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd27671e688455da30507409a3aa997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.253159  7336 tablet_service.cc:1511] Processing CreateTablet for tablet e466d58acb044afe83b14710ab8eaca9 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:40.255452  7336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e466d58acb044afe83b14710ab8eaca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.276742  7335 tablet_service.cc:1511] Processing CreateTablet for tablet 501ab39f68fc46dd8bcf3761f8a610cf (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:40.277941  7334 tablet_service.cc:1511] Processing CreateTablet for tablet cc2688d492204007bb197167fa954266 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:40.278915  7335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501ab39f68fc46dd8bcf3761f8a610cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.279842  7334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2688d492204007bb197167fa954266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.288811  7412 tablet_service.cc:1511] Processing CreateTablet for tablet bbd27671e688455da30507409a3aa997 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:40.290522  7412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd27671e688455da30507409a3aa997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.297050  7411 tablet_service.cc:1511] Processing CreateTablet for tablet e466d58acb044afe83b14710ab8eaca9 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:40.298877  7411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e466d58acb044afe83b14710ab8eaca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.299921  7333 tablet_service.cc:1511] Processing CreateTablet for tablet d166d5008c8c4d098bb1973379289560 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:40.301599  7333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d166d5008c8c4d098bb1973379289560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.302451  7486 tablet_service.cc:1511] Processing CreateTablet for tablet bbd27671e688455da30507409a3aa997 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:40.304163  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbd27671e688455da30507409a3aa997. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.305632  7481 tablet_service.cc:1511] Processing CreateTablet for tablet 4e44335ea170481ab59db4cca4c7c180 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:40.307345  7481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44335ea170481ab59db4cca4c7c180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.307449  7485 tablet_service.cc:1511] Processing CreateTablet for tablet e466d58acb044afe83b14710ab8eaca9 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:40.309093  7485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e466d58acb044afe83b14710ab8eaca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.309324  7410 tablet_service.cc:1511] Processing CreateTablet for tablet 501ab39f68fc46dd8bcf3761f8a610cf (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:40.311065  7410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501ab39f68fc46dd8bcf3761f8a610cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.311925  7484 tablet_service.cc:1511] Processing CreateTablet for tablet 501ab39f68fc46dd8bcf3761f8a610cf (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:40.313578  7484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 501ab39f68fc46dd8bcf3761f8a610cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.313174  7483 tablet_service.cc:1511] Processing CreateTablet for tablet cc2688d492204007bb197167fa954266 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:40.314673  7483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2688d492204007bb197167fa954266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.311921  7482 tablet_service.cc:1511] Processing CreateTablet for tablet d166d5008c8c4d098bb1973379289560 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:40.316342  7482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d166d5008c8c4d098bb1973379289560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.316578  7409 tablet_service.cc:1511] Processing CreateTablet for tablet cc2688d492204007bb197167fa954266 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:40.318033  7409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc2688d492204007bb197167fa954266. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.319890  7480 tablet_service.cc:1511] Processing CreateTablet for tablet ccee0354eaf24cd2937732a1b12ead42 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:40.319901  7332 tablet_service.cc:1511] Processing CreateTablet for tablet 4e44335ea170481ab59db4cca4c7c180 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:40.321498  7480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccee0354eaf24cd2937732a1b12ead42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.321614  7332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44335ea170481ab59db4cca4c7c180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.324355  7408 tablet_service.cc:1511] Processing CreateTablet for tablet d166d5008c8c4d098bb1973379289560 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:40.325958  7408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d166d5008c8c4d098bb1973379289560. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.326040  7478 tablet_service.cc:1511] Processing CreateTablet for tablet 06d88a6686704bf29105419087fffe9f (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:40.324350  7479 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3e8fc8428d42a8ab357ca69292f70d (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:40.327524  7478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d88a6686704bf29105419087fffe9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.331605  7479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3e8fc8428d42a8ab357ca69292f70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.331270  7331 tablet_service.cc:1511] Processing CreateTablet for tablet ccee0354eaf24cd2937732a1b12ead42 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:40.332862  7331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccee0354eaf24cd2937732a1b12ead42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.337479  7407 tablet_service.cc:1511] Processing CreateTablet for tablet 4e44335ea170481ab59db4cca4c7c180 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:40.339318  7407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e44335ea170481ab59db4cca4c7c180. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.344462  7406 tablet_service.cc:1511] Processing CreateTablet for tablet ccee0354eaf24cd2937732a1b12ead42 (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:40.345088  7404 tablet_service.cc:1511] Processing CreateTablet for tablet 06d88a6686704bf29105419087fffe9f (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:40.346341  7406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccee0354eaf24cd2937732a1b12ead42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.347918  7405 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3e8fc8428d42a8ab357ca69292f70d (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:40.351042  7405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3e8fc8428d42a8ab357ca69292f70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.352005  7330 tablet_service.cc:1511] Processing CreateTablet for tablet 0f3e8fc8428d42a8ab357ca69292f70d (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:40.352938  7404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d88a6686704bf29105419087fffe9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.353680  7330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f3e8fc8428d42a8ab357ca69292f70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.368906  7329 tablet_service.cc:1511] Processing CreateTablet for tablet 06d88a6686704bf29105419087fffe9f (DEFAULT_TABLE table=test-workload [id=1c471e4eeffb4cd6b69614f7b49a3999]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:40.370985  7329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06d88a6686704bf29105419087fffe9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:40.436364  7536 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:40.497123  7539 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.510752  7536 tablet_bootstrap.cc:654] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.510819  7539 tablet_bootstrap.cc:654] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.483655  7538 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:40.533774  7538 tablet_bootstrap.cc:654] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.535707  7536 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.536546  7536 ts_tablet_manager.cc:1403] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.101s	user 0.019s	sys 0.009s
I20260709 13:17:40.537593  7539 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:40.538292  7539 ts_tablet_manager.cc:1403] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.013s
I20260709 13:17:40.541198  7536 raft_consensus.cc:359] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.541965  7536 raft_consensus.cc:385] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.541558  7539 raft_consensus.cc:359] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.542343  7536 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.542477  7539 raft_consensus.cc:385] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.542943  7539 raft_consensus.cc:740] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.543726  7538 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:40.543357  7536 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.543857  7539 consensus_queue.cc:260] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.544358  7538 ts_tablet_manager.cc:1403] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.000s
I20260709 13:17:40.546793  7372 heartbeater.cc:499] Master 127.21.5.190:36863 was elected leader, sending a full tablet report...
I20260709 13:17:40.547060  7536 ts_tablet_manager.cc:1434] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:17:40.548615  7536 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:40.548344  7538 raft_consensus.cc:359] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.549197  7538 raft_consensus.cc:385] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.549516  7538 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.550580  7538 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.552696  7521 heartbeater.cc:499] Master 127.21.5.190:36863 was elected leader, sending a full tablet report...
I20260709 13:17:40.553915  7539 ts_tablet_manager.cc:1434] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 13:17:40.555395  7539 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.556172  7536 tablet_bootstrap.cc:654] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.562640  7539 tablet_bootstrap.cc:654] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.573194  7536 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.574199  7536 ts_tablet_manager.cc:1403] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.007s
I20260709 13:17:40.575537  7539 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:40.576339  7539 ts_tablet_manager.cc:1403] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.001s
I20260709 13:17:40.578526  7536 raft_consensus.cc:359] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.579490  7539 raft_consensus.cc:359] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.579879  7536 raft_consensus.cc:385] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.580518  7539 raft_consensus.cc:385] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.580685  7536 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.580965  7539 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.583549  7539 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.586733  7447 heartbeater.cc:499] Master 127.21.5.190:36863 was elected leader, sending a full tablet report...
I20260709 13:17:40.587328  7538 ts_tablet_manager.cc:1434] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.042s	user 0.014s	sys 0.015s
I20260709 13:17:40.587821  7539 ts_tablet_manager.cc:1434] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:17:40.588613  7538 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:40.588919  7539 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.582049  7536 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.592411  7536 ts_tablet_manager.cc:1434] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 13:17:40.594151  7536 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:40.599298  7539 tablet_bootstrap.cc:654] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.602406  7536 tablet_bootstrap.cc:654] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.612119  7538 tablet_bootstrap.cc:654] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.615494  7536 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.616377  7536 ts_tablet_manager.cc:1403] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.004s
I20260709 13:17:40.617856  7539 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:40.618554  7539 ts_tablet_manager.cc:1403] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.012s
I20260709 13:17:40.627893  7539 raft_consensus.cc:359] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.628600  7538 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:40.629490  7538 ts_tablet_manager.cc:1403] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.008s
I20260709 13:17:40.630251  7536 raft_consensus.cc:359] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.631275  7536 raft_consensus.cc:385] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.631642  7536 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.632578  7536 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.633194  7538 raft_consensus.cc:359] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.634052  7538 raft_consensus.cc:385] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.634367  7538 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.635257  7538 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.635736  7536 ts_tablet_manager.cc:1434] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 13:17:40.637209  7536 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:40.637737  7538 ts_tablet_manager.cc:1434] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 13:17:40.639093  7538 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:40.628778  7539 raft_consensus.cc:385] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.643383  7539 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.644424  7539 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.645223  7545 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.645946  7545 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.647593  7539 ts_tablet_manager.cc:1434] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.029s	user 0.011s	sys 0.002s
I20260709 13:17:40.649089  7539 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.649312  7545 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.650825  7536 tablet_bootstrap.cc:654] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.657282  7536 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.658046  7536 ts_tablet_manager.cc:1403] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.004s
I20260709 13:17:40.660193  7538 tablet_bootstrap.cc:654] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.661744  7536 raft_consensus.cc:359] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.662673  7539 tablet_bootstrap.cc:654] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.662937  7536 raft_consensus.cc:385] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.663353  7536 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.664508  7536 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.672108  7536 ts_tablet_manager.cc:1434] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:17:40.674060  7536 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
W20260709 13:17:40.681746  7293 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:40.682480  7545 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.683416  7545 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.684651  7536 tablet_bootstrap.cc:654] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.689031  7545 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.706483  7539 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:40.707525  7539 ts_tablet_manager.cc:1403] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.027s
I20260709 13:17:40.712210  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.707515  7538 tablet_bootstrap.cc:492] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:40.714424  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.714876  7538 ts_tablet_manager.cc:1403] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.076s	user 0.029s	sys 0.017s
I20260709 13:17:40.715431  7346 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 0.
I20260709 13:17:40.715457  7539 raft_consensus.cc:359] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.716606  7539 raft_consensus.cc:385] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.717039  7539 raft_consensus.cc:740] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.717181  7536 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.718050  7536 ts_tablet_manager.cc:1403] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.044s	user 0.022s	sys 0.013s
W20260709 13:17:40.718122  7385 leader_election.cc:343] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.718221  7539 consensus_queue.cc:260] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.718672  7538 raft_consensus.cc:359] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.719554  7538 raft_consensus.cc:385] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.719403  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:40.719990  7538 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.720522  7545 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:40.720964  7545 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.720939  7538 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.721645  7545 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.722153  7536 raft_consensus.cc:359] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.723276  7536 raft_consensus.cc:385] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.723687  7536 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.724784  7536 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.728519  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:40.730679  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
W20260709 13:17:40.732499  7385 leader_election.cc:343] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.733093  7385 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:40.734702  7536 ts_tablet_manager.cc:1434] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:17:40.736544  7536 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:40.737030  7545 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.741346  7539 ts_tablet_manager.cc:1434] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
W20260709 13:17:40.742460  7385 leader_election.cc:343] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.749078  7539 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.755728  7545 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.756678  7545 raft_consensus.cc:2749] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:40.758996  7539 tablet_bootstrap.cc:654] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.758826  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:40.759668  7346 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.760982  7536 tablet_bootstrap.cc:654] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.768601  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:40.765074  7543 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:40.774331  7385 leader_election.cc:343] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.774462  7543 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.774139  7346 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 1.
I20260709 13:17:40.777323  7538 ts_tablet_manager.cc:1434] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.062s	user 0.026s	sys 0.034s
I20260709 13:17:40.777464  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:40.778955  7557 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:40.780166  7557 raft_consensus.cc:697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:40.781383  7538 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:40.781698  7557 consensus_queue.cc:237] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.785202  7544 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.786085  7544 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.792196  7544 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.810045  7543 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.818199  7538 tablet_bootstrap.cc:654] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.810140  7539 tablet_bootstrap.cc:492] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:40.841205  7539 ts_tablet_manager.cc:1403] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.093s	user 0.006s	sys 0.025s
I20260709 13:17:40.841652  7544 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.842155  7536 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:40.842310  7544 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.845212  7539 raft_consensus.cc:359] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.846033  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.846306  7544 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.834794  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.846159  7539 raft_consensus.cc:385] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.836756  7245 catalog_manager.cc:5697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:40.847000  7347 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:40.847211  7346 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:40.834327  7543 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.848583  7543 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.849165  7459 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:40.850896  7544 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:40.851459  7544 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.851974  7544 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.852340  7543 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.847193  7539 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.854019  7539 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.861265  7539 ts_tablet_manager.cc:1434] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:17:40.864689  7539 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:40.864830  7544 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.865705  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.856575  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.867034  7422 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:40.868307  7544 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.857683  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.855544  7536 ts_tablet_manager.cc:1403] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.119s	user 0.030s	sys 0.035s
I20260709 13:17:40.861649  7538 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:40.862294  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.870488  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:40.868158  7419 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:40.856492  7459 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
W20260709 13:17:40.872529  7313 leader_election.cc:343] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.873467  7313 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:40.873714  7536 raft_consensus.cc:359] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.874285  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:40.872483  7538 ts_tablet_manager.cc:1403] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.091s	user 0.017s	sys 0.005s
I20260709 13:17:40.875152  7536 raft_consensus.cc:385] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.875401  7346 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.875723  7536 raft_consensus.cc:740] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
W20260709 13:17:40.876374  7462 leader_election.cc:343] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.876725  7536 consensus_queue.cc:260] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
W20260709 13:17:40.877544  7462 leader_election.cc:343] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.878355  7544 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:40.878805  7544 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.874648  7543 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:40.879283  7544 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.879663  7543 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.880252  7543 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.878840  7538 raft_consensus.cc:359] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:40.883029  7538 raft_consensus.cc:385] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:40.884595  7538 raft_consensus.cc:740] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:40.889832  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:40.889257  7538 consensus_queue.cc:260] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:40.890945  7495 raft_consensus.cc:2393] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 in current term 1: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:40.893095  7544 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.895623  7538 ts_tablet_manager.cc:1434] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:17:40.899317  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:40.899771  7570 raft_consensus.cc:493] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.898027  7539 tablet_bootstrap.cc:654] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.900871  7544 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.901820  7310 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:40.899333  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:40.900668  7570 raft_consensus.cc:515] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.903158  7419 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.908092  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:40.909279  7347 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:40.909919  7346 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:40.900707  7538 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:40.914005  7459 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:40.921792  7544 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:40.920075  7543 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.929570  7544 raft_consensus.cc:697] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:40.931195  7536 ts_tablet_manager.cc:1434] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.061s	user 0.018s	sys 0.020s
I20260709 13:17:40.931048  7544 consensus_queue.cc:237] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.933460  7543 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.934847  7572 raft_consensus.cc:2749] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:40.934656  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:40.936096  7421 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:40.939874  7310 leader_election.cc:343] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.943513  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.941843  7570 leader_election.cc:290] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:40.936554  7347 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:40.945350  7536 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
W20260709 13:17:40.947329  7462 leader_election.cc:343] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.954778  7572 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.957599  7572 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.961750  7572 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:40.955185  7421 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:40.967621  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:40.969746  7557 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:40.970968  7557 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:40.974030  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.974354  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:40.974929  7459 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:40.975296  7496 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:40.975548  7557 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
W20260709 13:17:40.976111  7459 leader_election.cc:343] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.976217  7544 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:40.976491  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:40.976824  7459 leader_election.cc:304] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:40.977035  7544 raft_consensus.cc:697] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:40.978153  7574 raft_consensus.cc:2749] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:40.978163  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:40.978595  7310 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:40.979549  7572 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:40.979967  7572 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:40.972381  7313 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:40.980602  7572 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:17:40.980914  7310 leader_election.cc:343] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:40.980003  7385 leader_election.cc:343] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.981963  7573 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:40.982167  7313 leader_election.cc:343] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.983158  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:40.982679  7573 raft_consensus.cc:697] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
W20260709 13:17:40.984767  7385 leader_election.cc:343] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:40.985275  7385 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:40.986104  7557 raft_consensus.cc:2749] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:40.974341  7419 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:40.985224  7573 consensus_queue.cc:237] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.991288  7536 tablet_bootstrap.cc:654] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:40.993955  7544 consensus_queue.cc:237] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:40.994746  7539 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:41.003274  7539 ts_tablet_manager.cc:1403] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.139s	user 0.008s	sys 0.025s
I20260709 13:17:41.005247  7538 tablet_bootstrap.cc:654] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.009969  7245 catalog_manager.cc:5697] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:41.011085  7539 raft_consensus.cc:359] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.014916  7539 raft_consensus.cc:385] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.015411  7539 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.016723  7539 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.018174  7538 tablet_bootstrap.cc:492] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:41.018527  7536 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:41.019052  7538 ts_tablet_manager.cc:1403] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.009s
I20260709 13:17:41.019198  7536 ts_tablet_manager.cc:1403] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.086s	user 0.007s	sys 0.013s
I20260709 13:17:41.019548  7539 ts_tablet_manager.cc:1434] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:17:41.020908  7539 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:41.022996  7536 raft_consensus.cc:359] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.023222  7538 raft_consensus.cc:359] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.024075  7536 raft_consensus.cc:385] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.024174  7538 raft_consensus.cc:385] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.024545  7536 raft_consensus.cc:740] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.001955  7572 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.026412  7536 consensus_queue.cc:260] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.029629  7539 tablet_bootstrap.cc:654] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.030377  7572 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.024681  7538 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.032260  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:41.032671  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:41.033178  7496 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.033547  7536 ts_tablet_manager.cc:1434] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
W20260709 13:17:41.034096  7313 leader_election.cc:343] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.034925  7536 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:41.036222  7538 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.042323  7538 ts_tablet_manager.cc:1434] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:17:41.043541  7536 tablet_bootstrap.cc:654] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.043628  7538 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:41.032253  7243 catalog_manager.cc:5697] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.049196  7243 catalog_manager.cc:5697] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.052134  7496 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:41.055780  7310 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.057003  7538 tablet_bootstrap.cc:654] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.057240  7572 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:41.057816  7572 raft_consensus.cc:697] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:41.058672  7536 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:41.059252  7536 ts_tablet_manager.cc:1403] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.003s
I20260709 13:17:41.058732  7572 consensus_queue.cc:237] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.060767  7539 tablet_bootstrap.cc:492] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:41.061367  7539 ts_tablet_manager.cc:1403] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.016s
I20260709 13:17:41.062778  7536 raft_consensus.cc:359] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.063719  7536 raft_consensus.cc:385] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.064062  7536 raft_consensus.cc:740] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.065017  7539 raft_consensus.cc:359] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.065061  7536 consensus_queue.cc:260] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.069705  7539 raft_consensus.cc:385] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.070063  7539 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.071492  7536 ts_tablet_manager.cc:1434] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:17:41.065936  7557 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.072798  7536 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:41.069041  7538 tablet_bootstrap.cc:492] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:41.072481  7557 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.075062  7557 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.076828  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.077795  7347 raft_consensus.cc:2393] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:41.077597  7243 catalog_manager.cc:5697] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:41.077567  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:41.071528  7539 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.083027  7385 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.083801  7539 ts_tablet_manager.cc:1434] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.003s
I20260709 13:17:41.084527  7557 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.085062  7539 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:41.085731  7544 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.086283  7544 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.091215  7574 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.091221  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.091828  7574 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
W20260709 13:17:41.093163  7462 leader_election.cc:343] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.094993  7538 ts_tablet_manager.cc:1403] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.002s
I20260709 13:17:41.095911  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.099167  7539 tablet_bootstrap.cc:654] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.099418  7346 raft_consensus.cc:2393] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 1: Already voted for candidate 3e9e84d2e507455aa80bc7595a1ef075 in this term.
I20260709 13:17:41.099838  7536 tablet_bootstrap.cc:654] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.097355  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.102968  7462 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:41.099265  7538 raft_consensus.cc:359] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.104071  7538 raft_consensus.cc:385] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.098276  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.104713  7538 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.105108  7347 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:41.106753  7459 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.094403  7557 raft_consensus.cc:2749] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:41.104067  7570 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.109304  7588 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:41.109851  7588 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.110268  7588 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.094425  7574 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.098464  7544 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.120651  7588 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.124781  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:41.125878  7588 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.129678  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:41.130844  7347 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.132754  7572 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.133427  7572 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.136590  7572 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.139242  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.140084  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:41.140635  7420 raft_consensus.cc:2468] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
W20260709 13:17:41.141675  7310 leader_election.cc:343] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.142731  7570 raft_consensus.cc:2749] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:41.106034  7538 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.143854  7419 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.146692  7538 ts_tablet_manager.cc:1434] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.051s	user 0.011s	sys 0.000s
I20260709 13:17:41.148029  7538 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:41.148103  7313 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.150372  7572 raft_consensus.cc:2804] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:41.152333  7572 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.153028  7572 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.153127  7419 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:41.153968  7536 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:41.154588  7536 ts_tablet_manager.cc:1403] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.082s	user 0.021s	sys 0.025s
I20260709 13:17:41.158152  7536 raft_consensus.cc:359] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.150436  7347 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:41.166005  7536 raft_consensus.cc:385] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.166397  7536 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.165125  7538 tablet_bootstrap.cc:654] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.159123  7462 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:41.167904  7570 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:41.168412  7570 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:41.169332  7570 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.167287  7536 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.173666  7539 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:41.177343  7539 ts_tablet_manager.cc:1403] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.093s	user 0.037s	sys 0.028s
I20260709 13:17:41.174549  7536 ts_tablet_manager.cc:1434] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.002s
I20260709 13:17:41.175199  7557 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:41.181170  7539 raft_consensus.cc:359] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.182160  7539 raft_consensus.cc:385] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.182539  7539 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.183502  7539 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.186146  7539 ts_tablet_manager.cc:1434] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:17:41.177685  7572 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.187492  7539 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:41.190079  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
W20260709 13:17:41.192018  7310 leader_election.cc:343] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.194859  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:41.195540  7419 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.195963  7539 tablet_bootstrap.cc:654] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.205669  7572 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.206422  7588 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.207239  7588 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.208748  7539 tablet_bootstrap.cc:492] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:41.209381  7539 ts_tablet_manager.cc:1403] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:17:41.211616  7419 raft_consensus.cc:2468] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:41.211802  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.215981  7347 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:41.217782  7459 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:41.212332  7588 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.213033  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.219046  7313 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.220302  7588 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:41.220753  7588 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:41.220965  7462 leader_election.cc:343] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.222229  7572 raft_consensus.cc:2804] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:41.221179  7588 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.214570  7539 raft_consensus.cc:359] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.224239  7572 raft_consensus.cc:697] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:41.223482  7244 catalog_manager.cc:5697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.225133  7539 raft_consensus.cc:385] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.226495  7539 raft_consensus.cc:740] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.225373  7572 consensus_queue.cc:237] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.228336  7539 consensus_queue.cc:260] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.218118  7538 tablet_bootstrap.cc:492] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:41.232282  7538 ts_tablet_manager.cc:1403] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.085s	user 0.019s	sys 0.030s
I20260709 13:17:41.232946  7539 ts_tablet_manager.cc:1434] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 13:17:41.241705  7558 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.242149  7538 raft_consensus.cc:359] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.243541  7538 raft_consensus.cc:385] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.246672  7538 raft_consensus.cc:740] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.246454  7588 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.247893  7538 consensus_queue.cc:260] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.253865  7538 ts_tablet_manager.cc:1434] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:17:41.256184  7538 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:41.256318  7588 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.256884  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
W20260709 13:17:41.259002  7462 leader_election.cc:343] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:41.256219  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:41.260630  7346 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.264855  7538 tablet_bootstrap.cc:654] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.253701  7244 catalog_manager.cc:5697] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:41.277148  7538 tablet_bootstrap.cc:492] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:41.277871  7538 ts_tablet_manager.cc:1403] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.005s
I20260709 13:17:41.279263  7346 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:41.280941  7459 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.281879  7588 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:41.282424  7588 raft_consensus.cc:697] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:41.282920  7538 raft_consensus.cc:359] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.283797  7538 raft_consensus.cc:385] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.283418  7588 consensus_queue.cc:237] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.284094  7538 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.284904  7538 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.287312  7538 ts_tablet_manager.cc:1434] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:41.288486  7538 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:41.294168  7538 tablet_bootstrap.cc:654] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:41.299664  7588 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.300420  7588 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:41.303030  7588 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.303872  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.302197  7538 tablet_bootstrap.cc:492] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:41.304965  7538 ts_tablet_manager.cc:1403] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.000s
I20260709 13:17:41.304831  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.305657  7420 raft_consensus.cc:2393] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 1: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:41.306689  7462 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:41.307448  7588 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.307276  7244 catalog_manager.cc:5697] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:41.308481  7538 raft_consensus.cc:359] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.309173  7538 raft_consensus.cc:385] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:41.309448  7538 raft_consensus.cc:740] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:41.310214  7538 consensus_queue.cc:260] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.312438  7538 ts_tablet_manager.cc:1434] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:17:41.316203  7588 raft_consensus.cc:2749] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:41.318423  7588 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:41.331297  7570 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.378166  7496 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.383551  7557 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.384371  7557 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.387243  7557 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.387490  7572 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.388689  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:41.389668  7496 raft_consensus.cc:2393] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:41.389458  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.391451  7385 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.392369  7557 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.402458  7543 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.411320  7557 raft_consensus.cc:2749] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:41.430091  7543 raft_consensus.cc:493] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.430826  7543 raft_consensus.cc:515] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.432758  7543 leader_election.cc:290] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.433863  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:41.433890  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:41.435050  7495 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:41.435053  7420 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:41.436507  7313 leader_election.cc:304] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:41.437326  7543 raft_consensus.cc:2804] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:41.437697  7543 raft_consensus.cc:493] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:41.438002  7543 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.444730  7543 raft_consensus.cc:515] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.446928  7543 leader_election.cc:290] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.448017  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:41.448244  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:41.448751  7420 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.448927  7495 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.455840  7420 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:41.455840  7495 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:41.457453  7313 leader_election.cc:304] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:41.458307  7543 raft_consensus.cc:2804] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:41.458869  7543 raft_consensus.cc:697] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:41.459692  7543 consensus_queue.cc:237] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.468271  7243 catalog_manager.cc:5697] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:41.621928  7570 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:41.621928  7543 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:17:41.626259  7588 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.658280  7573 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.670329  7570 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.720182  7545 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:41.721760  7545 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.723207  7588 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:17:41.738000  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:41.738689  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:41.740085  7345 raft_consensus.cc:2393] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:41.741961  7385 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.742887  7420 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:41.729626  7545 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:41.755355  7421 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:49318
I20260709 13:17:41.756697  7421 raft_consensus.cc:606] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:41.778333  7588 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.809010  7543 consensus_queue.cc:1048] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.818802  7558 raft_consensus.cc:2749] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:41.823794  7574 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.843149  7495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:41.843946  7495 raft_consensus.cc:606] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:41.855490  7495 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc2688d492204007bb197167fa954266"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:41.856276  7495 raft_consensus.cc:606] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:41.856045  7572 consensus_queue.cc:1048] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:41.880326  7347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d166d5008c8c4d098bb1973379289560"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:41.880956  7347 raft_consensus.cc:606] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.887840  7496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:41.888445  7496 raft_consensus.cc:606] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.895452  7496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:41.896086  7496 raft_consensus.cc:606] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.902135  7347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:41.902860  7347 raft_consensus.cc:606] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:41.908947  7347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06d88a6686704bf29105419087fffe9f"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:41.909543  7347 raft_consensus.cc:606] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:41.948403  7572 consensus_queue.cc:1048] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:41.963930  7572 consensus_queue.cc:1048] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:41.974470  7573 raft_consensus.cc:993] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:41.975126  7572 raft_consensus.cc:1081] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:41.977238  7496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06d88a6686704bf29105419087fffe9f"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:45240
I20260709 13:17:41.978096  7496 raft_consensus.cc:493] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:41.978439  7496 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:41.987323  7496 raft_consensus.cc:515] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.990307  7496 leader_election.cc:290] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:41.990888  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:41.991242  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:41.991817  7347 raft_consensus.cc:3055] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:41.991842  7419 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:41.992354  7347 raft_consensus.cc:740] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:41.993614  7347 consensus_queue.cc:260] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:41.995594  7347 raft_consensus.cc:3060] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.004720  7419 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:42.006912  7462 leader_election.cc:304] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:42.007571  7347 raft_consensus.cc:2468] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:42.008008  7574 raft_consensus.cc:2804] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.008647  7574 raft_consensus.cc:697] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:42.009913  7574 consensus_queue.cc:237] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.017822  7245 catalog_manager.cc:5697] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:42.115195  7544 raft_consensus.cc:993] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.115818  7574 raft_consensus.cc:1081] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.117654  7419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:42.118278  7419 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.118741  7419 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.126122  7419 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.128443  7419 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.130179  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:42.130086  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.131255  7347 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.131273  7496 raft_consensus.cc:3055] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.131860  7496 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:42.133323  7496 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.135146  7496 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.142264  7347 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.144181  7385 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:42.146073  7558 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.146207  7496 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.146803  7558 raft_consensus.cc:697] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:42.148229  7558 consensus_queue.cc:237] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.156348  7245 catalog_manager.cc:5697] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.164995  7570 raft_consensus.cc:993] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.165637  7544 raft_consensus.cc:1081] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.166409  7545 raft_consensus.cc:993] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.167109  7545 raft_consensus.cc:1081] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.167930  7347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:42.168682  7347 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.169121  7347 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.169024  7346 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44704
I20260709 13:17:42.169914  7346 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.170303  7346 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.176160  7347 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.177963  7346 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.178602  7347 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.179687  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:42.179731  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.180485  7419 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.180640  7496 raft_consensus.cc:3055] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.181142  7496 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:42.183290  7496 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.184449  7496 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.187474  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:42.188194  7421 raft_consensus.cc:3055] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.188555  7421 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:42.189615  7421 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.190135  7419 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.191424  7421 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.192600  7313 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:42.193758  7572 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.194296  7572 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:42.194391  7346 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.195508  7572 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.196630  7570 raft_consensus.cc:993] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.197211  7570 raft_consensus.cc:1081] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.198083  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.198822  7495 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.199690  7419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:42.200534  7419 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.200951  7419 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.206905  7570 raft_consensus.cc:993] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.207432  7574 raft_consensus.cc:1081] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:42.207497  7421 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.206650  7245 catalog_manager.cc:5697] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.208956  7313 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:42.199779  7496 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.209776  7346 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc2688d492204007bb197167fa954266"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:42.210603  7346 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.211079  7346 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.211241  7572 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.212103  7572 raft_consensus.cc:697] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:42.212631  7419 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.216141  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:42.216965  7347 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.217702  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.218421  7496 raft_consensus.cc:3055] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.218822  7496 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:42.213373  7572 consensus_queue.cc:237] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.219871  7496 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.221194  7496 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.221915  7346 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.225272  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:42.216203  7495 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.225925  7421 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.231724  7347 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.234966  7346 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.236367  7419 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.237108  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.237874  7495 raft_consensus.cc:3055] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.238252  7495 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:42.239622  7421 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.239455  7495 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.239938  7385 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:42.241055  7545 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.241211  7495 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.241606  7545 raft_consensus.cc:697] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:42.242519  7496 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.242823  7545 consensus_queue.cc:237] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.245340  7313 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:42.247680  7573 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.248327  7573 raft_consensus.cc:697] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:42.247892  7245 catalog_manager.cc:5697] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.249557  7573 consensus_queue.cc:237] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.252830  7495 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:42.258111  7573 raft_consensus.cc:993] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.259655  7573 raft_consensus.cc:1081] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.257093  7243 catalog_manager.cc:5697] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.264079  7419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49290
I20260709 13:17:42.264894  7419 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.265362  7419 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.268323  7627 raft_consensus.cc:993] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.269021  7627 raft_consensus.cc:1081] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:42.272024  7421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d166d5008c8c4d098bb1973379289560"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49290
I20260709 13:17:42.272804  7421 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:42.273240  7421 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.273614  7245 catalog_manager.cc:5697] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.275879  7419 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.280570  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.281322  7495 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.281345  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:42.282243  7347 raft_consensus.cc:3055] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.282586  7347 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:42.283702  7347 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.283766  7421 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.285234  7347 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.286764  7421 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.288285  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:42.289077  7346 raft_consensus.cc:3055] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:42.289466  7346 raft_consensus.cc:740] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:42.290212  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:42.290452  7346 consensus_queue.cc:260] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.278592  7419 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:42.292090  7346 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.290988  7496 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:42.294713  7495 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.296461  7385 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:42.297124  7347 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.297475  7545 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.298189  7545 raft_consensus.cc:697] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:42.299379  7545 consensus_queue.cc:237] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.302253  7496 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.304517  7385 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:42.305548  7545 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:42.306339  7545 raft_consensus.cc:697] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:42.307396  7346 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:42.307420  7545 consensus_queue.cc:237] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:42.308459  7245 catalog_manager.cc:5697] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.324644  7243 catalog_manager.cc:5697] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:42.484499  7346 raft_consensus.cc:1275] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.486311  7544 consensus_queue.cc:1048] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.496213  7419 raft_consensus.cc:1275] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.498189  7570 consensus_queue.cc:1048] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:42.532262  7346 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.534078  7545 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.544847  7496 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.547099  7545 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:42.634792  7419 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.636246  7627 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.645177  7496 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.647182  7573 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:42.668473  7496 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.669827  7573 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.681233  7495 raft_consensus.cc:1275] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.681384  7419 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.683063  7545 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.684322  7573 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:42.707131  7419 raft_consensus.cc:1275] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.708925  7346 raft_consensus.cc:1275] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.708887  7573 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.711232  7632 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.721500  7495 raft_consensus.cc:1275] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.723642  7543 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.749853  7495 raft_consensus.cc:1275] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.751405  7632 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.762458  7346 raft_consensus.cc:1275] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.764375  7632 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.835927  7495 raft_consensus.cc:1275] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.837563  7632 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:42.844087  7346 raft_consensus.cc:1275] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:42.846554  7545 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:44.941895  7243 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59670:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:44.944752  7243 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:45.005471  7334 tablet_service.cc:1511] Processing CreateTablet for tablet 26955d59ed3b4c91ae35bff0d6c45808 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:45.006989  7409 tablet_service.cc:1511] Processing CreateTablet for tablet 26955d59ed3b4c91ae35bff0d6c45808 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:45.008287  7334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26955d59ed3b4c91ae35bff0d6c45808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.008872  7478 tablet_service.cc:1511] Processing CreateTablet for tablet 26955d59ed3b4c91ae35bff0d6c45808 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:45.012277  7478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26955d59ed3b4c91ae35bff0d6c45808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.015594  7329 tablet_service.cc:1511] Processing CreateTablet for tablet 32c77064066849229154ee08fe743028 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:45.016718  7404 tablet_service.cc:1511] Processing CreateTablet for tablet b261c08a7c8a43d2a6fd239e5f8565be (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:45.018596  7337 tablet_service.cc:1511] Processing CreateTablet for tablet b261c08a7c8a43d2a6fd239e5f8565be (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:45.020465  7329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c77064066849229154ee08fe743028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.023505  7410 tablet_service.cc:1511] Processing CreateTablet for tablet 48ea9a7ba2e44f9b9bddce661a5a061e (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:45.025529  7486 tablet_service.cc:1511] Processing CreateTablet for tablet 32c77064066849229154ee08fe743028 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:45.028105  7483 tablet_service.cc:1511] Processing CreateTablet for tablet b261c08a7c8a43d2a6fd239e5f8565be (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:45.029152  7407 tablet_service.cc:1511] Processing CreateTablet for tablet a534d5e35ea341c591a61e94a851a293 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:45.030560  7479 tablet_service.cc:1511] Processing CreateTablet for tablet 48ea9a7ba2e44f9b9bddce661a5a061e (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:45.035045  7412 tablet_service.cc:1511] Processing CreateTablet for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:45.036078  7332 tablet_service.cc:1511] Processing CreateTablet for tablet a534d5e35ea341c591a61e94a851a293 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:45.037875  7332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a534d5e35ea341c591a61e94a851a293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.040503  7405 tablet_service.cc:1511] Processing CreateTablet for tablet 50cddde895ba41b7aac3346cbdb5ae6c (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:45.042562  7335 tablet_service.cc:1511] Processing CreateTablet for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:45.044399  7335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6211b62ab6f94bd7bd8575b3a8673e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.046928  7406 tablet_service.cc:1511] Processing CreateTablet for tablet 9539b8b5f9674ce99dbea364d81f55c7 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:45.047751  7330 tablet_service.cc:1511] Processing CreateTablet for tablet 50cddde895ba41b7aac3346cbdb5ae6c (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:45.048082  7333 tablet_service.cc:1511] Processing CreateTablet for tablet 48ea9a7ba2e44f9b9bddce661a5a061e (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:45.049476  7330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50cddde895ba41b7aac3346cbdb5ae6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.049969  7331 tablet_service.cc:1511] Processing CreateTablet for tablet 9539b8b5f9674ce99dbea364d81f55c7 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:45.051398  7485 tablet_service.cc:1511] Processing CreateTablet for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:45.052366  7484 tablet_service.cc:1511] Processing CreateTablet for tablet a534d5e35ea341c591a61e94a851a293 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:45.011791  7408 tablet_service.cc:1511] Processing CreateTablet for tablet 32c77064066849229154ee08fe743028 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:45.008935  7409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26955d59ed3b4c91ae35bff0d6c45808. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.054600  7482 tablet_service.cc:1511] Processing CreateTablet for tablet 50cddde895ba41b7aac3346cbdb5ae6c (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:45.058979  7406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9539b8b5f9674ce99dbea364d81f55c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.059455  7486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c77064066849229154ee08fe743028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.063445  7337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b261c08a7c8a43d2a6fd239e5f8565be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.063803  7483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b261c08a7c8a43d2a6fd239e5f8565be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.064386  7331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9539b8b5f9674ce99dbea364d81f55c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.065199  7408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32c77064066849229154ee08fe743028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.066468  7333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ea9a7ba2e44f9b9bddce661a5a061e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.070539  7481 tablet_service.cc:1511] Processing CreateTablet for tablet 9539b8b5f9674ce99dbea364d81f55c7 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:45.072072  7407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a534d5e35ea341c591a61e94a851a293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.075014  7405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50cddde895ba41b7aac3346cbdb5ae6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.077399  7336 tablet_service.cc:1511] Processing CreateTablet for tablet 82995eea6405421d941a76c887eb7bd1 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:45.079051  7410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ea9a7ba2e44f9b9bddce661a5a061e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.079282  7336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82995eea6405421d941a76c887eb7bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.080678  7404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b261c08a7c8a43d2a6fd239e5f8565be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.071424  7479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48ea9a7ba2e44f9b9bddce661a5a061e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.083233  7481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9539b8b5f9674ce99dbea364d81f55c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.087028  7484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a534d5e35ea341c591a61e94a851a293. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.091212  7411 tablet_service.cc:1511] Processing CreateTablet for tablet 82995eea6405421d941a76c887eb7bd1 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:45.089350  7412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6211b62ab6f94bd7bd8575b3a8673e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.091012  7482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50cddde895ba41b7aac3346cbdb5ae6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.099117  7411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82995eea6405421d941a76c887eb7bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.102871  7480 tablet_service.cc:1511] Processing CreateTablet for tablet 82995eea6405421d941a76c887eb7bd1 (DEFAULT_TABLE table=soft_deleted_table [id=97cf88786b1844d3a369214870965fde]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:45.102797  7485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6211b62ab6f94bd7bd8575b3a8673e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.107092  7480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82995eea6405421d941a76c887eb7bd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:45.128332  7656 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.150699  7657 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.168910  7656 tablet_bootstrap.cc:654] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.214774  7658 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.248909  7657 tablet_bootstrap.cc:654] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.263612  7656 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.264246  7656 ts_tablet_manager.cc:1403] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.136s	user 0.009s	sys 0.010s
I20260709 13:17:45.267904  7657 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.267767  7656 raft_consensus.cc:359] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.268425  7656 raft_consensus.cc:385] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.268513  7657 ts_tablet_manager.cc:1403] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.118s	user 0.014s	sys 0.007s
I20260709 13:17:45.268764  7656 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.269627  7656 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.271720  7657 raft_consensus.cc:359] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.272516  7657 raft_consensus.cc:385] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.272819  7657 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.273358  7658 tablet_bootstrap.cc:654] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.273627  7657 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.276484  7657 ts_tablet_manager.cc:1434] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:17:45.277633  7657 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.285645  7656 ts_tablet_manager.cc:1434] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:17:45.286379  7657 tablet_bootstrap.cc:654] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.287102  7656 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.292800  7657 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.293511  7657 ts_tablet_manager.cc:1403] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.005s
I20260709 13:17:45.296204  7658 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.296815  7658 ts_tablet_manager.cc:1403] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.082s	user 0.006s	sys 0.007s
I20260709 13:17:45.296883  7657 raft_consensus.cc:359] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.297730  7657 raft_consensus.cc:385] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.298061  7657 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.298834  7657 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.300537  7658 raft_consensus.cc:359] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.301327  7657 ts_tablet_manager.cc:1434] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:17:45.301417  7658 raft_consensus.cc:385] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.302945  7657 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.303138  7658 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.304040  7658 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.306145  7656 tablet_bootstrap.cc:654] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.310537  7657 tablet_bootstrap.cc:654] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.321729  7656 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.322288  7656 ts_tablet_manager.cc:1403] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.002s
I20260709 13:17:45.324034  7658 ts_tablet_manager.cc:1434] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.001s
I20260709 13:17:45.325489  7658 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.325775  7656 raft_consensus.cc:359] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.326936  7656 raft_consensus.cc:385] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.327265  7656 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.329617  7656 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.331161  7657 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.331831  7657 ts_tablet_manager.cc:1403] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.021s
I20260709 13:17:45.334966  7658 tablet_bootstrap.cc:654] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.337523  7657 raft_consensus.cc:359] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.338419  7657 raft_consensus.cc:385] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.338762  7657 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.339648  7657 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.342705  7656 ts_tablet_manager.cc:1434] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.000s
I20260709 13:17:45.348347  7656 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.350140  7657 ts_tablet_manager.cc:1434] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.005s
I20260709 13:17:45.354996  7657 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.357908  7656 tablet_bootstrap.cc:654] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.363150  7657 tablet_bootstrap.cc:654] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.372020  7658 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.372779  7658 ts_tablet_manager.cc:1403] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.019s
I20260709 13:17:45.372888  7656 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.373558  7656 ts_tablet_manager.cc:1403] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 13:17:45.373726  7657 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.374293  7657 ts_tablet_manager.cc:1403] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 13:17:45.376786  7658 raft_consensus.cc:359] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.377053  7656 raft_consensus.cc:359] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.377262  7657 raft_consensus.cc:359] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.377765  7658 raft_consensus.cc:385] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.377856  7656 raft_consensus.cc:385] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.378069  7657 raft_consensus.cc:385] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.378212  7658 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.378331  7656 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.378585  7657 raft_consensus.cc:740] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.379386  7656 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.379558  7657 consensus_queue.cc:260] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.379387  7658 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.382477  7656 ts_tablet_manager.cc:1434] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:17:45.382472  7658 ts_tablet_manager.cc:1434] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:17:45.383774  7658 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.383774  7656 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.388664  7657 ts_tablet_manager.cc:1434] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:17:45.390249  7657 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.392458  7656 tablet_bootstrap.cc:654] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.398118  7657 tablet_bootstrap.cc:654] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.404054  7658 tablet_bootstrap.cc:654] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.405556  7656 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.406241  7656 ts_tablet_manager.cc:1403] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.023s	user 0.011s	sys 0.008s
I20260709 13:17:45.410689  7656 raft_consensus.cc:359] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.411844  7656 raft_consensus.cc:385] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.412267  7656 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.413594  7644 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.414276  7644 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.418787  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.419970  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.420202  7346 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:45.422601  7459 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.422870  7644 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.424057  7644 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:17:45.424540  7462 leader_election.cc:343] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.424613  7644 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.425933  7644 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.426666  7656 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.431774  7656 ts_tablet_manager.cc:1434] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 13:17:45.434718  7657 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.435298  7656 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.435446  7658 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.436218  7658 ts_tablet_manager.cc:1403] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.024s
I20260709 13:17:45.440023  7658 raft_consensus.cc:359] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.440932  7658 raft_consensus.cc:385] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.441278  7658 raft_consensus.cc:740] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.441552  7644 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.435443  7657 ts_tablet_manager.cc:1403] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.045s	user 0.022s	sys 0.016s
I20260709 13:17:45.445250  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.446012  7346 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.450358  7573 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.456838  7671 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.457731  7671 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.451256  7573 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.442166  7658 consensus_queue.cc:260] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.466120  7656 tablet_bootstrap.cc:654] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.467267  7657 raft_consensus.cc:359] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.468153  7657 raft_consensus.cc:385] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.468482  7657 raft_consensus.cc:740] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.469345  7657 consensus_queue.cc:260] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.471228  7658 ts_tablet_manager.cc:1434] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.035s	user 0.010s	sys 0.000s
I20260709 13:17:45.471468  7644 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.473076  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.473894  7658 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.474589  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.475703  7657 ts_tablet_manager.cc:1434] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.010s
I20260709 13:17:45.476933  7657 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.480331  7656 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.481052  7656 ts_tablet_manager.cc:1403] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.046s	user 0.028s	sys 0.014s
I20260709 13:17:45.483790  7645 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:45.484055  7462 leader_election.cc:343] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.484444  7645 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
W20260709 13:17:45.484988  7313 leader_election.cc:343] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.484939  7656 raft_consensus.cc:359] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.486482  7656 raft_consensus.cc:385] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.486889  7656 raft_consensus.cc:740] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.486938  7645 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.488065  7656 consensus_queue.cc:260] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.489197  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.491022  7658 tablet_bootstrap.cc:654] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.490139  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.491623  7656 ts_tablet_manager.cc:1434] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:17:45.493083  7656 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
W20260709 13:17:45.493204  7385 leader_election.cc:343] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:17:45.494450  7385 leader_election.cc:343] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.494935  7385 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:45.496225  7645 raft_consensus.cc:2749] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.497918  7657 tablet_bootstrap.cc:654] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.498862  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
W20260709 13:17:45.508476  7310 leader_election.cc:343] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.508558  7671 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.509014  7310 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:45.510082  7675 raft_consensus.cc:2749] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.498168  7573 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.512254  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.510555  7656 tablet_bootstrap.cc:654] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.513067  7421 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:45.513237  7671 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.514087  7675 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.513989  7671 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.514693  7675 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.516952  7675 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.518329  7462 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.519207  7644 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.519603  7644 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.519943  7644 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.520928  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.522250  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.522944  7345 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
W20260709 13:17:45.523214  7459 leader_election.cc:343] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.524403  7459 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.527125  7672 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.527521  7672 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.527868  7672 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.529156  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.529739  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.530788  7495 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
W20260709 13:17:45.531379  7462 leader_election.cc:343] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.532864  7310 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.534204  7675 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.535579  7671 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.535985  7675 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.536482  7675 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.542073  7672 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.542447  7644 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.547148  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.538560  7346 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:45.549961  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.550827  7347 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.551182  7459 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:45.551767  7657 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.552109  7671 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.552413  7657 ts_tablet_manager.cc:1403] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.000s
I20260709 13:17:45.552675  7671 raft_consensus.cc:697] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:45.553588  7671 consensus_queue.cc:237] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.557415  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
W20260709 13:17:45.557430  7462 leader_election.cc:343] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.554952  7672 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.558238  7657 raft_consensus.cc:359] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.559518  7657 raft_consensus.cc:385] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.559437  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.560266  7421 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.561591  7658 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.562202  7658 ts_tablet_manager.cc:1403] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.089s	user 0.020s	sys 0.009s
I20260709 13:17:45.548143  7644 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.564121  7347 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
W20260709 13:17:45.564972  7459 leader_election.cc:343] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.568344  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.565487  7675 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.563673  7657 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.571959  7657 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.572907  7675 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.574558  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.574896  7421 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:45.575353  7657 ts_tablet_manager.cc:1434] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.023s	user 0.011s	sys 0.000s
I20260709 13:17:45.576263  7462 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:45.576640  7657 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.577632  7644 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:17:45.577690  7313 leader_election.cc:343] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.578275  7644 raft_consensus.cc:697] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:45.567366  7656 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.579044  7656 ts_tablet_manager.cc:1403] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.086s	user 0.011s	sys 0.011s
I20260709 13:17:45.579242  7644 consensus_queue.cc:237] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.585731  7657 tablet_bootstrap.cc:654] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.570413  7459 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
W20260709 13:17:45.587492  7313 leader_election.cc:343] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.567306  7658 raft_consensus.cc:359] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.588905  7658 raft_consensus.cc:385] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.589032  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:45.589465  7658 raft_consensus.cc:740] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.589844  7495 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.590834  7658 consensus_queue.cc:260] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.593964  7658 ts_tablet_manager.cc:1434] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.031s	user 0.003s	sys 0.006s
I20260709 13:17:45.594903  7656 raft_consensus.cc:359] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.595868  7656 raft_consensus.cc:385] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.596190  7656 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.597052  7656 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.597954  7244 catalog_manager.cc:5697] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.599830  7656 ts_tablet_manager.cc:1434] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 13:17:45.601039  7656 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.602138  7495 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:45.603881  7310 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:45.605334  7678 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.605893  7678 raft_consensus.cc:697] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:45.606875  7678 consensus_queue.cc:237] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.587497  7672 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.610111  7672 raft_consensus.cc:697] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:45.611387  7672 consensus_queue.cc:237] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.614216  7657 tablet_bootstrap.cc:492] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.615095  7657 ts_tablet_manager.cc:1403] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.006s
I20260709 13:17:45.615713  7658 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.619433  7657 raft_consensus.cc:359] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.620517  7657 raft_consensus.cc:385] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.620918  7657 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.621622  7671 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.621807  7657 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.622455  7671 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.626564  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.627406  7347 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:45.629288  7459 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.630227  7658 tablet_bootstrap.cc:654] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.630363  7644 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.630940  7644 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.631371  7644 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.635320  7671 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.636992  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
W20260709 13:17:45.638878  7462 leader_election.cc:343] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.637003  7657 ts_tablet_manager.cc:1434] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 13:17:45.641554  7657 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.642417  7644 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.645141  7644 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.646687  7242 catalog_manager.cc:5697] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:45.651553  7658 tablet_bootstrap.cc:492] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.651207  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.652515  7658 ts_tablet_manager.cc:1403] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.037s	user 0.015s	sys 0.014s
W20260709 13:17:45.653704  7462 leader_election.cc:343] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.651278  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.655512  7657 tablet_bootstrap.cc:654] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.655982  7347 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.656597  7658 raft_consensus.cc:359] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.657495  7658 raft_consensus.cc:385] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.657840  7658 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.658785  7656 tablet_bootstrap.cc:654] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.658730  7658 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.661458  7658 ts_tablet_manager.cc:1434] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:17:45.663210  7658 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.674314  7658 tablet_bootstrap.cc:654] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.681805  7656 tablet_bootstrap.cc:492] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.682467  7656 ts_tablet_manager.cc:1403] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.082s	user 0.015s	sys 0.000s
I20260709 13:17:45.686165  7656 raft_consensus.cc:359] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.687067  7656 raft_consensus.cc:385] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.687373  7656 raft_consensus.cc:740] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.688458  7656 consensus_queue.cc:260] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.691193  7656 ts_tablet_manager.cc:1434] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:17:45.692389  7656 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.694301  7645 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.694947  7645 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.700574  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.701637  7496 raft_consensus.cc:2393] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:45.701769  7656 tablet_bootstrap.cc:654] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.698429  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.705022  7645 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.706532  7385 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:45.692876  7243 catalog_manager.cc:5697] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.711128  7657 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.711969  7657 ts_tablet_manager.cc:1403] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.071s	user 0.015s	sys 0.002s
I20260709 13:17:45.711769  7645 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.716642  7657 raft_consensus.cc:359] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.717715  7657 raft_consensus.cc:385] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.718096  7657 raft_consensus.cc:740] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.710539  7243 catalog_manager.cc:5697] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.719200  7657 consensus_queue.cc:260] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.722888  7657 ts_tablet_manager.cc:1434] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.010s	user 0.012s	sys 0.000s
I20260709 13:17:45.724443  7657 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: Bootstrap starting.
I20260709 13:17:45.731200  7656 tablet_bootstrap.cc:492] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.731904  7656 ts_tablet_manager.cc:1403] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.040s	user 0.007s	sys 0.012s
I20260709 13:17:45.735091  7657 tablet_bootstrap.cc:654] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.735477  7656 raft_consensus.cc:359] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.736351  7656 raft_consensus.cc:385] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.736650  7656 raft_consensus.cc:740] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.746681  7656 consensus_queue.cc:260] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.748219  7347 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:45.749698  7656 ts_tablet_manager.cc:1434] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:17:45.751070  7656 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: Bootstrap starting.
I20260709 13:17:45.750605  7459 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:45.753221  7672 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.753783  7672 raft_consensus.cc:697] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:45.754705  7672 consensus_queue.cc:237] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.757740  7645 raft_consensus.cc:2749] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.759346  7656 tablet_bootstrap.cc:654] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.761952  7658 tablet_bootstrap.cc:492] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.762706  7658 ts_tablet_manager.cc:1403] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.100s	user 0.058s	sys 0.036s
I20260709 13:17:45.766783  7658 raft_consensus.cc:359] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.767730  7658 raft_consensus.cc:385] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.768106  7658 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.769052  7658 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.772059  7658 ts_tablet_manager.cc:1434] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 13:17:45.773221  7658 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.787427  7658 tablet_bootstrap.cc:654] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.792479  7645 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.793463  7645 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.799297  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.800277  7347 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 0.
I20260709 13:17:45.802750  7385 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:45.813464  7694 raft_consensus.cc:2804] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.822523  7694 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.828181  7694 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.822974  7645 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.831665  7242 catalog_manager.cc:5697] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:45.834444  7672 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.835345  7672 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.839280  7656 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: No bootstrap required, opened a new log
I20260709 13:17:45.840052  7656 ts_tablet_manager.cc:1403] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent bootstrapping tablet: real 0.089s	user 0.043s	sys 0.030s
I20260709 13:17:45.843799  7656 raft_consensus.cc:359] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.844625  7656 raft_consensus.cc:385] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.845059  7656 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.844729  7694 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.846804  7656 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.847275  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.848297  7347 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:45.848562  7694 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.849640  7675 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.849773  7656 ts_tablet_manager.cc:1434] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:45.850235  7675 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.852639  7675 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.853919  7694 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.854599  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.854563  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.862954  7459 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.863581  7495 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:45.864015  7644 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.862649  7672 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.865139  7644 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.865612  7644 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.865865  7310 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
W20260709 13:17:45.866979  7293 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:45.854643  7694 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.870738  7419 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:45.871616  7678 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.874145  7657 tablet_bootstrap.cc:492] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: No bootstrap required, opened a new log
I20260709 13:17:45.874903  7657 ts_tablet_manager.cc:1403] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: Time spent bootstrapping tablet: real 0.151s	user 0.040s	sys 0.023s
I20260709 13:17:45.872686  7678 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.876091  7678 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.876783  7694 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
W20260709 13:17:45.872575  7385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 60004us
I20260709 13:17:45.880869  7657 raft_consensus.cc:359] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.881734  7657 raft_consensus.cc:385] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.882018  7657 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.882513  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.882848  7657 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.883536  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.882513  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.884259  7420 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 0.
I20260709 13:17:45.884490  7347 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.885386  7657 ts_tablet_manager.cc:1434] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:17:45.883368  7495 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 0.
I20260709 13:17:45.890671  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:45.891503  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:45.891744  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.892257  7495 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.892481  7496 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 0.
I20260709 13:17:45.898346  7658 tablet_bootstrap.cc:492] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.899140  7658 ts_tablet_manager.cc:1403] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.126s	user 0.021s	sys 0.005s
I20260709 13:17:45.901477  7644 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.904366  7658 raft_consensus.cc:359] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.903019  7385 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.905262  7658 raft_consensus.cc:385] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.905671  7658 raft_consensus.cc:740] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.906253  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.906426  7699 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.907034  7699 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.906582  7658 consensus_queue.cc:260] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.907578  7346 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.907408  7699 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.908895  7658 ts_tablet_manager.cc:1434] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:17:45.910189  7658 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.911484  7644 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:45.912845  7678 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.914170  7345 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:45.919332  7573 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.919955  7658 tablet_bootstrap.cc:654] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:45.920502  7699 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.920143  7573 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.921856  7678 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.923094  7699 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.924425  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:45.921305  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:45.925231  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.925401  7496 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.924886  7573 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.926092  7494 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 0.
I20260709 13:17:45.925390  7345 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:45.927608  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:45.927563  7310 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.930114  7385 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:45.930941  7658 tablet_bootstrap.cc:492] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.931069  7699 raft_consensus.cc:2749] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:45.931711  7658 ts_tablet_manager.cc:1403] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.004s
I20260709 13:17:45.934844  7573 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:45.935480  7573 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:45.935977  7573 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.935608  7658 raft_consensus.cc:359] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.936363  7496 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:45.936486  7658 raft_consensus.cc:385] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.938072  7310 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.937162  7658 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.939421  7658 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.940296  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.941243  7421 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 in current term 1: Already voted for candidate 3e9e84d2e507455aa80bc7595a1ef075 in this term.
I20260709 13:17:45.942261  7658 ts_tablet_manager.cc:1434] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 13:17:45.942842  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.943228  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:45.943701  7658 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: Bootstrap starting.
I20260709 13:17:45.943961  7420 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.946904  7678 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.948168  7678 raft_consensus.cc:697] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:45.949369  7678 consensus_queue.cc:237] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.952271  7658 tablet_bootstrap.cc:654] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:45.955701  7313 leader_election.cc:343] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:45.956907  7347 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 1.
I20260709 13:17:45.958645  7385 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:45.959601  7699 raft_consensus.cc:2804] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.960147  7420 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:45.962010  7346 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:45.962218  7675 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:45.963131  7675 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.963761  7699 raft_consensus.cc:697] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:45.964952  7699 consensus_queue.cc:237] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.966127  7495 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 1.
I20260709 13:17:45.966984  7658 tablet_bootstrap.cc:492] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: No bootstrap required, opened a new log
I20260709 13:17:45.967586  7658 ts_tablet_manager.cc:1403] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.003s
I20260709 13:17:45.968257  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:45.967370  7459 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:45.969823  7644 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:45.970507  7644 raft_consensus.cc:697] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:45.971033  7675 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:45.971630  7644 consensus_queue.cc:237] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.972026  7658 raft_consensus.cc:359] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.972796  7658 raft_consensus.cc:385] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:45.973110  7658 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Initialized, Role: FOLLOWER
I20260709 13:17:45.973876  7658 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.975428  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:45.976480  7658 ts_tablet_manager.cc:1434] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:17:45.978083  7420 raft_consensus.cc:2393] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 in current term 1: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:45.980417  7313 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:45.982039  7675 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:45.990542  7242 catalog_manager.cc:5697] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.002317  7672 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:45.983497  7573 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:45.989197  7243 catalog_manager.cc:5697] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.015919  7573 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.014945  7675 raft_consensus.cc:2749] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.019439  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.020186  7420 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.019713  7244 catalog_manager.cc:5697] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:46.030133  7420 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:46.031658  7313 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.032706  7678 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:46.033265  7678 raft_consensus.cc:697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.033382  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.034173  7494 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.034220  7678 consensus_queue.cc:237] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.048164  7494 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:46.048012  7243 catalog_manager.cc:5697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 1 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.052007  7420 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.062155  7671 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.070415  7678 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.088356  7675 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.116706  7673 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.126320  7420 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.131606  7673 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.138717  7673 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.154733  7673 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.158097  7699 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:46.158861  7699 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.162742  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:46.164116  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:46.165477  7347 raft_consensus.cc:2393] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 1: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:46.166756  7385 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:46.167770  7694 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:46.168481  7699 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.178236  7694 raft_consensus.cc:2749] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:46.213765  7673 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.229154  7673 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.255373  7494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:46.256161  7494 raft_consensus.cc:606] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:46.262602  7494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32c77064066849229154ee08fe743028"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:46.263378  7494 raft_consensus.cc:606] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:46.269227  7494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:46.269757  7494 raft_consensus.cc:606] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:46.274956  7494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:46.275496  7494 raft_consensus.cc:606] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:46.283913  7346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:46.284482  7346 raft_consensus.cc:606] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:46.290798  7494 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:45244
I20260709 13:17:46.291415  7494 raft_consensus.cc:606] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:46.297590  7346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:46.298295  7346 raft_consensus.cc:606] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:46.303814  7346 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:44724
I20260709 13:17:46.304337  7346 raft_consensus.cc:606] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:46.390686  7694 consensus_queue.cc:1048] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.402947  7645 consensus_queue.cc:1048] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.438588  7673 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.446034  7672 raft_consensus.cc:993] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.446640  7671 raft_consensus.cc:1081] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.448589  7347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:46.449273  7347 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.449728  7347 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.454002  7671 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.458799  7675 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.460014  7347 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.462702  7347 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.463929  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.464284  7573 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.464743  7495 raft_consensus.cc:3055] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.465137  7495 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.466122  7495 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.469147  7495 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.470219  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.475098  7420 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.485692  7420 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.487268  7313 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.488372  7675 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.488950  7675 raft_consensus.cc:697] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.489945  7675 consensus_queue.cc:237] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.496181  7678 raft_consensus.cc:993] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:46.496800  7678 raft_consensus.cc:1081] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:46.501502  7672 raft_consensus.cc:993] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.501577  7496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:45240
I20260709 13:17:46.502199  7673 raft_consensus.cc:1081] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.502373  7496 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.502962  7496 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.503665  7705 raft_consensus.cc:993] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.504220  7705 raft_consensus.cc:1081] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.506953  7420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49290
I20260709 13:17:46.507611  7420 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.508093  7420 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.510241  7244 catalog_manager.cc:5697] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.507678  7347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32c77064066849229154ee08fe743028"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:46.513612  7347 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.514120  7347 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.520243  7420 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.521633  7419 raft_consensus.cc:1240] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:46.523300  7420 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.523845  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:46.524565  7346 raft_consensus.cc:3055] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.524881  7346 raft_consensus.cc:740] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.525707  7346 consensus_queue.cc:260] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.530066  7347 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.531566  7346 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.532248  7495 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.533905  7347 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.534178  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.534914  7495 raft_consensus.cc:3055] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.535259  7495 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.535738  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.536401  7420 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.536132  7495 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.537628  7495 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.539531  7494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.539961  7573 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:46.541390  7346 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.543220  7385 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.540277  7494 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.544420  7699 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.544932  7699 raft_consensus.cc:697] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:46.545832  7699 consensus_queue.cc:237] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.546289  7495 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.547879  7310 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:46.549269  7573 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.550146  7573 raft_consensus.cc:697] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.551750  7573 consensus_queue.cc:237] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.571143  7420 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.574086  7496 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.577531  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:46.578249  7346 raft_consensus.cc:3055] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.578594  7346 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.578663  7678 raft_consensus.cc:993] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.579218  7678 raft_consensus.cc:1081] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.579535  7346 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.579208  7494 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.581045  7346 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.580883  7244 catalog_manager.cc:5697] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.585207  7420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49290
I20260709 13:17:46.586074  7420 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.587653  7420 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.591116  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.591987  7421 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.596657  7573 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:46.598191  7420 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.599961  7672 raft_consensus.cc:993] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.600574  7672 raft_consensus.cc:1081] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.602335  7420 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.603565  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.604280  7495 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.609498  7345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:46.610226  7345 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.610740  7345 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.610028  7243 catalog_manager.cc:5697] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.606743  7496 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:46.614991  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:46.615890  7347 raft_consensus.cc:3055] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.616230  7347 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.618964  7421 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:46.619141  7347 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.620474  7347 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.620714  7462 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:46.621769  7673 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.622282  7673 raft_consensus.cc:697] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.623234  7673 consensus_queue.cc:237] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.627682  7495 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.628770  7493 raft_consensus.cc:1240] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:17:46.629213  7347 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.590553  7346 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:46.630806  7385 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.631803  7699 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.632318  7699 raft_consensus.cc:697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:46.633239  7699 consensus_queue.cc:237] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.633415  7345 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.637096  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.637882  7420 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.638058  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.638842  7493 raft_consensus.cc:3055] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.639178  7493 raft_consensus.cc:740] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.640106  7493 consensus_queue.cc:260] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.641408  7493 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.640487  7243 catalog_manager.cc:5697] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:46.637045  7345 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.646900  7420 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.647044  7244 catalog_manager.cc:5697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.648499  7313 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.649515  7678 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.650032  7678 raft_consensus.cc:697] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.650571  7493 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.650981  7678 consensus_queue.cc:237] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.660403  7243 catalog_manager.cc:5697] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.713923  7671 raft_consensus.cc:993] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.714519  7673 raft_consensus.cc:1081] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:46.715631  7672 raft_consensus.cc:993] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.716351  7673 raft_consensus.cc:1081] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:46.716809  7345 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:46.717583  7345 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.717947  7345 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.718266  7420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:46.718918  7420 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:46.719249  7420 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.726066  7345 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.726397  7420 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.729527  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:46.729678  7420 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.730201  7347 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.731182  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.731841  7493 raft_consensus.cc:3055] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.732146  7493 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.732967  7493 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.733938  7493 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.737591  7347 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.739126  7385 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.740178  7699 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.740016  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:46.740111  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:46.740939  7699 raft_consensus.cc:697] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:46.741110  7495 raft_consensus.cc:3055] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:46.741226  7420 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.741488  7493 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:46.740238  7345 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:46.741633  7495 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:46.742908  7495 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.741977  7699 consensus_queue.cc:237] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.744176  7495 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:46.750114  7420 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.751330  7495 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:46.751804  7313 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:46.752868  7678 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:17:46.753399  7678 raft_consensus.cc:697] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:46.754009  7243 catalog_manager.cc:5697] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.754365  7678 consensus_queue.cc:237] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:46.764432  7244 catalog_manager.cc:5697] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 2, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 2 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:46.972288  7420 raft_consensus.cc:1275] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:46.973851  7678 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:46.982579  7495 raft_consensus.cc:1275] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:46.985129  7573 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:47.013175  7420 raft_consensus.cc:1275] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.014840  7573 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.024178  7495 raft_consensus.cc:1275] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.026147  7678 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.042812  7493 raft_consensus.cc:1275] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.044456  7678 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.054545  7420 raft_consensus.cc:1275] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.056725  7705 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.065423  7495 raft_consensus.cc:1275] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.067332  7699 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:47.077361  7345 raft_consensus.cc:1275] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.079169  7694 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.120535  7420 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.122130  7673 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.132470  7345 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.135234  7673 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.153169  7345 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.154664  7694 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.163321  7493 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.165936  7645 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:47.235206  7345 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.235353  7493 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.236860  7645 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.237602  7694 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.275861  7420 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.277468  7705 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:47.288201  7493 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:17:47.290266  7705 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:49.088560  7737 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.089635  7737 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.092898  7737 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.197680  7737 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.198678  7737 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.201963  7737 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.273227  7740 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.278046  7738 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.282477  7739 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.283643  7739 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.287422  7739 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.296293  7739 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.283813  7738 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.297479  7739 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.274040  7740 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.302109  7739 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.303233  7739 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:49.303557  7738 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.304399  7740 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.304170  7739 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.305985  7742 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.306761  7742 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.307868  7739 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.310261  7742 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.310695  7744 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.313592  7739 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.311882  7744 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.314375  7743 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.313511  7740 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.316102  7743 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.316612  7740 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.317759  7744 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.318817  7743 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.319079  7740 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.311070  7742 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.315210  7739 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.322961  7739 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.323854  7742 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.325961  7742 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.338531  7737 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.339316  7737 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.341737  7737 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.338291  7744 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.343205  7744 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.345057  7745 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.345922  7745 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:49.346434  7744 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.348285  7744 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.348867  7744 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.352506  7737 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.353190  7737 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.353818  7746 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:49.354888  7746 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.355683  7737 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.351142  7744 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.358582  7746 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.350848  7742 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.382283  7742 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.367720  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.384963  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.386080  7340 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:49.386270  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.369551  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.369796  7739 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.372479  7343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.372905  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.374879  7341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.367722  7745 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.378283  7339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.378875  7747 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.379504  7338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.381385  7748 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.395792  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.425638  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.394415  7737 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.387554  7744 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.384994  7742 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.408177  7750 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.442929  7737 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.452664  7744 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.443946  7747 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.456758  7737 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.438026  7745 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.457738  7745 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.458526  7747 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.459147  7742 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.441638  7748 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.459822  7742 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.460915  7745 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.433033  7739 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.453239  7753 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.462589  7742 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.462990  7750 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.443009  7344 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.467983  7742 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.464779  7739 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.469633  7745 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.471105  7745 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.468664  7742 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.467509  7747 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.464829  7744 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.475145  7747 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.477664  7742 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.478111  7747 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.449448  7385 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:49.482725  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.481976  7746 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.469786  7753 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.483525  7493 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.483704  7746 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:49.484166  7746 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.469965  7750 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.487003  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.487861  7493 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.480046  7745 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.495409  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.497167  7752 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.497675  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.497797  7752 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.499222  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.500982  7421 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.490547  7749 raft_consensus.cc:493] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:49.490593  7737 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.502462  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.503248  7749 raft_consensus.cc:515] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.490851  7753 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.507017  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.509135  7421 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.512570  7749 leader_election.cc:290] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.513469  7737 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:49.516611  7737 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.517680  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.518735  7493 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.519558  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.525460  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.524983  7420 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.527583  7748 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.529291  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.530236  7493 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.531637  7746 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.532977  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.534057  7746 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.537266  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.538595  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.539141  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.539306  7492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.540113  7492 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 1.
I20260709 13:17:49.540553  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.533915  7752 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.541703  7492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.542483  7492 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.543555  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.543732  7492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.544605  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.547843  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.565259  7419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.551975  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.565956  7416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.566022  7419 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.546226  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06d88a6686704bf29105419087fffe9f" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.559767  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.549605  7420 raft_consensus.cc:1240] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Rejecting Update request from peer c5bbd435a7a74dceaa00d2cd66e601f2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.563436  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.568363  7417 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.558543  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.569437  7422 raft_consensus.cc:2468] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 1.
I20260709 13:17:49.545822  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.554203  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.572868  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.575314  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.575503  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.575901  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.576545  7490 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:49.577519  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.579711  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.580590  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.581598  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.582404  7421 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.583684  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.584421  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.585063  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.585342  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.586444  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.587157  7420 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 2.
I20260709 13:17:49.588449  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.590057  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:49.591863  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.592149  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.593540  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.594954  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.596371  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.599164  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.600618  7385 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.602035  7748 raft_consensus.cc:2749] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.602284  7737 raft_consensus.cc:2749] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.604782  7385 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.606151  7385 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.608508  7752 raft_consensus.cc:2749] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.607297  7746 raft_consensus.cc:2749] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.609701  7385 leader_election.cc:304] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.612511  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.611101  7385 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.614324  7385 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.614959  7746 raft_consensus.cc:2749] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.615187  7748 raft_consensus.cc:2749] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.615622  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.616619  7748 raft_consensus.cc:2749] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:49.617421  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.617986  7310 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.619315  7739 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.619495  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.619846  7739 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.620379  7739 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.621035  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.622727  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.623147  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.623867  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.624961  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:49.621953  7422 raft_consensus.cc:2393] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 in current term 3: Already voted for candidate 3e9e84d2e507455aa80bc7595a1ef075 in this term.
I20260709 13:17:49.626852  7462 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.628477  7755 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.628598  7462 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.629065  7755 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.629526  7755 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.630219  7753 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.630684  7753 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.631086  7753 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.629838  7462 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.633402  7745 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.633843  7745 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.634230  7745 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.640123  7462 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.641247  7750 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.641703  7750 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.641857  7462 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.642149  7750 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.643039  7462 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.642868  7753 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.643962  7742 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.644375  7742 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.644757  7742 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.618031  7385 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.653317  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:49.656126  7310 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.657609  7746 raft_consensus.cc:2749] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.658849  7744 raft_consensus.cc:2804] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:49.659350  7744 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.659003  7747 consensus_queue.cc:1059] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.659840  7744 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.659698  7742 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.658535  7740 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.661496  7740 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.660799  7755 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.661936  7740 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.662163  7742 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.660861  7754 raft_consensus.cc:3055] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.663287  7754 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:49.664361  7754 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.666539  7310 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.667975  7741 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.668417  7741 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.668800  7741 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.657769  7753 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.669680  7744 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
W20260709 13:17:49.672389  7385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:41951, user_credentials={real_user=slave}} blocked reactor thread for 55396.2us
I20260709 13:17:49.673462  7385 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.674316  7746 raft_consensus.cc:2749] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.674733  7755 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.674860  7462 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.675858  7385 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.675871  7755 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.676338  7462 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:49.676715  7755 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.677217  7755 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.677251  7742 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.677806  7742 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:49.678279  7742 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.678454  7744 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.679304  7746 raft_consensus.cc:2749] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.681516  7754 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.650072  7739 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.685475  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.685897  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.685511  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.687204  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.689831  7739 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.686995  7755 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.690595  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.692616  7310 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.697120  7742 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.699837  7742 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.699941  7741 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.700752  7743 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.699690  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.701321  7743 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.701560  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.701763  7743 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.703413  7755 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.705385  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.712661  7741 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.712435  7740 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.706461  7745 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
W20260709 13:17:49.714779  7459 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 57472.2us
I20260709 13:17:49.714849  7750 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.722302  7459 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.719717  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.715998  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.724658  7750 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.725757  7750 raft_consensus.cc:2749] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.713033  7743 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.726539  7459 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.726830  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.728413  7459 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.729177  7743 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.716689  7740 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.730461  7459 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.730659  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.716311  7310 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.731534  7754 raft_consensus.cc:2749] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.718472  7745 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.731568  7755 raft_consensus.cc:2749] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.733232  7459 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.733721  7310 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.734243  7755 raft_consensus.cc:2749] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.734716  7739 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.735180  7740 raft_consensus.cc:2804] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:17:49.735682  7740 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.736143  7740 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.736416  7459 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.737380  7755 raft_consensus.cc:2749] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.739383  7459 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.740265  7755 raft_consensus.cc:2749] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.742429  7459 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.743315  7755 raft_consensus.cc:2749] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:17:49.745481  7740 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.746151  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.730739  7750 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.746995  7750 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.747509  7750 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.747407  7310 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.749090  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.750675  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.751603  7740 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.751940  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.749292  7310 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.753669  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.760360  7339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.755344  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.756960  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.753235  7743 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.762104  7743 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.758576  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.762660  7743 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.763173  7310 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:49.764207  7741 raft_consensus.cc:2749] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.764390  7459 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.759181  7739 raft_consensus.cc:2749] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.765012  7741 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:49.765445  7741 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:49.765748  7755 raft_consensus.cc:2749] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.767560  7459 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.768386  7755 raft_consensus.cc:2749] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.765836  7741 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.769896  7459 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.770798  7755 raft_consensus.cc:2749] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.772396  7459 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.773255  7755 raft_consensus.cc:2749] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:49.774750  7459 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.775635  7755 raft_consensus.cc:2749] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.759634  7750 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.776985  7741 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.777057  7459 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.780333  7741 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.780694  7755 raft_consensus.cc:2749] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.780889  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.781416  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.782292  7459 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.783058  7750 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:49.784008  7755 raft_consensus.cc:2749] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.785218  7459 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.786281  7743 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.787667  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:49.788513  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.789846  7755 raft_consensus.cc:2749] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.790380  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.789849  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:49.791201  7743 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:49.792263  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.793157  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.793977  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.795342  7488 raft_consensus.cc:1240] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.796401  7493 raft_consensus.cc:1240] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:49.796404  7496 raft_consensus.cc:1240] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.796648  7489 raft_consensus.cc:1240] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.797972  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.799064  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:49.799221  7310 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.800848  7743 raft_consensus.cc:2749] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.801506  7310 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.801682  7459 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:49.803831  7755 raft_consensus.cc:2749] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.804175  7310 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.805200  7741 raft_consensus.cc:2749] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.805716  7310 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.805971  7741 raft_consensus.cc:2749] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.806865  7740 raft_consensus.cc:2749] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.807144  7310 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.808144  7740 raft_consensus.cc:2749] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:17:49.808770  7741 consensus_queue.cc:1059] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.809595  7740 consensus_queue.cc:1059] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:49.809918  7738 consensus_queue.cc:1059] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.810804  7740 raft_consensus.cc:3055] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:49.811174  7739 raft_consensus.cc:3055] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.810372  7743 consensus_queue.cc:1059] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.811630  7739 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.811646  7738 raft_consensus.cc:3055] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.812230  7738 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.812533  7741 raft_consensus.cc:3055] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.812963  7741 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.812726  7739 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.813124  7738 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.813865  7741 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.814851  7738 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.814368  7739 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.815095  7741 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.811213  7740 raft_consensus.cc:740] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.817317  7740 consensus_queue.cc:260] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:49.818966  7740 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.810055  7310 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.826030  7743 raft_consensus.cc:2749] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.826793  7310 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:49.827716  7743 raft_consensus.cc:2749] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:17:49.853259  7489 raft_consensus.cc:1240] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.854888  7739 consensus_queue.cc:1059] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.856089  7740 raft_consensus.cc:3055] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.856424  7740 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.857273  7740 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.858439  7740 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.860324  7344 raft_consensus.cc:1240] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.862346  7748 consensus_queue.cc:1059] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.863004  7489 raft_consensus.cc:1240] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.863775  7746 raft_consensus.cc:3055] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.864262  7746 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.865226  7746 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.865450  7739 consensus_queue.cc:1059] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.866783  7739 raft_consensus.cc:3055] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.867270  7739 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.868533  7739 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.870110  7739 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.870695  7344 raft_consensus.cc:1240] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.871246  7489 raft_consensus.cc:1240] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.872692  7748 consensus_queue.cc:1059] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.873929  7752 raft_consensus.cc:3055] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.866858  7746 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.874315  7752 raft_consensus.cc:740] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.875447  7752 consensus_queue.cc:260] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.877089  7752 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.873097  7743 consensus_queue.cc:1059] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.883425  7743 raft_consensus.cc:3055] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.883864  7743 raft_consensus.cc:740] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.884850  7743 consensus_queue.cc:260] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.886324  7743 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.896543  7489 raft_consensus.cc:1240] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 9caae85dad6340fab81354bfb82e3c91 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.898195  7743 consensus_queue.cc:1059] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.898672  7344 raft_consensus.cc:1240] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.899387  7739 raft_consensus.cc:3055] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.899760  7739 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:49.900484  7752 consensus_queue.cc:1059] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.900923  7739 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.902005  7746 raft_consensus.cc:3055] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.902383  7739 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.902449  7746 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.903985  7746 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.905654  7746 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.913178  7344 raft_consensus.cc:1240] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.915159  7748 consensus_queue.cc:1059] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:49.916437  7748 raft_consensus.cc:3055] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.916864  7748 raft_consensus.cc:740] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.917155  7344 raft_consensus.cc:1240] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.917866  7748 consensus_queue.cc:260] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.918725  7752 consensus_queue.cc:1059] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.919529  7748 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.919795  7746 raft_consensus.cc:3055] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.920288  7746 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.921211  7746 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.922479  7746 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:49.941745  7344 raft_consensus.cc:1240] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 1. Current term is 2. Ops: []
I20260709 13:17:49.943399  7746 consensus_queue.cc:1059] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:17:49.944481  7748 raft_consensus.cc:3055] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:17:49.944850  7748 raft_consensus.cc:740] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.945786  7748 consensus_queue.cc:260] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.947309  7748 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:17:49.956386  7344 raft_consensus.cc:1240] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:49.958102  7748 consensus_queue.cc:1059] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:49.959125  7746 raft_consensus.cc:3055] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:49.959468  7746 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:49.960291  7746 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:49.961465  7746 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:50.151394  7489 raft_consensus.cc:1240] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3. Ops: []
I20260709 13:17:50.152989  7746 consensus_queue.cc:1059] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:50.154032  7748 raft_consensus.cc:3055] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:17:50.154361  7748 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:50.155093  7748 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:50.156236  7748 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:17:50.321020  7517 debug-util.cc:398] Leaking SignalData structure 0x7b08001c1f20 after lost signal to thread 21529
W20260709 13:17:50.322479  7517 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8260 after lost signal to thread 7279
W20260709 13:17:50.324092  7517 debug-util.cc:398] Leaking SignalData structure 0x7b0800387140 after lost signal to thread 7371
W20260709 13:17:50.325503  7517 debug-util.cc:398] Leaking SignalData structure 0x7b080014de00 after lost signal to thread 7446
W20260709 13:17:50.326819  7517 debug-util.cc:398] Leaking SignalData structure 0x7b08002253a0 after lost signal to thread 7520
I20260709 13:17:50.656379  7242 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59670:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:17:50.680969  7242 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59670:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
W20260709 13:17:50.703450  7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.703666  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.707692  7229 catalog_manager.cc:5107] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): alter failed for tablet 26955d59ed3b4c91ae35bff0d6c45808 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]): Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.708801  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.709338  7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.711824  7408 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.712198  7485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45210: Illegal state: replica c5bbd435a7a74dceaa00d2cd66e601f2 is not leader of this config: current role FOLLOWER
I20260709 13:17:50.715068 21526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:17:50.717113  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.720844  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.721772  7408 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.736235  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.736326  7408 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.742238  7485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45210: Illegal state: replica c5bbd435a7a74dceaa00d2cd66e601f2 is not leader of this config: current role FOLLOWER
I20260709 13:17:50.744726  7344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:36412
I20260709 13:17:50.745453  7344 raft_consensus.cc:606] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:50.745947  7344 raft_consensus.cc:614] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:17:50.749387  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.758926  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.762643  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.763566  7408 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.767936  7410 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.767009  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:50.770179  7415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:50.770915  7415 raft_consensus.cc:606] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:50.771283  7415 raft_consensus.cc:614] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:50.772652 21526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:17:50.773311 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:17:50.781144  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:50.781378  7415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:50.782258  7415 raft_consensus.cc:606] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:50.782838  7415 raft_consensus.cc:614] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:50.785523  7344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:36412
I20260709 13:17:50.786115  7344 raft_consensus.cc:606] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:50.786391  7344 raft_consensus.cc:614] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:50.787608 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:50.788264 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:17:50.797060  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.809512  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.817862  7485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45210: Illegal state: replica c5bbd435a7a74dceaa00d2cd66e601f2 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.818924  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.823030  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.825532  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.829952  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.837303  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.877358  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.882932  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.890995  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.897063  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.902096  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.913805  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.913805  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.930965  7485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45210: Illegal state: replica c5bbd435a7a74dceaa00d2cd66e601f2 is not leader of this config: current role FOLLOWER
W20260709 13:17:50.951736  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:50.982797  7763 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:50.983527  7763 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:50.985803  7763 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:50.987037  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:50.986816  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:50.987818  7489 raft_consensus.cc:2468] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:50.989122  7310 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:50.989918  7763 raft_consensus.cc:2804] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:50.990348  7763 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:50.990692  7763 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:50.997776  7763 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:51.000602  7763 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Requested vote from peers c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.001253  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.001224  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.002380  7489 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.012710  7489 raft_consensus.cc:2468] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 3.
I20260709 13:17:51.014369  7310 leader_election.cc:304] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:51.015347  7763 raft_consensus.cc:2804] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.016973  7763 raft_consensus.cc:697] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 3 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:51.017931  7763 consensus_queue.cc:237] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
W20260709 13:17:51.031720  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.031827  7242 catalog_manager.cc:5697] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:51.042028  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.051164  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.067447  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.071753  7485 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45210: Illegal state: replica c5bbd435a7a74dceaa00d2cd66e601f2 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.073804  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.076361  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:51.082507  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.100492  7755 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.101306  7755 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.103888  7755 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.104679  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.105244  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.105664  7344 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.106040  7415 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.107221  7459 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.108139  7755 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.108557  7755 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.108883  7755 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.121929  7766 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.122295  7755 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.122867  7766 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.124800  7755 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.125509  7766 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.126261  7766 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.129079  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.129616  7766 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.128854  7767 raft_consensus.cc:493] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.129276  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.131120  7489 raft_consensus.cc:2393] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.130787  7767 raft_consensus.cc:515] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } }
I20260709 13:17:51.126323  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.133086  7415 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:51.133641  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.133812  7767 leader_election.cc:290] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.136034  7766 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.136633  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.130031  7342 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.137444  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbd27671e688455da30507409a3aa997" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.127180  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.138255  7489 raft_consensus.cc:2368] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3.
I20260709 13:17:51.138338  7344 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.139394  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.139757  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.140242  7493 raft_consensus.cc:2393] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.140424  7346 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.141970  7385 leader_election.cc:400] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) with higher term. Message: Invalid argument: T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3.
I20260709 13:17:51.142280  7385 leader_election.cc:403] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:17:51.143872  7385 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.145218  7767 raft_consensus.cc:3060] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.148299  7415 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:51.149250  7385 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.150170  7462 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.151597  7766 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.152281  7766 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.151726  7768 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.153473  7766 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.153541  7768 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.154198  7768 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.150259  7344 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:51.152865  7755 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:51.157047  7767 raft_consensus.cc:2749] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) with higher term. Message: Invalid argument: T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 2. Current term is 3.
I20260709 13:17:51.166365  7767 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.167158  7767 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.168610  7755 raft_consensus.cc:697] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:51.167369  7766 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.169869  7755 consensus_queue.cc:237] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.172832  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.172858  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.173980  7346 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.175158  7767 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.175797  7767 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.175841  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.173882  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.176460  7767 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.177275  7493 raft_consensus.cc:2393] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.165969  7768 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.179558  7770 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.180159  7770 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.180603  7770 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.180749  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.181572  7346 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.181888  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.185794  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.186883  7493 raft_consensus.cc:2393] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.188856  7385 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.192305  7766 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.193863  7766 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.195079  7767 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.195526  7344 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.196810  7768 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.195152  7766 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.198468  7766 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.201041  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.202154  7493 raft_consensus.cc:2393] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.190212  7770 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.204631  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.204892  7770 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.205647  7493 raft_consensus.cc:2393] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.207291  7385 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.208364  7770 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.208864  7770 raft_consensus.cc:697] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.209774  7770 consensus_queue.cc:237] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.210740  7772 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.210602  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.211367  7772 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.211879  7493 raft_consensus.cc:2393] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.213654  7772 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.197976  7342 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.215723  7766 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.217511  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.220764  7342 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.214389  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.218678  7766 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.219434  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.225365  7493 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:51.219771  7385 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.227073  7766 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.228927  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.230568  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.231551  7489 raft_consensus.cc:2393] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.230145  7242 catalog_manager.cc:5697] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.232605  7242 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 82995eea6405421d941a76c887eb7bd1 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.230208  7766 raft_consensus.cc:697] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.233810  7766 consensus_queue.cc:237] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.239379  7342 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.240840  7772 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.241575  7772 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.244079  7772 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.230067  7385 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.247509  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.248376  7342 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.248649  7770 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.249161  7770 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.249511  7770 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.251852  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.251446  7243 catalog_manager.cc:5697] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.252892  7489 raft_consensus.cc:2393] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.253201  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 26955d59ed3b4c91ae35bff0d6c45808 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.256059  7385 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.258747  7770 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.260416  7772 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.261091  7772 raft_consensus.cc:697] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.261124  7770 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.262480  7768 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.262188  7772 consensus_queue.cc:237] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.263060  7768 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.265185  7768 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.266161  7385 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.269322  7768 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.269789  7768 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.270941  7768 raft_consensus.cc:3060] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.277469  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.278424  7346 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.277664  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.279824  7342 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.281036  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.282045  7493 raft_consensus.cc:2393] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.283100  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.283949  7489 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.285454  7768 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.287637  7755 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:51.288004  7768 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.288300  7755 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.289001  7769 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:51.289788  7769 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.288982  7242 catalog_manager.cc:5697] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.291872  7242 catalog_manager.cc:5697] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.293269  7755 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.293478  7242 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.294651  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.295506  7415 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.296095  7344 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.297621  7462 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.297977  7769 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.298883  7769 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.299327  7769 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:51.299789  7769 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.300575  7768 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.301226  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.302214  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.303422  7768 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.303694  7346 raft_consensus.cc:2393] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 3: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
W20260709 13:17:51.308287  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.304061  7415 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.312448  7342 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:51.313247  7462 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.314981  7755 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.315722  7755 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.316409  7385 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.316673  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.317575  7493 raft_consensus.cc:2393] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.319046  7768 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.321889  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.322772  7766 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.323279  7766 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.323508  7776 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.323823  7766 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.324062  7776 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:51.324548  7776 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.324954  7755 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.327066  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
W20260709 13:17:51.327701  7410 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.327942  7415 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.329461  7462 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
W20260709 13:17:51.322438  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.331157  7755 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.331621  7755 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.332012  7755 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.315322  7489 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:51.322482  7769 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.333931  7766 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.303151  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.336871  7344 raft_consensus.cc:2393] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 3: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:51.337455  7766 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.335340  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.338196  7768 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.338892  7768 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.339421  7766 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.339202  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.340526  7769 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.340919  7766 raft_consensus.cc:697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.341809  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.341950  7766 consensus_queue.cc:237] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.343189  7768 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.345503  7776 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.348106  7776 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.339210  7385 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.351521  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.337559  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.352509  7493 raft_consensus.cc:2393] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.352857  7346 raft_consensus.cc:2393] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 3: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:51.354138  7339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.355299  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.356160  7493 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:51.356184  7768 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:51.357872  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.358903  7489 raft_consensus.cc:2393] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.358839  7768 raft_consensus.cc:697] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.360453  7776 raft_consensus.cc:493] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.361104  7776 raft_consensus.cc:515] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.363487  7776 leader_election.cc:290] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.361514  7768 consensus_queue.cc:237] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.369446  7415 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.372792  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.375052  7755 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.369661  7339 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
W20260709 13:17:51.370018  7334 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44700: Illegal state: replica 9caae85dad6340fab81354bfb82e3c91 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.362545  7385 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
W20260709 13:17:51.379436  7410 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.380286  7772 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.371381  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.380735  7772 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.381163  7347 raft_consensus.cc:2393] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c5bbd435a7a74dceaa00d2cd66e601f2 in current term 3: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:51.381011  7342 raft_consensus.cc:2468] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.381175  7772 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.382851  7755 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.386044  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.387730  7385 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.389196  7385 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.390473  7770 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.391041  7770 raft_consensus.cc:697] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.391255  7767 raft_consensus.cc:2804] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.392025  7770 consensus_queue.cc:237] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.394595  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48ea9a7ba2e44f9b9bddce661a5a061e" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.397024  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.398748  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.399950  7346 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.402446  7418 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.410115  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.412499  7344 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.414477  7385 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.416400  7781 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.416985  7781 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9caae85dad6340fab81354bfb82e3c91)
I20260709 13:17:51.417441  7781 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.419792  7462 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:51.408814  7767 raft_consensus.cc:697] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.421648  7755 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.446868  7768 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.430398  7772 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.447580  7768 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.434201  7755 raft_consensus.cc:697] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
W20260709 13:17:51.449011  7410 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.435216  7767 consensus_queue.cc:237] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.451367  7415 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.437238  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.463377  7462 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:51.474887  7755 consensus_queue.cc:237] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.481292  7768 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.479564  7242 catalog_manager.cc:5697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.468463  7776 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:51.500279  7489 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:51.502796  7346 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:51.491117  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.504637  7344 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.489029  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.506275  7493 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:51.506737  7385 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.507205  7776 raft_consensus.cc:697] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:51.484884  7768 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.513585  7766 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:17:51.514374  7776 consensus_queue.cc:237] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.508190  7769 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.523471  7769 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.512857  7770 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.526191  7415 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:51.530932  7770 raft_consensus.cc:493] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.531584  7770 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.539119  7781 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.537231  7769 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.548856  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.550311  7772 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.551640  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.552424  7496 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.553650  7418 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:51.515571  7768 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.555197  7784 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:17:51.548772  7344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.562475  7346 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 13:17:51.566339  7459 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 58357.6us
I20260709 13:17:51.561837  7781 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.570741  7344 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.567405  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:51.588409  7768 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.589641  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.576658  7766 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 13:17:51.581405  7342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.591017  7488 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.590482  7346 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.592114  7488 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 1.
I20260709 13:17:51.594208  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.594962  7346 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:51.596685  7385 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.597674  7766 raft_consensus.cc:2804] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:17:51.598062  7766 raft_consensus.cc:493] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.603484  7459 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:51.601004  7242 catalog_manager.cc:5697] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.607411  7242 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 50cddde895ba41b7aac3346cbdb5ae6c (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.608595  7242 catalog_manager.cc:5697] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.598423  7766 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:51.610916  7242 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.613571  7242 catalog_manager.cc:5697] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 3, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:51.616951  7490 raft_consensus.cc:2393] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 3: Already voted for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in this term.
I20260709 13:17:51.618961  7462 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:51.616992  7789 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 3e9e84d2e507455aa80bc7595a1ef075)
I20260709 13:17:51.619693  7243 catalog_manager.cc:5697] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 3, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.620805  7496 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:51.622367  7385 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.622849  7769 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.620836  7789 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.623318  7769 raft_consensus.cc:493] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.623414  7768 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:51.623845  7769 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.624150  7462 leader_election.cc:304] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:51.624053  7768 raft_consensus.cc:697] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:51.625499  7242 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 48ea9a7ba2e44f9b9bddce661a5a061e (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.605000  7755 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.627465  7791 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 13:17:51.633313  7418 raft_consensus.cc:1275] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:51.635017  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.634896  7768 consensus_queue.cc:237] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.635910  7787 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.655037  7794 consensus_queue.cc:1048] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:51.639528  7496 raft_consensus.cc:1275] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:51.647662  7422 raft_consensus.cc:2368] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 for earlier term 3. Current term is 4.
I20260709 13:17:51.649346  7791 raft_consensus.cc:2749] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:17:51.650358  7244 catalog_manager.cc:5697] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 3, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:51.637128  7793 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.629364  7776 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:51.674161  7776 raft_consensus.cc:697] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:51.675359  7776 consensus_queue.cc:237] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.667982  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 26955d59ed3b4c91ae35bff0d6c45808 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:51.677489  7769 raft_consensus.cc:515] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.645910  7755 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.681928  7347 raft_consensus.cc:1275] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:51.653760  7787 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.687229  7794 consensus_queue.cc:1048] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:51.682832  7793 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.695109  7489 raft_consensus.cc:1240] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:51.690935  7769 leader_election.cc:290] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.700748  7789 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.710353  7793 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.677717  7783 mvcc.cc:204] Tried to move back new op lower bound from 7305638180691521536 to 7305638179717259264. Current Snapshot: MvccSnapshot[applied={T|T < 7305638180691521536}]
I20260709 13:17:51.716096  7787 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.721815  7801 tablet.cc:1724] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:51.786870  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.793037  7797 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.725342  7800 tablet.cc:1724] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:51.794962  7797 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.796819  7772 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:51.800773  7797 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.794044  7767 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:17:51.766232  7448 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:26955d59ed3b4c91ae35bff0d6c45808)
W20260709 13:17:51.778479  7757 log.cc:927] Time spent T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:17:51.691581  7462 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34207, user_credentials={real_user=slave}} blocked reactor thread for 67533.6us
I20260709 13:17:51.811249  7313 leader_election.cc:400] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) with higher term. Message: Invalid argument: T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 for earlier term 3. Current term is 4.
I20260709 13:17:51.846869  7313 leader_election.cc:403] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:17:51.848002  7313 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.130:34207, user_credentials={real_user=slave}} blocked reactor thread for 70022.6us
I20260709 13:17:51.850476  7796 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.811306  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.812944  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:51.856650  7347 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.860056  7345 raft_consensus.cc:1275] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:51.815124  7755 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.820772  7342 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:51.821977  7422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.820915  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.825271  7767 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.761446  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:51.881698  7340 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.884045  7767 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.885592  7422 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 3.
I20260709 13:17:51.831200  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:51.831373  7343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
W20260709 13:17:51.836047  7790 log.cc:927] Time spent T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 13:17:51.893630  7343 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.827219  7787 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.895401  7787 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.874156  7421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32c77064066849229154ee08fe743028" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:51.876248  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.879182  7418 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 3.
W20260709 13:17:51.904301  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.891332  7755 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:51.880189  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.906881  7755 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.878968  7459 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.908718  7776 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.909176  7776 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.909596  7776 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.912492  7459 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.914386  7490 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 2.
I20260709 13:17:51.917723  7313 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.923159  7791 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.923429  7755 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:51.923867  7791 raft_consensus.cc:493] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.924335  7791 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.929203  7776 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:51.932653  7420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.937158  7420 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 3.
I20260709 13:17:51.937260  7787 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:51.932540  7343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
W20260709 13:17:51.907161  7232 catalog_manager.cc:5107] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): alter failed for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]): Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:51.938846  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:51.938953  7343 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.939888  7417 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:51.941160  7459 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:51.946797  7797 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.956884  7787 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.959672  7787 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.979748  7787 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.979271  7800 tablet.cc:1724] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:51.982790  7313 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:51.984362  7805 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:17:51.961761  7448 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:26955d59ed3b4c91ae35bff0d6c45808)
I20260709 13:17:51.965716  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
W20260709 13:17:51.978801  7385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 161326us
I20260709 13:17:51.984949  7313 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:51.993466  7789 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:51.994001  7789 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.994407  7789 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:51.995975  7340 raft_consensus.cc:2468] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:51.984861  7805 raft_consensus.cc:493] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:51.999063  7805 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:17:51.999765  7410 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
W20260709 13:17:52.001454  7522 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:26955d59ed3b4c91ae35bff0d6c45808)
I20260709 13:17:52.003882  7459 leader_election.cc:304] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:52.005254  7755 raft_consensus.cc:2804] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.005936  7755 raft_consensus.cc:697] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:51.952860  7797 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:52.012012  7412 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Illegal state: replica 3e9e84d2e507455aa80bc7595a1ef075 is not leader of this config: current role FOLLOWER
I20260709 13:17:52.013388  7796 raft_consensus.cc:2749] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) with higher term. Message: Invalid argument: T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 for earlier term 3. Current term is 4.
I20260709 13:17:52.014247  7769 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:52.015092  7769 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.019690  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:52.023823  7340 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 3.
I20260709 13:17:52.025816  7459 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:52.027304  7755 consensus_queue.cc:237] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.027899  7776 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.028484  7784 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:52.028927  7784 raft_consensus.cc:493] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.009192  7797 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.030639  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.032871  7791 raft_consensus.cc:515] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.033608  7340 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.036167  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.029357  7784 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.033641  7242 catalog_manager.cc:5697] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.026384  7766 raft_consensus.cc:515] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.041018  7242 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.043146  7766 leader_election.cc:290] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.044379  7791 leader_election.cc:290] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.046922  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.039513  7343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc2688d492204007bb197167fa954266" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.051772  7343 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.052109  7490 raft_consensus.cc:1240] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.055353  7489 raft_consensus.cc:1275] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:52.056614  7769 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.057451  7490 raft_consensus.cc:1240] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.057801  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:52.057890  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075" is_pre_election: true
I20260709 13:17:52.055353  7346 raft_consensus.cc:1275] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:17:52.060582  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:52.061303  7344 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:52.061352  7490 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 2.
I20260709 13:17:52.062184  7770 raft_consensus.cc:515] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.063073  7489 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:52.064853  7346 raft_consensus.cc:1275] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:52.067188  7347 raft_consensus.cc:1240] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: [3.3-3.4]
I20260709 13:17:52.068133  7341 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:52.074177  7770 leader_election.cc:290] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.076651  7787 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.078107  7808 consensus_queue.cc:1059] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.078928  7806 consensus_queue.cc:1059] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.080389  7767 consensus_queue.cc:1059] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.082556  7772 raft_consensus.cc:3055] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.081465  7385 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:52.083133  7767 raft_consensus.cc:3055] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.083542  7767 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.083137  7772 raft_consensus.cc:740] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.085191  7767 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.085912  7415 raft_consensus.cc:1275] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:17:52.086844  7767 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.088025  7784 raft_consensus.cc:515] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.088809  7768 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.086278  7772 consensus_queue.cc:260] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.090562  7385 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:52.091795  7781 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:52.092474  7781 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.089612  7766 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:52.093591  7766 raft_consensus.cc:493] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:52.093688  7781 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.093962  7805 raft_consensus.cc:515] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.096076  7345 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 4: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
W20260709 13:17:52.094532  7806 consensus_queue.cc:1175] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9caae85dad6340fab81354bfb82e3c91" responder_term: 3 status { last_received { term: 2 index: 2 } last_committed_idx: 2 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:17:52.079030  7770 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 13:17:52.099634  7797 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.102937 21526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:52.103391  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.106418  7345 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.112104  7811 tablet.cc:1724] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.117698  7772 raft_consensus.cc:3060] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.118242  7806 raft_consensus.cc:3055] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.118753  7806 raft_consensus.cc:740] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.119313  7244 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 26955d59ed3b4c91ae35bff0d6c45808 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.119942  7806 consensus_queue.cc:260] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.121609  7806 raft_consensus.cc:3060] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.095418  7766 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.125660  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.126379  7415 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.114213  7343 raft_consensus.cc:2468] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.137488  7243 catalog_manager.cc:5697] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
W20260709 13:17:52.145995  7385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 53640.5us
I20260709 13:17:52.148098  7786 consensus_queue.cc:1048] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.149298  7339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.149796  7781 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.149963  7786 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.151021  7459 leader_election.cc:304] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:52.154043  7347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.154794  7347 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:17:52.150117  7339 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.155791  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e44335ea170481ab59db4cca4c7c180" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.145941  7805 leader_election.cc:290] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.151746  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 82995eea6405421d941a76c887eb7bd1 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.152370  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.144647  7797 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.162724  7417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.163657  7787 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.165580  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.165575  7341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.167071  7341 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.166393  7489 raft_consensus.cc:3060] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.124869  7789 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.176604  7347 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:52.145035  7755 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:17:52.188230  7784 leader_election.cc:290] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.191195  7345 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.195690  7342 raft_consensus.cc:1240] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Rejecting Update request from peer c5bbd435a7a74dceaa00d2cd66e601f2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.196108  7459 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:52.194864  7811 tablet.cc:1724] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.188585  7415 raft_consensus.cc:2468] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 4.
I20260709 13:17:52.205494  7422 raft_consensus.cc:1240] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Rejecting Update request from peer c5bbd435a7a74dceaa00d2cd66e601f2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.207726  7421 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:52.208644  7421 raft_consensus.cc:606] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:52.208863  7416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.210285  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.189067  7755 raft_consensus.cc:2804] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.203418  7385 leader_election.cc:304] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:52.225476  7490 raft_consensus.cc:1240] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.227630  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a534d5e35ea341c591a61e94a851a293" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.228648  7496 raft_consensus.cc:3060] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:17:52.218259  7784 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.243805  7814 raft_consensus.cc:2804] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:17:52.244562  7814 raft_consensus.cc:697] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.245728  7814 consensus_queue.cc:237] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.252763  7418 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:52.253716  7418 raft_consensus.cc:606] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:52.258355  7789 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.267668  7496 raft_consensus.cc:2468] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 3.
I20260709 13:17:52.279407  7496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.286708  7784 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.288218  7784 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.220221  7340 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.294405  7310 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91; no voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:52.295105  7342 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.296149  7805 raft_consensus.cc:2749] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:52.263200  7755 raft_consensus.cc:697] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.298185  7313 leader_election.cc:304] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:52.259800  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d166d5008c8c4d098bb1973379289560" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.301364  7346 raft_consensus.cc:1240] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.308692  7459 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:52.306730  7415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:52.316277  7816 consensus_queue.cc:1059] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 13:17:52.317625  7488 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.307101  7805 raft_consensus.cc:2804] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.309844  7490 raft_consensus.cc:3060] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.306893  7806 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 13:17:52.329334  7805 raft_consensus.cc:697] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:52.329545  7755 consensus_queue.cc:237] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.330899  7805 consensus_queue.cc:237] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.351029  7821 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.350998  7819 consensus_queue.cc:1059] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:17:52.339727  7339 raft_consensus.cc:2468] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.355945  7343 raft_consensus.cc:1240] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: [3.3-3.4]
I20260709 13:17:52.357934  7416 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.356103  7459 leader_election.cc:304] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:52.359220  7801 tablet.cc:1724] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.345144  7772 consensus_queue.cc:1059] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:17:52.366179  7772 raft_consensus.cc:3055] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.372841  7772 raft_consensus.cc:740] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.374538  7772 consensus_queue.cc:260] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.360849  7821 raft_consensus.cc:697] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.365818  7808 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.394598  7766 raft_consensus.cc:515] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.396343  7417 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9caae85dad6340fab81354bfb82e3c91 in current term 4: Already voted for candidate 3e9e84d2e507455aa80bc7595a1ef075 in this term.
W20260709 13:17:52.408826  7459 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.129:41907, user_credentials={real_user=slave}} blocked reactor thread for 52858.3us
I20260709 13:17:52.411409  7819 raft_consensus.cc:2804] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.411444  7816 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:17:52.426542  7819 raft_consensus.cc:697] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.427495  7767 raft_consensus.cc:993] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: : Instructing follower c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:52.407563  7821 consensus_queue.cc:237] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.395750  7341 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:52.428171  7767 raft_consensus.cc:1081] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Signalling peer c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
W20260709 13:17:52.410970  7462 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.130:34207, user_credentials={real_user=slave}} blocked reactor thread for 51003.1us
I20260709 13:17:52.430471  7496 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:45224
I20260709 13:17:52.431164  7496 raft_consensus.cc:493] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:52.431591  7496 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.436031  7776 raft_consensus.cc:3055] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.435053  7800 tablet.cc:1724] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.436573  7776 raft_consensus.cc:740] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.437718  7776 consensus_queue.cc:260] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.438534  7827 raft_consensus.cc:3055] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.439013  7827 raft_consensus.cc:740] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.439559  7385 leader_election.cc:304] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:52.440109  7827 consensus_queue.cc:260] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.441639  7827 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.443895  7781 raft_consensus.cc:2804] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.444574  7781 raft_consensus.cc:697] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.403975  7772 raft_consensus.cc:3060] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.407109  7489 raft_consensus.cc:2468] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 4.
I20260709 13:17:52.411966  7415 raft_consensus.cc:606] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:52.459062  7341 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.459610  7781 consensus_queue.cc:237] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.474764  7415 raft_consensus.cc:614] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:52.476184  7496 raft_consensus.cc:515] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.489535  7496 leader_election.cc:290] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:52.502912  7495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.504055  7495 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 4: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:52.504990  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.505820  7346 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.458820  7766 leader_election.cc:290] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.454517  7343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9539b8b5f9674ce99dbea364d81f55c7" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:52.491698  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26955d59ed3b4c91ae35bff0d6c45808" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.529325  7776 raft_consensus.cc:3060] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.529558  7824 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 13:17:52.532160  7490 raft_consensus.cc:2468] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:52.529385  7340 raft_consensus.cc:1275] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.532698  7244 catalog_manager.cc:5697] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 3 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.534540  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.507476  7310 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
W20260709 13:17:52.543470  7310 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.5.131:41951, user_credentials={real_user=slave}} blocked reactor thread for 63338.1us
I20260709 13:17:52.537017  7341 raft_consensus.cc:1240] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 LEADER]: Rejecting Update request from peer c5bbd435a7a74dceaa00d2cd66e601f2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.546319  7343 raft_consensus.cc:2393] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 4: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:52.551007  7342 raft_consensus.cc:1240] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.553105  7805 raft_consensus.cc:2804] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.555575  7385 leader_election.cc:304] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075; no voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:52.564054  7346 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.554901  7805 raft_consensus.cc:697] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:52.570236  7805 consensus_queue.cc:237] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.571095  7806 raft_consensus.cc:2749] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:17:52.454089  7819 consensus_queue.cc:237] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.491254  7833 tablet.cc:1724] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.584422  7459 leader_election.cc:304] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:52.565115  7418 raft_consensus.cc:3055] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.591024  7824 raft_consensus.cc:2804] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:52.591770  7824 raft_consensus.cc:697] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:52.566112  7797 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:17:52.581429  7768 consensus_queue.cc:1059] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 13:17:52.590181  7418 raft_consensus.cc:740] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.615427  7811 tablet.cc:1724] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.592888  7824 consensus_queue.cc:237] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.669384  7786 raft_consensus.cc:993] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:52.670212  7786 raft_consensus.cc:1081] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:52.672936  7346 raft_consensus.cc:1275] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:52.681334  7781 raft_consensus.cc:3055] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:17:52.682032  7781 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.700170  7415 raft_consensus.cc:1275] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:52.700794  7415 pending_rounds.cc:85] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting all ops after (but not including) 2
I20260709 13:17:52.701089  7415 pending_rounds.cc:107] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:52.701498  7415 raft_consensus.cc:2889] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.702067  7415 pending_rounds.cc:107] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted ALTER_SCHEMA_OP operation due to leader change: 3.4
I20260709 13:17:52.702123  7418 consensus_queue.cc:260] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.700322  7808 consensus_queue.cc:1048] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:17:52.704807  7346 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44704
I20260709 13:17:52.686131  7781 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.708629  7781 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.712620  7420 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b261c08a7c8a43d2a6fd239e5f8565be"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:57264
W20260709 13:17:52.712775  7415 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Aborted: Op aborted by new leader
I20260709 13:17:52.713505  7420 raft_consensus.cc:606] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:52.713944  7420 raft_consensus.cc:614] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:17:52.714311  7415 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:49280 (request call id 42) took 1274 ms. Trace:
I20260709 13:17:52.715266 21526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 4
I20260709 13:17:52.716315 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:52.720472  7490 raft_consensus.cc:1240] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Rejecting Update request from peer 3e9e84d2e507455aa80bc7595a1ef075 for earlier term 3. Current term is 4. Ops: []
I20260709 13:17:52.704162  7418 raft_consensus.cc:3060] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:52.727517  7420 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:52.728901  7800 tablet.cc:1724] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.736935  7847 tablet.cc:1724] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.738360  7346 raft_consensus.cc:493] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:52.730005  7420 raft_consensus.cc:606] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:52.742719  7346 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.745070  7493 raft_consensus.cc:1275] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:17:52.747570 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:52.748049  7786 consensus_queue.cc:1048] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:17:52.750438  7768 raft_consensus.cc:993] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:52.751202  7766 raft_consensus.cc:1081] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:52.753696  7338 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44704
I20260709 13:17:52.754317  7338 raft_consensus.cc:493] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:52.754756  7338 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.748471 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:52.754887  7346 raft_consensus.cc:515] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.764681  7420 raft_consensus.cc:1275] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.766785  7789 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.715214  7415 rpcz_store.cc:276] 0709 13:17:51.439843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.441601 (+  1758us) service_pool.cc:224] Handling call
0709 13:17:51.445353 (+  3752us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b580066ea98. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f1803c78db1 00007f1803db8b71 00007f1803db8eee 0000000000375939 00007f180be2c1e0 00007f180bef50c4 00007f180bef5131 00007f180beb2ae0 00007f180be9b6a9 00007f180be9ace4 00007f18053275f4
0709 13:17:52.714082 (+1268729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:51.530911 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 13:17:51.653410 (+122499us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 13:17:51.653476 (+    66us) alter_schema_op.cc:66] Acquired schema lock
0709 13:17:51.653557 (+    81us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 13:17:51.653740 (+   183us) alter_schema_op.cc:128] START. Timestamp: 1783603071653646
0709 13:17:51.653778 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 13:17:51.654872 (+  1094us) log.cc:844] Serialized 204 byte log entry
0709 13:17:52.712525 (+1057653us) op_driver.cc:464] REPLICATION: finished
0709 13:17:52.712619 (+    94us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0709 13:17:52.712653 (+    34us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"spinlock_wait_cycles":4408448,"child_traces":[["op",{"mutex_wait_us":4009,"prepare.queue_time_us":71277,"prepare.run_cpu_time_us":3631,"prepare.run_wall_time_us":128468,"raft.queue_time_us":1449,"raft.run_cpu_time_us":1677,"raft.run_wall_time_us":1676,"replication_time_us":1048604,"spinlock_wait_cycles":93056,"thread_start_us":37838,"threads_started":1}]]}
I20260709 13:17:52.784974  7244 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 50cddde895ba41b7aac3346cbdb5ae6c (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.785717  7244 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 48ea9a7ba2e44f9b9bddce661a5a061e (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.790587  7345 raft_consensus.cc:1275] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.791021  7801 tablet.cc:1724] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:52.793169  7827 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.814483  7824 consensus_queue.cc:1048] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.832065  7339 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.846809  7827 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.849031  7346 leader_election.cc:290] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.853322  7416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.856216  7416 raft_consensus.cc:3055] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:52.856637  7416 raft_consensus.cc:740] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.856700  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "50cddde895ba41b7aac3346cbdb5ae6c" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.857532  7489 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.857695  7416 consensus_queue.cc:260] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.859414  7416 raft_consensus.cc:3060] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.867262  7418 raft_consensus.cc:2468] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 4.
I20260709 13:17:52.868856  7416 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:52.870401  7313 leader_election.cc:304] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:52.871464  7805 raft_consensus.cc:2804] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:52.871804  7415 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:52.871999  7805 raft_consensus.cc:697] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:52.872363  7415 pending_rounds.cc:85] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Aborting all ops after (but not including) 2
I20260709 13:17:52.872534  7493 raft_consensus.cc:1275] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:52.873102  7493 pending_rounds.cc:85] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Aborting all ops after (but not including) 2
I20260709 13:17:52.872665  7415 pending_rounds.cc:107] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:52.873373  7493 pending_rounds.cc:107] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:52.873781  7493 raft_consensus.cc:2889] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.875705  7338 raft_consensus.cc:515] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.876767  7415 raft_consensus.cc:2889] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.873234  7805 consensus_queue.cc:237] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.884280  7824 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.884694  7244 catalog_manager.cc:5697] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 3 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.886669  7415 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:52.896739  7415 raft_consensus.cc:3055] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:52.890509  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccee0354eaf24cd2937732a1b12ead42" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:52.897245  7415 raft_consensus.cc:740] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:52.897675  7490 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.898600  7415 consensus_queue.cc:260] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.900148  7415 raft_consensus.cc:3060] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:52.901087  7489 raft_consensus.cc:2468] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:52.892570  7338 leader_election.cc:290] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:52.884130  7801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:49280 (request call id 44) took 1024 ms. Trace:
I20260709 13:17:52.904500  7346 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.914505  7801 rpcz_store.cc:276] 0709 13:17:51.860062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:51.860408 (+   346us) service_pool.cc:224] Handling call
0709 13:17:52.884043 (+1023635us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:51.862825 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 13:17:51.863558 (+   733us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 13:17:51.863623 (+    65us) alter_schema_op.cc:66] Acquired schema lock
0709 13:17:51.863707 (+    84us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 13:17:51.863895 (+   188us) alter_schema_op.cc:128] START. Timestamp: 1783603071863811
0709 13:17:51.863929 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:17:51.864953 (+  1024us) log.cc:844] Serialized 204 byte log entry
0709 13:17:52.651223 (+786270us) op_driver.cc:464] REPLICATION: finished
0709 13:17:52.790607 (+139384us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b54009cc8a8. stack: 00007f1803dbe343 00007f1803dbe240 00007f180374d1d4 00007f180d48b304 00007f180d48b299 00007f180d2999e1 00007f180d29864c 00007f180a914845 00007f180a918d61 00007f180a918d19 00007f180a918ca9 00007f180a918c71 00007f180a917e9d 000000000038f354 000000000038f199 00007f1803df0475
0709 13:17:52.790808 (+   201us) alter_schema_op.cc:133] APPLY ALTER-SCHEMA: Starting
0709 13:17:52.793084 (+  2276us) log.cc:844] Serialized 5 byte log entry
0709 13:17:52.841590 (+ 48506us) tablet_metadata.cc:665] Metadata flushed
0709 13:17:52.882951 (+ 41361us) log.cc:844] Serialized 14 byte log entry
0709 13:17:52.883823 (+   872us) alter_schema_op.cc:73] Released schema lock
0709 13:17:52.883866 (+    43us) alter_schema_op.cc:186] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":24842,"mutex_wait_us":10452,"prepare.queue_time_us":493,"prepare.run_cpu_time_us":3128,"prepare.run_wall_time_us":13544,"raft.queue_time_us":13010,"raft.run_cpu_time_us":2195,"raft.run_wall_time_us":2192,"replication_time_us":763963,"spinlock_wait_cycles":20040576,"thread_start_us":76725,"threads_started":1}]]}
I20260709 13:17:52.922923  7415 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:52.924611  7418 raft_consensus.cc:1275] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:52.925132  7418 pending_rounds.cc:85] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting all ops after (but not including) 2
I20260709 13:17:52.925398  7418 pending_rounds.cc:107] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:52.925689  7418 raft_consensus.cc:2889] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.925736  7313 leader_election.cc:304] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:52.926784  7789 raft_consensus.cc:2804] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:52.891543  7805 consensus_queue.cc:1048] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.911494  7490 raft_consensus.cc:2468] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:52.927309  7789 raft_consensus.cc:697] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:52.928385  7789 consensus_queue.cc:237] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:52.929064  7784 consensus_queue.cc:1048] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.930645  7784 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.905696  7244 catalog_manager.cc:5697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 4, leader changed from 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.912869  7416 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:52.932734  7416 pending_rounds.cc:85] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting all ops after (but not including) 2
I20260709 13:17:52.933022  7416 pending_rounds.cc:107] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:52.933269  7416 raft_consensus.cc:2889] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:52.934916  7244 catalog_manager.cc:5697] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 3 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.935892  7824 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.937601  7244 catalog_manager.cc:5697] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 3 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.939769  7244 catalog_manager.cc:5697] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 3 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 4 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:52.942226  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.963976  7242 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.965437  7242 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.966933  7242 catalog_manager.cc:5697] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 3 to 4, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 4 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:52.969030  7242 catalog_manager.cc:5697] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 4 to 5, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 5 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:52.971079  7242 catalog_manager.cc:5697] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 2 to 4, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 4 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:52.972800  7242 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet 9539b8b5f9674ce99dbea364d81f55c7 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.973513  7242 catalog_manager.cc:5697] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } }
I20260709 13:17:52.974846  7242 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet a534d5e35ea341c591a61e94a851a293 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.977662  7490 raft_consensus.cc:1275] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.979628  7814 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:17:52.992280  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet a534d5e35ea341c591a61e94a851a293 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:52.994930  7345 raft_consensus.cc:1275] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:52.998039  7824 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:52.999987  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.001729  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.003006  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet a534d5e35ea341c591a61e94a851a293 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.009375  7800 tablet.cc:1724] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.015477  7847 tablet.cc:1724] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.027633  7801 tablet.cc:1724] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.037683  7346 raft_consensus.cc:1275] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:17:53.039479  7786 consensus_queue.cc:1048] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:53.063381  7244 catalog_manager.cc:5697] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } } }
I20260709 13:17:53.062728  7833 tablet.cc:1724] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.075656  7800 tablet.cc:1724] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.083050  7418 raft_consensus.cc:1275] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:17:53.083674  7418 pending_rounds.cc:85] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Aborting all ops after (but not including) 2
I20260709 13:17:53.083957  7418 pending_rounds.cc:107] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:17:53.084242  7418 raft_consensus.cc:2889] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:17:53.084653  7418 pending_rounds.cc:107] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Aborting uncommitted ALTER_SCHEMA_OP operation due to leader change: 3.4
W20260709 13:17:53.085263  7418 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:49280: Aborted: Op aborted by new leader
I20260709 13:17:53.085877  7418 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:49280 (request call id 49) took 1090 ms. Trace:
I20260709 13:17:53.093537  7847 tablet.cc:1724] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.095463  7489 raft_consensus.cc:1275] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:53.086292  7418 rpcz_store.cc:276] 0709 13:17:51.995556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:17:52.001494 (+  5938us) service_pool.cc:224] Handling call
0709 13:17:53.085803 (+1084309us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:17:52.010097 (+     0us) alter_schema_op.cc:104] PREPARE ALTER-SCHEMA: Starting
0709 13:17:52.010896 (+   799us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0709 13:17:52.010930 (+    34us) alter_schema_op.cc:66] Acquired schema lock
0709 13:17:52.011008 (+    78us) alter_schema_op.cc:119] PREPARE ALTER-SCHEMA: finished
0709 13:17:52.011159 (+   151us) alter_schema_op.cc:128] START. Timestamp: 1783603072011076
0709 13:17:52.011184 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:17:52.012090 (+   906us) log.cc:844] Serialized 204 byte log entry
0709 13:17:53.085097 (+1073007us) op_driver.cc:464] REPLICATION: finished
0709 13:17:53.085147 (+    50us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0709 13:17:53.085175 (+    28us) alter_schema_op.cc:171] AlterSchemaCommitCallback: op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":277,"prepare.run_cpu_time_us":2538,"prepare.run_wall_time_us":2537,"raft.queue_time_us":10791,"raft.run_cpu_time_us":976,"raft.run_wall_time_us":975,"replication_time_us":1073726}]]}
W20260709 13:17:53.097539  7232 catalog_manager.cc:5107] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): alter failed for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]): Aborted: Op aborted by new leader
I20260709 13:17:53.098728  7784 consensus_queue.cc:1048] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:53.099449  7805 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:53.110841  7418 raft_consensus.cc:1275] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:53.113075  7805 consensus_queue.cc:1048] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:53.123847  7847 tablet.cc:1724] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.132078  7778 mvcc.cc:204] Tried to move back new op lower bound from 7305638187375992832 to 7305638185264340992. Current Snapshot: MvccSnapshot[applied={T|T < 7305638187375992832}]
I20260709 13:17:53.133796  7833 tablet.cc:1724] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.136570  7800 tablet.cc:1724] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.147130  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.148154  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet a534d5e35ea341c591a61e94a851a293 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.149149  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet 6211b62ab6f94bd7bd8575b3a8673e76 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.164551  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.172958  7242 catalog_manager.cc:5697] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 4 to 5, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 5 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:53.174432  7242 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.200233  7418 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:53.202092  7784 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:53.207405  7244 catalog_manager.cc:5789] TS 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.212445  7243 catalog_manager.cc:5789] TS 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.214883  7345 raft_consensus.cc:1275] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:17:53.217271  7784 consensus_queue.cc:1048] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:53.260214  7244 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet 32c77064066849229154ee08fe743028 (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.360795  7800 tablet.cc:1724] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.371008  7833 tablet.cc:1724] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.375352  7847 tablet.cc:1724] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:53.463011  7489 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:53.464696  7789 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:53.480233  7418 raft_consensus.cc:1275] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:53.484665  7805 consensus_queue.cc:1048] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:53.502452  7242 catalog_manager.cc:5789] TS c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951) does not have the latest schema for tablet b261c08a7c8a43d2a6fd239e5f8565be (table soft_deleted_table [id=97cf88786b1844d3a369214870965fde]). Expected version 1 got 0
I20260709 13:17:53.513339  7489 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:53.515012  7787 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:53.527328  7416 raft_consensus.cc:1275] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:53.529160  7805 consensus_queue.cc:1048] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:53.799449 21526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:53.825664  7489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:46428
I20260709 13:17:53.826457  7489 raft_consensus.cc:606] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Received request to transfer leadership to TS 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:53.829641  7489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:46428
I20260709 13:17:53.830353  7489 raft_consensus.cc:606] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:53.831729 21526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:17:53.832404 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:53.839865  7489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:46428
I20260709 13:17:53.840631  7489 raft_consensus.cc:606] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:53.843973  7489 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
mode: GRACEFUL
new_leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:46428
I20260709 13:17:53.844671  7489 raft_consensus.cc:606] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Received request to transfer leadership to TS 3e9e84d2e507455aa80bc7595a1ef075
I20260709 13:17:53.845793 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:17:53.846643 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:53.881938  7784 raft_consensus.cc:993] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:53.882695  7784 raft_consensus.cc:1081] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:53.885013  7416 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:53.885834  7416 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.886278  7416 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.893078  7784 raft_consensus.cc:993] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:53.893905  7784 raft_consensus.cc:1081] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:53.896121  7418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:53.896782  7418 raft_consensus.cc:493] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.897171  7418 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.898517  7416 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.901379  7416 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:53.903375  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:53.904235  7345 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.905218  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:53.906013  7489 raft_consensus.cc:3055] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:53.906520  7489 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:53.906147  7418 raft_consensus.cc:515] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.907577  7489 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.908950  7489 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.909013  7418 leader_election.cc:290] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:53.909826  7346 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:53.910346  7346 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.916028  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82995eea6405421d941a76c887eb7bd1" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:53.916911  7490 raft_consensus.cc:3055] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:53.917321  7490 raft_consensus.cc:740] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:53.918286  7490 consensus_queue.cc:260] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.919696  7490 raft_consensus.cc:3060] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.922541  7489 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:53.924516  7385 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:53.925554  7786 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:53.926203  7786 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:53.927186  7786 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.929662  7490 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:53.934525  7385 leader_election.cc:304] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:53.935837  7814 raft_consensus.cc:2804] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:53.936614  7814 raft_consensus.cc:697] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:53.938035  7814 consensus_queue.cc:237] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.940168  7346 raft_consensus.cc:2468] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:53.943338  7866 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:53.944864  7866 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.947365  7866 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:53.947046  7242 catalog_manager.cc:5697] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 4 to 5, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:53.949339  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91" is_pre_election: true
I20260709 13:17:53.950361  7340 raft_consensus.cc:2393] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 4: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:53.953073  7493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" is_pre_election: true
I20260709 13:17:53.952239  7345 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:53.953884  7493 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:53.965250  7385 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:53.966689  7866 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:17:53.967180  7866 raft_consensus.cc:493] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c5bbd435a7a74dceaa00d2cd66e601f2)
I20260709 13:17:53.967588  7866 raft_consensus.cc:3060] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:17:53.974694  7784 raft_consensus.cc:993] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:53.975417  7797 raft_consensus.cc:1081] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Signalling peer 9caae85dad6340fab81354bfb82e3c91 to start an election
I20260709 13:17:53.977656  7340 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76"
dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
 from {username='slave'} at 127.0.0.1:44718
I20260709 13:17:53.978420  7340 raft_consensus.cc:493] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:53.978933  7340 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.978075  7866 raft_consensus.cc:515] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.991183  7866 leader_election.cc:290] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:53.994637  7340 raft_consensus.cc:515] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:53.998792  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:53.999001  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6211b62ab6f94bd7bd8575b3a8673e76" candidate_uuid: "9caae85dad6340fab81354bfb82e3c91" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:53.999634  7418 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:53.999699  7490 raft_consensus.cc:3055] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:54.000216  7490 raft_consensus.cc:740] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:54.001307  7490 consensus_queue.cc:260] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.001652  7489 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:54.002689  7490 raft_consensus.cc:3060] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:54.003731  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f3e8fc8428d42a8ab357ca69292f70d" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:54.004742  7345 raft_consensus.cc:2393] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 3e9e84d2e507455aa80bc7595a1ef075 in current term 4: Already voted for candidate 9caae85dad6340fab81354bfb82e3c91 in this term.
I20260709 13:17:54.016462  7489 raft_consensus.cc:2468] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 4.
I20260709 13:17:54.016989  7418 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:54.018396  7385 leader_election.cc:304] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 9caae85dad6340fab81354bfb82e3c91
I20260709 13:17:54.018698  7313 leader_election.cc:304] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:54.014593  7340 leader_election.cc:290] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [CANDIDATE]: Term 5 election: Requested vote from peers 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:54.019630  7786 raft_consensus.cc:2804] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:17:54.019776  7805 raft_consensus.cc:2804] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:54.020344  7786 raft_consensus.cc:697] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:54.020524  7805 raft_consensus.cc:697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Becoming Leader. State: Replica: 9caae85dad6340fab81354bfb82e3c91, State: Running, Role: LEADER
I20260709 13:17:54.021721  7805 consensus_queue.cc:237] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.023445  7490 raft_consensus.cc:2468] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9caae85dad6340fab81354bfb82e3c91 in term 5.
I20260709 13:17:54.021557  7786 consensus_queue.cc:237] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.042472  7243 catalog_manager.cc:5697] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 reported cstate change: term changed from 4 to 5, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129). New cstate: current_term: 5 leader_uuid: "9caae85dad6340fab81354bfb82e3c91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.050809  7242 catalog_manager.cc:5697] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 4 to 5, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.053642  7242 catalog_manager.cc:5697] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 3 to 4, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 4 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.167672  7883 tablet.cc:1724] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:54.168411  7884 tablet.cc:1724] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:54.171635  7885 tablet.cc:1724] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:17:54.199404  7824 raft_consensus.cc:993] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: : Instructing follower 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:54.200037  7824 raft_consensus.cc:1081] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Signalling peer 3e9e84d2e507455aa80bc7595a1ef075 to start an election
I20260709 13:17:54.202064  7418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
 from {username='slave'} at 127.0.0.1:49304
I20260709 13:17:54.202868  7418 raft_consensus.cc:493] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:54.203261  7418 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:54.201061  7243 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: soft_deleted_table [id=97cf88786b1844d3a369214870965fde] alter complete (version 1)
I20260709 13:17:54.210402  7418 raft_consensus.cc:515] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.213374  7418 leader_election.cc:290] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131:41951)
I20260709 13:17:54.214543  7340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:54.215180  7490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e466d58acb044afe83b14710ab8eaca9" candidate_uuid: "3e9e84d2e507455aa80bc7595a1ef075" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
I20260709 13:17:54.215402  7340 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:54.215830  7490 raft_consensus.cc:3055] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:17:54.216187  7490 raft_consensus.cc:740] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:54.217115  7490 consensus_queue.cc:260] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.218426  7490 raft_consensus.cc:3060] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:17:54.224655  7340 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:54.226215  7385 leader_election.cc:304] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e9e84d2e507455aa80bc7595a1ef075, 9caae85dad6340fab81354bfb82e3c91; no voters: 
I20260709 13:17:54.227257  7786 raft_consensus.cc:2804] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:17:54.227830  7786 raft_consensus.cc:697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Becoming Leader. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:54.228766  7786 consensus_queue.cc:237] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:54.229887  7490 raft_consensus.cc:2468] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e9e84d2e507455aa80bc7595a1ef075 in term 5.
I20260709 13:17:54.245672  7243 catalog_manager.cc:5697] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 reported cstate change: term changed from 4 to 5, leader changed from c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131) to 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130). New cstate: current_term: 5 leader_uuid: "3e9e84d2e507455aa80bc7595a1ef075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:54.375978  7490 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:54.377533  7786 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.392777  7340 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:54.395745  7814 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:54.420517  7490 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:54.422564  7789 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:54.435011  7418 raft_consensus.cc:1275] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Refusing update from remote peer 9caae85dad6340fab81354bfb82e3c91: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:54.437647  7876 consensus_queue.cc:1048] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:17:54.457336  7489 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:54.460918  7814 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:17:54.463208  7340 raft_consensus.cc:1275] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:17:54.466878  7872 consensus_queue.cc:1048] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:54.580870  7490 raft_consensus.cc:1275] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:54.582438  7872 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:17:54.587096  7340 raft_consensus.cc:1275] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:17:54.589257  7872 consensus_queue.cc:1048] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:17:54.717024  7490 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:54.718648  7872 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:54.723703  7340 raft_consensus.cc:1275] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Refusing update from remote peer 3e9e84d2e507455aa80bc7595a1ef075: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:17:54.726389  7872 consensus_queue.cc:1048] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:17:54.888698 21526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:54.897060 21526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:17:54.897809 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:54.907987  7418 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
mode: GRACEFUL
new_leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:57264
I20260709 13:17:54.908866  7418 raft_consensus.cc:606] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Received request to transfer leadership to TS c5bbd435a7a74dceaa00d2cd66e601f2
I20260709 13:17:54.910760 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:17:54.911404 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:55.444862  7872 raft_consensus.cc:993] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: : Instructing follower c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:55.445438  7814 raft_consensus.cc:1081] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Signalling peer c5bbd435a7a74dceaa00d2cd66e601f2 to start an election
I20260709 13:17:55.447531  7490 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf"
dest_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2"
 from {username='slave'} at 127.0.0.1:45224
I20260709 13:17:55.448282  7490 raft_consensus.cc:493] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:17:55.448724  7490 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:55.460533  7490 raft_consensus.cc:515] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:55.463074  7490 leader_election.cc:290] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 6 election: Requested vote from peers 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907), 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207)
I20260709 13:17:55.463953  7345 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "9caae85dad6340fab81354bfb82e3c91"
I20260709 13:17:55.464185  7418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "501ab39f68fc46dd8bcf3761f8a610cf" candidate_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: true dest_uuid: "3e9e84d2e507455aa80bc7595a1ef075"
I20260709 13:17:55.464845  7418 raft_consensus.cc:3055] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:17:55.465205  7418 raft_consensus.cc:740] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 3e9e84d2e507455aa80bc7595a1ef075, State: Running, Role: LEADER
I20260709 13:17:55.465430  7345 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:55.466290  7418 consensus_queue.cc:260] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:55.467939  7418 raft_consensus.cc:3060] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:17:55.471805  7345 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 6.
I20260709 13:17:55.473194  7459 leader_election.cc:304] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9caae85dad6340fab81354bfb82e3c91, c5bbd435a7a74dceaa00d2cd66e601f2; no voters: 
I20260709 13:17:55.474012  7824 raft_consensus.cc:2804] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:17:55.474489  7824 raft_consensus.cc:697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 6 LEADER]: Becoming Leader. State: Replica: c5bbd435a7a74dceaa00d2cd66e601f2, State: Running, Role: LEADER
I20260709 13:17:55.475311  7824 consensus_queue.cc:237] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } }
I20260709 13:17:55.477232  7418 raft_consensus.cc:2468] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5bbd435a7a74dceaa00d2cd66e601f2 in term 6.
I20260709 13:17:55.484052  7244 catalog_manager.cc:5697] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 reported cstate change: term changed from 5 to 6, leader changed from 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130) to c5bbd435a7a74dceaa00d2cd66e601f2 (127.21.5.131). New cstate: current_term: 6 leader_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c5bbd435a7a74dceaa00d2cd66e601f2" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 41951 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:55.960638  7345 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 6 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:55.962525  7824 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9caae85dad6340fab81354bfb82e3c91" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 41907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:55.984095  7416 raft_consensus.cc:1275] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 6 FOLLOWER]: Refusing update from remote peer c5bbd435a7a74dceaa00d2cd66e601f2: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:17:55.986141  7824 consensus_queue.cc:1048] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e9e84d2e507455aa80bc7595a1ef075" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:17:55.986786 21526 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:17:56.072497 21526 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:17:56.073298 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:17:56.085573 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:17:56.086246 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:17:57.122799 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
W20260709 13:17:57.129007  7385 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 13:17:57.136695  7385 consensus_peers.cc:597] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.146705  7459 consensus_peers.cc:597] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.156877  7385 consensus_peers.cc:597] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.162480  7385 consensus_peers.cc:597] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.171463  7385 consensus_peers.cc:597] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.175866 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:57.177060 21526 tablet_replica.cc:333] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.178654 21526 raft_consensus.cc:2243] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.179766 21526 raft_consensus.cc:2272] T 6211b62ab6f94bd7bd8575b3a8673e76 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.183504 21526 tablet_replica.cc:333] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.184527 21526 raft_consensus.cc:2243] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.185350 21526 raft_consensus.cc:2272] T 26955d59ed3b4c91ae35bff0d6c45808 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.188355 21526 tablet_replica.cc:333] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.189301 21526 raft_consensus.cc:2243] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.189965 21526 raft_consensus.cc:2272] T 48ea9a7ba2e44f9b9bddce661a5a061e P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.193563 21526 tablet_replica.cc:333] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.194345 21526 raft_consensus.cc:2243] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.194955 21526 raft_consensus.cc:2272] T a534d5e35ea341c591a61e94a851a293 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.197542 21526 tablet_replica.cc:333] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.198240 21526 raft_consensus.cc:2243] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.199277 21526 raft_consensus.cc:2272] T 4e44335ea170481ab59db4cca4c7c180 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.202092 21526 tablet_replica.cc:333] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.202852 21526 raft_consensus.cc:2243] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.203575 21526 raft_consensus.cc:2272] T 82995eea6405421d941a76c887eb7bd1 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.206171 21526 tablet_replica.cc:333] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.206990 21526 raft_consensus.cc:2243] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.207561 21526 raft_consensus.cc:2272] T b261c08a7c8a43d2a6fd239e5f8565be P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.210078 21526 tablet_replica.cc:333] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.210817 21526 raft_consensus.cc:2243] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.211337 21526 raft_consensus.cc:2272] T d166d5008c8c4d098bb1973379289560 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.213866 21526 tablet_replica.cc:333] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.214545 21526 raft_consensus.cc:2243] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.215696 21526 raft_consensus.cc:2272] T 50cddde895ba41b7aac3346cbdb5ae6c P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.218163 21526 tablet_replica.cc:333] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.218883 21526 raft_consensus.cc:2243] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.219408 21526 raft_consensus.cc:2272] T cc2688d492204007bb197167fa954266 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.221876 21526 tablet_replica.cc:333] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.222496 21526 raft_consensus.cc:2243] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.223088 21526 raft_consensus.cc:2272] T 32c77064066849229154ee08fe743028 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.225934 21526 tablet_replica.cc:333] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.226755 21526 raft_consensus.cc:2243] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.227311 21526 raft_consensus.cc:2272] T 0f3e8fc8428d42a8ab357ca69292f70d P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.230203 21526 tablet_replica.cc:333] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.230919 21526 raft_consensus.cc:2243] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.231801 21526 raft_consensus.cc:2272] T ccee0354eaf24cd2937732a1b12ead42 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.234112 21526 tablet_replica.cc:333] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.234684 21526 raft_consensus.cc:2243] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.235623 21526 raft_consensus.cc:2272] T bbd27671e688455da30507409a3aa997 P 9caae85dad6340fab81354bfb82e3c91 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.237970 21526 tablet_replica.cc:333] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.238497 21526 raft_consensus.cc:2243] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.239450 21526 raft_consensus.cc:2272] T 9539b8b5f9674ce99dbea364d81f55c7 P 9caae85dad6340fab81354bfb82e3c91 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.241798 21526 tablet_replica.cc:333] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.242354 21526 raft_consensus.cc:2243] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.242821 21526 raft_consensus.cc:2272] T 501ab39f68fc46dd8bcf3761f8a610cf P 9caae85dad6340fab81354bfb82e3c91 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.245553 21526 tablet_replica.cc:333] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.246150 21526 raft_consensus.cc:2243] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.246783 21526 raft_consensus.cc:2272] T 06d88a6686704bf29105419087fffe9f P 9caae85dad6340fab81354bfb82e3c91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.249536 21526 tablet_replica.cc:333] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91: stopping tablet replica
I20260709 13:17:57.250128 21526 raft_consensus.cc:2243] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.250671 21526 raft_consensus.cc:2272] T e466d58acb044afe83b14710ab8eaca9 P 9caae85dad6340fab81354bfb82e3c91 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:57.276176  7459 consensus_peers.cc:597] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.300571 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
I20260709 13:17:57.344342 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
I20260709 13:17:57.388898 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:57.389837 21526 tablet_replica.cc:333] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.390856 21526 raft_consensus.cc:2243] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.391530 21526 raft_consensus.cc:2272] T 6211b62ab6f94bd7bd8575b3a8673e76 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.394155 21526 tablet_replica.cc:333] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.394979 21526 raft_consensus.cc:2243] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.395557 21526 raft_consensus.cc:2272] T 50cddde895ba41b7aac3346cbdb5ae6c P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.398072 21526 tablet_replica.cc:333] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.398883 21526 raft_consensus.cc:2243] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.399906 21526 raft_consensus.cc:2272] T a534d5e35ea341c591a61e94a851a293 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.402657 21526 tablet_replica.cc:333] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.403414 21526 raft_consensus.cc:2243] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.403963 21526 raft_consensus.cc:2272] T b261c08a7c8a43d2a6fd239e5f8565be P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.406510 21526 tablet_replica.cc:333] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.407370 21526 raft_consensus.cc:2243] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.408385 21526 raft_consensus.cc:2272] T d166d5008c8c4d098bb1973379289560 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.410930 21526 tablet_replica.cc:333] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.411715 21526 raft_consensus.cc:2243] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.412263 21526 raft_consensus.cc:2272] T cc2688d492204007bb197167fa954266 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.414832 21526 tablet_replica.cc:333] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.415634 21526 raft_consensus.cc:2243] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.416193 21526 raft_consensus.cc:2272] T 32c77064066849229154ee08fe743028 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.418596 21526 tablet_replica.cc:333] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.419297 21526 raft_consensus.cc:2243] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.420257 21526 raft_consensus.cc:2272] T 0f3e8fc8428d42a8ab357ca69292f70d P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.423039 21526 tablet_replica.cc:333] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.423776 21526 raft_consensus.cc:2243] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.424901 21526 raft_consensus.cc:2272] T 82995eea6405421d941a76c887eb7bd1 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.428246 21526 tablet_replica.cc:333] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
W20260709 13:17:57.428110  7462 consensus_peers.cc:597] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Couldn't send request to peer 3e9e84d2e507455aa80bc7595a1ef075. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.429262 21526 raft_consensus.cc:2243] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.429859 21526 raft_consensus.cc:2272] T 4e44335ea170481ab59db4cca4c7c180 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.432477 21526 tablet_replica.cc:333] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.433161 21526 raft_consensus.cc:2243] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.434096 21526 raft_consensus.cc:2272] T 48ea9a7ba2e44f9b9bddce661a5a061e P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.436697 21526 tablet_replica.cc:333] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.437377 21526 raft_consensus.cc:2243] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.437891 21526 raft_consensus.cc:2272] T 26955d59ed3b4c91ae35bff0d6c45808 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.440485 21526 tablet_replica.cc:333] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.441152 21526 raft_consensus.cc:2243] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.441737 21526 raft_consensus.cc:2272] T 9539b8b5f9674ce99dbea364d81f55c7 P 3e9e84d2e507455aa80bc7595a1ef075 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.444317 21526 tablet_replica.cc:333] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.444964 21526 raft_consensus.cc:2243] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.445456 21526 raft_consensus.cc:2272] T 501ab39f68fc46dd8bcf3761f8a610cf P 3e9e84d2e507455aa80bc7595a1ef075 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.447935 21526 tablet_replica.cc:333] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.448613 21526 raft_consensus.cc:2243] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.449113 21526 raft_consensus.cc:2272] T ccee0354eaf24cd2937732a1b12ead42 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.451761 21526 tablet_replica.cc:333] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.452379 21526 raft_consensus.cc:2243] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.452925 21526 raft_consensus.cc:2272] T bbd27671e688455da30507409a3aa997 P 3e9e84d2e507455aa80bc7595a1ef075 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.455484 21526 tablet_replica.cc:333] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.456108 21526 raft_consensus.cc:2243] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.456753 21526 raft_consensus.cc:2272] T 06d88a6686704bf29105419087fffe9f P 3e9e84d2e507455aa80bc7595a1ef075 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.459725 21526 tablet_replica.cc:333] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075: stopping tablet replica
I20260709 13:17:57.460487 21526 raft_consensus.cc:2243] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.461776 21526 raft_consensus.cc:2272] T e466d58acb044afe83b14710ab8eaca9 P 3e9e84d2e507455aa80bc7595a1ef075 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:57.469126  7459 consensus_peers.cc:597] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:17:57.472040  7459 consensus_peers.cc:597] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.510107 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
W20260709 13:17:57.541431  7462 consensus_peers.cc:597] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Couldn't send request to peer 3e9e84d2e507455aa80bc7595a1ef075. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.556453 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
I20260709 13:17:57.595253 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:17:57.596261 21526 tablet_replica.cc:333] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.597352 21526 raft_consensus.cc:2243] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.598023 21526 raft_consensus.cc:2272] T 6211b62ab6f94bd7bd8575b3a8673e76 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:57.598978  7462 consensus_peers.cc:597] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 3e9e84d2e507455aa80bc7595a1ef075 (127.21.5.130:34207): Couldn't send request to peer 3e9e84d2e507455aa80bc7595a1ef075. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:34207: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.600903 21526 tablet_replica.cc:333] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.601888 21526 raft_consensus.cc:2243] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.602481 21526 raft_consensus.cc:2272] T a534d5e35ea341c591a61e94a851a293 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:57.604684  7459 consensus_peers.cc:597] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.605538 21526 tablet_replica.cc:333] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.606287 21526 raft_consensus.cc:2243] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.606940 21526 raft_consensus.cc:2272] T 50cddde895ba41b7aac3346cbdb5ae6c P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 13:17:57.611007  7459 consensus_peers.cc:597] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 -> Peer 9caae85dad6340fab81354bfb82e3c91 (127.21.5.129:41907): Couldn't send request to peer 9caae85dad6340fab81354bfb82e3c91. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:41907: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:17:57.611536 21526 tablet_replica.cc:333] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.612277 21526 raft_consensus.cc:2243] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.612798 21526 raft_consensus.cc:2272] T 0f3e8fc8428d42a8ab357ca69292f70d P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.615345 21526 tablet_replica.cc:333] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.615995 21526 raft_consensus.cc:2243] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.616892 21526 raft_consensus.cc:2272] T 32c77064066849229154ee08fe743028 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.619236 21526 tablet_replica.cc:333] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.619812 21526 raft_consensus.cc:2243] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.620338 21526 raft_consensus.cc:2272] T 48ea9a7ba2e44f9b9bddce661a5a061e P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.622586 21526 tablet_replica.cc:333] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.623250 21526 raft_consensus.cc:2243] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.624147 21526 raft_consensus.cc:2272] T 26955d59ed3b4c91ae35bff0d6c45808 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.626631 21526 tablet_replica.cc:333] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.627322 21526 raft_consensus.cc:2243] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.627844 21526 raft_consensus.cc:2272] T ccee0354eaf24cd2937732a1b12ead42 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.630259 21526 tablet_replica.cc:333] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.631013 21526 raft_consensus.cc:2243] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.631563 21526 raft_consensus.cc:2272] T bbd27671e688455da30507409a3aa997 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.634070 21526 tablet_replica.cc:333] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.634788 21526 raft_consensus.cc:2243] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.635329 21526 raft_consensus.cc:2272] T 9539b8b5f9674ce99dbea364d81f55c7 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.637645 21526 tablet_replica.cc:333] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.638250 21526 raft_consensus.cc:2243] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 6 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.639248 21526 raft_consensus.cc:2272] T 501ab39f68fc46dd8bcf3761f8a610cf P c5bbd435a7a74dceaa00d2cd66e601f2 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.641533 21526 tablet_replica.cc:333] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.642172 21526 raft_consensus.cc:2243] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.642669 21526 raft_consensus.cc:2272] T e466d58acb044afe83b14710ab8eaca9 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.645385 21526 tablet_replica.cc:333] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.645948 21526 raft_consensus.cc:2243] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.646868 21526 raft_consensus.cc:2272] T b261c08a7c8a43d2a6fd239e5f8565be P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.649366 21526 tablet_replica.cc:333] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.650156 21526 raft_consensus.cc:2243] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.650738 21526 raft_consensus.cc:2272] T d166d5008c8c4d098bb1973379289560 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.653119 21526 tablet_replica.cc:333] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.653667 21526 raft_consensus.cc:2243] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.654522 21526 raft_consensus.cc:2272] T cc2688d492204007bb197167fa954266 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.656873 21526 tablet_replica.cc:333] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.657577 21526 raft_consensus.cc:2243] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.658051 21526 raft_consensus.cc:2272] T 82995eea6405421d941a76c887eb7bd1 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.660406 21526 tablet_replica.cc:333] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.661002 21526 raft_consensus.cc:2243] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:17:57.661509 21526 raft_consensus.cc:2272] T 4e44335ea170481ab59db4cca4c7c180 P c5bbd435a7a74dceaa00d2cd66e601f2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.663956 21526 tablet_replica.cc:333] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2: stopping tablet replica
I20260709 13:17:57.664549 21526 raft_consensus.cc:2243] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.665529 21526 raft_consensus.cc:2272] T 06d88a6686704bf29105419087fffe9f P c5bbd435a7a74dceaa00d2cd66e601f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.728055 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:17:57.770841 21526 master.cc:562] Master@127.21.5.190:36863 shutting down...
I20260709 13:17:57.801330 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:17:57.802171 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:17:57.802591 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1439561648a74e079c3e5754923a4f27: stopping tablet replica
I20260709 13:17:57.826483 21526 master.cc:584] Master@127.21.5.190:36863 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (18360 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:17:57.896624 21526 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.5.190:35245
I20260709 13:17:57.897926 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:57.905355  7916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:57.905586  7917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:57.905958  7919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:57.907645 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:57.908938 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:57.909147 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:57.909355 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603077909318 us; error 0 us; skew 500 ppm
I20260709 13:17:57.909940 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:57.912719 21526 webserver.cc:533] Webserver started at http://127.21.5.190:35953/ using document root <none> and password file <none>
I20260709 13:17:57.913272 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:57.913483 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:57.913790 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:57.915166 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/master-0-root/instance:
uuid: "9bf0ca574fd34304a11e2647cd080d34"
format_stamp: "Formatted at 2026-07-09 13:17:57 on dist-test-slave-h0rz"
I20260709 13:17:57.919967 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:17:57.923702  7924 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:57.924515 21526 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.004s	sys 0.000s
I20260709 13:17:57.924947 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/master-0-root
uuid: "9bf0ca574fd34304a11e2647cd080d34"
format_stamp: "Formatted at 2026-07-09 13:17:57 on dist-test-slave-h0rz"
I20260709 13:17:57.925258 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:57.944140 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:57.945660 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:57.992321 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.190:35245
I20260709 13:17:57.992460  7975 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.190:35245 every 8 connection(s)
I20260709 13:17:57.996882  7976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.010403  7976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: Bootstrap starting.
I20260709 13:17:58.016489  7976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.021620  7976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: No bootstrap required, opened a new log
I20260709 13:17:58.024345  7976 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER }
I20260709 13:17:58.024787  7976 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:58.024988  7976 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bf0ca574fd34304a11e2647cd080d34, State: Initialized, Role: FOLLOWER
I20260709 13:17:58.025593  7976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER }
I20260709 13:17:58.026051  7976 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:17:58.026269  7976 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:17:58.026494  7976 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:58.032452  7976 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER }
I20260709 13:17:58.033144  7976 leader_election.cc:304] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bf0ca574fd34304a11e2647cd080d34; no voters: 
I20260709 13:17:58.034591  7976 leader_election.cc:290] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:17:58.035063  7979 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:58.036660  7979 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9bf0ca574fd34304a11e2647cd080d34, State: Running, Role: LEADER
I20260709 13:17:58.037510  7979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER }
I20260709 13:17:58.038040  7976 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:17:58.042459  7980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bf0ca574fd34304a11e2647cd080d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER } }
I20260709 13:17:58.043612  7980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:58.042712  7981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bf0ca574fd34304a11e2647cd080d34. Latest consensus state: current_term: 1 leader_uuid: "9bf0ca574fd34304a11e2647cd080d34" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bf0ca574fd34304a11e2647cd080d34" member_type: VOTER } }
I20260709 13:17:58.045217  7981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [sys.catalog]: This master's current role is: LEADER
I20260709 13:17:58.047144  7987 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:17:58.056433  7987 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:17:58.060130 21526 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:17:58.068051  7987 catalog_manager.cc:1383] Generated new cluster ID: 0278336d84174e67a8398dc98e1222d0
I20260709 13:17:58.068549  7987 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:17:58.089375  7987 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:17:58.091042  7987 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:17:58.112218  7987 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: Generated new TSK 0
I20260709 13:17:58.113020  7987 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:17:58.127746 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.135063  7997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.136160  7998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.137961  8000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.138285 21526 server_base.cc:1061] running on GCE node
I20260709 13:17:58.139598 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.139822 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.140002 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603078139985 us; error 0 us; skew 500 ppm
I20260709 13:17:58.140504 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.143105 21526 webserver.cc:533] Webserver started at http://127.21.5.129:34571/ using document root <none> and password file <none>
I20260709 13:17:58.143582 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.143745 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.143977 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.145257 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-0-root/instance:
uuid: "1b980f88bc014fc6be417a3736b2431a"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.150215 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20260709 13:17:58.154829  8005 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.155787 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:17:58.156178 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-0-root
uuid: "1b980f88bc014fc6be417a3736b2431a"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.156502 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.183104 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.184633 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.186465 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.189178 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.189455 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.189744 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.189952 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.247949 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.129:33275
I20260709 13:17:58.248080  8067 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.129:33275 every 8 connection(s)
I20260709 13:17:58.255196 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.264241  8072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.265527  8073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.271706 21526 server_base.cc:1061] running on GCE node
W20260709 13:17:58.273084  8075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.275055 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.275420 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.275694 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603078275674 us; error 0 us; skew 500 ppm
I20260709 13:17:58.276507 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.278245  8068 heartbeater.cc:344] Connected to a master server at 127.21.5.190:35245
I20260709 13:17:58.278718  8068 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.279935  8068 heartbeater.cc:507] Master 127.21.5.190:35245 requested a full tablet report, sending...
I20260709 13:17:58.280387 21526 webserver.cc:533] Webserver started at http://127.21.5.130:45033/ using document root <none> and password file <none>
I20260709 13:17:58.281235 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.281500 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.281836 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.283464  7941 ts_manager.cc:194] Registered new tserver with Master: 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275)
I20260709 13:17:58.283748 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-1-root/instance:
uuid: "1065249cd7854a589e85734eea0cb7d7"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.286507  7941 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57944
I20260709 13:17:58.291164 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:17:58.295428  8080 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.296375 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:17:58.296753 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-1-root
uuid: "1065249cd7854a589e85734eea0cb7d7"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.297096 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.315088 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.316689 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.318521 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.321249 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.321455 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.321795 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.321967 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.370280 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.130:44953
I20260709 13:17:58.370467  8142 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.130:44953 every 8 connection(s)
I20260709 13:17:58.376288 21526 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:17:58.387691  8146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:17:58.389009  8147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.394652 21526 server_base.cc:1061] running on GCE node
W20260709 13:17:58.396274  8149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:17:58.396739  8143 heartbeater.cc:344] Connected to a master server at 127.21.5.190:35245
I20260709 13:17:58.397325  8143 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.397786 21526 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:17:58.398134 21526 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:17:58.398352 21526 hybrid_clock.cc:648] HybridClock initialized: now 1783603078398333 us; error 0 us; skew 500 ppm
I20260709 13:17:58.398413  8143 heartbeater.cc:507] Master 127.21.5.190:35245 requested a full tablet report, sending...
I20260709 13:17:58.399309 21526 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:17:58.401378  7941 ts_manager.cc:194] Registered new tserver with Master: 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:17:58.402495 21526 webserver.cc:533] Webserver started at http://127.21.5.131:46321/ using document root <none> and password file <none>
I20260709 13:17:58.403228 21526 fs_manager.cc:362] Metadata directory not provided
I20260709 13:17:58.403436 21526 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:17:58.403292  7941 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57950
I20260709 13:17:58.403877 21526 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:17:58.405432 21526 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-2-root/instance:
uuid: "ca5ec37bf26843779baebb0fd163eb89"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.410782 21526 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 13:17:58.414965  8154 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.416121 21526 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:17:58.416538 21526 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-2-root
uuid: "ca5ec37bf26843779baebb0fd163eb89"
format_stamp: "Formatted at 2026-07-09 13:17:58 on dist-test-slave-h0rz"
I20260709 13:17:58.416898 21526 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskurpRMQ/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602506348444-21526-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:17:58.429390 21526 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:17:58.430967 21526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:17:58.432502 21526 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:17:58.435189 21526 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:17:58.435454 21526 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.435716 21526 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:17:58.435901 21526 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:17:58.485507 21526 rpc_server.cc:307] RPC server started. Bound to: 127.21.5.131:43473
I20260709 13:17:58.485626  8216 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.5.131:43473 every 8 connection(s)
I20260709 13:17:58.503155  8217 heartbeater.cc:344] Connected to a master server at 127.21.5.190:35245
I20260709 13:17:58.503824  8217 heartbeater.cc:461] Registering TS with master...
I20260709 13:17:58.504778  8217 heartbeater.cc:507] Master 127.21.5.190:35245 requested a full tablet report, sending...
I20260709 13:17:58.507231  7941 ts_manager.cc:194] Registered new tserver with Master: ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:58.507750 21526 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017442043s
I20260709 13:17:58.508948  7941 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57952
I20260709 13:17:58.538102  7941 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57960:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:17:58.541496  7941 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:17:58.623550  8108 tablet_service.cc:1511] Processing CreateTablet for tablet 10e88db52a2f47e0b370e51feecef5c9 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:58.625324  8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e88db52a2f47e0b370e51feecef5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.625425  8033 tablet_service.cc:1511] Processing CreateTablet for tablet 10e88db52a2f47e0b370e51feecef5c9 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:58.627144  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e88db52a2f47e0b370e51feecef5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.628484  8032 tablet_service.cc:1511] Processing CreateTablet for tablet 71104e08d54d484b8ffc160110130f09 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:58.631347  8031 tablet_service.cc:1511] Processing CreateTablet for tablet 20206412d4034cfeab74a505acba50bf (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:58.632565  8107 tablet_service.cc:1511] Processing CreateTablet for tablet 71104e08d54d484b8ffc160110130f09 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:58.634179  8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71104e08d54d484b8ffc160110130f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.631904  8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71104e08d54d484b8ffc160110130f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.639325  8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20206412d4034cfeab74a505acba50bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.638984  8106 tablet_service.cc:1511] Processing CreateTablet for tablet 20206412d4034cfeab74a505acba50bf (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:58.640841  8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20206412d4034cfeab74a505acba50bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.644375  8182 tablet_service.cc:1511] Processing CreateTablet for tablet 10e88db52a2f47e0b370e51feecef5c9 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:17:58.645031  8181 tablet_service.cc:1511] Processing CreateTablet for tablet 71104e08d54d484b8ffc160110130f09 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:17:58.646405  8182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10e88db52a2f47e0b370e51feecef5c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.647189  8181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 71104e08d54d484b8ffc160110130f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.659148  8030 tablet_service.cc:1511] Processing CreateTablet for tablet 7608af48fc5947b195079c23e77ae939 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:58.660926  8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7608af48fc5947b195079c23e77ae939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.663829  8180 tablet_service.cc:1511] Processing CreateTablet for tablet 20206412d4034cfeab74a505acba50bf (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:17:58.665688  8180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20206412d4034cfeab74a505acba50bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.673388  8105 tablet_service.cc:1511] Processing CreateTablet for tablet 7608af48fc5947b195079c23e77ae939 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:58.675513  8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7608af48fc5947b195079c23e77ae939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.681141  8029 tablet_service.cc:1511] Processing CreateTablet for tablet 6687da6ebcda4a1d9f350d992d631bf2 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:58.683251  8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6687da6ebcda4a1d9f350d992d631bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.688179  8104 tablet_service.cc:1511] Processing CreateTablet for tablet 6687da6ebcda4a1d9f350d992d631bf2 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:58.690518  8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6687da6ebcda4a1d9f350d992d631bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.707660  8233 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:58.708958  8179 tablet_service.cc:1511] Processing CreateTablet for tablet 7608af48fc5947b195079c23e77ae939 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:17:58.710827  8179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7608af48fc5947b195079c23e77ae939. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.715675  8233 tablet_bootstrap.cc:654] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.722172  8232 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:58.734589  8178 tablet_service.cc:1511] Processing CreateTablet for tablet 6687da6ebcda4a1d9f350d992d631bf2 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:17:58.735455  8031 tablet_service.cc:1511] Processing CreateTablet for tablet 9afc2afc03d24282b30770b16fcb90d7 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:58.737113  8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9afc2afc03d24282b30770b16fcb90d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.741732  8103 tablet_service.cc:1511] Processing CreateTablet for tablet 9afc2afc03d24282b30770b16fcb90d7 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:58.743602  8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9afc2afc03d24282b30770b16fcb90d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.742036  8033 tablet_service.cc:1511] Processing CreateTablet for tablet d8e0e96c35b24bd6907e0e8a77b02ab6 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:58.747351  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e0e96c35b24bd6907e0e8a77b02ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.747335  8177 tablet_service.cc:1511] Processing CreateTablet for tablet 9afc2afc03d24282b30770b16fcb90d7 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:17:58.757361  8177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9afc2afc03d24282b30770b16fcb90d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.760394  8178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6687da6ebcda4a1d9f350d992d631bf2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.760448  8108 tablet_service.cc:1511] Processing CreateTablet for tablet d8e0e96c35b24bd6907e0e8a77b02ab6 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:58.762036  8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e0e96c35b24bd6907e0e8a77b02ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.760496  8232 tablet_bootstrap.cc:654] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.787539  8234 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:58.813375  8180 tablet_service.cc:1511] Processing CreateTablet for tablet d8e0e96c35b24bd6907e0e8a77b02ab6 (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:17:58.817193  8180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8e0e96c35b24bd6907e0e8a77b02ab6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.833614  8104 tablet_service.cc:1511] Processing CreateTablet for tablet a014352cb2b14c8db9ddf4de9884caec (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:58.834398  8033 tablet_service.cc:1511] Processing CreateTablet for tablet a014352cb2b14c8db9ddf4de9884caec (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:58.835525  8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a014352cb2b14c8db9ddf4de9884caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.835389  8182 tablet_service.cc:1511] Processing CreateTablet for tablet a014352cb2b14c8db9ddf4de9884caec (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:17:58.837047  8182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a014352cb2b14c8db9ddf4de9884caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.840413  8030 tablet_service.cc:1511] Processing CreateTablet for tablet a6b065971a594587baa2cc73228d3e2b (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:58.835965  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a014352cb2b14c8db9ddf4de9884caec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.849377  8178 tablet_service.cc:1511] Processing CreateTablet for tablet a6b065971a594587baa2cc73228d3e2b (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:58.851225  8178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b065971a594587baa2cc73228d3e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.865141  8232 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:58.868214  8232 ts_tablet_manager.cc:1403] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.147s	user 0.011s	sys 0.018s
I20260709 13:17:58.871966  8232 raft_consensus.cc:359] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.866786  8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b065971a594587baa2cc73228d3e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.867527  8233 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:58.879443  8233 ts_tablet_manager.cc:1403] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.172s	user 0.037s	sys 0.052s
I20260709 13:17:58.879657  8103 tablet_service.cc:1511] Processing CreateTablet for tablet a6b065971a594587baa2cc73228d3e2b (DEFAULT_TABLE table=test-workload [id=294bfff39d284f5cbdcc12f47eff79e3]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:17:58.881330  8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a6b065971a594587baa2cc73228d3e2b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:17:58.882918  8233 raft_consensus.cc:359] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.883878  8233 raft_consensus.cc:385] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:58.884217  8233 raft_consensus.cc:740] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:58.884325  8234 tablet_bootstrap.cc:654] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.885432  8232 raft_consensus.cc:385] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:58.885380  8233 consensus_queue.cc:260] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.885970  8232 raft_consensus.cc:740] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:58.887363  8232 consensus_queue.cc:260] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.913316  8068 heartbeater.cc:499] Master 127.21.5.190:35245 was elected leader, sending a full tablet report...
I20260709 13:17:58.913269  8238 raft_consensus.cc:493] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:58.917716  8238 raft_consensus.cc:515] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.920323  8232 ts_tablet_manager.cc:1434] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.004s
I20260709 13:17:58.921742  8232 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:58.936587  8234 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:58.937345  8234 ts_tablet_manager.cc:1403] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.150s	user 0.018s	sys 0.000s
I20260709 13:17:58.941512  8234 raft_consensus.cc:359] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.942440  8234 raft_consensus.cc:385] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:58.942828  8234 raft_consensus.cc:740] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:58.943857  8234 consensus_queue.cc:260] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:58.968833  8233 ts_tablet_manager.cc:1434] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.089s	user 0.027s	sys 0.038s
I20260709 13:17:58.969950  8217 heartbeater.cc:499] Master 127.21.5.190:35245 was elected leader, sending a full tablet report...
I20260709 13:17:58.970355  8233 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:58.971267  8143 heartbeater.cc:499] Master 127.21.5.190:35245 was elected leader, sending a full tablet report...
I20260709 13:17:58.978889  8233 tablet_bootstrap.cc:654] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.980576  8234 ts_tablet_manager.cc:1434] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.043s	user 0.018s	sys 0.012s
I20260709 13:17:58.982230  8234 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:58.969918  8232 tablet_bootstrap.cc:654] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:58.990115  8234 tablet_bootstrap.cc:654] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:58.991490  8218 tablet.cc:2406] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:17:58.997455  8238 leader_election.cc:290] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.021530  8232 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.022374  8232 ts_tablet_manager.cc:1403] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.101s	user 0.015s	sys 0.004s
I20260709 13:17:59.026436  8232 raft_consensus.cc:359] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.027345  8232 raft_consensus.cc:385] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.027680  8232 raft_consensus.cc:740] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.030539  8232 consensus_queue.cc:260] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.031265  8233 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.032169  8233 ts_tablet_manager.cc:1403] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.062s	user 0.034s	sys 0.016s
I20260709 13:17:59.035691  8233 raft_consensus.cc:359] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.036706  8233 raft_consensus.cc:385] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.037124  8233 raft_consensus.cc:740] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.038013  8233 consensus_queue.cc:260] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.040649  8233 ts_tablet_manager.cc:1434] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:17:59.041565  8232 ts_tablet_manager.cc:1434] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:17:59.041996  8233 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.050655  8233 tablet_bootstrap.cc:654] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:59.054183  7989 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:17:59.055065  8234 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.055771  8234 ts_tablet_manager.cc:1403] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.074s	user 0.033s	sys 0.030s
I20260709 13:17:59.056401  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20206412d4034cfeab74a505acba50bf" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.059741  8234 raft_consensus.cc:359] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.060683  8234 raft_consensus.cc:385] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.061018  8234 raft_consensus.cc:740] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.062036  8234 consensus_queue.cc:260] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.063450  8118 raft_consensus.cc:2468] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.064881  8234 ts_tablet_manager.cc:1434] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:17:59.066213  8234 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.067040  8006 leader_election.cc:304] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.068653  8238 raft_consensus.cc:2804] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.069188  8238 raft_consensus.cc:493] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.069638  8238 raft_consensus.cc:3060] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.073853  8234 tablet_bootstrap.cc:654] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.076267  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20206412d4034cfeab74a505acba50bf" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
W20260709 13:17:59.078720  8008 leader_election.cc:343] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.085431  8234 tablet_bootstrap.cc:492] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.086107  8234 ts_tablet_manager.cc:1403] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:17:59.087707  8232 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.089624  8234 raft_consensus.cc:359] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.092969  8234 raft_consensus.cc:385] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.093366  8234 raft_consensus.cc:740] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.095007  8234 consensus_queue.cc:260] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.091894  8238 raft_consensus.cc:515] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.100517  8232 tablet_bootstrap.cc:654] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.100287  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20206412d4034cfeab74a505acba50bf" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.101289  8118 raft_consensus.cc:3060] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.102016  8234 ts_tablet_manager.cc:1434] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.006s
I20260709 13:17:59.103520  8234 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.109602  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20206412d4034cfeab74a505acba50bf" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
W20260709 13:17:59.112159  8008 leader_election.cc:343] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.113117  8238 leader_election.cc:290] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.118360  8233 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.119380  8233 ts_tablet_manager.cc:1403] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.045s
I20260709 13:17:59.120163  8118 raft_consensus.cc:2468] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.121835  8006 leader_election.cc:304] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.123474  8238 raft_consensus.cc:2804] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.124078  8238 raft_consensus.cc:697] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:17:59.124238  8232 tablet_bootstrap.cc:492] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.125097  8232 ts_tablet_manager.cc:1403] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.010s
I20260709 13:17:59.123792  8233 raft_consensus.cc:359] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.125687  8233 raft_consensus.cc:385] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.126168  8233 raft_consensus.cc:740] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.125353  8238 consensus_queue.cc:237] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.127645  8234 tablet_bootstrap.cc:654] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.127331  8233 consensus_queue.cc:260] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.130366  8233 ts_tablet_manager.cc:1434] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 13:17:59.131691  8233 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.143209  8232 raft_consensus.cc:359] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.144333  8232 raft_consensus.cc:385] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.144708  8232 raft_consensus.cc:740] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.145793  8232 consensus_queue.cc:260] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.146739  8234 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.147701  8234 ts_tablet_manager.cc:1403] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.044s	user 0.017s	sys 0.006s
I20260709 13:17:59.148928  8232 ts_tablet_manager.cc:1434] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 13:17:59.150352  8232 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.156937  8233 tablet_bootstrap.cc:654] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.157775  8232 tablet_bootstrap.cc:654] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.157927  8234 raft_consensus.cc:359] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.158949  8234 raft_consensus.cc:385] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.159292  8234 raft_consensus.cc:740] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.160207  8234 consensus_queue.cc:260] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.163075  8234 ts_tablet_manager.cc:1434] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 13:17:59.162019  7938 catalog_manager.cc:5697] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.171146  8234 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.173640  8233 tablet_bootstrap.cc:492] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.174665  8233 ts_tablet_manager.cc:1403] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.043s	user 0.018s	sys 0.006s
I20260709 13:17:59.180058  8234 tablet_bootstrap.cc:654] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.184000  8233 raft_consensus.cc:359] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.185050  8233 raft_consensus.cc:385] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.185415  8233 raft_consensus.cc:740] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.186069  8232 tablet_bootstrap.cc:492] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.186738  8232 ts_tablet_manager.cc:1403] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.020s
I20260709 13:17:59.186399  8233 consensus_queue.cc:260] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.189145  8233 ts_tablet_manager.cc:1434] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:17:59.190412  8233 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.190372  8232 raft_consensus.cc:359] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.191541  8232 raft_consensus.cc:385] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.191897  8232 raft_consensus.cc:740] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.192940  8232 consensus_queue.cc:260] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.195948  8232 ts_tablet_manager.cc:1434] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:17:59.197269  8232 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.200107  8233 tablet_bootstrap.cc:654] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.202445  8234 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.203157  8234 ts_tablet_manager.cc:1403] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.032s	user 0.022s	sys 0.003s
I20260709 13:17:59.204227  8238 raft_consensus.cc:493] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.205029  8238 raft_consensus.cc:515] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.206548  8234 raft_consensus.cc:359] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.208420  8234 raft_consensus.cc:385] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.208745  8234 raft_consensus.cc:740] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.209216  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e88db52a2f47e0b370e51feecef5c9" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.209972  8118 raft_consensus.cc:2468] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.209710  8234 consensus_queue.cc:260] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.211381  8006 leader_election.cc:304] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.211393  8238 leader_election.cc:290] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.213047  8238 raft_consensus.cc:2804] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.213445  8238 raft_consensus.cc:493] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.213819  8238 raft_consensus.cc:3060] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.215504  8234 ts_tablet_manager.cc:1434] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:17:59.218416  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e88db52a2f47e0b370e51feecef5c9" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.222110  8192 raft_consensus.cc:2468] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.222406  8234 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.213807  8232 tablet_bootstrap.cc:654] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.226600  8239 raft_consensus.cc:493] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.234927  8234 tablet_bootstrap.cc:654] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.234481  8239 raft_consensus.cc:515] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.238915  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.239717  8118 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.241272  8006 leader_election.cc:304] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.243423  8239 leader_election.cc:290] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.244230  8239 raft_consensus.cc:2804] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.244692  8239 raft_consensus.cc:493] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.244807  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.245977  8192 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.245242  8239 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.253145  8233 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.253753  8234 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.254122  8233 ts_tablet_manager.cc:1403] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.064s	user 0.015s	sys 0.031s
I20260709 13:17:59.254395  8234 ts_tablet_manager.cc:1403] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.013s
I20260709 13:17:59.254513  8232 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.252684  8238 raft_consensus.cc:515] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.255216  8232 ts_tablet_manager.cc:1403] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.000s
I20260709 13:17:59.256999  8238 leader_election.cc:290] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.258415  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e88db52a2f47e0b370e51feecef5c9" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.259091  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10e88db52a2f47e0b370e51feecef5c9" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.259438  8192 raft_consensus.cc:3060] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.259755  8118 raft_consensus.cc:3060] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.261224  8232 raft_consensus.cc:359] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.262123  8232 raft_consensus.cc:385] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.262425  8232 raft_consensus.cc:740] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.263772  8241 raft_consensus.cc:493] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.264415  8241 raft_consensus.cc:515] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.266027  8239 raft_consensus.cc:515] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.269145  8241 leader_election.cc:290] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.263453  8232 consensus_queue.cc:260] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.269407  8192 raft_consensus.cc:2468] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.271166  8008 leader_election.cc:304] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.272619  8261 raft_consensus.cc:2804] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.276211  8232 ts_tablet_manager.cc:1434] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:17:59.278401  8232 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.280332  8234 raft_consensus.cc:359] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.281302  8234 raft_consensus.cc:385] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.281535  8237 raft_consensus.cc:493] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.282655  8237 raft_consensus.cc:515] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.287055  8237 leader_election.cc:290] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:17:59.288514  8239 leader_election.cc:290] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.292843  8118 raft_consensus.cc:2468] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.295589  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.297827  8232 tablet_bootstrap.cc:654] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.298084  8117 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.307586  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.308634  8191 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.310213  8083 leader_election.cc:304] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.311316  8241 raft_consensus.cc:2804] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.311755  8241 raft_consensus.cc:493] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.311868  8232 tablet_bootstrap.cc:492] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.312158  8241 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.312631  8232 ts_tablet_manager.cc:1403] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.007s
I20260709 13:17:59.281723  8234 raft_consensus.cc:740] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.313679  8233 raft_consensus.cc:359] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.316579  8232 raft_consensus.cc:359] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.317484  8233 raft_consensus.cc:385] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.317618  8232 raft_consensus.cc:385] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.317521  8261 raft_consensus.cc:697] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:17:59.316984  8117 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.318172  8232 raft_consensus.cc:740] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.317992  8233 raft_consensus.cc:740] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.319821  8261 consensus_queue.cc:237] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.322328  8233 consensus_queue.cc:260] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.323458  8006 leader_election.cc:304] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.324795  8265 raft_consensus.cc:2804] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.325680  8265 raft_consensus.cc:697] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:17:59.326891  8265 consensus_queue.cc:237] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.320082  8232 consensus_queue.cc:260] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.330441  8233 ts_tablet_manager.cc:1434] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 13:17:59.331902  8233 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.334398  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.335355  8043 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.337275  8232 ts_tablet_manager.cc:1434] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.000s
I20260709 13:17:59.339295  8232 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.337991  8241 raft_consensus.cc:515] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.345108  8241 leader_election.cc:290] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.346187  8239 raft_consensus.cc:493] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.346868  8239 raft_consensus.cc:515] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.347656  8233 tablet_bootstrap.cc:654] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.350950  8192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.350550  8234 consensus_queue.cc:260] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.351613  8192 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.351778  8270 raft_consensus.cc:493] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.352346  8270 raft_consensus.cc:515] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.352947  8232 tablet_bootstrap.cc:654] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.355122  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:17:59.355826  8043 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.357213  8270 leader_election.cc:290] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.360304  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.361451  8191 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.362423  8234 ts_tablet_manager.cc:1434] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.108s	user 0.009s	sys 0.005s
I20260709 13:17:59.362241  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.363065  8117 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.363974  8239 leader_election.cc:290] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.366302  8241 raft_consensus.cc:493] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.368371  8083 leader_election.cc:304] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.372745  8006 leader_election.cc:304] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.372638  8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.377468  8042 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.372136  8234 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.380388  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.381347  8191 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.386650  8265 raft_consensus.cc:2804] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.375097  8270 raft_consensus.cc:2804] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.387804  8270 raft_consensus.cc:493] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.386890  8234 tablet_bootstrap.cc:654] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.388161  8270 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.387327  7940 catalog_manager.cc:5697] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.393371  8043 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.394950  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.395861  8190 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.373044  8241 raft_consensus.cc:515] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.397828  8270 raft_consensus.cc:515] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.402660  8261 raft_consensus.cc:493] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.403223  8261 raft_consensus.cc:515] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.406060  8261 leader_election.cc:290] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.406213  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.407058  8189 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.399703  8192 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.412334  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.412544  8191 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.413548  8117 raft_consensus.cc:2393] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca5ec37bf26843779baebb0fd163eb89 in current term 1: Already voted for candidate 1065249cd7854a589e85734eea0cb7d7 in this term.
I20260709 13:17:59.415688  8232 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.415839  8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.416405  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:17:59.416772  8232 ts_tablet_manager.cc:1403] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.078s	user 0.015s	sys 0.008s
I20260709 13:17:59.412273  8188 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.400581  8241 leader_election.cc:290] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.419164  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.420851  8232 raft_consensus.cc:359] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.421743  8232 raft_consensus.cc:385] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.422088  8233 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
W20260709 13:17:59.422395  8008 leader_election.cc:343] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.422678  8234 tablet_bootstrap.cc:492] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.422853  8233 ts_tablet_manager.cc:1403] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.001s
I20260709 13:17:59.422930  8189 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.423413  8234 ts_tablet_manager.cc:1403] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.060s	user 0.006s	sys 0.008s
I20260709 13:17:59.406962  8081 leader_election.cc:304] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.424336  8270 leader_election.cc:290] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.422065  8232 raft_consensus.cc:740] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.412326  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.387355  8265 raft_consensus.cc:493] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:17:59.426666  8083 leader_election.cc:343] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.426899  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
W20260709 13:17:59.427533  8081 leader_election.cc:343] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.427851  8083 leader_election.cc:304] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.428514  8043 raft_consensus.cc:2393] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ca5ec37bf26843779baebb0fd163eb89 in current term 1: Already voted for candidate 1065249cd7854a589e85734eea0cb7d7 in this term.
I20260709 13:17:59.428949  8241 raft_consensus.cc:2804] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.428177  8081 leader_election.cc:304] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7; no voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.429919  8232 consensus_queue.cc:260] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.431174  8270 raft_consensus.cc:2804] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.431218  8155 leader_election.cc:304] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca5ec37bf26843779baebb0fd163eb89; no voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a
I20260709 13:17:59.432451  8237 raft_consensus.cc:2749] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.432821  8232 ts_tablet_manager.cc:1434] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:17:59.427286  8265 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.434729  8241 raft_consensus.cc:697] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:17:59.435829  8241 consensus_queue.cc:237] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.437129  8276 raft_consensus.cc:2749] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.437593  8234 raft_consensus.cc:359] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.440294  8234 raft_consensus.cc:385] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.440665  8234 raft_consensus.cc:740] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.441547  8234 consensus_queue.cc:260] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
W20260709 13:17:59.442185  8006 leader_election.cc:343] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.442827  8006 leader_election.cc:304] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a; no voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.444167  8261 raft_consensus.cc:2749] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.445835  8234 ts_tablet_manager.cc:1434] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.002s
I20260709 13:17:59.437180  8233 raft_consensus.cc:359] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.438592  8232 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.446888  8270 raft_consensus.cc:697] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:17:59.447269  8233 raft_consensus.cc:385] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.448041  8233 raft_consensus.cc:740] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.448138  8270 consensus_queue.cc:237] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.449162  8233 consensus_queue.cc:260] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.454857  8233 ts_tablet_manager.cc:1434] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.003s
I20260709 13:17:59.455677  8234 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.456362  8233 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.457228  7940 catalog_manager.cc:5697] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.474964  8234 tablet_bootstrap.cc:654] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.475093  8265 raft_consensus.cc:515] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.476644  8041 raft_consensus.cc:2393] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1065249cd7854a589e85734eea0cb7d7 in current term 1: Already voted for candidate 1b980f88bc014fc6be417a3736b2431a in this term.
I20260709 13:17:59.478909  8265 leader_election.cc:290] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.479306  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.480443  8189 raft_consensus.cc:2393] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b980f88bc014fc6be417a3736b2431a in current term 1: Already voted for candidate 1065249cd7854a589e85734eea0cb7d7 in this term.
I20260709 13:17:59.481370  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.482990  8006 leader_election.cc:304] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a; no voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.483249  8233 tablet_bootstrap.cc:654] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.484583  8265 raft_consensus.cc:2749] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.489485  8232 tablet_bootstrap.cc:654] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.492941  7939 catalog_manager.cc:5697] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.497431  8233 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.498184  8233 ts_tablet_manager.cc:1403] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.008s
I20260709 13:17:59.499894  8234 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.500671  8232 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.501302  8232 ts_tablet_manager.cc:1403] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.000s
I20260709 13:17:59.502067  8233 raft_consensus.cc:359] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.500686  8234 ts_tablet_manager.cc:1403] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.015s
I20260709 13:17:59.502990  8233 raft_consensus.cc:385] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.504256  8233 raft_consensus.cc:740] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.504974  8232 raft_consensus.cc:359] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.505313  8233 consensus_queue.cc:260] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.506161  8232 raft_consensus.cc:385] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.506594  8232 raft_consensus.cc:740] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.507452  8232 consensus_queue.cc:260] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.509902  8232 ts_tablet_manager.cc:1434] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:17:59.511994  8233 ts_tablet_manager.cc:1434] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:17:59.513417  8233 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.519136  8232 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:17:59.520437  8265 raft_consensus.cc:493] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.521387  8233 tablet_bootstrap.cc:654] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.521097  8265 raft_consensus.cc:515] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.507362  8234 raft_consensus.cc:359] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.524837  8234 raft_consensus.cc:385] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.524657  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.525552  8265 leader_election.cc:290] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.525862  8117 raft_consensus.cc:2468] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:17:59.525547  8234 raft_consensus.cc:740] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.524932  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.527379  8234 consensus_queue.cc:260] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.528118  8232 tablet_bootstrap.cc:654] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
W20260709 13:17:59.528334  8008 leader_election.cc:343] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.529946  8006 leader_election.cc:304] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.530679  8234 ts_tablet_manager.cc:1434] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:17:59.531386  8265 raft_consensus.cc:2804] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.532035  8265 raft_consensus.cc:493] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.532512  8234 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:17:59.532507  8265 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.539824  8261 raft_consensus.cc:493] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.540608  8232 tablet_bootstrap.cc:492] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:17:59.540488  8261 raft_consensus.cc:515] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.541198  8232 ts_tablet_manager.cc:1403] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:17:59.543215  8234 tablet_bootstrap.cc:654] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.544478  8232 raft_consensus.cc:359] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.545290  8232 raft_consensus.cc:385] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.545565  8232 raft_consensus.cc:740] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.547101  8261 leader_election.cc:290] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.549101  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
W20260709 13:17:59.550946  8008 leader_election.cc:343] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.550170  7940 catalog_manager.cc:5697] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.553681  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
W20260709 13:17:59.555769  8006 leader_election.cc:343] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:17:59.556326  8006 leader_election.cc:304] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a; no voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:17:59.557734  8261 raft_consensus.cc:2749] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:17:59.560075  8234 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:17:59.561030  8234 ts_tablet_manager.cc:1403] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.009s
I20260709 13:17:59.546273  8232 consensus_queue.cc:260] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.564270  8232 ts_tablet_manager.cc:1434] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:17:59.566216  8234 raft_consensus.cc:359] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.567658  8234 raft_consensus.cc:385] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.568146  8234 raft_consensus.cc:740] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.569573  8234 consensus_queue.cc:260] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.573725  8234 ts_tablet_manager.cc:1434] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:17:59.575316  8265 raft_consensus.cc:515] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.579588  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.580335  8117 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.580940  8189 raft_consensus.cc:3060] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.584161  8265 leader_election.cc:290] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.586148  8233 tablet_bootstrap.cc:492] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.586954  8233 ts_tablet_manager.cc:1403] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.074s	user 0.034s	sys 0.033s
I20260709 13:17:59.594106  8270 raft_consensus.cc:493] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.592087  8117 raft_consensus.cc:2468] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.588184  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.590374  8233 raft_consensus.cc:359] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.600189  8233 raft_consensus.cc:385] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.599781  8270 raft_consensus.cc:515] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.600908  8006 leader_election.cc:304] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.601315  8265 consensus_queue.cc:1048] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:59.602039  8261 raft_consensus.cc:2804] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.600553  8233 raft_consensus.cc:740] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.602584  8261 raft_consensus.cc:697] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:17:59.604486  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.605321  8041 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.604918  8261 consensus_queue.cc:237] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.607147  8081 leader_election.cc:304] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.608001  8241 raft_consensus.cc:2804] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.608297  8241 raft_consensus.cc:493] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.608484  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.608528  8241 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.609438  8191 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.603437  8233 consensus_queue.cc:260] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.613016  8190 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.614523  8233 ts_tablet_manager.cc:1434] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.009s
I20260709 13:17:59.602835  8270 leader_election.cc:290] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.618551  8233 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:17:59.619683  8241 raft_consensus.cc:515] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.624155  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:17:59.624945  8041 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.625016  8190 raft_consensus.cc:2468] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:17:59.633793  8233 tablet_bootstrap.cc:654] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:17:59.635135  8041 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.636626  8081 leader_election.cc:304] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.637526  8270 raft_consensus.cc:2804] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.638103  8270 raft_consensus.cc:697] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:17:59.639101  8270 consensus_queue.cc:237] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.640611  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.641441  8190 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.642803  8241 leader_election.cc:290] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.652419  7938 catalog_manager.cc:5697] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.656865  8190 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.657249  8238 consensus_queue.cc:1048] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:59.671777  8233 tablet_bootstrap.cc:492] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:17:59.672708  8233 ts_tablet_manager.cc:1403] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.057s	user 0.019s	sys 0.002s
I20260709 13:17:59.676839  8233 raft_consensus.cc:359] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.677899  8233 raft_consensus.cc:385] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:17:59.678262  8233 raft_consensus.cc:740] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:17:59.679394  8233 consensus_queue.cc:260] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.688967  8233 ts_tablet_manager.cc:1434] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:17:59.707646  7938 catalog_manager.cc:5697] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.727933  8239 consensus_queue.cc:1048] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:59.742134  8239 consensus_queue.cc:1048] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:59.836418  8237 raft_consensus.cc:493] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.836966  8237 raft_consensus.cc:515] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.838990  8237 leader_election.cc:290] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:17:59.840133  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.840257  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:17:59.841264  8041 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:17:59.841300  8117 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:17:59.842795  8155 leader_election.cc:304] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.843597  8237 raft_consensus.cc:2804] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.844040  8237 raft_consensus.cc:493] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.844494  8237 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.846124  8270 raft_consensus.cc:493] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:17:59.846793  8270 raft_consensus.cc:515] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.849375  8270 leader_election.cc:290] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.850271  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:17:59.850837  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:17:59.851322  8041 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.851630  8191 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:17:59.852806  8081 leader_election.cc:304] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:17:59.853716  8270 raft_consensus.cc:2804] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:17:59.854153  8270 raft_consensus.cc:493] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:17:59.854008  8237 raft_consensus.cc:515] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.854681  8270 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.856604  8237 leader_election.cc:290] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:17:59.857607  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:17:59.857815  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:17:59.858326  8041 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.858484  8117 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.864022  8270 raft_consensus.cc:515] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.866429  8270 leader_election.cc:290] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:17:59.867092  8117 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:17:59.867678  8041 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:17:59.867867  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:17:59.868578  8191 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.868651  8155 leader_election.cc:304] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.869115  8239 consensus_queue.cc:1048] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:59.869810  8237 raft_consensus.cc:2804] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.870074  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:17:59.870738  8041 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:17:59.871773  8237 raft_consensus.cc:697] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:17:59.872850  8237 consensus_queue.cc:237] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.881280  8191 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.883276  8041 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:17:59.883682  8083 leader_election.cc:304] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:17:59.884729  8270 raft_consensus.cc:2804] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:17:59.885390  8270 raft_consensus.cc:697] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:17:59.886546  8270 consensus_queue.cc:237] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:17:59.901752  8261 consensus_queue.cc:1048] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:17:59.904278  7940 catalog_manager.cc:5697] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:17:59.908931  7939 catalog_manager.cc:5697] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:17:59.936432  8270 consensus_queue.cc:1048] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:17:59.949573  8241 consensus_queue.cc:1048] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.020498  8241 consensus_queue.cc:1048] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.034471  8241 consensus_queue.cc:1048] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.037828  8261 consensus_queue.cc:1048] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.052898  8238 consensus_queue.cc:1048] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.182111  8241 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.200585  8241 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.203711  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "71104e08d54d484b8ffc160110130f09"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:53366
I20260709 13:18:00.204430  8117 raft_consensus.cc:606] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS 1b980f88bc014fc6be417a3736b2431a
I20260709 13:18:00.216933  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7608af48fc5947b195079c23e77ae939"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:53366
I20260709 13:18:00.217634  8117 raft_consensus.cc:606] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS 1b980f88bc014fc6be417a3736b2431a
I20260709 13:18:00.233920  8041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:18:00.234826  8041 raft_consensus.cc:606] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:00.242468  8041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:18:00.243286  8041 raft_consensus.cc:606] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:00.263686  8191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:42656
I20260709 13:18:00.264354  8191 raft_consensus.cc:606] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:00.270452  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:53366
I20260709 13:18:00.271147  8117 raft_consensus.cc:606] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS ca5ec37bf26843779baebb0fd163eb89
I20260709 13:18:00.315650  8237 consensus_queue.cc:1048] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.324414  8297 raft_consensus.cc:993] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.325261  8237 raft_consensus.cc:1081] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.327291  8118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53356
I20260709 13:18:00.327952  8118 raft_consensus.cc:493] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.328383  8118 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.331929  8297 consensus_queue.cc:1048] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.336704  8118 raft_consensus.cc:515] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.338270  8117 raft_consensus.cc:1240] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Rejecting Update request from peer ca5ec37bf26843779baebb0fd163eb89 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:00.340850  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:00.341228  8118 leader_election.cc:290] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:00.341627  8043 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.342074  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a014352cb2b14c8db9ddf4de9884caec" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:00.342783  8191 raft_consensus.cc:3055] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.343124  8191 raft_consensus.cc:740] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:00.344076  8191 consensus_queue.cc:260] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.345074  8297 consensus_queue.cc:1059] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.347842  8191 raft_consensus.cc:3060] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.351058  8043 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.352339  8081 leader_election.cc:304] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:00.353183  8241 raft_consensus.cc:2804] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.353688  8241 raft_consensus.cc:697] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.354604  8241 consensus_queue.cc:237] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.355194  8191 raft_consensus.cc:2468] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.363065  7940 catalog_manager.cc:5697] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.386754  8241 consensus_queue.cc:1048] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:00.395525  8278 raft_consensus.cc:993] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:00.396241  8278 raft_consensus.cc:1081] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:00.398170  8189 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:42648
I20260709 13:18:00.399287  8189 raft_consensus.cc:493] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.399748  8189 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.403832  8241 consensus_queue.cc:1048] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:00.409474  8189 raft_consensus.cc:515] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.413573  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:00.415211  8043 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.415684  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a6b065971a594587baa2cc73228d3e2b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:00.416476  8117 raft_consensus.cc:3055] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.416849  8117 raft_consensus.cc:740] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.418354  8117 consensus_queue.cc:260] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.420207  8117 raft_consensus.cc:3060] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.413964  8189 leader_election.cc:290] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:00.425125  8043 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 2.
I20260709 13:18:00.426959  8155 leader_election.cc:304] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:00.428014  8297 raft_consensus.cc:2804] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.428421  8117 raft_consensus.cc:2468] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 2.
I20260709 13:18:00.428589  8297 raft_consensus.cc:697] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 2 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:00.429723  8297 consensus_queue.cc:237] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.437487  7940 catalog_manager.cc:5697] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:00.446376  8261 raft_consensus.cc:993] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.446964  8238 raft_consensus.cc:1081] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.448558  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:18:00.449117  8117 raft_consensus.cc:493] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.449447  8117 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.456713  8117 raft_consensus.cc:515] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.458778  8117 leader_election.cc:290] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:00.459800  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:00.460093  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6687da6ebcda4a1d9f350d992d631bf2" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:00.460578  8043 raft_consensus.cc:3055] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.460722  8189 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.460911  8043 raft_consensus.cc:740] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:00.461808  8043 consensus_queue.cc:260] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.463222  8043 raft_consensus.cc:3060] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.467432  8189 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.469013  8083 leader_election.cc:304] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:00.469955  8241 raft_consensus.cc:2804] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.470134  8043 raft_consensus.cc:2468] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.470417  8241 raft_consensus.cc:697] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.471493  8241 consensus_queue.cc:237] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.478132  8261 raft_consensus.cc:993] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.478798  8238 raft_consensus.cc:1081] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:00.480489  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:18:00.480217  7940 catalog_manager.cc:5697] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from 1b980f88bc014fc6be417a3736b2431a (127.21.5.129) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.481108  8117 raft_consensus.cc:493] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.481532  8117 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.489516  8117 raft_consensus.cc:515] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.492321  8117 leader_election.cc:290] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:00.492799  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:00.492929  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9afc2afc03d24282b30770b16fcb90d7" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:00.493856  8043 raft_consensus.cc:3055] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.493942  8189 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.494290  8043 raft_consensus.cc:740] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:00.496713  8278 raft_consensus.cc:993] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:00.496481  8043 consensus_queue.cc:260] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.497272  8278 raft_consensus.cc:1081] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:00.497870  8043 raft_consensus.cc:3060] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.499373  8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "71104e08d54d484b8ffc160110130f09"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:45806
I20260709 13:18:00.500082  8041 raft_consensus.cc:493] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.500461  8041 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.502224  8189 raft_consensus.cc:2468] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.503630  8083 leader_election.cc:304] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:00.504464  8278 raft_consensus.cc:2804] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.504973  8278 raft_consensus.cc:697] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.505803  8278 consensus_queue.cc:237] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.507053  8043 raft_consensus.cc:2468] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:00.510277  8041 raft_consensus.cc:515] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.513595  8041 leader_election.cc:290] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:00.514925  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:00.515883  8189 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.514890  7940 catalog_manager.cc:5697] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from 1b980f88bc014fc6be417a3736b2431a (127.21.5.129) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.520512  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "71104e08d54d484b8ffc160110130f09" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:00.521322  8117 raft_consensus.cc:3055] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.521662  8117 raft_consensus.cc:740] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.522557  8117 consensus_queue.cc:260] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.523821  8117 raft_consensus.cc:3060] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.525437  8189 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:00.526386  8191 raft_consensus.cc:1240] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Rejecting Update request from peer 1065249cd7854a589e85734eea0cb7d7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:18:00.527066  8008 leader_election.cc:304] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:00.528318  8238 raft_consensus.cc:2804] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.528839  8238 raft_consensus.cc:697] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:00.529772  8238 consensus_queue.cc:237] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.534168  8117 raft_consensus.cc:2468] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:00.542340  7939 catalog_manager.cc:5697] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.585968  8278 raft_consensus.cc:993] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:00.586537  8241 raft_consensus.cc:1081] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:00.588414  8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7608af48fc5947b195079c23e77ae939"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:45806
I20260709 13:18:00.588992  8041 raft_consensus.cc:493] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:00.589275  8041 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.595566  8041 raft_consensus.cc:515] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.597666  8041 leader_election.cc:290] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:00.598645  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:00.598960  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:00.599449  8117 raft_consensus.cc:3055] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:00.599730  8189 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.599874  8117 raft_consensus.cc:740] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:00.600816  8117 consensus_queue.cc:260] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.601948  8117 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:00.606383  8189 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:00.607856  8008 leader_election.cc:304] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:00.608724  8238 raft_consensus.cc:2804] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:00.608984  8117 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:00.609179  8238 raft_consensus.cc:697] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:00.610272  8238 consensus_queue.cc:237] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:00.617972  7939 catalog_manager.cc:5697] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:00.770483  8041 raft_consensus.cc:1275] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.772156  8241 consensus_queue.cc:1048] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:00.783615  8189 raft_consensus.cc:1275] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.785282  8276 consensus_queue.cc:1048] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:00.914830  8041 raft_consensus.cc:1275] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.916565  8276 consensus_queue.cc:1048] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:00.926335  8191 raft_consensus.cc:1275] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.928647  8278 consensus_queue.cc:1048] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:00.921379  8189 raft_consensus.cc:1275] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.935408  8238 consensus_queue.cc:1048] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:00.953583  8117 raft_consensus.cc:1275] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.955296  8238 consensus_queue.cc:1048] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:00.975121  8041 raft_consensus.cc:1275] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.976581  8278 consensus_queue.cc:1048] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:00.986276  8191 raft_consensus.cc:1275] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.988858  8278 consensus_queue.cc:1048] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:00.997313  8117 raft_consensus.cc:1275] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:00.998735  8238 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:01.004808  8191 raft_consensus.cc:1275] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:01.006944  8261 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:01.007380  8117 raft_consensus.cc:1275] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:01.009593  8297 consensus_queue.cc:1048] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:01.029053  8041 raft_consensus.cc:1275] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:01.031390  8323 consensus_queue.cc:1048] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:03.313717  7940 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57970:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:18:03.318270  7940 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:18:03.388500  8033 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdca70093f940099388f36c968b1cac (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:03.389982  8103 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdca70093f940099388f36c968b1cac (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:03.391021  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdca70093f940099388f36c968b1cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.391398  8178 tablet_service.cc:1511] Processing CreateTablet for tablet 9cdca70093f940099388f36c968b1cac (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:18:03.392313  8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdca70093f940099388f36c968b1cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.393404  8178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9cdca70093f940099388f36c968b1cac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.393512  8030 tablet_service.cc:1511] Processing CreateTablet for tablet d658590c916e46a3b06ed414a0622af0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:03.414597  8030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d658590c916e46a3b06ed414a0622af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.423327  8032 tablet_service.cc:1511] Processing CreateTablet for tablet 77d45ecd2f3345bd92acb90edbd95eb0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:03.425315  8032 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d45ecd2f3345bd92acb90edbd95eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.437458  8108 tablet_service.cc:1511] Processing CreateTablet for tablet dd703609ff9a4095b953ec5774ebbd0b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:03.400262  8182 tablet_service.cc:1511] Processing CreateTablet for tablet d658590c916e46a3b06ed414a0622af0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:03.410737  8029 tablet_service.cc:1511] Processing CreateTablet for tablet dd703609ff9a4095b953ec5774ebbd0b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:03.441322  8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd703609ff9a4095b953ec5774ebbd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.411669  8177 tablet_service.cc:1511] Processing CreateTablet for tablet 6a17070010424c10896ecfeefbb687c1 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:03.411954  8105 tablet_service.cc:1511] Processing CreateTablet for tablet 6a17070010424c10896ecfeefbb687c1 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:03.443535  8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a17070010424c10896ecfeefbb687c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.429860  8179 tablet_service.cc:1511] Processing CreateTablet for tablet dd703609ff9a4095b953ec5774ebbd0b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:18:03.412639  8031 tablet_service.cc:1511] Processing CreateTablet for tablet 6a17070010424c10896ecfeefbb687c1 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:18:03.449764  8031 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a17070010424c10896ecfeefbb687c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.412720  8104 tablet_service.cc:1511] Processing CreateTablet for tablet d658590c916e46a3b06ed414a0622af0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:18:03.454087  8104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d658590c916e46a3b06ed414a0622af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.457320  8180 tablet_service.cc:1511] Processing CreateTablet for tablet 77d45ecd2f3345bd92acb90edbd95eb0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:03.458673  8028 tablet_service.cc:1511] Processing CreateTablet for tablet 849222e5c2d74fa5b133370d90e6d268 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:03.460979  8028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849222e5c2d74fa5b133370d90e6d268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.467140  8108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd703609ff9a4095b953ec5774ebbd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.477525  8177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a17070010424c10896ecfeefbb687c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.482182  8180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d45ecd2f3345bd92acb90edbd95eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.516254  8107 tablet_service.cc:1511] Processing CreateTablet for tablet 77d45ecd2f3345bd92acb90edbd95eb0 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:18:03.518397  8107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77d45ecd2f3345bd92acb90edbd95eb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.528214  8182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d658590c916e46a3b06ed414a0622af0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.536099  8181 tablet_service.cc:1511] Processing CreateTablet for tablet 849222e5c2d74fa5b133370d90e6d268 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:03.566874  8337 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:03.568976  8179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd703609ff9a4095b953ec5774ebbd0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.599570  8181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849222e5c2d74fa5b133370d90e6d268. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:18:03.602943  8083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:43473, user_credentials={real_user=slave}} blocked reactor thread for 74119.2us
I20260709 13:18:03.606665  8106 tablet_service.cc:1511] Processing CreateTablet for tablet 849222e5c2d74fa5b133370d90e6d268 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:18:03.637861  8340 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:03.643437  8106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849222e5c2d74fa5b133370d90e6d268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.644843  8033 tablet_service.cc:1511] Processing CreateTablet for tablet 30feeaac6048464d9513b6da16bce336 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:03.647820  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30feeaac6048464d9513b6da16bce336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.664353  8337 tablet_bootstrap.cc:654] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.668251  8176 tablet_service.cc:1511] Processing CreateTablet for tablet 30feeaac6048464d9513b6da16bce336 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:03.670181  8176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30feeaac6048464d9513b6da16bce336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.677670  8103 tablet_service.cc:1511] Processing CreateTablet for tablet 30feeaac6048464d9513b6da16bce336 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:18:03.679306  8103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30feeaac6048464d9513b6da16bce336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.720769  8033 tablet_service.cc:1511] Processing CreateTablet for tablet a3683256adef493fbb1545c6189ae91b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:03.722558  8033 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3683256adef493fbb1545c6189ae91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.745265  8105 tablet_service.cc:1511] Processing CreateTablet for tablet a3683256adef493fbb1545c6189ae91b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:03.747123  8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3683256adef493fbb1545c6189ae91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.749702  8029 tablet_service.cc:1511] Processing CreateTablet for tablet e9f73b8edc6e4b2b90ac0554c85ddc59 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:03.751529  8029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f73b8edc6e4b2b90ac0554c85ddc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.759152  8340 tablet_bootstrap.cc:654] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.759882  8176 tablet_service.cc:1511] Processing CreateTablet for tablet a3683256adef493fbb1545c6189ae91b (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:18:03.761629  8176 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3683256adef493fbb1545c6189ae91b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.764493  8341 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:03.782478  8341 tablet_bootstrap.cc:654] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.798151  8105 tablet_service.cc:1511] Processing CreateTablet for tablet e9f73b8edc6e4b2b90ac0554c85ddc59 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:03.799309  8181 tablet_service.cc:1511] Processing CreateTablet for tablet e9f73b8edc6e4b2b90ac0554c85ddc59 (DEFAULT_TABLE table=soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:18:03.799932  8105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f73b8edc6e4b2b90ac0554c85ddc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.800945  8181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9f73b8edc6e4b2b90ac0554c85ddc59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:18:03.799793  8340 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:03.806064  8340 ts_tablet_manager.cc:1403] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.169s	user 0.014s	sys 0.004s
I20260709 13:18:03.829388  8337 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:03.833544  8341 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:03.839123  8341 ts_tablet_manager.cc:1403] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.075s	user 0.014s	sys 0.010s
I20260709 13:18:03.839294  8337 ts_tablet_manager.cc:1403] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.273s	user 0.006s	sys 0.058s
I20260709 13:18:03.841033  8340 raft_consensus.cc:359] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.842113  8340 raft_consensus.cc:385] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.842711  8341 raft_consensus.cc:359] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.843441  8341 raft_consensus.cc:385] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.843705  8341 raft_consensus.cc:740] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.844556  8341 consensus_queue.cc:260] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.847589  8341 ts_tablet_manager.cc:1434] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:18:03.848861  8341 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:03.842440  8340 raft_consensus.cc:740] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.853655  8340 consensus_queue.cc:260] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.854940  8337 raft_consensus.cc:359] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.855860  8337 raft_consensus.cc:385] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.856170  8337 raft_consensus.cc:740] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.856956  8340 ts_tablet_manager.cc:1434] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.047s	user 0.005s	sys 0.003s
I20260709 13:18:03.858333  8337 consensus_queue.cc:260] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.859530  8340 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:03.864840  8341 tablet_bootstrap.cc:654] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.871805  8337 ts_tablet_manager.cc:1434] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.003s
I20260709 13:18:03.874955  8340 tablet_bootstrap.cc:654] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.885169  8337 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:03.890237  8340 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:03.891304  8340 ts_tablet_manager.cc:1403] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.012s
I20260709 13:18:03.896235  8340 raft_consensus.cc:359] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.896971  8340 raft_consensus.cc:385] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.897303  8340 raft_consensus.cc:740] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.899211  8337 tablet_bootstrap.cc:654] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.905010  8337 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:03.905171  8341 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:03.905601  8337 ts_tablet_manager.cc:1403] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.005s
I20260709 13:18:03.906111  8341 ts_tablet_manager.cc:1403] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.058s	user 0.023s	sys 0.019s
I20260709 13:18:03.910341  8341 raft_consensus.cc:359] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.910085  8340 consensus_queue.cc:260] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.911383  8341 raft_consensus.cc:385] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.912323  8341 raft_consensus.cc:740] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.913314  8341 consensus_queue.cc:260] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.913360  8337 raft_consensus.cc:359] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.913932  8340 ts_tablet_manager.cc:1434] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 13:18:03.915506  8340 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:03.916395  8341 ts_tablet_manager.cc:1434] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 13:18:03.917541  8337 raft_consensus.cc:385] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.918013  8337 raft_consensus.cc:740] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.919150  8337 consensus_queue.cc:260] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.923995  8340 tablet_bootstrap.cc:654] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.924799  8341 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:03.933190  8337 ts_tablet_manager.cc:1434] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.005s
I20260709 13:18:03.936606  8337 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:03.936626  8340 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:03.937551  8340 ts_tablet_manager.cc:1403] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:18:03.937634  8341 tablet_bootstrap.cc:654] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.941562  8340 raft_consensus.cc:359] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.942422  8340 raft_consensus.cc:385] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.942791  8340 raft_consensus.cc:740] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.943643  8340 consensus_queue.cc:260] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.946532  8340 ts_tablet_manager.cc:1434] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:18:03.947962  8340 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:03.948546  8341 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:03.949308  8341 ts_tablet_manager.cc:1403] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.006s
I20260709 13:18:03.950304  8337 tablet_bootstrap.cc:654] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.953723  8341 raft_consensus.cc:359] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.954917  8341 raft_consensus.cc:385] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.955322  8341 raft_consensus.cc:740] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:03.956561  8340 tablet_bootstrap.cc:654] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.956310  8341 consensus_queue.cc:260] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.959735  8341 ts_tablet_manager.cc:1434] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:18:03.961282  8341 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:03.969010  8326 raft_consensus.cc:493] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.970361  8340 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:03.970026  8326 raft_consensus.cc:515] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.971040  8340 ts_tablet_manager.cc:1403] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 13:18:03.971657  8341 tablet_bootstrap.cc:654] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:03.974944  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:03.975241  8340 raft_consensus.cc:359] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.976403  8340 raft_consensus.cc:385] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:03.976806  8340 raft_consensus.cc:740] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
W20260709 13:18:03.978284  8006 leader_election.cc:343] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:03.977929  8340 consensus_queue.cc:260] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.988235  8340 ts_tablet_manager.cc:1434] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:18:03.990110  8340 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:03.992534  8339 raft_consensus.cc:493] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:03.993522  8339 raft_consensus.cc:515] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:03.996255  8341 tablet_bootstrap.cc:492] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:03.997143  8341 ts_tablet_manager.cc:1403] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.036s	user 0.028s	sys 0.004s
I20260709 13:18:04.000965  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.001385  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.001262  8341 raft_consensus.cc:359] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.002562  8341 raft_consensus.cc:385] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.003041  8341 raft_consensus.cc:740] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.002282  8041 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.006837  8341 consensus_queue.cc:260] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.006855  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.008273  8189 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.010342  8341 ts_tablet_manager.cc:1434] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.002s
I20260709 13:18:04.011811  8341 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:04.011415  8081 leader_election.cc:304] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:04.013017  8338 raft_consensus.cc:2804] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.013456  8338 raft_consensus.cc:493] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.013831  8338 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.013763  8008 leader_election.cc:343] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.014842  8008 leader_election.cc:304] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a; no voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:18:04.016278  8347 raft_consensus.cc:2749] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:03.998126  8339 leader_election.cc:290] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.018559  8326 leader_election.cc:290] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.022110  8341 tablet_bootstrap.cc:654] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.007946  8340 tablet_bootstrap.cc:654] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.035877  8341 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:04.036669  8341 ts_tablet_manager.cc:1403] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.006s
I20260709 13:18:04.037695  8338 raft_consensus.cc:515] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.041678  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.043015  8338 leader_election.cc:290] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.043224  8341 raft_consensus.cc:359] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.044348  8341 raft_consensus.cc:385] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.044795  8341 raft_consensus.cc:740] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.045902  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.045958  8341 consensus_queue.cc:260] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.047104  8041 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.048174  8340 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:04.048838  8340 ts_tablet_manager.cc:1403] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.059s	user 0.006s	sys 0.010s
I20260709 13:18:04.049654  8341 ts_tablet_manager.cc:1434] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:18:04.051046  8341 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:04.052634  8340 raft_consensus.cc:359] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.053110  8190 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.053705  8340 raft_consensus.cc:385] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.054157  8340 raft_consensus.cc:740] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.055060  8340 consensus_queue.cc:260] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.057789  8340 ts_tablet_manager.cc:1434] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:18:04.062002  8341 tablet_bootstrap.cc:654] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.066828  8340 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:04.079030  8347 raft_consensus.cc:493] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.080062  8347 raft_consensus.cc:515] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.081143  8340 tablet_bootstrap.cc:654] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.085728  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d658590c916e46a3b06ed414a0622af0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.093428  8189 raft_consensus.cc:2468] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:18:04.095099  8008 leader_election.cc:304] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.087435  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d658590c916e46a3b06ed414a0622af0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.083091  8341 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:04.097110  8341 ts_tablet_manager.cc:1403] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.005s
I20260709 13:18:04.087239  8347 leader_election.cc:290] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.098021  8347 raft_consensus.cc:2804] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.095197  8340 tablet_bootstrap.cc:492] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:04.098781  8347 raft_consensus.cc:493] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.099011  8340 ts_tablet_manager.cc:1403] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.040s	user 0.018s	sys 0.004s
I20260709 13:18:04.099377  8347 raft_consensus.cc:3060] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.103233  8340 raft_consensus.cc:359] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
W20260709 13:18:04.105021  8006 leader_election.cc:343] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.108243  8341 raft_consensus.cc:359] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.109217  8341 raft_consensus.cc:385] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.109536  8341 raft_consensus.cc:740] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.104202  8340 raft_consensus.cc:385] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.110984  8340 raft_consensus.cc:740] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.110538  8341 consensus_queue.cc:260] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.112437  8340 consensus_queue.cc:260] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.113507  8341 ts_tablet_manager.cc:1434] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:18:04.118634  8341 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
W20260709 13:18:04.120115  7989 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:18:04.110107  8190 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:18:04.125869  8083 leader_election.cc:304] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.127061  8338 raft_consensus.cc:2804] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.127664  8338 raft_consensus.cc:697] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:04.129021  8338 consensus_queue.cc:237] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.130453  8041 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:18:04.137307  8340 ts_tablet_manager.cc:1434] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:18:04.140333  8337 tablet_bootstrap.cc:492] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.141687  8337 ts_tablet_manager.cc:1403] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.205s	user 0.085s	sys 0.107s
I20260709 13:18:04.141376  8347 raft_consensus.cc:515] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.138695  8341 tablet_bootstrap.cc:654] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.145308  8340 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:04.151362  8337 raft_consensus.cc:359] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.152343  8337 raft_consensus.cc:385] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.145196  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d658590c916e46a3b06ed414a0622af0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.152742  8337 raft_consensus.cc:740] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.151376  7941 catalog_manager.cc:5697] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.153891  8337 consensus_queue.cc:260] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.155097  8118 raft_consensus.cc:3060] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.157254  8337 ts_tablet_manager.cc:1434] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 13:18:04.158947  8337 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.160055  8340 tablet_bootstrap.cc:654] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.167388  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d658590c916e46a3b06ed414a0622af0" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.168358  8190 raft_consensus.cc:3060] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.170120  8337 tablet_bootstrap.cc:654] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.171836  8118 raft_consensus.cc:2468] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:18:04.177242  8347 leader_election.cc:290] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.177313  8341 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:04.179349  8341 ts_tablet_manager.cc:1403] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.061s	user 0.021s	sys 0.009s
I20260709 13:18:04.182161  8190 raft_consensus.cc:2468] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:18:04.184521  8008 leader_election.cc:304] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.187400  8341 raft_consensus.cc:359] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.190343  8347 raft_consensus.cc:2804] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.191787  8347 raft_consensus.cc:697] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:04.193274  8347 consensus_queue.cc:237] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.196146  8341 raft_consensus.cc:385] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.196871  8341 raft_consensus.cc:740] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.197846  8341 consensus_queue.cc:260] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.201452  8341 ts_tablet_manager.cc:1434] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.007s
I20260709 13:18:04.202733  8341 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:04.209941  8337 tablet_bootstrap.cc:492] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.210702  8337 ts_tablet_manager.cc:1403] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.052s	user 0.038s	sys 0.006s
I20260709 13:18:04.212029  8341 tablet_bootstrap.cc:654] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.214572  8337 raft_consensus.cc:359] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.215469  8337 raft_consensus.cc:385] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.215813  8337 raft_consensus.cc:740] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.216933  8337 consensus_queue.cc:260] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.220763  8337 ts_tablet_manager.cc:1434] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:18:04.222095  8340 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:04.222398  8337 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.223320  8340 ts_tablet_manager.cc:1403] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.078s	user 0.026s	sys 0.011s
I20260709 13:18:04.227988  8338 raft_consensus.cc:493] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.229032  8338 raft_consensus.cc:515] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.233939  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.234903  8041 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.236738  8081 leader_election.cc:304] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:04.237890  8339 raft_consensus.cc:2804] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.238346  8339 raft_consensus.cc:493] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.238782  8339 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.240764  8337 tablet_bootstrap.cc:654] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.246960  8278 raft_consensus.cc:493] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.247711  8278 raft_consensus.cc:515] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.252575  8278 leader_election.cc:290] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.253203  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30feeaac6048464d9513b6da16bce336" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.254320  8190 raft_consensus.cc:2468] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.254678  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30feeaac6048464d9513b6da16bce336" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.255528  8041 raft_consensus.cc:2468] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.257052  8081 leader_election.cc:304] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:04.259419  8338 leader_election.cc:290] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.260210  8338 raft_consensus.cc:2804] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.260627  8338 raft_consensus.cc:493] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.261219  8338 raft_consensus.cc:3060] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.261767  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.262683  8189 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 0.
I20260709 13:18:04.270780  8347 raft_consensus.cc:493] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.271832  8347 raft_consensus.cc:515] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.272116  8338 raft_consensus.cc:515] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.274317  8339 raft_consensus.cc:515] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.276057  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.280895  8337 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.281612  8337 ts_tablet_manager.cc:1403] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.060s	user 0.017s	sys 0.009s
I20260709 13:18:04.276742  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30feeaac6048464d9513b6da16bce336" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.282235  8041 raft_consensus.cc:3060] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.283162  8339 leader_election.cc:290] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.284799  8297 raft_consensus.cc:493] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.285392  8297 raft_consensus.cc:515] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.277835  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.285869  8337 raft_consensus.cc:359] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.287204  8190 raft_consensus.cc:2468] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:18:04.287261  8337 raft_consensus.cc:385] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.287771  8337 raft_consensus.cc:740] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.278458  8347 leader_election.cc:290] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.288023  8297 leader_election.cc:290] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.289064  8337 consensus_queue.cc:260] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.279639  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30feeaac6048464d9513b6da16bce336" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.289388  8008 leader_election.cc:304] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.291486  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.292395  8043 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.292618  8042 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.290264  8189 raft_consensus.cc:3060] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.294731  8006 leader_election.cc:343] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.296622  8347 raft_consensus.cc:2804] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.297331  8347 raft_consensus.cc:493] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.298053  8347 raft_consensus.cc:3060] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:18:04.299427  8155 leader_election.cc:343] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.302693  7940 catalog_manager.cc:5697] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.307722  8341 tablet_bootstrap.cc:492] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:04.278185  8340 raft_consensus.cc:359] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.305696  8041 raft_consensus.cc:2468] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:18:04.308687  8347 raft_consensus.cc:515] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.308794  8341 ts_tablet_manager.cc:1403] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.106s	user 0.029s	sys 0.015s
I20260709 13:18:04.311654  8337 ts_tablet_manager.cc:1434] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.030s	user 0.010s	sys 0.000s
I20260709 13:18:04.312170  8340 raft_consensus.cc:385] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.312959  8337 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.313233  8081 leader_election.cc:304] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:04.313606  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.314220  8339 raft_consensus.cc:2804] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.314386  8191 raft_consensus.cc:3060] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.314889  8339 raft_consensus.cc:697] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:04.315960  8339 consensus_queue.cc:237] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.312610  8340 raft_consensus.cc:740] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.317621  8340 consensus_queue.cc:260] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.319854  8190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.320785  8190 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.321854  8340 ts_tablet_manager.cc:1434] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.001s
I20260709 13:18:04.322795  8347 leader_election.cc:290] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.323103  8337 tablet_bootstrap.cc:654] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.297222  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.323863  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.325692  8340 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
W20260709 13:18:04.326469  8006 leader_election.cc:343] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.323715  8341 raft_consensus.cc:359] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.308645  8189 raft_consensus.cc:2468] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
W20260709 13:18:04.327616  8155 leader_election.cc:343] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.328282  8155 leader_election.cc:304] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ca5ec37bf26843779baebb0fd163eb89; no voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a
I20260709 13:18:04.329294  8297 raft_consensus.cc:2749] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:04.327665  8341 raft_consensus.cc:385] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.331272  8341 raft_consensus.cc:740] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.335425  8341 consensus_queue.cc:260] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.338164  8341 ts_tablet_manager.cc:1434] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.004s
I20260709 13:18:04.334204  8191 raft_consensus.cc:2468] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:18:04.338984  8340 tablet_bootstrap.cc:654] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.339962  8008 leader_election.cc:304] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:04.341432  8347 raft_consensus.cc:2804] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.342003  8347 raft_consensus.cc:697] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:04.342969  8347 consensus_queue.cc:237] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.345008  8338 leader_election.cc:290] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.349346  8341 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: Bootstrap starting.
I20260709 13:18:04.355706  8337 tablet_bootstrap.cc:492] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.356846  8337 ts_tablet_manager.cc:1403] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.044s	user 0.015s	sys 0.021s
I20260709 13:18:04.359266  8190 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:18:04.360706  8043 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 1.
I20260709 13:18:04.362465  8083 leader_election.cc:304] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.361405  8337 raft_consensus.cc:359] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.363627  8339 raft_consensus.cc:2804] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.369349  8339 raft_consensus.cc:697] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:04.364355  8337 raft_consensus.cc:385] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.370361  8337 raft_consensus.cc:740] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.370471  8339 consensus_queue.cc:237] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.372910  8337 consensus_queue.cc:260] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.379796  7941 catalog_manager.cc:5697] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.382555  8341 tablet_bootstrap.cc:654] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.384585  8337 ts_tablet_manager.cc:1434] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 13:18:04.391500  8337 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.397812  8340 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:04.398900  8340 ts_tablet_manager.cc:1403] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.074s	user 0.019s	sys 0.003s
I20260709 13:18:04.400612  8337 tablet_bootstrap.cc:654] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.403571  8340 raft_consensus.cc:359] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.405099  8340 raft_consensus.cc:385] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.405552  8340 raft_consensus.cc:740] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.406724  8340 consensus_queue.cc:260] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.408972  8297 raft_consensus.cc:493] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.409783  8297 raft_consensus.cc:515] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.410506  8340 ts_tablet_manager.cc:1434] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:18:04.414312  8340 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: Bootstrap starting.
I20260709 13:18:04.416966  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.418076  8043 raft_consensus.cc:2468] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:18:04.419790  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.423154  8155 leader_election.cc:304] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.435227  8376 raft_consensus.cc:493] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.434484  8337 tablet_bootstrap.cc:492] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.438153  8337 ts_tablet_manager.cc:1403] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.047s	user 0.008s	sys 0.018s
I20260709 13:18:04.414382  8297 leader_election.cc:290] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.439628  8297 raft_consensus.cc:2804] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.442188  8297 raft_consensus.cc:493] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.437906  8376 raft_consensus.cc:515] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.444594  8297 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.446208  8337 raft_consensus.cc:359] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.448966  8337 raft_consensus.cc:385] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:18:04.444937  8155 leader_election.cc:343] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:18:04.449863  8337 raft_consensus.cc:740] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.453194  8337 consensus_queue.cc:260] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.458981  8337 ts_tablet_manager.cc:1434] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.002s
I20260709 13:18:04.456562  8341 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: No bootstrap required, opened a new log
I20260709 13:18:04.459070  8340 tablet_bootstrap.cc:654] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.459854  8341 ts_tablet_manager.cc:1403] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: Time spent bootstrapping tablet: real 0.111s	user 0.029s	sys 0.013s
I20260709 13:18:04.455229  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.460368  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.461450  8118 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:18:04.463874  8341 raft_consensus.cc:359] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.467443  8341 raft_consensus.cc:385] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.466446  7941 catalog_manager.cc:5697] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.460711  8043 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:18:04.467749  8341 raft_consensus.cc:740] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.468796  8341 consensus_queue.cc:260] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.469424  8155 leader_election.cc:304] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.459116  8376 leader_election.cc:290] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.470595  8377 raft_consensus.cc:2804] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.471081  8377 raft_consensus.cc:493] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.471427  8341 ts_tablet_manager.cc:1434] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:18:04.465792  8337 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.471498  8377 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.469152  7941 catalog_manager.cc:5697] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 1 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.481294  8337 tablet_bootstrap.cc:654] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.482208  8377 raft_consensus.cc:515] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.484856  8377 leader_election.cc:290] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.476127  8297 raft_consensus.cc:515] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.486922  8340 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: No bootstrap required, opened a new log
I20260709 13:18:04.487732  8340 ts_tablet_manager.cc:1403] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: Time spent bootstrapping tablet: real 0.074s	user 0.018s	sys 0.001s
I20260709 13:18:04.487731  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.488741  8043 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.489012  8297 leader_election.cc:290] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.491578  8340 raft_consensus.cc:359] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.492254  8340 raft_consensus.cc:385] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.492431  8340 raft_consensus.cc:740] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.493032  8340 consensus_queue.cc:260] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.495404  8340 ts_tablet_manager.cc:1434] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:18:04.497983  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.498903  8118 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.499032  8337 tablet_bootstrap.cc:492] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.499856  8337 ts_tablet_manager.cc:1403] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.000s
I20260709 13:18:04.508677  8118 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.509150  8337 raft_consensus.cc:359] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.510035  8337 raft_consensus.cc:385] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.510337  8337 raft_consensus.cc:740] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.511123  8337 consensus_queue.cc:260] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.511813  8043 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.513422  8337 ts_tablet_manager.cc:1434] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:18:04.514739  8337 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: Bootstrap starting.
I20260709 13:18:04.514508  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.515683  8155 leader_election.cc:304] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.516605  8297 raft_consensus.cc:2804] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.517110  8297 raft_consensus.cc:697] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:04.518018  8297 consensus_queue.cc:237] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.521178  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.522025  8118 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.522246  8043 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.530380  8337 tablet_bootstrap.cc:654] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: Neither blocks nor log segments found. Creating new log.
I20260709 13:18:04.532238  8118 raft_consensus.cc:2468] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.533836  8155 leader_election.cc:304] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.534825  8376 raft_consensus.cc:2804] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.535405  8376 raft_consensus.cc:697] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:04.536381  8376 consensus_queue.cc:237] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.539517  8043 raft_consensus.cc:2468] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.541438  8339 consensus_queue.cc:1048] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.543313  8347 raft_consensus.cc:493] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.545855  8347 raft_consensus.cc:515] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.553084  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.554275  8118 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:18:04.556457  8006 leader_election.cc:304] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:04.558199  8347 leader_election.cc:290] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.559007  8358 raft_consensus.cc:2804] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.559541  8358 raft_consensus.cc:493] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.559892  8358 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.559909  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.564034  8191 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 0.
I20260709 13:18:04.559852  8337 tablet_bootstrap.cc:492] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: No bootstrap required, opened a new log
I20260709 13:18:04.571288  8337 ts_tablet_manager.cc:1403] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: Time spent bootstrapping tablet: real 0.057s	user 0.025s	sys 0.011s
I20260709 13:18:04.574718  8337 raft_consensus.cc:359] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.575526  8337 raft_consensus.cc:385] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:18:04.575814  8337 raft_consensus.cc:740] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Initialized, Role: FOLLOWER
I20260709 13:18:04.573809  7941 catalog_manager.cc:5697] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.570384  8358 raft_consensus.cc:515] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.577651  7941 catalog_manager.cc:5697] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.579815  8337 consensus_queue.cc:260] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.583127  8337 ts_tablet_manager.cc:1434] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:18:04.578998  8358 leader_election.cc:290] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.580595  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:04.580590  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.583859  8191 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.587438  8339 consensus_queue.cc:1048] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.583887  8118 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.598109  8191 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:18:04.599998  8008 leader_election.cc:304] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.601261  8358 raft_consensus.cc:2804] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.601853  8358 raft_consensus.cc:697] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:04.603094  8358 consensus_queue.cc:237] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.604827  8347 consensus_queue.cc:1048] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.606542  8118 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 1.
I20260709 13:18:04.608297  8376 raft_consensus.cc:493] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.608878  8376 raft_consensus.cc:515] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.611373  8376 leader_election.cc:290] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.613598  8116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f73b8edc6e4b2b90ac0554c85ddc59" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7" is_pre_election: true
I20260709 13:18:04.613968  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f73b8edc6e4b2b90ac0554c85ddc59" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.614571  8116 raft_consensus.cc:2468] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:18:04.614787  8043 raft_consensus.cc:2468] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 0.
I20260709 13:18:04.616554  8155 leader_election.cc:304] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.619181  8376 raft_consensus.cc:2804] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:18:04.619650  8376 raft_consensus.cc:493] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:18:04.620102  8376 raft_consensus.cc:3060] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.624857  8339 raft_consensus.cc:493] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:18:04.625626  8339 raft_consensus.cc:515] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.629469  8339 leader_election.cc:290] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:04.631803  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a" is_pre_election: true
I20260709 13:18:04.634546  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd703609ff9a4095b953ec5774ebbd0b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca5ec37bf26843779baebb0fd163eb89" is_pre_election: true
I20260709 13:18:04.635639  8191 raft_consensus.cc:2393] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1065249cd7854a589e85734eea0cb7d7 in current term 1: Already voted for candidate 1b980f88bc014fc6be417a3736b2431a in this term.
I20260709 13:18:04.635083  7940 catalog_manager.cc:5697] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 1 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:04.636715  8376 raft_consensus.cc:515] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.637118  8083 leader_election.cc:304] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7; no voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89
I20260709 13:18:04.638562  8339 raft_consensus.cc:3060] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.654063  8339 raft_consensus.cc:2749] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:18:04.663522  8358 consensus_queue.cc:1048] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.673604  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f73b8edc6e4b2b90ac0554c85ddc59" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:04.675179  8041 raft_consensus.cc:3060] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.673604  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9f73b8edc6e4b2b90ac0554c85ddc59" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:04.677879  8118 raft_consensus.cc:3060] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:18:04.672479  8376 leader_election.cc:290] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:04.695402  8118 raft_consensus.cc:2468] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.696384  8041 raft_consensus.cc:2468] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 1.
I20260709 13:18:04.697429  8155 leader_election.cc:304] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:04.698525  8377 raft_consensus.cc:2804] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:18:04.699133  8377 raft_consensus.cc:697] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:04.700138  8377 consensus_queue.cc:237] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:04.709704  7941 catalog_manager.cc:5697] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 1 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:04.767692  8339 consensus_queue.cc:1048] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:04.784129  8338 consensus_queue.cc:1048] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.875828  8390 consensus_queue.cc:1048] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.890184  8390 consensus_queue.cc:1048] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.903748  8338 consensus_queue.cc:1048] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.911623  8338 consensus_queue.cc:1048] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.929710  8297 consensus_queue.cc:1048] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.945348  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9cdca70093f940099388f36c968b1cac"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:53366
I20260709 13:18:04.946118  8117 raft_consensus.cc:606] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS 1b980f88bc014fc6be417a3736b2431a
I20260709 13:18:04.953508  8377 consensus_queue.cc:1048] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.960155  8376 consensus_queue.cc:1048] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.970032  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:53366
I20260709 13:18:04.970808  8117 raft_consensus.cc:606] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS 1b980f88bc014fc6be417a3736b2431a
I20260709 13:18:04.985944  8297 consensus_queue.cc:1048] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:04.989951  8191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:42656
I20260709 13:18:04.990568  8191 raft_consensus.cc:606] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:04.999965  8041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:45824
I20260709 13:18:05.000679  8041 raft_consensus.cc:606] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:05.010793  8191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
mode: GRACEFUL
new_leader_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:42656
I20260709 13:18:05.011523  8191 raft_consensus.cc:606] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Received request to transfer leadership to TS 1065249cd7854a589e85734eea0cb7d7
I20260709 13:18:05.092338  8399 raft_consensus.cc:993] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:05.092955  8278 raft_consensus.cc:1081] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:05.094893  8376 consensus_queue.cc:1048] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:18:05.094931  8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9cdca70093f940099388f36c968b1cac"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:45806
I20260709 13:18:05.095748  8041 raft_consensus.cc:493] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:05.096187  8041 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.105187  8041 raft_consensus.cc:515] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.108158  8041 leader_election.cc:290] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:05.108633  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:05.109522  8118 raft_consensus.cc:3055] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.109910  8118 raft_consensus.cc:740] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:05.110878  8118 consensus_queue.cc:260] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.112154  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9cdca70093f940099388f36c968b1cac" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:05.112445  8404 consensus_queue.cc:1048] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:05.112942  8191 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.112197  8118 raft_consensus.cc:3060] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.122503  8191 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:05.124379  8008 leader_election.cc:304] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:05.125221  8118 raft_consensus.cc:2468] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:05.125465  8390 raft_consensus.cc:2804] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.126024  8390 raft_consensus.cc:697] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:05.127074  8390 consensus_queue.cc:237] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.138269  8390 consensus_queue.cc:1048] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:05.142016  7939 catalog_manager.cc:5697] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.163866  8358 consensus_queue.cc:1048] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:18:05.175859  8358 raft_consensus.cc:993] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.176450  8358 raft_consensus.cc:1081] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.178321  8118 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53352
I20260709 13:18:05.179056  8118 raft_consensus.cc:493] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:05.179457  8118 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.189015  8118 raft_consensus.cc:515] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.192378  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:05.192590  8118 leader_election.cc:290] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:05.193243  8041 raft_consensus.cc:3055] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.193611  8041 raft_consensus.cc:740] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:05.192449  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "849222e5c2d74fa5b133370d90e6d268" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:05.194346  8191 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.194589  8041 consensus_queue.cc:260] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.196127  8041 raft_consensus.cc:3060] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.203233  8191 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.204172  8041 raft_consensus.cc:2468] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.204867  8083 leader_election.cc:304] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:05.205919  8399 raft_consensus.cc:2804] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.206416  8399 raft_consensus.cc:697] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:05.207265  8399 consensus_queue.cc:237] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.215453  7941 catalog_manager.cc:5697] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from 1b980f88bc014fc6be417a3736b2431a (127.21.5.129) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.316901  8339 raft_consensus.cc:993] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:05.317494  8399 raft_consensus.cc:1081] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer 1b980f88bc014fc6be417a3736b2431a to start an election
I20260709 13:18:05.319278  8041 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
 from {username='slave'} at 127.0.0.1:45806
I20260709 13:18:05.319841  8041 raft_consensus.cc:493] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:05.320155  8041 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.325807  8041 raft_consensus.cc:515] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.327580  8041 leader_election.cc:290] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Requested vote from peers 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:05.328717  8118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:05.328845  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a17070010424c10896ecfeefbb687c1" candidate_uuid: "1b980f88bc014fc6be417a3736b2431a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:05.329555  8118 raft_consensus.cc:3055] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.329684  8191 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.329952  8118 raft_consensus.cc:740] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:05.330943  8118 consensus_queue.cc:260] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.332140  8118 raft_consensus.cc:3060] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.333879  8377 raft_consensus.cc:993] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.334506  8404 raft_consensus.cc:1081] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.336539  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53356
I20260709 13:18:05.337188  8117 raft_consensus.cc:493] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:05.337538  8117 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.338364  8191 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:05.339597  8118 raft_consensus.cc:2468] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b980f88bc014fc6be417a3736b2431a in term 2.
I20260709 13:18:05.339836  8008 leader_election.cc:304] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b980f88bc014fc6be417a3736b2431a, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:05.340906  8358 raft_consensus.cc:2804] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.341409  8358 raft_consensus.cc:697] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Becoming Leader. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:05.342257  8358 consensus_queue.cc:237] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.345602  8117 raft_consensus.cc:515] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.348780  8117 leader_election.cc:290] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:05.348820  8041 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:05.349623  8041 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.349835  8191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77d45ecd2f3345bd92acb90edbd95eb0" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:05.350505  8191 raft_consensus.cc:3055] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.350852  8191 raft_consensus.cc:740] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:05.352919  8191 consensus_queue.cc:260] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.354418  8191 raft_consensus.cc:3060] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.354115  7941 catalog_manager.cc:5697] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to 1b980f88bc014fc6be417a3736b2431a (127.21.5.129). New cstate: current_term: 2 leader_uuid: "1b980f88bc014fc6be417a3736b2431a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.358922  8041 raft_consensus.cc:2468] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.360456  8081 leader_election.cc:304] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:05.361315  8339 raft_consensus.cc:2804] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.361821  8339 raft_consensus.cc:697] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:05.362699  8339 consensus_queue.cc:237] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.363288  8191 raft_consensus.cc:2468] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.372149  7941 catalog_manager.cc:5697] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.448740  8377 raft_consensus.cc:993] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: : Instructing follower 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.449342  8404 raft_consensus.cc:1081] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Signalling peer 1065249cd7854a589e85734eea0cb7d7 to start an election
I20260709 13:18:05.451223  8117 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
 from {username='slave'} at 127.0.0.1:53356
I20260709 13:18:05.451905  8117 raft_consensus.cc:493] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:05.452301  8117 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.459023  8117 raft_consensus.cc:515] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.461277  8117 leader_election.cc:290] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131:43473)
I20260709 13:18:05.462471  8043 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:05.463522  8043 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.462700  8189 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3683256adef493fbb1545c6189ae91b" candidate_uuid: "1065249cd7854a589e85734eea0cb7d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
I20260709 13:18:05.466385  8189 raft_consensus.cc:3055] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:05.466849  8189 raft_consensus.cc:740] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:05.467870  8189 consensus_queue.cc:260] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.469507  8189 raft_consensus.cc:3060] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:05.476102  8043 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.477942  8081 leader_election.cc:304] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, 1b980f88bc014fc6be417a3736b2431a; no voters: 
I20260709 13:18:05.479251  8339 raft_consensus.cc:2804] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:05.479456  8189 raft_consensus.cc:2468] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1065249cd7854a589e85734eea0cb7d7 in term 2.
I20260709 13:18:05.479866  8339 raft_consensus.cc:697] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Becoming Leader. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:05.481324  8339 consensus_queue.cc:237] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:05.489845  7940 catalog_manager.cc:5697] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 reported cstate change: term changed from 1 to 2, leader changed from ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131) to 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130). New cstate: current_term: 2 leader_uuid: "1065249cd7854a589e85734eea0cb7d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: UNKNOWN } } }
I20260709 13:18:05.626438  8189 raft_consensus.cc:1275] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.627955  8347 consensus_queue.cc:1048] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.636684  8118 raft_consensus.cc:1275] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.639611  8347 consensus_queue.cc:1048] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.685756  8189 raft_consensus.cc:1275] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.687343  8339 consensus_queue.cc:1048] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.698122  8043 raft_consensus.cc:1275] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.701094  8339 consensus_queue.cc:1048] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.730770  8118 raft_consensus.cc:1275] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.732263  8347 consensus_queue.cc:1048] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.740566  8189 raft_consensus.cc:1275] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1b980f88bc014fc6be417a3736b2431a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.742027  8347 consensus_queue.cc:1048] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.908883  8041 raft_consensus.cc:1275] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.910990  8339 consensus_queue.cc:1048] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.921255  8191 raft_consensus.cc:1275] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.924343  8339 consensus_queue.cc:1048] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:05.986946  8041 raft_consensus.cc:1275] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.988281  8339 consensus_queue.cc:1048] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:05.996227  8191 raft_consensus.cc:1275] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Refusing update from remote peer 1065249cd7854a589e85734eea0cb7d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:05.997761  8278 consensus_queue.cc:1048] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:08.018996  7940 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57970:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:18:08.044171  7940 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:57970:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:18:08.079433 21526 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:18:08.244010  8444 log.cc:927] Time spent T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Append to log took a long time: real 0.072s	user 0.004s	sys 0.000s
W20260709 13:18:08.299208  8441 log.cc:927] Time spent T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 13:18:08.277068  8083 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.131:43473, user_credentials={real_user=slave}} blocked reactor thread for 107371us
W20260709 13:18:08.304250  8440 log.cc:927] Time spent T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
W20260709 13:18:08.315002  8442 log.cc:927] Time spent T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Append to log took a long time: real 0.134s	user 0.004s	sys 0.000s
W20260709 13:18:08.341941  8081 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:33275, user_credentials={real_user=slave}} blocked reactor thread for 80748.5us
I20260709 13:18:08.341437  8459 tablet.cc:1724] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:18:08.404860  8155 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.5.129:33275, user_credentials={real_user=slave}} blocked reactor thread for 71447.9us
I20260709 13:18:08.444007  8469 tablet.cc:1724] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.412849  8458 tablet.cc:1724] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.464191  8041 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7608af48fc5947b195079c23e77ae939"
dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
mode: GRACEFUL
new_leader_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:41536
I20260709 13:18:08.465258  8041 raft_consensus.cc:606] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Received request to transfer leadership to TS ca5ec37bf26843779baebb0fd163eb89
W20260709 13:18:08.527220  8465 log.cc:927] Time spent T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 13:18:08.586359  8478 tablet.cc:1724] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.594743  8482 tablet.cc:1724] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.612011  8483 tablet.cc:1724] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.612131  8480 tablet.cc:1724] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.571576  8475 tablet.cc:1724] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.578384  8479 tablet.cc:1724] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.539846  8472 tablet.cc:1724] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.653105  8486 tablet.cc:1724] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.613837  8484 tablet.cc:1724] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.693288  8483 tablet.cc:1724] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.698099  8458 tablet.cc:1724] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.735826  8469 tablet.cc:1724] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.742980  8459 tablet.cc:1724] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.748791  7940 catalog_manager.cc:5789] TS 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953) does not have the latest schema for tablet 30feeaac6048464d9513b6da16bce336 (table soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]). Expected version 1 got 0
I20260709 13:18:08.749912  7940 catalog_manager.cc:5789] TS 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953) does not have the latest schema for tablet 849222e5c2d74fa5b133370d90e6d268 (table soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]). Expected version 1 got 0
I20260709 13:18:08.761392  7940 catalog_manager.cc:5789] TS 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953) does not have the latest schema for tablet a3683256adef493fbb1545c6189ae91b (table soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d]). Expected version 1 got 0
I20260709 13:18:08.762090  8475 tablet.cc:1724] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.767549  8480 tablet.cc:1724] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.760880  8484 tablet.cc:1724] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.772611  8455 raft_consensus.cc:993] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: : Instructing follower ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:08.773293  8455 raft_consensus.cc:1081] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Signalling peer ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:08.776120  8186 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7608af48fc5947b195079c23e77ae939"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:42638
I20260709 13:18:08.776893  8186 raft_consensus.cc:493] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:08.777289  8186 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.782881  8486 tablet.cc:1724] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.789587  8483 tablet.cc:1724] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.797981  8480 tablet.cc:1724] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.803480  8479 tablet.cc:1724] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.830891  8117 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6"
dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
mode: GRACEFUL
new_leader_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:52194
I20260709 13:18:08.831641  8117 raft_consensus.cc:606] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Received request to transfer leadership to TS ca5ec37bf26843779baebb0fd163eb89
I20260709 13:18:08.836761 21526 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:18:08.836515  8482 tablet.cc:1724] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.837628 21526 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:18:08.840535  8472 tablet.cc:1724] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.841639  8186 raft_consensus.cc:515] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.849747  8470 raft_consensus.cc:993] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: : Instructing follower ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:08.850349  8186 leader_election.cc:290] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 3 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:08.850689  8474 raft_consensus.cc:1081] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Signalling peer ca5ec37bf26843779baebb0fd163eb89 to start an election
I20260709 13:18:08.857815  8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:08.859021  8040 raft_consensus.cc:3055] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:18:08.859375  8040 raft_consensus.cc:740] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1b980f88bc014fc6be417a3736b2431a, State: Running, Role: LEADER
I20260709 13:18:08.860277  8040 consensus_queue.cc:260] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.852167  8458 tablet.cc:1724] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.861610  8040 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.861263  8469 tablet.cc:1724] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:18:08.867051  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7608af48fc5947b195079c23e77ae939" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:08.867856  8117 raft_consensus.cc:3060] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:18:08.892647  8186 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6"
dest_uuid: "ca5ec37bf26843779baebb0fd163eb89"
 from {username='slave'} at 127.0.0.1:42648
I20260709 13:18:08.893535  8186 raft_consensus.cc:493] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:18:08.894001  8186 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.883795  8117 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 3.
I20260709 13:18:08.897310  8155 leader_election.cc:304] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:08.898403  8466 raft_consensus.cc:2804] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:18:08.899057  8466 raft_consensus.cc:697] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 3 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:08.899672  8040 raft_consensus.cc:2468] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 3.
I20260709 13:18:08.900143  8466 consensus_queue.cc:237] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.904186  8186 raft_consensus.cc:515] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.907575  8186 leader_election.cc:290] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 2 election: Requested vote from peers 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275), 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953)
I20260709 13:18:08.909936  8117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1065249cd7854a589e85734eea0cb7d7"
I20260709 13:18:08.910629  8117 raft_consensus.cc:3055] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:18:08.910957  8117 raft_consensus.cc:740] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1065249cd7854a589e85734eea0cb7d7, State: Running, Role: LEADER
I20260709 13:18:08.911849  8117 consensus_queue.cc:260] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.913357  8117 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.918663  8040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8e0e96c35b24bd6907e0e8a77b02ab6" candidate_uuid: "ca5ec37bf26843779baebb0fd163eb89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1b980f88bc014fc6be417a3736b2431a"
I20260709 13:18:08.920506  8040 raft_consensus.cc:3060] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:18:08.924100  8117 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 2.
I20260709 13:18:08.925881  8155 leader_election.cc:304] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1065249cd7854a589e85734eea0cb7d7, ca5ec37bf26843779baebb0fd163eb89; no voters: 
I20260709 13:18:08.926947  8466 raft_consensus.cc:2804] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:18:08.927054  7941 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: soft_deleted_table [id=902bc639f78e483f8cc5306fc9afc42d] alter complete (version 1)
I20260709 13:18:08.927531  8466 raft_consensus.cc:697] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 2 LEADER]: Becoming Leader. State: Replica: ca5ec37bf26843779baebb0fd163eb89, State: Running, Role: LEADER
I20260709 13:18:08.928529  8466 consensus_queue.cc:237] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } }
I20260709 13:18:08.930081  8040 raft_consensus.cc:2468] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca5ec37bf26843779baebb0fd163eb89 in term 2.
I20260709 13:18:08.945546  7940 catalog_manager.cc:5697] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 2 to 3, leader changed from 1b980f88bc014fc6be417a3736b2431a (127.21.5.129) to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 3 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:08.946964  7940 catalog_manager.cc:5697] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 reported cstate change: term changed from 1 to 2, leader changed from 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130) to ca5ec37bf26843779baebb0fd163eb89 (127.21.5.131). New cstate: current_term: 2 leader_uuid: "ca5ec37bf26843779baebb0fd163eb89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5ec37bf26843779baebb0fd163eb89" member_type: VOTER last_known_addr { host: "127.21.5.131" port: 43473 } health_report { overall_health: HEALTHY } } }
I20260709 13:18:09.298606  8117 raft_consensus.cc:1275] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 3 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:09.300263  8466 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:09.312709  8040 raft_consensus.cc:1275] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 3 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:18:09.315616  8466 consensus_queue.cc:1048] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:18:09.372188  8040 raft_consensus.cc:1275] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:09.373612  8466 consensus_queue.cc:1048] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b980f88bc014fc6be417a3736b2431a" member_type: VOTER last_known_addr { host: "127.21.5.129" port: 33275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:18:09.384655  8117 raft_consensus.cc:1275] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Refusing update from remote peer ca5ec37bf26843779baebb0fd163eb89: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:18:09.387620  8506 consensus_queue.cc:1048] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1065249cd7854a589e85734eea0cb7d7" member_type: VOTER last_known_addr { host: "127.21.5.130" port: 44953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:18:09.909583 21526 tablet_server.cc:179] TabletServer@127.21.5.129:0 shutting down...
W20260709 13:18:09.912130  8081 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 12 similar messages]
W20260709 13:18:09.916826  8081 consensus_peers.cc:597] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:09.924949  8155 consensus_peers.cc:597] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:09.948915 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:09.949941 21526 tablet_replica.cc:333] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.951084 21526 raft_consensus.cc:2243] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.951835 21526 raft_consensus.cc:2272] T 849222e5c2d74fa5b133370d90e6d268 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.954478 21526 tablet_replica.cc:333] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.955181 21526 raft_consensus.cc:2243] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.955636 21526 raft_consensus.cc:2272] T a3683256adef493fbb1545c6189ae91b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.957883 21526 tablet_replica.cc:333] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.958546 21526 raft_consensus.cc:2243] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:09.959540 21526 raft_consensus.cc:2272] T 6a17070010424c10896ecfeefbb687c1 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.961704 21526 tablet_replica.cc:333] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.962325 21526 raft_consensus.cc:2243] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.962888 21526 raft_consensus.cc:2272] T 30feeaac6048464d9513b6da16bce336 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.965283 21526 tablet_replica.cc:333] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.965907 21526 raft_consensus.cc:2243] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:09.966933 21526 raft_consensus.cc:2272] T dd703609ff9a4095b953ec5774ebbd0b P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.969702 21526 tablet_replica.cc:333] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.970337 21526 raft_consensus.cc:2243] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.970917 21526 raft_consensus.cc:2272] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.973229 21526 tablet_replica.cc:333] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.973889 21526 raft_consensus.cc:2243] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:09.975005 21526 raft_consensus.cc:2272] T d658590c916e46a3b06ed414a0622af0 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.977486 21526 tablet_replica.cc:333] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.978124 21526 raft_consensus.cc:2243] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:09.979200 21526 raft_consensus.cc:2272] T 10e88db52a2f47e0b370e51feecef5c9 P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.981752 21526 tablet_replica.cc:333] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.982407 21526 raft_consensus.cc:2243] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:09.983322 21526 raft_consensus.cc:2272] T 9cdca70093f940099388f36c968b1cac P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.985486 21526 tablet_replica.cc:333] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.986181 21526 raft_consensus.cc:2243] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.986773 21526 raft_consensus.cc:2272] T 9afc2afc03d24282b30770b16fcb90d7 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.989166 21526 tablet_replica.cc:333] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.989807 21526 raft_consensus.cc:2243] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.990278 21526 raft_consensus.cc:2272] T 7608af48fc5947b195079c23e77ae939 P 1b980f88bc014fc6be417a3736b2431a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.994081 21526 tablet_replica.cc:333] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.994789 21526 raft_consensus.cc:2243] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.995350 21526 raft_consensus.cc:2272] T a6b065971a594587baa2cc73228d3e2b P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:09.997737 21526 tablet_replica.cc:333] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:09.998364 21526 raft_consensus.cc:2243] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:09.998924 21526 raft_consensus.cc:2272] T 6687da6ebcda4a1d9f350d992d631bf2 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.001226 21526 tablet_replica.cc:333] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:10.001912 21526 raft_consensus.cc:2243] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.002370 21526 raft_consensus.cc:2272] T a014352cb2b14c8db9ddf4de9884caec P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.005007 21526 tablet_replica.cc:333] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:10.005551 21526 raft_consensus.cc:2243] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.006251 21526 raft_consensus.cc:2272] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.009068 21526 tablet_replica.cc:333] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:10.009641 21526 raft_consensus.cc:2243] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.010579 21526 raft_consensus.cc:2272] T 71104e08d54d484b8ffc160110130f09 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.013181 21526 tablet_replica.cc:333] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:10.013867 21526 raft_consensus.cc:2243] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.014345 21526 raft_consensus.cc:2272] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1b980f88bc014fc6be417a3736b2431a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.016589 21526 tablet_replica.cc:333] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a: stopping tablet replica
I20260709 13:18:10.017069 21526 raft_consensus.cc:2243] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.018064 21526 raft_consensus.cc:2272] T 20206412d4034cfeab74a505acba50bf P 1b980f88bc014fc6be417a3736b2431a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:10.023638  8081 consensus_peers.cc:597] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:10.061601  8081 consensus_peers.cc:597] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:33275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.065613 21526 tablet_server.cc:196] TabletServer@127.21.5.129:0 shutdown complete.
I20260709 13:18:10.111788 21526 tablet_server.cc:179] TabletServer@127.21.5.130:0 shutting down...
W20260709 13:18:10.147053  8081 consensus_peers.cc:597] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:33275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:10.152192  8155 consensus_peers.cc:597] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Couldn't send request to peer 1065249cd7854a589e85734eea0cb7d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.157011 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:10.158063 21526 tablet_replica.cc:333] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.158900 21526 raft_consensus.cc:2243] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.159888 21526 raft_consensus.cc:2272] T 849222e5c2d74fa5b133370d90e6d268 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.162295 21526 tablet_replica.cc:333] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.163064 21526 raft_consensus.cc:2243] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.164060 21526 raft_consensus.cc:2272] T 77d45ecd2f3345bd92acb90edbd95eb0 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.166563 21526 tablet_replica.cc:333] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.167439 21526 raft_consensus.cc:2243] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.168083 21526 raft_consensus.cc:2272] T dd703609ff9a4095b953ec5774ebbd0b P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.170490 21526 tablet_replica.cc:333] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.171231 21526 raft_consensus.cc:2243] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.172183 21526 raft_consensus.cc:2272] T a3683256adef493fbb1545c6189ae91b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.174521 21526 tablet_replica.cc:333] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.175277 21526 raft_consensus.cc:2243] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.176235 21526 raft_consensus.cc:2272] T 30feeaac6048464d9513b6da16bce336 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.178748 21526 tablet_replica.cc:333] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.179412 21526 raft_consensus.cc:2243] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.179917 21526 raft_consensus.cc:2272] T e9f73b8edc6e4b2b90ac0554c85ddc59 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.182124 21526 tablet_replica.cc:333] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.182889 21526 raft_consensus.cc:2243] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.183399 21526 raft_consensus.cc:2272] T d658590c916e46a3b06ed414a0622af0 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.185508 21526 tablet_replica.cc:333] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.186122 21526 raft_consensus.cc:2243] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.186695 21526 raft_consensus.cc:2272] T 7608af48fc5947b195079c23e77ae939 P 1065249cd7854a589e85734eea0cb7d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.190393 21526 tablet_replica.cc:333] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.191159 21526 raft_consensus.cc:2243] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.191725 21526 raft_consensus.cc:2272] T 6a17070010424c10896ecfeefbb687c1 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.194682 21526 tablet_replica.cc:333] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
W20260709 13:18:10.194990  8155 consensus_peers.cc:597] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Couldn't send request to peer 1065249cd7854a589e85734eea0cb7d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.195443 21526 raft_consensus.cc:2243] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.196091 21526 raft_consensus.cc:2272] T a6b065971a594587baa2cc73228d3e2b P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.198562 21526 tablet_replica.cc:333] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.199280 21526 raft_consensus.cc:2243] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.200238 21526 raft_consensus.cc:2272] T 6687da6ebcda4a1d9f350d992d631bf2 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.202852 21526 tablet_replica.cc:333] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.203493 21526 raft_consensus.cc:2243] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.203984 21526 raft_consensus.cc:2272] T 10e88db52a2f47e0b370e51feecef5c9 P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.206568 21526 tablet_replica.cc:333] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.207232 21526 raft_consensus.cc:2243] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.207696 21526 raft_consensus.cc:2272] T 9cdca70093f940099388f36c968b1cac P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.210160 21526 tablet_replica.cc:333] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.210796 21526 raft_consensus.cc:2243] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.211838 21526 raft_consensus.cc:2272] T 9afc2afc03d24282b30770b16fcb90d7 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.214445 21526 tablet_replica.cc:333] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.215112 21526 raft_consensus.cc:2243] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.215571 21526 raft_consensus.cc:2272] T d8e0e96c35b24bd6907e0e8a77b02ab6 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.218585 21526 tablet_replica.cc:333] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.219194 21526 raft_consensus.cc:2243] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.220115 21526 raft_consensus.cc:2272] T a014352cb2b14c8db9ddf4de9884caec P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.222636 21526 tablet_replica.cc:333] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.223158 21526 raft_consensus.cc:2243] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.223623 21526 raft_consensus.cc:2272] T 20206412d4034cfeab74a505acba50bf P 1065249cd7854a589e85734eea0cb7d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.226171 21526 tablet_replica.cc:333] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7: stopping tablet replica
I20260709 13:18:10.226963 21526 raft_consensus.cc:2243] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.227770 21526 raft_consensus.cc:2272] T 71104e08d54d484b8ffc160110130f09 P 1065249cd7854a589e85734eea0cb7d7 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:18:10.240665  8155 consensus_peers.cc:597] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:33275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:10.265448  8155 consensus_peers.cc:597] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Couldn't send request to peer 1065249cd7854a589e85734eea0cb7d7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:44953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:18:10.273237  8155 consensus_peers.cc:597] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1b980f88bc014fc6be417a3736b2431a (127.21.5.129:33275): Couldn't send request to peer 1b980f88bc014fc6be417a3736b2431a. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.129:33275: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.276609 21526 tablet_server.cc:196] TabletServer@127.21.5.130:0 shutdown complete.
I20260709 13:18:10.320890 21526 tablet_server.cc:179] TabletServer@127.21.5.131:0 shutting down...
I20260709 13:18:10.370791 21526 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:18:10.371704 21526 tablet_replica.cc:333] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
W20260709 13:18:10.371703  8155 consensus_peers.cc:597] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 -> Peer 1065249cd7854a589e85734eea0cb7d7 (127.21.5.130:44953): Couldn't send request to peer 1065249cd7854a589e85734eea0cb7d7. Status: Network error: Client connection negotiation failed: client connection to 127.21.5.130:44953: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:18:10.373046 21526 raft_consensus.cc:2243] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.373660 21526 raft_consensus.cc:2272] T a3683256adef493fbb1545c6189ae91b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.376113 21526 tablet_replica.cc:333] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.376811 21526 raft_consensus.cc:2243] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.377404 21526 raft_consensus.cc:2272] T dd703609ff9a4095b953ec5774ebbd0b P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.379793 21526 tablet_replica.cc:333] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.380481 21526 raft_consensus.cc:2243] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.381027 21526 raft_consensus.cc:2272] T 77d45ecd2f3345bd92acb90edbd95eb0 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.383358 21526 tablet_replica.cc:333] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.384030 21526 raft_consensus.cc:2243] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.385042 21526 raft_consensus.cc:2272] T e9f73b8edc6e4b2b90ac0554c85ddc59 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.387431 21526 tablet_replica.cc:333] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.388099 21526 raft_consensus.cc:2243] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.388633 21526 raft_consensus.cc:2272] T d658590c916e46a3b06ed414a0622af0 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.390923 21526 tablet_replica.cc:333] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.391559 21526 raft_consensus.cc:2243] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.392061 21526 raft_consensus.cc:2272] T 30feeaac6048464d9513b6da16bce336 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.394240 21526 tablet_replica.cc:333] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.394907 21526 raft_consensus.cc:2243] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.395418 21526 raft_consensus.cc:2272] T 849222e5c2d74fa5b133370d90e6d268 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.397635 21526 tablet_replica.cc:333] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.398274 21526 raft_consensus.cc:2243] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.398825 21526 raft_consensus.cc:2272] T 9afc2afc03d24282b30770b16fcb90d7 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.401258 21526 tablet_replica.cc:333] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.401888 21526 raft_consensus.cc:2243] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.402872 21526 raft_consensus.cc:2272] T 7608af48fc5947b195079c23e77ae939 P ca5ec37bf26843779baebb0fd163eb89 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.405426 21526 tablet_replica.cc:333] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.406098 21526 raft_consensus.cc:2243] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.406644 21526 raft_consensus.cc:2272] T 71104e08d54d484b8ffc160110130f09 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.409129 21526 tablet_replica.cc:333] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.409744 21526 raft_consensus.cc:2243] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.410262 21526 raft_consensus.cc:2272] T a014352cb2b14c8db9ddf4de9884caec P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.412982 21526 tablet_replica.cc:333] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.413637 21526 raft_consensus.cc:2243] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.414641 21526 raft_consensus.cc:2272] T d8e0e96c35b24bd6907e0e8a77b02ab6 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.417480 21526 tablet_replica.cc:333] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.418179 21526 raft_consensus.cc:2243] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.418787 21526 raft_consensus.cc:2272] T 6a17070010424c10896ecfeefbb687c1 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.421010 21526 tablet_replica.cc:333] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.421638 21526 raft_consensus.cc:2243] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.422648 21526 raft_consensus.cc:2272] T a6b065971a594587baa2cc73228d3e2b P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.425213 21526 tablet_replica.cc:333] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.425942 21526 raft_consensus.cc:2243] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.426568 21526 raft_consensus.cc:2272] T 6687da6ebcda4a1d9f350d992d631bf2 P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.429340 21526 tablet_replica.cc:333] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.430045 21526 raft_consensus.cc:2243] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.430578 21526 raft_consensus.cc:2272] T 20206412d4034cfeab74a505acba50bf P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.433467 21526 tablet_replica.cc:333] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.434198 21526 raft_consensus.cc:2243] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.434953 21526 raft_consensus.cc:2272] T 9cdca70093f940099388f36c968b1cac P ca5ec37bf26843779baebb0fd163eb89 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.437305 21526 tablet_replica.cc:333] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89: stopping tablet replica
I20260709 13:18:10.437901 21526 raft_consensus.cc:2243] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:18:10.438387 21526 raft_consensus.cc:2272] T 10e88db52a2f47e0b370e51feecef5c9 P ca5ec37bf26843779baebb0fd163eb89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.479578 21526 tablet_server.cc:196] TabletServer@127.21.5.131:0 shutdown complete.
I20260709 13:18:10.526015 21526 master.cc:562] Master@127.21.5.190:35245 shutting down...
I20260709 13:18:10.553257 21526 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:18:10.554034 21526 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:18:10.554415 21526 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9bf0ca574fd34304a11e2647cd080d34: stopping tablet replica
I20260709 13:18:10.578375 21526 master.cc:584] Master@127.21.5.190:35245 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (12743 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (31103 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (584103 ms total)
[  PASSED  ] 6 tests.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver
[  FAILED  ] LeaderRebalancerTest.RestartTserver

 2 FAILED TESTS
I20260709 13:18:10.643358 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 615 messages since previous log ~18 seconds ago
I20260709 13:18:10.643612 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 615 messages since previous log ~18 seconds ago
I20260709 13:18:10.643775 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 611 messages since previous log ~18 seconds ago
I20260709 13:18:10.643985 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~40 seconds ago
I20260709 13:18:10.644160 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~40 seconds ago
I20260709 13:18:10.644335 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 7 messages since previous log ~93 seconds ago
I20260709 13:18:10.644583 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 13:18:10.644867 21526 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~11 seconds ago
